Introducing JsonSrcGen 1.1

JsonSrcGen 1.1 has been released. This is the first release that includes UTF8 support.

Why UTF8?

UTF8 support is a big deal for a Json serializer. In c# strings are UTF16, but the vast majority of real JSON usage is UTF8, files, rest API's all use UTF8. This means a JSON serializer that uses .net strings has to do a double conversion, first from UTF8 to a .Net string (UTF16) then from the .net string to the json object. If a JSON serializer can work with UTF16 directly then a large amount of time can be saved.

Features

• UTF8 Support
• Decimal type support
• Struct and read only struct support

Code Examples

// To Json UTF8
ReadOnlySpan<byte> json = convert.ToJsonUtf8(new MyType(){MyProperty = "Some value"});

// From Json UTF8
ReadOnlySpan<byte> utf8Json = Encoding.Utf8.GetBytes("{\"MyProperty\:\"Some value\"}");
convert.FromJson(utf8Json, myType);

Benchmarks

The code for the following benchmarks can be found in github. The following class is used for these benchmarks:

public class JsonTestClass
{
    public string FirstName{get;set;}
    public string LastName{get;set;}
    public int Age{get;set;}
    public bool Registered {get;set;}
}

How to get it

Please checkout the JsonSrcGen project on github: https://github.com/trampster/JsonSrcGen

JsonSrcGen is available as a nuget package:

https://www.nuget.org/packages/JsonSrcGen

Thanks to...

A special thanks goes out to the people who have contributed to JsonSrcGen for this release.

@trampster - Daniel Hughes

@hugobritobh - Hugo de Brito V. R. Alves

@sirh3e - Marvin Huber

@Youssef1313 - Youssef Victor

How to contribute

• Testing/bug reports
• Submit pull requests
• Github sponsorship

 

 Introducing JsonSrcGen 1.0

JsonSrcGen 1.0.3 has been released. JsonSrcGen 1.0 is a production ready stable release. 

Features

• Compile time code generation using c# Source Generators
• Serialization to and from strings
• Custom serializers
• Support for all common .net types including Lists, Arrays and Dictionarys
• High performance in both startup and runtime

Whats Next?

JsonSrcGen 1.0 will only get bug fixes going forward. Work will start on JsonSrcGen 1.1 which will add the ability to serialise to an from utf8 data. In real world applications it is more likely that you will need to work with utf8 than with strings, so JsonSrcGen 1.1 should provide a significant boost for those use cases.

Please checkout the JsonSrcGen project on github: https://github.com/trampster/JsonSrcGen

JsonSrcGen is available as a nuget package:

https://www.nuget.org/packages/JsonSrcGen

 

JsonSrcGen 1.0.1 RC 2 Released

JsonSrcGen 1.0.1 RC 2 has been released, This is the second release candidate leading up to the first production ready release. All going well this will be the last release candidate before the first stable release.

New Features

• Added JsonOptionalAttribute to specify a property should be set to default if not present in the JSON during deserialisation.

Bug Fixes

• Fixed deserialising empty objects

JsonOptionalAttribute

Added JsonOptionalAttribute to specify a property should be set to default if not present in the JSON during deserialisation. This is only necessary if you are reusing an object for multiple deserialisations.

public class MyJsonType
{
    [JsonOptional]
    string MyProperty{get;set;}
}

Real Json Testing program

If you would like to ensure your Json API works with JsonSrcGen then you can submit your tests to be included in our RealJsonTests folder via a merge request. The JsonSrcGen developers will ensure that any tests included here pass for each new release.

Tests must meet the following criteria:

• Use nunit
• Use local json data (no external API calls from the tests)

API's are eligible for free inclusion under any of the following conditions:

• The API is available to the public free of charge.
• The server or client code is available under an OSI approved opensource licence.

If your API does not meet the above conditions please contacts the maintainers to discus how you can support the development in exchange for having your tests included.

Call to testing

Please test JsonSrcGen against your Json and raise bug reports for any problem you find.

Please checkout the JsonSrcGen project on github: https://github.com/trampster/JsonSrcGen

JsonSrcGen is available as a nuget package:

https://www.nuget.org/packages/JsonSrcGen

JsonSrcGen 1.0.0.1 RC 1 Released

JsonSrcGen 1.0.0.1 RC 1 has been released, This is the first release candidate leading up to the first production ready release. JsonSrcGen is now feature complete.

Changes

• Serialises from ReadOnlySpan<char> instead of from string

New Features

• Support skipping null values when serialising
• Support DateTimeOffset

Serialises from ReadOnlySpan<char> instead of from string

From Json conversions now serialize from ReadOnlySpan<char> instead of string. Strings can be converted to ReadOnlySpan<char> very cheaply but converting a ReadOnlySpan<char> to a string is expensive because it requires allocating memory. c# will automatically convert string to ReadOnlySpan<char> so you can continue to use strings.

Support skipping null values when serialising

The attribute [JsonIgnoreNull] can be added to a class to instruct JsonSrcGen to skip serializing null values. 

Real Json Testing program

If you would like to ensure your Json API works with JsonSrcGen then you can submit your tests to be included in our RealJsonTests folder via a merge request. The JsonSrcGen developers will ensure that any tests included here pass for each new release.

Tests must meet the following criteria:

• Use nunit
• Use local json data (no external API calls from the tests)

API's are eligible for free inclusion under any of the following conditions:

• The API is available to the public free of charge.
• The server or client code is available under an OSI approved opensource licence.

If your API does not meet the above conditions please contacts the maintainers to discus how you can support the development in exchange for having your tests included.

Call to testing

Please test JsonSrcGen against your Json and raise bug reports for any problem you find.

Please checkout the JsonSrcGen project on github: https://github.com/trampster/JsonSrcGen

JsonSrcGen is available as a nuget package:

https://www.nuget.org/packages/JsonSrcGen

JsonSrcGen 0.2.0 Alpha Released

JsonSrcGen 0.2.0 alpha has been published. This release contains the following changes:

Breaking Changes

• Serialise to ReadOnlySpan<char> instead of string
• JsonSrcGenConvert renamed to JsonConverter

New Features

• Support for Json Values
• Support for Custom Converters
• Support for char

Serialise to ReadOnlySpan<char> instead of string

To Json conversions now produce ReadOnlySpan<char> instead of string, is produces a significant speedup as it allows us to avoid the memory allocation required to create the String. The ReadOnlySpan reuses the same memory for each ToJson conversion within a thread. Because of this the ReadOnlySpan must be consumed before calling ToJson again or the data it points at will change.

Support for Json Values

Simple Json Values can not be convertered to and from Json. To do this you must specify a JsonValue attribute at the solution level as follows:

[assembly: JsonValue(typeof(int))]

...

var converter = new JsonConverter

ReadOnlySpan<char> json = converter.ToJson(1456);

int value = converter.FromJson(0, "1456"); 

Support for Custom Converters

Custom Converters allow you to provide a custom conversion code for a type. To do so you must implement ICustomConverter<T> and add the CustomConverter attribute to your class.

[CustomConverter(typeof(int))]
public class CustomCaseStringConverter : ICustomConverter<int>
{
    public void ToJson(IJsonBuilder builder, int value)
    {
        // Write your json to the builder here
    }

    public ReadOnlySpan<char> FromJson(ReadOnlySpan<char> json, ref int value)
    {
        // Read the Json from the json span here
    }
}

Please checkout the JsonSrcGen project on github: https://github.com/trampster/JsonSrcGen

JsonSrcGen is available as a nuget package:

https://www.nuget.org/packages/JsonSrcGen

JsonSrcGen + CoreRT = Pure Magic

In my previous post I talked about how using a Source Generator allows us to make a reflection free Json generator. This has serious advantages when it comes deployment size and startup time.

These advantages become even bigger when paired with an AOT or Ahead Of Time compiler like CoreRT. CoreRT can produce impressively small binaries but cannot do Reflection.Emit and has only limited support for reflection. But seeing as JsonSrcGen doesn't do any reflection the two are a match made in heaven.

For me this produces a binary that is only 2.1 MB and will startup and serialise a simple Json class in only 5 ms.

In order to see just how good this is here a comparison to what you can do with the officially supported .NET Json and self contained publishing options. Which included the new Trimming and Ready To Run options.

How to use JsonSrcGen with CoreRT

1. Create a new .NET 5 console project

dotnet new console

2. Add the CoreRT Package Source

CoreRT requires a custom package source to be added to your nuget.config. I didn't have one so added one using: 

dotnet new nuget

Then add the dotnet-core Package Source to your nuget config so it looks as follows:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="dotnet-core" value="https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json" />
    <add key="nuget" value="https://api.nuget.org/v3/index.json" />
  </packageSources>
</configuration>

3. Add the CoreRT and JsonSrcGen nuget packages

dotnet add package JsonSrcGen

dotnet add package Microsoft.DotNet.ILCompiler

4. Add the following lines to you .csproj file

<LangVersion>preview</LangVersion>
<IlcDisableReflection>true</IlcDisableReflection>
<IlcInvariantGlobalization>true</IlcInvariantGlobalization>

The first line is required because Source Generators are currently in preview, this will not be required once they are released.

The Second line tells CoreRT that we do not need reflection, this is a significant saving in binary size.

The final line tells CoreRT that we do not need localisation support.

5. Install prerequisites

What you need here depends on your platform, but for me on linux I needed the following:

sudo apt-get install clang zlib1g-dev libkrb5-dev libtinfo5

6. Then build the project to get your output

dotnet publish -r linux-x64 -c Release

The -r linux-x64 tells CoreRT to build for linux 64 bit, if you are on a different platform you will need to change this.

7. If you are on linux use the strip command to get rid of debug information

This step is only required on linux, because the debug information ends up in the binary on linux.

strip bin/Release/net5.0/linux-x64/publish/CoreRTSample

Please checkout the JsonSrcGen project on github: https://github.com/trampster/JsonSrcGen

You can find a JsonSrcGen CoreRT sample project in our samples folder

Introducing JsonSrcGen

Over the last month I've been working on a new Json Library for .net that takes advantage of the new c# Source Generator feature that is being previewed in .NET 5. I've called this JsonSrcGen which is short for Json Source Generator.

Why yet another Json library?

Source Generators allow us to generate optimal Json serialization code at compile time. Up until now Json Libraries have made heavy use of Reflection and Reflection.Emit to produce serialization code at runtime. But this approach has some draw backs.

• Startup times are slow because of the reflection and time to emit the serialization code.
• AOT (Ahead of Time) platforms like Xamarin.iOS, Blazor and CoreRT struggle with Reflection and cannot do Reflection.Emit. Even if they can do reflection they can get significant size savings if they exclude it
• They require slow runtime lookups to match Types to generated Serializers

How do I use it?

Source Generators currently require .NET 5 and LangVersion set to 'preview'

<TargetFramework>net5.0</TargetFramework>
<LangVersion>preview</LangVersion>

Add a nuget reference to our nuget package

Annotate your c# class with JsonSrcGen Attributes:

[Json]
public class MyType
{
    public int Age {get;set}

    [JsonName("my_property")]
    public string MyProperty {get;set}   

    [JsonIgnore]
    public string IgnoredProperty {get;set;}
}

Then use JsonSrcGenConvert instance to convert to and from Json strings.

var converter = new JsonSrcGenConvert();

//convert from Json
MyType myType = new MyType();
converter.FromJson(myType, jsonString);

//convert to json
jsonString = converter.ToJson(new MyType());

Notice that when converting from Json you supply an already instantiated instance of the type, this allows you to reuse type instances and thus reduce memory allocations. Which adds up to a significant performance boost while deserializing.

How fast is it?

I am not going to make any claims about the performance of JsonSrcGen. Many .net serializers have made claims about being the fastest only to have their claims age very poorly. Instead I will just say that JsonSrcGen has a strong focus on performance. Below is a benchmark of serialization and deserialization of a simple class. However I strongly advise you to benchmark against your own types and your own use case to determine which is best for you.

How do I get it?

JsonSrcGen is available on Nuget in here

The code is available on github here under the MIT license.

JsonSrcGen is currently in alpha quality and should not be used for production code. However feel free to try it out and report any issues you have with it.