It is difficult to build much without needing to deal with json at some point. In the world of F# there are several options. Today I want to lay out some of the popular choices and offer some some compare and contrast. These libraries all have their own strengths and weaknesses, and bringing it all into a single place to highlight these is a useful point of reference for decision making and general understanding.
First, a couple disclaimers. What I’m presenting is not an exhaustive list of libraries, but it is what I have found are commonly referenced and used. The benchmarks provided are intended to be representative, but should be taken with a grain of salt (as with all benchmarks). Other systems and data may see slightly different results than what I’m showing here. Most of these libraries have the ability to support custom serialization/deserialization and all kinds of options. The goal is not to dive into a bunch of rabbit holes, but what does typical usage looks like. If there is another library you’d like to see, an error I made, or a performance aspect I missed, hit me up and I’ll make corrections/additions. The goal is to offer an honest representation, and it is important to get this as correct as possible. Second, the libraries I’ll be investigating are below. The first three libraries are F#-specific, the last two are the common json encoders in the .NET ecosystem that everyone knows.
The benchmarks are run on Linux, using .NET 7. Below are the library versions.
1 | # BenchmarkDotNet |
In the spirit of testing common use cases, below are the data types I’ll be serializing/deserializing to/from json. I also include some code samples. This at least helps level-set the type of data being handled.
1 | // Discriminated Union and Record types |
One of the characteristics that really sticks out when serializing F# objects to json is how the library handles F#-specific types, namely Discriminated Unions (DU) and Tuples. This can be a point of taste, but I prefer the way FSharpLu
, Thoth
, and Fleece
handle DUs. Even for these libraries there are are variations. FSharpLu and Thoth auto-encode without any extra effort, while Fleece requires writing encoders/decoders for anything beyond primitive types. This falls into the balance of more/less power and is a specific library design choice. Fleece is inspired by Aeson. This typically means more work upfront, but offers power regarding data representation. Without putting too fine a point on it, this can be a significant advantage.
In the spirit of investigating variations, I also include multiple Thoth methods. The first is Auto encoding/decoding, which is what most people will use. The second uses pre-cached encoders/decoders by type. Caching requires an extra step prior to usage, but can offer a performance boost. The third is manual encoding/decoding in a similar vein to Fleece. This allows control of the encoding/decoding json, and potentially a performance boost. It is nice to have this as an capability, although the Fleece ergonomics are a bit nicer.
Then there are the traditional .NET json libraries. Newtonsoft.Json
auto-encodes out of the box, although the default encoding of some types feels awkward to my F# eyes. System.Text.Json
also auto-encodes, except for Discriminated Unions. If you try to use it with a Discriminated Union, you’ll get a compile error. To overcome this, I use FSharp.SystemTextJson
to add DU support. I think its great this library exists, but it does highlight a functionality gap for System.Text.Json
.
What does all of this mean in practice? Time for some examples of where these libraries differ. Primitive types like int and string aren’t interesting; they all encode the same way, so basically “nothing to see here”. To that end, I’ll focus on the Customer record type specified above. I’ll start where they all encode data in the same manner. For reference, here is where they all match serialization (missing are Industry and Stats fields):1
2
3
4
5
6
7
8
9
10{
"Id": "1d765765-9ad9-46e1-989e-b3e4fa2ff3e6",
"Name": "Foo Inc.",
"Created": "2023-03-03T16:49:07.8710678Z",
"Contacts": [
"[email protected]",
"[email protected]"
],
"Notes": "This is a note"
}
Using the above as a baseline, I can now dig into the serialization differences. First up is the Industry DU. As a refresher, Industry can be one of several fixed types (e.g. Finance, Manufacturing, etc) or an Other
value with a custom string. Below are both encodings, broken down by library. FSharpLu, Thoth, and Fleece are the most intuitive. Fleece is a special case because I chose to encode Other
as Other=<value>
to show some customization (more on how that happens later), but Fleece could look however makes the most sense to you. The Newtonsoft and System.Text.Json serializations are a bit more verbose. It is a more complex type, so in general this is fine, but the representation does feel a little weird. I think when interacting with external systems this could potentially be an issue, but in the grand scheme of things, this isn’t a really a deal breaker.
1 | ### Industry = Finance |
1 | ### Industry = Other("Recycling") |
Now on to the Stats tuple. Four of the five encode it as an array. From a data structure perspective this is interesting and mostly pragmatic representation that maps to other languages without too much hassle. Newtonsoft is interesting in that it has the most honest data representation, but feels kind of clunky when encoding. With that said, I prefer this method for the following reason. What I don’t show here is the case when the tuple is mixed types (e.g. int * int * string
), in that case the other libraries show [10, 20, "30"]
, which could be an issue if the decoder expected an array to have a single type.
1 | ### Stats = 10, 20, 30 |
Now that I’ve looked at the results, it is time to see how the serialization/deserialize happens. Libraries are more than just the results, they also include the developer ergonomics. In addition to caveats mentioned above, some of these libraries have multiple ways to do things, I’ve just picked what is most convenient for my tests. Now is a good time to mention the above type definitions were abbreviated for readability; its now to bring out all the gruesome details. The most notable blocks are the Fleece codec and Thoth decoder/encoders, both of which are for manual encoding support. I’ve commented where library-supporting code exists. You’ll may also notice static member vs. module code organization; this to to match the respective library conventions, even though within a singular codebase it falls on the side of inconsistency. Frankly, its a lot of extra code to add to a type, but that is the cost of hand encoding/decoding. It may be too much to stomach for some, but sometimes the flexibility is worth the effort. Just to be clear, the Thoth-specific decoder/encoder modules are only required for manual usage, when using Auto options, these are not necessary.
1 | type Email = string |
The actual serialization and deserialization for all libraries is simple and straight-forward. Manual encoding when using Thoth takes a bit more effort, but not much. They all support custom options, but this is the bare bones method to get data converted back and forth.
1 | // FSharpLu |
There is one more major piece to discuss, performance. This is where BenchmarkDotNet comes into play. I investigate several different scenarios. I focus mostly on different object types: int list, string list, Customer list, and Customer. An additional variation is list size, for this I run the tests with size: 1, 100, and 1000. The combination of these tests will provide some insight regarding performance (for both speed and memory). For the most part, the benchmark code is straight forward. The only variation to note is that I try to keep the serialized json consistent, but I do need multiple versions of the json strings, since the libraries support slightly different encodings. Beyond that, they are all operating on the same data. Fair warning, the benchmark results are pretty hefty, but it’s the only way to see all the different breakdowns.
### System:
BenchmarkDotNet=v0.13.4, OS=ubuntu 20.04
Intel Core i7-9750H CPU 2.60GHz, 1 CPU, 12 logical and 6 physical cores
.NET SDK=7.0.101
[Host] : .NET 7.0.1 (7.0.122.56804), X64 RyuJIT AVX2 DEBUG
DefaultJob : .NET 7.0.1 (7.0.122.56804), X64 RyuJIT AVX2
### Serialization:
Method | ListSize | Mean | Error | StdDev | Median | Gen0 | Gen1 | Gen2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
IntListFSharpLu | 1 | 7,043.0 ns | 43.89 ns | 41.05 ns | 7,033.7 ns | 0.6 | - | - | 4167 B |
IntListThoth | 1 | 1,194.4 ns | 7.02 ns | 6.22 ns | 1,197.0 ns | 0.1 | - | - | 1168 B |
IntListThothCached | 1 | 406.7 ns | 3.48 ns | 3.25 ns | 405.5 ns | 0.1 | - | - | 1039 B |
IntListThothManual | 1 | 164.1 ns | 0.27 ns | 0.24 ns | 164.1 ns | 0.0 | - | - | 408 B |
IntListFleece | 1 | 1,174.0 ns | 6.37 ns | 5.65 ns | 1,172.6 ns | 0.9 | 0.0 | - | 5832 B |
IntListNewtonsoft | 1 | 389.2 ns | 0.55 ns | 0.43 ns | 389.2 ns | 0.2 | 0.0 | - | 1447 B |
IntListTextJson | 1 | 316.4 ns | 1.62 ns | 1.51 ns | 316.2 ns | 0.0 | - | - | 136 B |
StringListFSharpLu | 1 | 23,558.4 ns | 32.47 ns | 30.37 ns | 23,567.7 ns | 2.7 | - | - | 17161 B |
StringListThoth | 1 | 17,798.6 ns | 34.58 ns | 30.65 ns | 17,790.3 ns | 2.2 | - | - | 14276 B |
StringListThothCached | 1 | 17,000.0 ns | 28.15 ns | 26.34 ns | 16,994.6 ns | 2.2 | - | - | 14144 B |
StringListThothManual | 1 | 16,396.5 ns | 12.94 ns | 10.81 ns | 16,398.5 ns | 2.1 | - | - | 13216 B |
StringListFleece | 1 | 17,498.7 ns | 20.93 ns | 16.34 ns | 17,500.4 ns | 2.9 | - | - | 18841 B |
StringListNewtonsoft | 1 | 16,627.6 ns | 17.37 ns | 14.51 ns | 16,625.7 ns | 2.2 | - | - | 14320 B |
StringListTextJson | 1 | 16,270.4 ns | 44.64 ns | 41.76 ns | 16,262.5 ns | 2.0 | - | - | 13032 B |
RecordListFSharpLu | 1 | 48,328.4 ns | 189.01 ns | 176.80 ns | 48,287.4 ns | 2.4 | - | - | 15483 B |
RecordListThoth | 1 | 84,474.2 ns | 307.36 ns | 287.50 ns | 84,523.1 ns | 4.7 | - | - | 30290 B |
RecordListThothCached | 1 | 44,234.5 ns | 150.27 ns | 140.56 ns | 44,270.8 ns | 3.2 | 0.1 | - | 20479 B |
RecordListThothManual | 1 | 45,829.3 ns | 63.85 ns | 59.72 ns | 45,842.2 ns | 3.7 | 0.1 | - | 23980 B |
RecordListFleece | 1 | 12,916.2 ns | 29.86 ns | 23.31 ns | 12,924.2 ns | 3.4 | 0.1 | - | 21768 B |
RecordListNewtonsoft | 1 | 4,928.5 ns | 20.42 ns | 18.11 ns | 4,922.4 ns | 0.7 | - | - | 4871 B |
RecordListTextJson | 1 | 4,409.8 ns | 12.55 ns | 11.12 ns | 4,413.0 ns | 0.3 | - | - | 2247 B |
OneRecordFSharpLu | 1 | 43,885.5 ns | 111.43 ns | 98.78 ns | 43,885.5 ns | 2.3 | - | - | 14553 B |
OneRecordThoth | 1 | 83,886.9 ns | 264.00 ns | 246.94 ns | 83,845.5 ns | 4.6 | - | - | 29321 B |
OneRecordThothCached | 1 | 43,521.3 ns | 306.90 ns | 287.08 ns | 43,645.5 ns | 3.1 | 0.1 | - | 19608 B |
OneRecordThothManual | 1 | 48,191.4 ns | 94.14 ns | 83.45 ns | 48,174.8 ns | 3.6 | 0.1 | - | 23070 B |
OneRecordFleece | 1 | 11,719.4 ns | 34.92 ns | 29.16 ns | 11,727.1 ns | 3.2 | 0.1 | - | 20439 B |
OneRecordNewtonsoft | 1 | 4,362.4 ns | 38.07 ns | 35.61 ns | 4,375.7 ns | 0.7 | - | - | 4587 B |
OneRecordTextJson | 1 | 4,088.9 ns | 9.79 ns | 8.68 ns | 4,085.0 ns | 0.3 | - | - | 1963 B |
IntListFSharpLu | 100 | 20,188.5 ns | 86.07 ns | 80.51 ns | 20,171.1 ns | 3.4 | 0.1 | - | 21864 B |
IntListThoth | 100 | 14,860.5 ns | 42.98 ns | 38.10 ns | 14,859.4 ns | 3.4 | 0.1 | - | 21745 B |
IntListThothCached | 100 | 13,736.9 ns | 44.20 ns | 41.35 ns | 13,720.9 ns | 3.4 | 0.1 | - | 21607 B |
IntListThothManual | 100 | 9,308.6 ns | 15.10 ns | 12.61 ns | 9,305.5 ns | 2.8 | 0.1 | - | 17952 B |
IntListFleece | 100 | 16,534.7 ns | 52.73 ns | 46.74 ns | 16,519.1 ns | 5.8 | 0.2 | - | 36727 B |
IntListNewtonsoft | 100 | 7,372.3 ns | 6.85 ns | 6.07 ns | 7,371.9 ns | 1.3 | 0.0 | - | 8741 B |
IntListTextJson | 100 | 3,831.8 ns | 4.95 ns | 4.63 ns | 3,832.3 ns | 0.7 | - | - | 4469 B |
StringListFSharpLu | 100 | 1,649,890.3 ns | 2,182.75 ns | 2,041.75 ns | 1,649,718.0 ns | 208.9 | 13.6 | - | 1315335 B |
StringListThoth | 100 | 1,603,423.3 ns | 3,536.47 ns | 2,953.12 ns | 1,602,939.4 ns | 208.9 | 11.7 | - | 1313783 B |
StringListThothCached | 100 | 1,558,011.5 ns | 3,719.41 ns | 3,479.14 ns | 1,558,608.2 ns | 208.9 | 13.6 | - | 1313298 B |
StringListThothManual | 100 | 1,609,062.4 ns | 2,173.69 ns | 2,033.27 ns | 1,609,041.0 ns | 207.0 | 19.5 | - | 1299154 B |
StringListFleece | 100 | 1,701,875.8 ns | 3,307.47 ns | 3,093.81 ns | 1,701,224.7 ns | 212.8 | 13.6 | - | 1336981 B |
StringListNewtonsoft | 100 | 1,663,057.9 ns | 2,827.21 ns | 2,506.25 ns | 1,662,869.2 ns | 207.0 | 11.7 | - | 1303146 B |
StringListTextJson | 100 | 1,572,979.7 ns | 2,086.97 ns | 1,850.05 ns | 1,573,096.4 ns | 205.0 | 11.7 | - | 1293890 B |
RecordListFSharpLu | 100 | 709,046.5 ns | 1,413.84 ns | 1,322.50 ns | 709,459.4 ns | 89.8 | 27.3 | - | 568895 B |
RecordListThoth | 100 | 4,123,862.4 ns | 20,371.95 ns | 19,055.93 ns | 4,118,491.6 ns | 304.6 | 210.9 | - | 1914472 B |
RecordListThothCached | 100 | 4,068,735.7 ns | 18,677.86 ns | 17,471.28 ns | 4,060,941.0 ns | 296.8 | 195.3 | - | 1901524 B |
RecordListThothManual | 100 | 4,531,003.4 ns | 11,349.26 ns | 10,616.10 ns | 4,534,660.2 ns | 351.5 | 140.6 | - | 2249710 B |
RecordListFleece | 100 | 857,710.4 ns | 2,601.90 ns | 2,433.82 ns | 857,026.0 ns | 206.0 | 0.9 | - | 1294299 B |
RecordListNewtonsoft | 100 | 405,181.7 ns | 701.05 ns | 621.47 ns | 405,120.1 ns | 50.7 | 13.6 | - | 321528 B |
RecordListTextJson | 100 | 362,255.9 ns | 631.07 ns | 559.42 ns | 362,328.9 ns | 34.6 | 8.3 | - | 220188 B |
IntListFSharpLu | 1000 | 133,347.3 ns | 1,257.91 ns | 1,176.65 ns | 133,482.3 ns | 28.8 | 5.3 | - | 182022 B |
IntListThoth | 1000 | 134,059.9 ns | 306.36 ns | 255.82 ns | 134,002.1 ns | 31.2 | 3.4 | - | 196388 B |
IntListThothCached | 1000 | 128,716.7 ns | 1,423.84 ns | 1,331.86 ns | 127,905.5 ns | 31.0 | 2.4 | - | 196170 B |
IntListThothManual | 1000 | 93,161.8 ns | 454.78 ns | 425.40 ns | 93,178.4 ns | 27.4 | 0.1 | - | 172760 B |
IntListFleece | 1000 | 149,448.3 ns | 386.25 ns | 342.40 ns | 149,418.3 ns | 44.1 | 9.7 | - | 277991 B |
IntListNewtonsoft | 1000 | 68,508.4 ns | 220.08 ns | 195.09 ns | 68,464.8 ns | 12.2 | 1.3 | - | 77127 B |
IntListTextJson | 1000 | 34,069.5 ns | 107.37 ns | 89.66 ns | 34,113.2 ns | 6.9 | 0.4 | - | 43871 B |
StringListFSharpLu | 1000 | 17,584,797.3 ns | 29,524.67 ns | 27,617.39 ns | 17,584,905.4 ns | 2062.5 | 125.0 | 31.2 | 13116871 B |
StringListThoth | 1000 | 16,840,644.9 ns | 28,439.32 ns | 25,210.72 ns | 16,842,766.9 ns | 2062.5 | 125.0 | 31.2 | 13110095 B |
StringListThothCached | 1000 | 16,990,921.6 ns | 51,412.23 ns | 45,575.60 ns | 16,988,783.6 ns | 2062.5 | 125.0 | 31.2 | 13108271 B |
StringListThothManual | 1000 | 16,557,071.7 ns | 146,791.20 ns | 137,308.58 ns | 16,601,215.2 ns | 2062.5 | 31.2 | - | 12984789 B |
StringListFleece | 1000 | 17,261,490.7 ns | 55,897.70 ns | 52,286.74 ns | 17,263,192.8 ns | 2125.0 | 93.7 | 31.2 | 13385711 B |
StringListNewtonsoft | 1000 | 16,650,742.4 ns | 50,595.94 ns | 44,851.98 ns | 16,638,779.1 ns | 2062.5 | 156.2 | 31.2 | 13019946 B |
StringListTextJson | 1000 | 16,210,066.4 ns | 18,023.81 ns | 15,977.64 ns | 16,215,392.3 ns | 2031.2 | 62.5 | - | 12938117 B |
RecordListFSharpLu | 1000 | 7,418,860.9 ns | 16,006.82 ns | 14,972.79 ns | 7,413,755.7 ns | 789.0 | 757.8 | 195.3 | 5522664 B |
RecordListThoth | 1000 | 45,465,135.8 ns | 385,734.45 ns | 360,816.24 ns | 45,402,331.0 ns | 2818.1 | 727.2 | 454.5 | 18928991 B |
RecordListThothCached | 1000 | 48,735,891.0 ns | 831,080.08 ns | 1,243,921.53 ns | 47,969,807.8 ns | 2818.1 | 727.2 | 454.5 | 18902199 B |
RecordListThothManual | 1000 | 54,555,776.3 ns | 125,263.21 ns | 117,171.29 ns | 54,558,445.9 ns | 3600.0 | 700.0 | 200.0 | 22406518 B |
RecordListFleece | 1000 | 13,469,301.0 ns | 100,449.31 ns | 89,045.70 ns | 13,458,451.0 ns | 1843.7 | 843.7 | 578.1 | 12810464 B |
RecordListNewtonsoft | 1000 | 4,360,206.6 ns | 30,850.12 ns | 28,857.22 ns | 4,368,718.1 ns | 546.8 | 375.0 | 140.6 | 3208284 B |
RecordListTextJson | 1000 | 3,924,525.0 ns | 16,541.52 ns | 14,663.63 ns | 3,920,796.3 ns | 406.2 | 257.8 | 140.6 | 2222653 B |
### Deserialization:
Method | ListSize | Mean | Error | StdDev | Gen0 | Gen1 | Gen2 | Allocated |
---|---|---|---|---|---|---|---|---|
IntListFSharpLu | 1 | 15.6 μs | 0.1459 μs | 0.1365 μs | 1.4954 | - | - | 9.29 KB |
IntListThoth | 1 | 17.9 μs | 0.0496 μs | 0.0440 μs | 1.4343 | - | - | 8.88 KB |
IntListThothCached | 1 | 16.9 μs | 0.1128 μs | 0.1055 μs | 1.4038 | - | - | 8.74 KB |
IntListThothManual | 1 | 10.1 μs | 0.0488 μs | 0.0457 μs | 1.2360 | - | - | 7.66 KB |
IntListFleece | 1 | 10.6 μs | 0.0179 μs | 0.0159 μs | 1.0071 | - | - | 6.26 KB |
IntListNewtonsoft | 1 | 9.0 μs | 0.0284 μs | 0.0265 μs | 1.0986 | - | - | 6.79 KB |
IntListTextJson | 1 | 8.5 μs | 0.0106 μs | 0.0088 μs | 0.6714 | - | - | 4.2 KB |
StringListFSharpLu | 1 | 33.7 μs | 0.1217 μs | 0.1138 μs | 3.6011 | - | - | 22.17 KB |
StringListThoth | 1 | 35.5 μs | 0.0508 μs | 0.0450 μs | 3.4790 | - | - | 21.56 KB |
StringListThothCached | 1 | 32.7 μs | 0.0909 μs | 0.0850 μs | 3.4790 | - | - | 21.42 KB |
StringListThothManual | 1 | 30.5 μs | 0.0669 μs | 0.0593 μs | 3.4790 | - | - | 21.43 KB |
StringListFleece | 1 | 30.0 μs | 0.0498 μs | 0.0441 μs | 3.1128 | - | - | 19.09 KB |
StringListNewtonsoft | 1 | 27.1 μs | 0.0321 μs | 0.0268 μs | 3.1738 | - | - | 19.56 KB |
StringListTextJson | 1 | 26.0 μs | 0.0560 μs | 0.0524 μs | 2.7466 | - | - | 16.99 KB |
RecordListFSharpLu | 1 | 103.2 μs | 0.2298 μs | 0.2037 μs | 4.8828 | - | - | 30.07 KB |
RecordListThoth | 1 | 209.4 μs | 0.3070 μs | 0.2872 μs | 10.9863 | 0.2441 | - | 68.71 KB |
RecordListThothCached | 1 | 163.9 μs | 0.4874 μs | 0.4559 μs | 9.5215 | 0.2441 | - | 58.5 KB |
RecordListThothManual | 1 | 72.3 μs | 0.1873 μs | 0.1752 μs | 4.0283 | - | - | 24.73 KB |
RecordListFleece | 1 | 24.1 μs | 0.0792 μs | 0.0741 μs | 3.4485 | 0.0916 | - | 21.25 KB |
RecordListNewtonsoft | 1 | 12.2 μs | 0.0365 μs | 0.0342 μs | 1.5717 | 0.0153 | - | 9.65 KB |
RecordListTextJson | 1 | 11.9 μs | 0.0402 μs | 0.0376 μs | 0.5035 | - | - | 3.16 KB |
OneRecordFSharpLu | 1 | 95.5 μs | 0.2836 μs | 0.2652 μs | 4.5166 | - | - | 28.38 KB |
OneRecordThoth | 1 | 182.2 μs | 0.5150 μs | 0.4817 μs | 8.7891 | - | - | 54.73 KB |
OneRecordThothCached | 1 | 137.7 μs | 0.3629 μs | 0.3031 μs | 7.0801 | - | - | 44.69 KB |
OneRecordThothManual | 1 | 66.1 μs | 0.1749 μs | 0.1460 μs | 3.7842 | - | - | 23.21 KB |
OneRecordFleece | 1 | 21.1 μs | 0.0523 μs | 0.0464 μs | 3.1738 | 0.0916 | - | 19.46 KB |
OneRecordNewtonsoft | 1 | 11.4 μs | 0.0185 μs | 0.0164 μs | 1.4954 | 0.0153 | - | 9.21 KB |
OneRecordTextJson | 1 | 10.5 μs | 0.0313 μs | 0.0293 μs | 0.4425 | - | - | 2.76 KB |
IntListFSharpLu | 100 | 41.6 μs | 0.1170 μs | 0.1094 μs | 5.3711 | 0.1831 | - | 33.05 KB |
IntListThoth | 100 | 131.1 μs | 1.2809 μs | 1.1981 μs | 10.7422 | 0.4883 | - | 66.87 KB |
IntListThothCached | 100 | 131.9 μs | 1.1325 μs | 1.0039 μs | 10.7422 | 0.4883 | - | 66.72 KB |
IntListThothManual | 100 | 54.6 μs | 0.0609 μs | 0.0570 μs | 9.0942 | 0.4272 | - | 55.76 KB |
IntListFleece | 100 | 53.1 μs | 0.1576 μs | 0.1474 μs | 9.3994 | 0.3052 | - | 57.82 KB |
IntListNewtonsoft | 100 | 34.5 μs | 0.0915 μs | 0.0811 μs | 4.9438 | 0.1221 | - | 30.55 KB |
IntListTextJson | 100 | 30.7 μs | 0.0543 μs | 0.0481 μs | 4.2419 | 0.1221 | - | 26.05 KB |
StringListFSharpLu | 100 | 1,763.7 μs | 2.6119 μs | 2.4432 μs | 210.9375 | 15.6250 | - | 1303.58 KB |
StringListThoth | 100 | 1,895.9 μs | 3.5673 μs | 3.1623 μs | 216.7969 | 17.5781 | - | 1329.64 KB |
StringListThothCached | 100 | 1,774.0 μs | 6.9559 μs | 6.5066 μs | 216.7969 | 17.5781 | - | 1329.27 KB |
StringListThothManual | 100 | 1,761.7 μs | 34.6178 μs | 35.5499 μs | 212.8906 | 7.8125 | - | 1308.76 KB |
StringListFleece | 100 | 1,773.9 μs | 5.0636 μs | 4.4888 μs | 216.7969 | 19.5313 | - | 1333.79 KB |
StringListNewtonsoft | 100 | 1,715.8 μs | 6.6255 μs | 6.1975 μs | 210.9375 | 15.6250 | - | 1300.96 KB |
StringListTextJson | 100 | 1,841.5 μs | 2.5186 μs | 2.1032 μs | 210.9375 | 15.6250 | - | 1299.17 KB |
RecordListFSharpLu | 100 | 1,506.4 μs | 3.7216 μs | 3.4812 μs | 156.2500 | 50.7813 | - | 969.71 KB |
RecordListThoth | 100 | 10,375.6 μs | 84.5533 μs | 79.0912 μs | 718.7500 | 531.2500 | 15.6250 | 4452.71 KB |
RecordListThothCached | 100 | 10,250.9 μs | 34.5145 μs | 32.2849 μs | 718.7500 | 562.5000 | 15.6250 | 4442.12 KB |
RecordListThothManual | 100 | 1,599.8 μs | 4.6668 μs | 4.1370 μs | 189.4531 | 89.8438 | - | 1164.58 KB |
RecordListFleece | 100 | 498.4 μs | 0.6963 μs | 0.6173 μs | 48.8281 | 1.9531 | - | 305.02 KB |
RecordListNewtonsoft | 100 | 854.5 μs | 3.7517 μs | 3.5094 μs | 81.0547 | 22.4609 | - | 502.68 KB |
RecordListTextJson | 100 | 979.8 μs | 2.1825 μs | 2.0416 μs | 48.8281 | 11.7188 | - | 304.22 KB |
IntListFSharpLu | 1000 | 256.8 μs | 0.4628 μs | 0.3864 μs | 39.5508 | 6.3477 | - | 245.56 KB |
IntListThoth | 1000 | 1,158.0 μs | 1.9641 μs | 1.8372 μs | 95.7031 | 37.1094 | - | 588.79 KB |
IntListThothCached | 1000 | 1,159.7 μs | 3.3701 μs | 2.8142 μs | 95.7031 | 39.0625 | - | 588.44 KB |
IntListThothManual | 1000 | 454.2 μs | 0.6121 μs | 0.5426 μs | 79.1016 | 1.9531 | - | 486.03 KB |
IntListFleece | 1000 | 443.5 μs | 0.5548 μs | 0.5189 μs | 84.4727 | 1.9531 | - | 520.01 KB |
IntListNewtonsoft | 1000 | 264.2 μs | 1.9403 μs | 1.8149 μs | 39.5508 | 9.2773 | - | 243.05 KB |
IntListTextJson | 1000 | 233.1 μs | 0.8043 μs | 0.7129 μs | 35.8887 | 8.0566 | - | 221.16 KB |
StringListFSharpLu | 1000 | 16,952.5 μs | 31.5653 μs | 27.9819 μs | 2093.7500 | 31.2500 | - | 12948.04 KB |
StringListThoth | 1000 | 18,092.8 μs | 201.8233 μs | 188.7856 μs | 2156.2500 | 125.0000 | 31.2500 | 13214.55 KB |
StringListThothCached | 1000 | 18,802.7 μs | 81.7247 μs | 76.4453 μs | 2156.2500 | 125.0000 | 31.2500 | 13213.3 KB |
StringListThothManual | 1000 | 17,291.1 μs | 183.9874 μs | 172.1019 μs | 2093.7500 | 281.2500 | 31.2500 | 13002.46 KB |
StringListFleece | 1000 | 18,058.9 μs | 51.8915 μs | 40.5135 μs | 2156.2500 | 156.2500 | 31.2500 | 13305.53 KB |
StringListNewtonsoft | 1000 | 17,507.9 μs | 66.8627 μs | 62.5434 μs | 2093.7500 | 31.2500 | - | 12945.42 KB |
StringListTextJson | 1000 | 16,826.3 μs | 37.8847 μs | 33.5838 μs | 2093.7500 | 31.2500 | - | 12950.66 KB |
RecordListFSharpLu | 1000 | 16,157.2 μs | 68.4580 μs | 64.0357 μs | 1406.2500 | 500.0000 | 250.0000 | 9433.67 KB |
RecordListThoth | 1000 | 106,187.4 μs | 633.2908 μs | 592.3806 μs | 7000.0000 | 750.0000 | 500.0000 | 44255.41 KB |
RecordListThothCached | 1000 | 108,517.2 μs | 608.9917 μs | 539.8552 μs | 7200.0000 | 800.0000 | 400.0000 | 44214.61 KB |
RecordListThothManual | 1000 | 21,465.3 μs | 120.3867 μs | 112.6098 μs | 1750.0000 | 1500.0000 | 718.7500 | 11445.14 KB |
RecordListFleece | 1000 | 5,433.1 μs | 98.3105 μs | 91.9597 μs | 632.8125 | 500.0000 | 359.3750 | 3292.54 KB |
RecordListNewtonsoft | 1000 | 9,820.1 μs | 18.1347 μs | 15.1433 μs | 796.8750 | 781.2500 | 140.6250 | 4995.52 KB |
RecordListTextJson | 1000 | 10,365.1 μs | 26.4786 μs | 22.1108 μs | 531.2500 | 515.6250 | 125.0000 | 3064.77 KB |
There are a couple trends to gleen from the above results. At a high level, F#-specific libraries incur a performance cost. Newtonsoft and System.Text.Json are typically 5x-10x faster than FSharpLu and Thoth, and 2x-3x faster than Fleece. Datatype impacts performance differences, but the trends hold for the most part (with the exception of string lists, where string handling dominates encoding). Library memory usage holds similar trends, just not as pronounced in most cases. Memory usage does show its impact on the larger ListSize (1000), where speed and the GC take a pretty hard hit.
Looking at the results a bit closer, Newtonsoft and System.Text.Json are in the same ballpark, but System.Text.Json nearly always had a slight lead in performance. On average, Thoth is the slowest, although cached encodings offer a small boost. Manual encoding for Thoth certainly makes it more competitive with FSharpLu and Fleece. Speaking of which, FSharpLu and Fleece are steadily in the middle of the pack. Fleece typically has performance several times faster, trending it closer to Newtonsoft and System.Text.Json performance. I suspect this is a result of manual encoding versus reflection, but just a hunch.
What does all this mean? Well, it depends (I hope you weren’t expecting a simple answer). Clearly there is a performance difference between the libraries, sometimes up to an order of magnitude (that is more the exception than the rule). When talking nanoseconds and microseconds, most use cases honestly aren’t going to see a noticeable difference. This is where ergonomics and flexibility need to factor into a decision. Everything has its trade off, and finding what works for your specific case is a process. If straight-up performance matters most, there are some clear leaders. Admittedly, the benchmark results surprised me. I didn’t expect the F#-specific libraries to have such a large gap. I also expected manual encoding to offer a better performance benefit over reflection than it did. As they say, you learn something new every day, and today is no exception. So there you have it, a comparison with some of the major json encoding libraries supported in F#. Seeing all the trade-offs in one place can assist in decisions moving forward and understanding what makes the best library for particular use cases. A better grasp on all the impact of all components in a system will help make any project stronger. As I mentioned in the beginning, if you see any errors or opportunities for improvement, let me know. Until next time, stay calm and keep encoding…