JsonParser & Json Serialization Issues

In the brainCloud C#/Unity client library, starting with version 5.9.3, we've done a rewrite on parts of the BrainCloudComms class in order to improve on memory management and CPU cycle usage. In this update we added a new tool to this library in order to handle incoming Json response bundles from brainCloud: JsonParser.

Let's quickly explain the old process and what has changed with JsonParser.

The BrainCloudComms class might receive multiple responses to process at once and certain responses also need additional processing for certain background tasks to be fulfilled (such as triggering reward events). Originally, we used the included JsonFx utility to split the responses and to process them. However, the SuccessCallback and FailureCallback callbacks are expecting a jsonResponse string to be passed through them. So the way that this works is as follows:

As you can see, we originally deserialized a response bundle and then reserialized each individual response before firing off a SuccessCallback. This lead to a large memory overhead, allocating over 10k objects for even simple Json responses!

That's where JsonParser comes in. Instead of deserializing then reserializing we're now simply manipulating strings using StringBuilder and grabbing the relevant data. Even simply parsing through the Json response bundle to split off the various responses led to a massive gain in CPU usage:

Our memory comparison doesn't look half-bad too! Here's how it looks after processing a 87.91 KB Json response bundle:

With these findings we integrated JsonParser fully into BrainCloudComms. It became available for clients starting with the 5.9.3 release of the brainCloud C#/Unity client library.

Additionally, JsonParser was extended to be able to parse Jsons quickly for any need. You can grab any string value, arrays, or other value types. Best of all, you can pass a params string[] hierarchy so it can look deep into the Json for the exact value you need.

If you just need a value or two and don't want to deserialize a whole object, try using JsonParser! You can use it in any script by including the BrainCloud.Common namespace.

For all of our testing and integration with the brainCloud C#/Unity client library, we've been using JsonFx whenever we do need do deserialize Jsons. Even when we deserialize directly into serializable classes we never encountered any issues.

This is because JsonFx isn't strict about how to handle Json fields, particularly with number fields.

Additionally, when you do serialize data using JsonFx to create Json strings, it will also normalize number fields. If a number field originally looked like this:

JsonFx will take yourValue and turn it into this upon serialization:

And if you're serializing into a class where yourValue is an int? JsonFx will map it to that int no problem.

We found this to be the case with scientific notation values as well where:

Becomes:

We also found that Unity's JsonUtility also exhibits similar behaviour.

Well, you might be here because it is an issue for your app that you've been working on or updating. And it is likely because you're not using JsonFx, you're using a different deserializer such as Newtonsoft.Json.NET, or LitJson. And these deserializers are more strict about how to handle number values. If yourValue is 1.0, then your field better be a float or a double or else!

But here's the rub:

brainCloud has always sent these raw Json values to your client apps. In our C#/Unity client library, using JsonFx to deserialize and reserialize that data before 5.9.3 had the unintended consequence of normalizing number values. And often in your Cloud Code, if you are not specific with your JavaScript objects, number values become doubles by default when serializing them. That's because in JavaScript, a number is a number.

On top of that when you see in your logs on the brainCloud portal, the value of yourValue looks like this:

Now we admit this one is confusing. Why does the Json string in the brainCloud portal logs show an int value here but then in the app you're receiving a double value?

Well it's simply due to the portal's prettifying function. It's also normalizing number values here. We did not consider this to be an issue until we have found clients finding this to be a discrepancy. So we are looking into how to make this more clear in the future.

If you are using JsonFX or Unity's JsonUtility classes, you are fine and shouldn't be experiencing this.

However, introduced in version 5.9.6, we've added a flag: JSON_COMPATIBILITY_FLAG. If you add this to your conditional compilation symbols in Visual Studio or as a Scripting Define symbol in Unity, then it will activate a step before processing incoming Json response bundles. In this step, we will use JsonFx to deserialize then reserialize the response bundles to normalize the values you receive.

For those of you who elect to use your own Json deserializers then this will work for you.

This does come with a tradeoff though. You will be reintroducing the old behaviour that does eat up a significant amount of overhead each time a response bundle is processed. This can work in a pinch for sure though, and if you're upgrading from a version from before 5.9.3 your app will still experience the overall improvements we've made.

The best way to handle this is to develop your app with the raw Json string data in-mind. You can instead have your fields be float or double and have a method to cast them to be int. Or use custom deserialization methods such as JsonConverter for Newtonsoft.Json.Net. Or start using JsonParser where it makes sense to not deserialize whole objects!

We hope this provides some insight into why we introduced JsonParser and how you can avoid butting your head against the raw Jsons you're now seeing.

If you have any questions or experiencing any other issues, you can always ask on the brainCloud Forums or give us a shout on our support system (hit the "Ask Support" button on the top-right in the brainCloud Portal)!