c# - standard - asp.net core json serialization




.Net Core 3.0 JsonSerializer populate existing object (5)

I'm preparing a migration from ASP.NET Core 2.2 to 3.0.

As I don't use any more advanced JSON features (but maybe one as described below), and 3.0 now comes with a built-in namespace/classes for JSON, System.Text.Json , I decided to see if I could drop the previous default Newtonsoft.Json . Do note, I'm aware that System.Text.Json will not completely replace Newtonsoft.Json .

I managed to do that everywhere, e.g.

var obj = JsonSerializer.Parse<T>(jsonstring);

var jsonstring = JsonSerializer.ToString(obj);

but in one place, where I populate an existing object.

With Newtonsoft.Json one can do

JsonConvert.PopulateObject(jsonstring, obj);

The built-in System.Text.Json namespace has some additional classes, like JsonDocumnet , JsonElement and Utf8JsonReader , though I can't find any that take an existing object as a parameter.

Nor am I experienced enough to see how to make use of the existing one's.

There might be a possible upcoming feature in .Net Core (thanks to Mustafa Gursel for the link), but meanwhile (and what if it doesn't),...

...I now wonder, is it possible to achieve something similar as what one can do with PopulateObject ?

I mean, is it possible with any of the other System.Text.Json classes to accomplish the same, and update/replace only the properties set?,... or some other clever workaround?

Here is a sample of how it could look like (and it need to be generic as the object passed into the deserialization method is of type <T> ). I have 2 Json string's to be parsed into an object, where the first have some default properties set, and the second some, e.g.

Note, a property value can be of any other type than a string .

Json string 1:

{
  "Title": "Startpage",
  "Link": "/index",
}

Json string 2:

{
  "Head": "Latest news"
  "Link": "/news"
}

Using the 2 Json strings above, I want an object resulting in:

{
  "Title": "Startpage",
  "Head": "Latest news",
  "Link": "/news"
}

As seen in above sample, if properties in the 2nd has values/is set, it replace values in the 1st (as with "Head" and "Link"), if not, existing value persist (as with "Title")


Here is some sample code that does it. It's using the new Utf8JsonReader struct so it populates the object at the same time it parses it. It supports JSON/CLR types equivalence, nested objects (creates if they don't exist), lists and arrays.

var populator = new JsonPopulator();
var obj = new MyClass();
populator.PopulateObject(obj, "{\"Title\":\"Startpage\",\"Link\":\"/index\"}");
populator.PopulateObject(obj, "{\"Head\":\"Latest news\",\"Link\":\"/news\"}");

public class MyClass
{
    public string Title { get; set; }
    public string Head { get; set; }
    public string Link { get; set; }
}

Note it doesn't support all of what you would probably expect, but you can override or customize it. Things that could be added: 1) naming convention. You'd have to override the GetProperty method. 2) dictionaries or expando objects. 3) performance can be improved because it uses Reflection instead of MemberAccessor/delegate techniques

public class JsonPopulator
{
    public void PopulateObject(object obj, string jsonString, JsonSerializerOptions options = null) => PopulateObject(obj, jsonString != null ? Encoding.UTF8.GetBytes(jsonString) : null, options);
    public virtual void PopulateObject(object obj, ReadOnlySpan<byte> jsonData, JsonSerializerOptions options = null)
    {
        options ??= new JsonSerializerOptions();
        var state = new JsonReaderState(new JsonReaderOptions { AllowTrailingCommas = options.AllowTrailingCommas, CommentHandling = options.ReadCommentHandling, MaxDepth = options.MaxDepth });
        var reader = new Utf8JsonReader(jsonData, isFinalBlock: true, state);
        new Worker(this, reader, obj, options);
    }

    protected virtual PropertyInfo GetProperty(ref Utf8JsonReader reader, JsonSerializerOptions options, object obj, string propertyName)
    {
        if (obj == null)
            throw new ArgumentNullException(nameof(obj));

        if (propertyName == null)
            throw new ArgumentNullException(nameof(propertyName));

        var prop = obj.GetType().GetProperty(propertyName);
        return prop;
    }

    protected virtual bool SetPropertyValue(ref Utf8JsonReader reader, JsonSerializerOptions options, object obj, string propertyName)
    {
        if (obj == null)
            throw new ArgumentNullException(nameof(obj));

        if (propertyName == null)
            throw new ArgumentNullException(nameof(propertyName));

        var prop = GetProperty(ref reader, options, obj, propertyName);
        if (prop == null)
            return false;

        if (!TryReadPropertyValue(ref reader, options, prop.PropertyType, out var value))
            return false;

        prop.SetValue(obj, value);
        return true;
    }

    protected virtual bool TryReadPropertyValue(ref Utf8JsonReader reader, JsonSerializerOptions options, Type propertyType, out object value)
    {
        if (propertyType == null)
            throw new ArgumentNullException(nameof(reader));

        if (reader.TokenType == JsonTokenType.Null)
        {
            value = null;
            return !propertyType.IsValueType || Nullable.GetUnderlyingType(propertyType) != null;
        }

        if (propertyType == typeof(object)) { value = ReadValue(ref reader); return true; }
        if (propertyType == typeof(string)) { value = JsonSerializer.ReadValue<JsonElement>(ref reader, options).GetRawText(); return true; }
        if (propertyType == typeof(int) && reader.TryGetInt32(out var i32)) { value = i32; return true; }
        if (propertyType == typeof(long) && reader.TryGetInt64(out var i64)) { value = i64; return true; }
        if (propertyType == typeof(DateTime) && reader.TryGetDateTime(out var dt)) { value = dt; return true; }
        if (propertyType == typeof(DateTimeOffset) && reader.TryGetDateTimeOffset(out var dto)) { value = dto; return true; }
        if (propertyType == typeof(Guid) && reader.TryGetGuid(out var guid)) { value = guid; return true; }
        if (propertyType == typeof(decimal) && reader.TryGetDecimal(out var dec)) { value = dec; return true; }
        if (propertyType == typeof(double) && reader.TryGetDouble(out var dbl)) { value = dbl; return true; }
        if (propertyType == typeof(float) && reader.TryGetSingle(out var sgl)) { value = sgl; return true; }
        if (propertyType == typeof(uint) && reader.TryGetUInt32(out var ui32)) { value = ui32; return true; }
        if (propertyType == typeof(ulong) && reader.TryGetUInt64(out var ui64)) { value = ui64; return true; }
        if (propertyType == typeof(byte[]) && reader.TryGetBytesFromBase64(out var bytes)) { value = bytes; return true; }

        if (propertyType == typeof(bool))
        {
            if (reader.TokenType == JsonTokenType.False || reader.TokenType == JsonTokenType.True)
            {
                value = reader.GetBoolean();
                return true;
            }
        }

        // fallback here
        return TryConvertValue(ref reader, propertyType, out value);
    }

    protected virtual object ReadValue(ref Utf8JsonReader reader)
    {
        switch (reader.TokenType)
        {
            case JsonTokenType.False: return false;
            case JsonTokenType.True: return true;
            case JsonTokenType.Null: return null;
            case JsonTokenType.String: return reader.GetString();

            case JsonTokenType.Number: // is there a better way?
                if (reader.TryGetInt32(out var i32))
                    return i32;

                if (reader.TryGetInt64(out var i64))
                    return i64;

                if (reader.TryGetUInt64(out var ui64)) // uint is already handled by i64
                    return ui64;

                if (reader.TryGetSingle(out var sgl))
                    return sgl;

                if (reader.TryGetDouble(out var dbl))
                    return dbl;

                if (reader.TryGetDecimal(out var dec))
                    return dec;

                break;
        }
        throw new NotSupportedException();
    }

    // we're here when json types & property types don't match exactly
    protected virtual bool TryConvertValue(ref Utf8JsonReader reader, Type propertyType, out object value)
    {
        if (propertyType == null)
            throw new ArgumentNullException(nameof(reader));

        if (propertyType == typeof(bool))
        {
            if (reader.TryGetInt64(out var i64)) // one size fits all
            {
                value = i64 != 0;
                return true;
            }
        }

        // TODO: add other conversions

        value = null;
        return false;
    }

    protected virtual object CreateInstance(ref Utf8JsonReader reader, Type propertyType)
    {
        if (propertyType.GetConstructor(Type.EmptyTypes) == null)
            return null;

        // TODO: handle custom instance creation
        try
        {
            return Activator.CreateInstance(propertyType);
        }
        catch
        {
            // swallow
            return null;
        }
    }

    private class Worker
    {
        private readonly Stack<WorkerProperty> _properties = new Stack<WorkerProperty>();
        private readonly Stack<object> _objects = new Stack<object>();

        public Worker(JsonPopulator populator, Utf8JsonReader reader, object obj, JsonSerializerOptions options)
        {
            _objects.Push(obj);
            WorkerProperty prop;
            WorkerProperty peek;
            while (reader.Read())
            {
                switch (reader.TokenType)
                {
                    case JsonTokenType.PropertyName:
                        prop = new WorkerProperty();
                        prop.PropertyName = Encoding.UTF8.GetString(reader.ValueSpan);
                        _properties.Push(prop);
                        break;

                    case JsonTokenType.StartObject:
                    case JsonTokenType.StartArray:
                        if (_properties.Count > 0)
                        {
                            object child = null;
                            var parent = _objects.Peek();
                            PropertyInfo pi = null;
                            if (parent != null)
                            {
                                pi = populator.GetProperty(ref reader, options, parent, _properties.Peek().PropertyName);
                                if (pi != null)
                                {
                                    child = pi.GetValue(parent); // mimic ObjectCreationHandling.Auto
                                    if (child == null && pi.CanWrite)
                                    {
                                        if (reader.TokenType == JsonTokenType.StartArray)
                                        {
                                            if (!typeof(IList).IsAssignableFrom(pi.PropertyType))
                                                break;  // don't create if we can't handle it
                                        }

                                        if (reader.TokenType == JsonTokenType.StartArray && pi.PropertyType.IsArray)
                                        {
                                            child = Activator.CreateInstance(typeof(List<>).MakeGenericType(pi.PropertyType.GetElementType())); // we can't add to arrays...
                                        }
                                        else
                                        {
                                            child = populator.CreateInstance(ref reader, pi.PropertyType);
                                            if (child != null)
                                            {
                                                pi.SetValue(parent, child);
                                            }
                                        }
                                    }
                                }
                            }

                            if (reader.TokenType == JsonTokenType.StartObject)
                            {
                                _objects.Push(child);
                            }
                            else if (child != null) // StartArray
                            {
                                peek = _properties.Peek();
                                peek.IsArray = pi.PropertyType.IsArray;
                                peek.List = (IList)child;
                                peek.ListPropertyType = GetListElementType(child.GetType());
                                peek.ArrayPropertyInfo = pi;
                            }
                        }
                        break;

                    case JsonTokenType.EndObject:
                        _objects.Pop();
                        if (_properties.Count > 0)
                        {
                            _properties.Pop();
                        }
                        break;

                    case JsonTokenType.EndArray:
                        if (_properties.Count > 0)
                        {
                            prop = _properties.Pop();
                            if (prop.IsArray)
                            {
                                var array = Array.CreateInstance(GetListElementType(prop.ArrayPropertyInfo.PropertyType), prop.List.Count); // array is finished, convert list into a real array
                                prop.List.CopyTo(array, 0);
                                prop.ArrayPropertyInfo.SetValue(_objects.Peek(), array);
                            }
                        }
                        break;

                    case JsonTokenType.False:
                    case JsonTokenType.Null:
                    case JsonTokenType.Number:
                    case JsonTokenType.String:
                    case JsonTokenType.True:
                        peek = _properties.Peek();
                        if (peek.List != null)
                        {
                            if (populator.TryReadPropertyValue(ref reader, options, peek.ListPropertyType, out var item))
                            {
                                peek.List.Add(item);
                            }
                            break;
                        }

                        prop = _properties.Pop();
                        var current = _objects.Peek();
                        if (current != null)
                        {
                            populator.SetPropertyValue(ref reader, options, current, prop.PropertyName);
                        }
                        break;
                }
            }
        }

        private static Type GetListElementType(Type type)
        {
            if (type.IsArray)
                return type.GetElementType();

            foreach (Type iface in type.GetInterfaces())
            {
                if (!iface.IsGenericType) continue;
                if (iface.GetGenericTypeDefinition() == typeof(IDictionary<,>)) return iface.GetGenericArguments()[1];
                if (iface.GetGenericTypeDefinition() == typeof(IList<>)) return iface.GetGenericArguments()[0];
                if (iface.GetGenericTypeDefinition() == typeof(ICollection<>)) return iface.GetGenericArguments()[0];
                if (iface.GetGenericTypeDefinition() == typeof(IEnumerable<>)) return iface.GetGenericArguments()[0];
            }
            return typeof(object);
        }
    }

    private class WorkerProperty
    {
        public string PropertyName;
        public IList List;
        public Type ListPropertyType;
        public bool IsArray;
        public PropertyInfo ArrayPropertyInfo;

        public override string ToString() => PropertyName;
    }
}

I am not sure if this will fix your problem, but it should work as a temporary workaround. All I did was write a simple class with a populateobject method in it.

public class MyDeserializer
{
    public static string PopulateObject(string[] jsonStrings)
    {
        Dictionary<string, object> fullEntity = new Dictionary<string, object>();

        if (jsonStrings != null && jsonStrings.Length > 0)
        {
            for (int i = 0; i < jsonStrings.Length; i++)
            {

                var myEntity = JsonSerializer.Parse<Dictionary<string, object>>(jsonStrings[i]);

                foreach (var key in myEntity.Keys)
                {
                    if (!fullEntity.ContainsKey(key))
                    {
                        fullEntity.Add(key, myEntity[key]);
                    }
                    else
                    {
                        fullEntity[key] = myEntity[key];
                    }
                }
            }
        }

        return JsonSerializer.ToString(fullEntity);
    }    
}

I put it into a console app for testing purposes. Below is the entire app if you would like to test it yourself.

using System;
using System.Text.Json;
using System.IO;
using System.Text.Json.Serialization;

namespace JsonQuestion1
{
    class Program
    {
        static void Main(string[] args)
        {
            // Only used for testing
            string path = @"C:\Users\Path\To\JsonFiles";
            string st1 = File.ReadAllText(path + @"\st1.json");
            string st2 = File.ReadAllText(path + @"\st2.json");
            // Only used for testing ^^^

            string myObject = MyDeserializer.PopulateObject(new[] { st1, st2 } );

            Console.WriteLine(myObject);
            Console.ReadLine();

        }
    }

    public class MyDeserializer
    {
    public static string PopulateObject(string[] jsonStrings)
    {
        Dictionary<string, object> fullEntity = new Dictionary<string, object>();

        if (jsonStrings != null && jsonStrings.Length > 0)
        {
            for (int i = 0; i < jsonStrings.Length; i++)
            {

                var myEntity = JsonSerializer.Parse<Dictionary<string, object>>(jsonStrings[i]);

                foreach (var key in myEntity.Keys)
                {
                    if (!fullEntity.ContainsKey(key))
                    {
                        fullEntity.Add(key, myEntity[key]);
                    }
                    else
                    {
                        fullEntity[key] = myEntity[key];
                    }
                }
            }
        }

            return JsonSerializer.ToString(fullEntity);
      }
    }
}

Json File Contents:

st1.json

{
    "Title": "Startpage",
    "Link": "/index"
}

st2.json

{
  "Title": "Startpage",
  "Head": "Latest news",
  "Link": "/news"
}

If its just one usage and you don't want to add extra dependencies / lots of code, you don't mind a bit of inefficiency and I've not missed something obvious, you can just use:

    private static T ParseWithTemplate<T>(T template, string input) 
    {
        var ignoreNulls = new JsonSerializerOptions() { IgnoreNullValues = true };
        var templateJson = JsonSerializer.ToString(template, ignoreNulls);
        var combinedData = templateJson.TrimEnd('}') + "," + input.TrimStart().TrimStart('{');
        return JsonSerializer.Parse<T>(combinedData);
    }

If you already use AutoMapper in your project or don't mind having dependency on it, you can merge objects in a following way:

var configuration = new MapperConfiguration(cfg => cfg
    .CreateMap<Model, Model>()
    .ForAllMembers(opts => opts.Condition((src, dest, srcMember) => srcMember != default)));
var mapper = configuration.CreateMapper();

var destination = new Model {Title = "Startpage", Link = "/index"};
var source = new Model {Head = "Latest news", Link = "/news"};

mapper.Map(source, destination);

class Model
{
    public string Head { get; set; }
    public string Title { get; set; }
    public string Link { get; set; }
}

The workaround can also be as simple as this:

using System;
using System.Linq;
using System.Text.Json.Serialization;

namespace ConsoleApp
{
    public class Model
    {
        public string Title { get; set; }
        public string Head { get; set; }
        public string Link { get; set; }
    }

    class Program
    {
        static void Main(string[] args)
        {
            var model = new Model();

            Console.WriteLine(JsonSerializer.ToString(model));

            var json1 = "{ \"Title\": \"Startpage\", \"Link\": \"/index\" }";

            Map<Model>(model, json1);

            Console.WriteLine(JsonSerializer.ToString(model));

            var json2 = "{ \"Head\": \"Latest news\", \"Link\": \"/news\" }";

            Map<Model>(model, json2);

            Console.WriteLine(JsonSerializer.ToString(model));

            Console.ReadKey();
        }

        private static T Map<T>(T obj, string jsonString) where T : class
        {
            var newObj = JsonSerializer.Parse<T>(jsonString);

            foreach (var property in newObj.GetType().GetProperties())
            {
                if (obj.GetType().GetProperties().Any(x => x.Name == property.Name && property.GetValue(newObj) != null))
                {
                    property.SetValue(obj, property.GetValue(newObj));
                }
            }

            return obj;
        }
    }
}

Output:





asp.net-core-3.0