February 1, 2013

Moving No Schema up the Stack with C# and Dynamic Types

One of the significant benefits of working with Couchbase Server 2.0 is its flexible schema.  Documents are stored as JSON, allowing for implicitly structured records that impose no order on each other.  In the real world, that "implicit structure" comes from your application.  When you create a new user in your application, the associated document is a JSON serialized version of your domain object.

public class User
{
    [JsonIgnore]
    public string Id { get; set; }
   
    [JsonProperty("username")]
    public string Username { get; set; }
   
    [JsonProperty("password")]
    public string Password { get; set; }
   
    [JsonProperty("type")]
    public string Type { get { return "user"; } }
}

//stored as { "username" : "hmoody", "password" : "b3cca" }

While this approach is common in applications where documents are read into and written from well defined domain objects, there are cases where it the structure of the documents is intentionally less well defined.  In such cases, it might not be feasible to have a domain object per document type. 

While in languages such as Python or Ruby, it might be common to use a less object-oriented approach (e.g., dictionaries or hashes), in strongly typed languages such as C# or Java, it's far more common to represent application data using strongly typed data objects, often called plain old (Java|C#) objects.  However, it is certainly possible to use dynamic language approaches with these languages. 

When a user asked recently on StackOverflow how to pull JSON documents out of Couchbase and into loosely typed C# objects, I proposed two options.  For the rest of this post, I'll describe a few basic extension methods you could use to take a similar approach with your data.

The first approach is to store Dictionary<string, object> instances.  Dictionaries naturally map to JSON structures, so they're a natural choice for working with documents in a less structured way.  Nested dictionaries also map well to complex JSON structures. 

var user1 = new Dictionary<string, object>
{
    { "username", "jzablocki" },
    { "preferences", new Dictionary<string, object>
        {
            { "theme",  "green"},
            { "timezone",  "EST" }
        }
    }
};

To save and read this document, we'll add extension methods to save and retrieve dictionaries.  These methods will live inside a new static class named CouchbaseDynamicExtensions.

public static class CouchbaseDynamicExtensions { ... }

The first method will simply wrap the standard ExecuteStore method, but will encapsulate a few things.  First, it will take care of serializing the Dictionary to JSON using the Newtonsoft.JSON library.  You can modify the serializer settings to support camel casing or other JSON formatting options.  I've left the defaults in place.  Second, I've encapsulated the IStoreOperationResponse details into an arguably simpler Tuple return.  Since tuples are commonly returned in dynamic languages, and I'm trying to be more dynamic, this seemed like an appropriate approach.

public static Tuple<bool, int, string> StoreDictionary(this ICouchbaseClient client, StoreMode storeMode,
                                            string key, Dictionary<string, object> dictionary)
{
    var json = JsonConvert.SerializeObject(dictionary);
    var result = client.ExecuteStore(storeMode, key, json);

    if (!result.Success)
    {
        if (result.Exception != null) throw result.Exception;

        return Tuple.Create(false, result.StatusCode.HasValue ? result.StatusCode.Value : -1, result.Message);
    }

    return Tuple.Create(true, 0, string.Empty);
}

Getting the Dictionary back from the stored JSON simply reverses the process.  Again, I'm wrapping the IGetOperationResult in a Tuple and taking care of deserializing the stored JSON into a Dictionary<string, object> instance.

public static Tuple<bool, int, string, Dictionary<string, object>> GetDictionary(this ICouchbaseClient client, string key)
{
    var result = client.ExecuteGet<string>(key);

    if (!result.Success)
    {
        if (result.Exception != null) throw result.Exception;

        return Tuple.Create<bool, int, string, Dictionary<string, object>>
                    (false, result.StatusCode.HasValue ? result.StatusCode.Value : -1, result.Message, null);
    }

    var dict = JsonConvert.DeserializeObject<Dictionary<string, object>>(result.Value);
    return Tuple.Create(true, 0, string.Empty, dict);
}

Saving and retreiving is straightforward (be sure to add a using to your extension class namespace).

var result = client.StoreDictionary(StoreMode.Set, "user_1", user1);
if (result.Item1)
{
   var dict = client.GetDictionary("user_1").Item4;
   Console.WriteLine(dict); //should be output of Dictionary.ToString()
}

A more interesting approach would be to take advantage of C#'s new dynamic typing and the ExpandoObject class.  These features allow developers to instruct the compiler to perform type checking at runtime, not at compile time.  Working with JSON documents is a great use case for dynamics.

The dynamic extension methods are almost identical, except where before there were dictionaries, now there are dynamic types. 

public static Tuple<bool, int, string> StoreDynamic(this ICouchbaseClient client, StoreMode storeMode,
                                            string key, ExpandoObject obj)
{
    var json = JsonConvert.SerializeObject(obj);
    var result = client.ExecuteStore(storeMode, key, json);

    if (!result.Success)
    {
        if (result.Exception != null) throw result.Exception as Exception;

        return Tuple.Create(false, result.StatusCode.HasValue ? result.StatusCode.Value : -1, result.Message);
    }

    return Tuple.Create(true, 0, string.Empty);
}

public static Tuple<bool, int, string, ExpandoObject> GetDynamic(this ICouchbaseClient client, string key)
{
    var result = client.ExecuteGet<string>(key);

    if (!result.Success)
    {
        if (result.Exception != null) throw result.Exception;

        return Tuple.Create<bool, int, string, ExpandoObject>
                    (false, result.StatusCode.HasValue ? result.StatusCode.Value : -1, result.Message, null);
    }

    var obj = JsonConvert.DeserializeObject<ExpandoObject>(result.Value);
    return Tuple.Create(true, 0, string.Empty, obj);
}

Using dynamic instances in your code, you can then save and retrieve data to and from Couchbase Server.  Note that you could also read any JSON document into an ExpandoObject using the code approach below.  To test that, you can call GetDynamic with a key of "user_1."

dynamic user2 = new ExpandoObject();
user2.Username = "jzablocki";
user2.Preferences = new ExpandoObject();
user2.Preferences.Theme = "green";
user2.Preferences.TimeZone = "EST";

client.StoreDynamic(StoreMode.Set, "user_2", user2 as ExpandoObject);
var getResult = client.GetDynamic("user_2");
if (getResult.Item1)
{
    dynamic item = getResult.Item4;
    Console.WriteLine(item.Preferences.Theme);
}

There are alternate approaches toward the dynamic extensions that require casting, because of the limitations of calling an extension method with dynamic arguments.  To simplify things, I've stuck with ExpandoObject arguments. 

C# obviously isn't a purely dynamic language, and therefore some of the methods are a less terse than they would be in their purely late bound counterparts.  However, these methods demonstrate that you don't have to give up the full richness of a schemaless document-oriented database just because your language is statically typed, as long as that language is C# of course...

Comments