Today we are releasing a Beta 2 of the Couchbase .NET SDK 2.0.   This is an exciting update in our continued commitment to the .NET platform, and the .NET community.  We've even managed to squeeze in a few new features before the GA Release in response to customer demand.   I'm excited to announce the inclusion of Bulk Methods and a sneak peak at Asynchronous Views and N1QL queries!

What’s in this release?

Feature-wise it builds on the previous releases (DP1, DP2 and Beta) and adds the following:

  • Bulk methods: Upsert and Get
  • Asynchronous Views
  • Asynchronous N1QL queries (Experimental – N1QL is still DP3)
  • Common Data Types – consistent typing of values amongst SDK’s
  • Management API – for managing Clusters, Buckets and Views

Bulk Methods: Upsert and Get

Bulk methods allow the application to send a set of keys in a single request and then wait for the results. For example:

using(var cluster = new Cluster())
{
    using (var bucket = cluster.OpenBucket())
    {
        var items = new Dictionary<string, dynamic>
        {
            {“Key1”, “string”},
            {“Key2”, new {Foo = “Bar”, Baz = 2}},
            {“Key3”, 2},
            {“Key4”, 5.8},
            {“Key5”, new[] {0x00, 0x00}}
         }
        var multiUpsert = bucket.Upsert(items);
        foreach (var item in multiUpsert)
        {
            Assert.IsTrue(item.Value.Success);
        }
   }
}

Above we are creating a Cluster object, and opening the default Bucket. After that we build a dictionary in with it’s keys defined as string and it’s values defined as a dynamic Type. We then add a number of keys and associated values; notice that each value is a different Type: a string, an anonymous Type, an integer, a decimal and finally a byte array. We are then calling IBucket.Upsert(…) and passing in the dictionary. We are then iterating through the results checking to see if each operation was successful.

Behind the scenes we are leveraging the Task Parallel Library to fire off each operation in parallel. For this overload the default ParallelOptions are used, but there are overloads that allow you to pass in your own ParallelOptions (including your own CancellationToken) and to control the size of each partition of keys that will be processed per iteration.

Asynchronous Views

Although support for Asynchronous CRUD (InsertAsync, UpsertAsync, etc) operations didn’t make it into the GA, Asynchronous Views did. The implementation is based off of the Task Asynchrony Pattern (TAP) that is familiar to most .NET developers and supports async/await keywords introduced in .NET 4.0:

public async void QueryAsync()
{
        using (var bucket = _cluster.OpenBucket(“beer-sample”))
        {
            var query = bucket.
                CreateQuery(“beer”, “brewery_beers”).
                Limit(10);

            var result = await bucket.QueryAsync<dynamic>(query);
            foreach (var row in result.Rows)
            {
                Console.WriteLine(row);
            }
        }
}

Here we are opening the “beer-sample” bucket from an already created Cluster object and then creating a query that will target the “brewery_beers” View that is defined by the “beer” design document. Importantly notice that the we are using the await keyword to signal an asynchronous wait and that calling method has the async keyword defined, also the name of the method that is executing the query called is postfixed with “Async”: QueryAsync(IViewQuery query). What is happening here is that the query request will be fired off an a ThreadPool thread and not off of the main thread. Cool stuff.

Asynchronous N1QL Queries (Experimental)

Similar to asynchronous Views, we have added overloads for executing N1QL queries asynchronously using async/await keywords:

public async void QueryN1QLAsync()
{
    using (var bucket = _cluster.OpenBucket())
    {
         const string query = “SELECT * “ +
                                 “FROM tutorial “ +
                                 “WHERE fname = ‘Ian'”;

          var result = await bucket.QueryAsync<dynamic>(query);
          foreach (var row in result.Rows)
          {
                Console.WriteLine(row);
           }
       }
 }

Once again we have added the “async” keyword to the signature of the method executing the query and then used the “await” keyword to do an asynchronous wait while query is executing off of the main thread and on a ThreadPool thread. Once the call returns, we iterate through the results just like any other N1QL query.

Common Data Types

It’s fairly common an enterprise environment to have one or more teams of developers using more than one platform to deliver software solutions satisfying the business needs of the organization. For example, a tier written in Java exposing core services to third-parties and another tier written in .NET consuming the data pushed in by the Java tier. In situations like this where one platform is writing data into Couchbase and another platform is reading that data, it’s important that underlying data type of each item stored in Couchbase are treated the same across both platforms and their respective SDK’s.

In the past, their has been some inconsistency on how the various SDK’s handled transcoding of types, but that changes in 2.0 SDK’s. Now when one SDK, say Python, stores a document or value, when another platform reads that value, it will be transcoded as the same type. This done by simply treating a subset of types as either JSON, strings, binary data or as a “private” type for backwards compatibility. This type information is stored as a “Common Flag” within the documents metadata and used by the SDK’s to determine how to handle transcoding consistently.

Management API

The last major feature released in GA is an all new Management API for performing Cluster and Bucket level tasks such as creating and removing Buckets, Design Document and Views, and adding and removing nodes from a Cluster.

The management API can be broken down into two discrete types of operations: those which are performed on a Bucket and those which are performed on the Cluster. Each are represented by their own respective manager objects, that are created by the Cluster object’s factory methods.

Here is an example of adding a node to a Cluster:

using (var cluster = new Cluster(configuration))
{
    var clusterManager = cluster.CreateManager(“Administrator”, “password”);
    var result = clusterManager.AddNode(“192.168.56.103”);
    Assert.IsTrue(result.Success);
}

First we create a Cluster object, which is the same object we use to open Buckets, passing in a configuration that tells the Cluster where the remote cluster exists. Then we call the CreateManager(…) factory method passing in our cluster’s administrative username and password. Once we have authenticated, we add a new node to the cluster and check to see that the request was honored as expected.

In the next example, a Bucket level administrative task will be performed: added a new Design Document and View to a Bucket.

using (var cluster = new Cluster(configuration))
{
    using (var bucket = cluster.OpenBucket())
    {
        var manager = bucket.CreateManager(“Administrator”, “”);
        var designDoc = File.ReadAllText(@”Data\DesignDocs\by_field.json”);
        var result = manager.InsertDesignDocument(“by_field”, designDoc);
        Assert.IsTrue(result.Success);
    }
}

Here we are creating a Cluster object, then opening the default Bucket and using a factory method to create the BucketManager object. Once we have the manager, we open a file containing a Design Document and View as JSON and use the InsertDesignDocument(..) method to add the Design Document “by_field” to the Bucket. Besides inserting a design document, additional methods exist for updating an existing design document, getting a design document or all of the design documents and for removing a design document.

What’s Changed?

Between Beta and Beta 2, a number of changes have occurred within the SDK. For the most part these are either additive or internal and will not impact users which have started using the 2.0 SDK. However, there are a couple of changes which are breaking changes, most notably:

A number of other minor changes mainly involving the renaming and/or reordering of parameters have also occurred.  

  • Couchase.CouchbaseCluster has been renamed to Couchbase.Cluster
  • CouchbaseBucket and MemchachedBucket have been moved from Couchbase.Core.Buckets to the main Couchbase namespace.

What’s not in this release?

While we strive to get as many features as possible into a release, there are always more features than cycles available it seems and some features did not make their way into GA. Most notably:

  • Replica reads: allowing the client to read from one or more replicas. This is primarily used for rebalance scenarios when a “Not My VBucket” could be encountered.
  • Asynchronous CRUD operations using async/await
  • Pluggable JSON parser: the ability to use the JSON serializer/deserializer of your choice. Currently the client relies on Newtonsoft.JSON.
  • Per connection TCP heartbeat settings: currently this is configured at the OS level via KeepAliveTime on Windows: http://technet.microsoft.com/en-us/library/cc782936%28WS.10%29.aspx

How to Get It

The binaries our available in S3, the source code is on Github, and the packages are available on Nuget:

  • Binaries are here.
  • The repo is here.
  • The NuGet packages are here.

Acknowledgments

Being an Open Source project, the Couchbase .NET SDK depends upon contributions no matter how small, from the community. An extra special thanks to the following who submitted pull requests and/or issued bug reports:

  • Matt Nischan
  • Vojta Jakubec

Author

Posted by Jeff Morris, Senior Software Engineer, Couchbase

Jeff Morris is a Senior Software Engineer at Couchbase. Prior to joining Couchbase, Jeff spent six years at Source Interlink as an Enterprise Web Architect. Jeff is responsible for the development of Couchbase SDKs and how to integrate with N1QL (query language).

All Posts

2 Comments

  1. Doron November 6, 2014 at 11:00 am

    Great job!
    What about LINQ? I it\’s not mentioned anywhere in this post, not even in the What’s not in this release? section.
    I hope it wasn\’t left behind…

    1. Jeff Morris November 7, 2014 at 1:47 am

      Hi Doron –

      The Linq2Couchbase project is alive and kicking, it\’s a separate layer over the SDK though. You can find it here: https://github.com/couchbasela…

      Note that I still haven\’t updated the nuget packages to beta2, so it probably won\’t work until it\’s updated; there are some structural changes will break it.

      Thanks!,

      Jeff

Leave a reply