Blog Post

Python SDK 1.1.0 Released

Mark Nunberg of Couchbase Published

The Python SDK for Couchbase has just been release with a new version -- 1.1.0. This blog post will go into some of the enhancements that have gone into this release.

The 1.1.0 SDK utilizes a newer version of libcouchbase (2.1.x) with additional features, bugfixes, and performance improvements. Note that it is also possible to use the older 1.0.0 SDK with the newer libcouchbase.

Durability Requirements

In 1.1.0, you can pass extra arguments to the various mutator methods (e.g. 'set', 'add', etc.) which will instruct the SDK to verify that the item has been persisted or replicated to a certain number of nodes before continuing.

Using this feature allows your application to proceed with a higher degree of confidence for critical data.

Normal Couchbase Server replication and persistence is asynchronous for performance reasons: The server will consider an operation successful once it is successfuly stored in the key/vBucket master's memory cache. The item is then placed in a replication queue and a persistence queue (which on normal occasions only take between several milliseconds to several seconds to be flushed).

Durability requirements allow the SDK to change the semantics of an operation's success by placing synchronization demands on the aforementioned queues. This is useful to avoid a race condition where the item is stored in a master node, and that node is immediately failed over before the item has been flushed from the various queues.

To use this feature, you need only specify the 'persist_to' or 'replicate_to' parameters for a storage operation; so for example:

cb.set("key", "value", persist_to=1, replicate_to=2)

Will only return successfuly once "key" has been placed on the master's disk, and has been propagated to at least two replicas.

Item API

In 1.0.0, all operations return a new instance of 'Result', or a subclass thereof. While this behavior is quite consistent and easy to deal with when handling API responses, it involved extra work in some applications where various parameters needed to be copied out from the 'Result' object into an application-specific data structure.

The 'Item' API was introduced to allow the best of both worlds. You can now define your own data structure and subclass it from 'Item' (which itself is also a 'Result' subclass). Using the Item API involves the following

  1. Define your own subclass of 'Item' (or just use the base class as-is)
  2. Create an instance of the class and give it a key. This key corresponds to the key in Couchbase Server which will be used for data operations.
  3. Create an 'ItemCollection' instance. This can be done with either the 'ItemSequence' collection (which just contains a list of items) or an ItemOptionDict (which can contain per-item per-operation options).
  4. Pass the 'ItemCollection' as the 'kv' parameter to one of the '*_multi' methods.
  5. Once the API method is complete, each Item will have its relevant information updated. For example after performing a 'get', each Item's 'value' and 'cas' properties will be updated with the values retrieved from the server.

See https://gist.github.com/mnunberg/6799897 for a full example of using the 'Item' API

gevent Support

gevent is an event/IO library making use of the greenlets coroutine system. One of the key features in gevent is the ability to intercept all Python I/O calls which would normally block, and proxy them into non-blocking variants. While this works for most Python modules, this does not work by default for C extensions which perform I/O outside of Python.

To this effect, experimental support was added to 1.1.0 for gevent. What this mainly does is reroute the libcouchbase I/O functions to Python itself using libcouchbase's "IOPS" I/O plugin system. Once this is done, the blocking routines (now being called as pure Python) are now proxied by gevent.

To enable this experimental feature, simply pass "experimental_gevent_support=True"  to the Connection (or Couchbase) constructor. Note that you will still need to monkey-patch the socket methods.