September 23, 2010

Membase Server Beta 4 is here, with memcached buckets!

We NorthScalers have been hard at work and are proud to release Membase Server Beta 4, our final Beta release ahead of our general availability release. Go and grab it here! In addition to support for 64-bit Windows, we think you'll be particularly excited by a major new feature in the release: memcached buckets! Introducing Memcached Buckets You now can create buckets in your Membase Server cluster that behave exactly like memcached, which means you can use Membase Server as a drop-in replacement for your existing memcached setup. In a single cluster you can now share the resources between memcached buckets and membase buckets. Let's look at the differences between memcached and membase bucket types:

Fundamentally, membase buckets are designed as permanent data stores. Once you put a key-value (KV) pair into a membase bucket it will remain there until you remove it (or the time-to-live expires). In a membase bucket, data will be written to disk, so your store can grow, constrained only by the available disk space. In addition, membase buckets offer replication; further, they are using vbuckets to allow data to be moved between nodes as cluster topology changes. On the other hand, memcached buckets follow the memcached semantics: they are fundamentally designed as caches, not permanent data stores. As the cache runs out of available cache memory, items are evicted from the cache, based on a least-recently-used (LRU) policy. As a result, your application needs to be able to cope with the expected behavior, namely that an item stored in the cache may not be available at some later point. If that's not the behavior you want, then you need to consider membase buckets as an alternative. And, just like memcached, memcached buckets do not persist data to disk and there is no replication between nodes. When you add a node, keys served from the new node are no longer accessible from the old node, as they do not get transferred from the old node to the new one. There will be cache misses and the KV pairs will need to be set again on the new node - again, these are the normal, expected behaviors of a memcached setup. Memory quota allocation for memcached buckets is identical to the that of the current NorthScale Memcached Server product. A fixed amount of memory per node is allocated for use by the memcached bucket, so adding or removing nodes will change the size of the memcached bucket. This is different from membase buckets, where the quota stays unchanged as the number of nodes changes, but we chose to keep memcached bucket behavior consistent with what current memcached users are accustomed to. The new setup wizard lets you configure the default bucket when creating a new cluster, so you can start with a memcached bucket as the only bucket, and expand from there as your needs dictate. A Quick Word on Disk Quotas Based on our own experience and feedback from users, we took a hard second look at our disk quota system for membase buckets. Ultimately, we decided to remove that option. We believe this change brings the product more in line with typical database behavior: we now only return errors and run out of disk space when you actually run out of disk space :) Of course we are still showing disk usage per bucket and across the cluster, so that you can keep an eye on overall resource usage. Enjoy Beta 4 and let us know what you think!