ForestDB was introduced as an efficient storage engine for Global Secondary Indexes in Couchbase 4.0. The constant innovation, willingness to experiment and the challenge to beat our own metrics drives us at Couchbase and it is with this resolve and determination that the Indexing team has been working on a bleeding edge storage engine. This system is for efficiently storing indexes so that queries that leverage indexes are truly blazing fast and there is minimum overhead of managing the indexes as well. Voila Plasma!

Plasma presents a highly scalable and performant key-value storage engine. Though Memory Optimized Indexes, introduced in Couchbase 4.5, scaled linearly with CPU cores and DRAM, having fully memory resident indexes is prohibitively expensive. Having lock-free data structures to effectively utilize multi-core CPUs and a memory first architecture that persists only when required, helps us create a system that scales almost linearly and exploits the current trends in SSD and Flash storage. Persistent snapshots, that run in the background, help us create recovery points that can be used to rollback in case of failure. There have been numerous innovations in the way we create and manage Persistence Snapshots in Plasma that creates minimal overhead on the disk and CPUs and also on the other processes running on the indexing node, thereby significantly reducing write amplification.

Though I, along with our developers, would like to go on and on about the technical details of the Plasma Storage Engine, we would like you to take Plasma for a spin [Click Here to Download]. Do note that Plasma is the storage engine in the backend when you choose ‘Standard Global Secondary Indexes’ as part of Index Settings while setting up the cluster.  

Early performance numbers of Plasma look highly encouraging. Compared to ForestDB, Plasma performance numbers(while being used as a storage engine for Couchbase Indexes) are in a different and a higher echelon. Plasma is light years ahead in terms of Latency and Throughput, while the Disk Footprints, Initial and Incremental Load Times have reduced by significant multiples when compared with ForestDB.

Our Couchbase customers can try out the Couchbase Server 5.0 and experience the power and magic of Plasma. Plasma is only available with Couchbase EE.   For our existing EE customers, we will be providing a seamless upgrade path from ForestDB to Plasma(details coming soon).

We would love to hear from you. You can leave your comments at our Forum. Read the sequel to this blog post here.

[Credits to the lead developer Sarath Lakshman who worked on the Nitro Storage Engine earlier, implemented Memory Optimized Indexes and took it up a notch further up with Plasma.]

Posted by Venkat Subramanian, Product Manager

Venkat dabbles in product development and product management and has been developing data/analytics platforms & products. Significant chunk of his experience has been with Oracle, where he transitioned from being an Engineer in Oracle’s Enterprise Manager team to Product Manager for Oracle's BI/Analytics suite of products. He has worked in startups in the past helping develop machine-learning/NLP products and distributed decisioning systems. He lurks around at @venkasub.

Leave a reply