Today, Thumbtack Technology published a blog post highlighting the final results of the NoSQL benchmark (link).

In June, the databases were benchmarked on 4 physical servers. I could see MongoDB and DataStax in the rearview mirror (link). Then, the databases were benchmarked on 4, 6, and 8 physical servers.

The performance tests were executed with two workloads.

  • Read Intensive (95% read / 5% write)
  • Balanced (50% read / 50% write)

Read Intensive

MongoDB scaled from 130K ops / second with 4 servers to 227K ops / second with 8 servers. The average latency was less than a millisecond. The latency is great, the throughput is not.

Apache Cassandra (DataStax) scaled from 60K ops / second with 4 servers to 99K ops / second with 8 servers. The average latency was below 3ms. The latency is not good, neither is the throughput.

Couchbase Server scaled from 903K ops / second with 4 servers to 1.71 million ops / second with 8 servers. The average latency was less than a millisecond. The latency is great, the throughput is great.

Balanced

MongoDB scaled from 50K ops / second with 4 servers to 85K ops / second with 8 servers. The average latency was less than a 2ms. The latency is pretty good, the throughput is pretty bad.

Apache Cassandra (DataStax) scaled from 53K ops / second with 4 servers to 89K ops / second with 8 servers. The average latency was below a millisecond. The latency is great, the througput is not.

Couchbase Server scaled from 800K ops / second with 4 servers to 1.24 million ops / second with 8 servers. The average latency was less than two milliseconds. The latency is pretty good, the throughput is great.

Conclusion

Apache Cassandra (DataStax) meets low latency requirements for write requests, at low throughput. It's not engineered for read performance. It does not leverage memory effectively (link). Apache Cassandra was engineered for an era when servers had little memory and spinning disks.

MongoDB meets low latency requirements for read requests, at low throughput. It's not engineered for write performance. It's limited by a master / slave topology, database-wide locking (one per db per instance), and more. MongoDB was engineered for… well, I'm not sure what it was engineered for.

Couchbase Server meets low latency requirements for read and write requests, at any throughput. Unlike MongoDB, it doesn't have topology and locking issues. Unlike Apache Cassandra, it leverages memory effectively with a integrated, managed object cache. Couchbase Server was engineered for read and write performance.

I can't see MongoDB and DataStax in the rearview. Not any more. Where art thou?

It's all in the white paper (here).

See Doug's take on the benchmark results (here).

Hacker News (link)
Reddit (link)

Posted by Shane Johnson, Director, Product Marketing, Couchbase

Shane K Johnson was the Director of Product Marketing at Couchbase. Prior to Couchbase, he occupied various roles in developing and evangelism with a background in Java and distributed systems. He has consulted with organizations in the financial, retail, telecommunications, and media industries to draft and implement architectures that relied on distributed systems for data and analysis.

One Comment

  1. We may need a Couchbase section if/after we develop content for that NoSQL DB, too.

Leave a reply