What I’m learning about Couchbase is that while it aims to provide sensible defaults, it also provides options to make trade-offs. The trade-off I’m going to focus on in this blog post is Scan Consistency for N1QL queries.

N1QL is SQL for JSON Documents

Couchbase 4.5 introduces a new Scan Consistency option: AtPlus. This joins the existing options to make a total of three:

  • NotBounded (default)

  • AtPlus (new to Couchbase Server 4.5)

  • RequestPlus

This blog post is going to review NotBounded and RequestPlus, as well as introduce AtPlus. I’ll also provide a code sample that you can use to try out all three yourself.

Not Bounded

This is the default behavior.

A N1QL query that is using “Not Bounded” Scan Consistency means that it will not wait for any indexes to finish updating before running the query and returning results. Let’s say you have documents A,B,C in a bucket. At the time of the query, only A and B are indexed. With Not Bounded, only documents A and B will be returned.

Since the query is not waiting on any indexing, this is the best option for performance. Document C will show up once it has been indexed. This shouldn’t take long, but if you have just created C, for instance, and then immediately query for a full list of documents, C may not show up. To demonstrate with the travel-sample bucket:

What I would expect to see when running this code is:

  1. Initial count: N

  2. Count after insert: N (still)

  3. Waiting

  4. Count after waiting: N+1

I set the wait time at 5 seconds, which is probably overkill. I use Thread.Sleep just for demonstration purposes, but obviously that’s a clunky tactic for a real app. That brings us to RequestPlus.

RequestPlus

This scan consistency option provides nearly the opposite of Not Bounded. It will wait until all document changes and index updates (up until the query was run) are processed before it runs the query.

Here’s a simplified example sequence of events when using RequestPlus

  1. Document C is created

  2. N1QL query to get all documents is executed (A,B,C all exist in the bucket)

  3. N1QL query is put on hold because at least one document needs to be indexed (document C).

  4. Indexing process is completed. Document C is now indexed.

  5. N1QL query executes, returning A,B,C.

This gives us completeness in the query results, at the expense of performance. In some cases, it doesn’t make sense to wait for everything to be indexed. And that brings us to AtPlus.

AtPlus (new to Couchbase 4.5)

This scan consistency option provides something of a middle ground between RequestPlus and Not Bounded. This is also a brand new scan consistency option for Couchbase 4.5.

With AtPlus, you’ll have to do a little more work in your code, but in return you’ll get better performance than using RequestPlus. Instead of waiting for an entire index to complete (which could be multiple documents), it will just wait for the documents that you specify to be indexed before running the query. This is sometimes known as “read your own write” or RYOW.

With this example, the N1QL query will wait only for the new document to be indexed. It will not wait for anything else. In a high-volume system, this could provide a good balance of increased performance and completeness of results.

Note that at the time of writing this post, you’ll need to make sure that you explicitly set UseEnhancedDurability to true when setting up your ClientConfiguration:

Conclusion

With N1QL, there’s a spectrum of speed and completeness. With Couchbase Server 4.5, you now have three options. NotBounded for raw speed, RequestPlus for up-to-now completeness, and AtPlus sits in between.

Please leave a comment, ping me on Twitter, or email me (matthew.groves AT couchbase DOT com).

Posted by Matthew Groves, Developer Advocate

Matthew is a Developer Advocate for Couchbase, and lives in the Central Ohio area. He has experience as a web developer as a consultant, in-house developer, and product developer. He has been a regular speaker at conferences and user groups all over the United States, and he has written AOP in .NET for Manning Books. He has experience in C# and .NET, but also with other web-related tools and technologies like JavaScript and PHP. You can find him on Twitter at @mgroves.

Leave a reply