One of the advantages of Open Source is the ability to stay close to the users. This has been hugely useful as we’ve developed Couchbase over the years. Of course, like any fast growing project and community, every once in awhile you need to revisit and re-tune core infrastructure to help yourself go faster.
That is our intent with SDK-RFCs and today we would like to solicit input.
TL;DR: Let us know what you think of the plans for the new Sub-Document API and the new Index Management interface.
More on the Challenge
As Couchbase Server 3.0 shipped, we also shipped a bevy of new SDKs that were setting us up for future releases. Each of these intentionally broke the API. We shipped previews it is all Open Source under an Apache 2.0 license, but we never really knew until a “dot O” version shipped and became recommended if the change would be acceptable. Looking back, it’s worked out very well as almost everyone writing code for Couchbase has made the move.
One of our objectives in this move was to make the API more common, while still allowing for idioms in the individual platforms developers have chosen. The first step there was to collaboratively determine what we thought the right level of detail was in the API, then get some feedback from others.
In effect, we were drawing up the artist’s renderings with draft blueprints.
This generally worked well. Many of the artifacts helped us get to the level of detail and commonality we desired. Still, we felt we could improve by making sure the interfaces being defined for co-projects were actually acceptable to those co-projects. That sounds obvious, but there were some problem areas. We also felt that we should make more room for anyone and everyone to comment. While most people developing on Couchbase won’t have the time to jump into design conversations, being Open will put more innovative eyeballs on our thoughts and the discussion and both the discussions and artifacts can give others insight into the thinking and design decisions for many years to come.
The Infrastructure
Our goals were for anyone to be able to jump into the discussion, for the artifacts to be publicly available and discoverable and for the process to follow patterns developers were familiar with.
There’s a bit of an overlap here with IETF RFCs, but with a modern twist. The modern twist is that rather than mailing lists being the social playing field for discourse, developer conversations– especially open ones– now happen on Github.
We’ve set up is a new repository under the couchbaselabs organization named SDK-RFCs. The first SDK-RFC is the SDK-RFC process because we’re meta like that. In effect, we open an issue to track the need, then put together a pull request to document the approach. All the while we maintain a README that lists everything in development and complete.
As mentioned in the README, we know we’re not alone in this. There are of course the well known IETF RFCs, Rust RFCs and my former Sun colleague Bryan Cantrill and the folks at Joyent had a similar need and are doing a great job inspiring discussion in Joyent RFDs.
Your Turn
Two SDK-RFCs in particular are pretty far along and ripe for input. We would love to hear your voice.
The first is on Sub-Document operations. There are times app developers want to modify a part of a document without retrieving it and storing it. Toward that end a set of protocol extensions and an API are being developed for the lowest level of operations and other APIs and components to Couchbase will be able to take advantage of these primitives in their own way in the future. This SDK-RFC covers a common set of API that we’ll put into the SDKs. It will be directly part of the public interface if you want the high efficiency this interface affords you. The new capabilities in the system will also serve to speed up other parts of Couchbase like N1QL over time.
The second is on Index Management. When building continuous integration or test scaffolding, indexes are often needed. The REST API is of course already available, but surfacing it in an API makes it easier to be used.
Please peruse these RFCs and comment on either the Sub-Document or the Index Management pull requests!
Credit for developing the process goes to everyone in the SDK engineering team here at Couchbase. Michael kickstarted the move from our onepagers to SDK-RFC and Mark took on the first one with Sub-Document. It was a good one to test the system, because of its wide reach. Brett and I worked with the team to refine the process and tools a bit and the whole team (everyone above plus Sergey, Jeff, Todd and Simon) jumped in with contributions and comments.
