We saw in a previous post how easy is to set up a Cross Data Center Replication (XDCR), today let’s go a little bit deeper to understand what makes XDCR such a great feature.
First of all, XDCR allows you to replicate data between different sized clusters, which makes it an excellent option for disaster and recovery plan. Apart from that, it is a simple yet powerful way to bring data closer to your users.
The replication is made from Memory-to-Memory. Thus all writes are saved in the memory first and then put in a replication queue, which sends it over the network through multiple threads. So, the whole performance is only limited by your network speed.
It is also topology aware, and therefore whenever you add or remove nodes from the source cluster, no action needs to be taken on the destination cluster. It will reestablish the connection and handle everything automatically.
Moreover, it also provides rack zone awareness, which helps to protect against multi-node failure events by separating active data and its replicas across “groups” which can then be mapped such that they occupy different racks, zones, or VM hosts.
Replications are made on the bucket level (between buckets of two or more clusters), and can be configured as follows.
- Unidirectional: Only the data written in one of the clusters is replicated, it is used when you want to configure a standby cluster for example.
- Bidirectional: (also known as active-to-active deployment) where both clusters can write data and all changes are synchronized between them. In summary, a bidirectional mapping is just two unidirectional replications pointing to each other.
- Hybrid: A combination of Bidirectional and Unidirectional topologies.
Thanks to DCP, you can also pause the replication at any time, and once you resume it, the recovery starts at the most recent checkpoint.
Database Change Protocol
Database Change Protocol (DCP) is a high-performance streaming protocol we use internally to communicate the state of data using an ordered changelog. It is robust and resilient in the face of transitory errors, for example, if the communication is interrupted, DCP is capable of resuming from the exact point of the last successful update once the connectivity is back.
It is also optimized to send only necessary data. For example, if there are several changes in a document, just the most recent version is marked to be replicated.
XDCR relies on DCP to propagate changes. This way it guarantees the same document will be replicated among all clusters regardless of connectivity problems.
Let’s also highlight another two exciting features of XDCR: Conflict Resolution and Data Filtering:
A conflict is where the same document is modified in two different locations before it has been synchronized between the locations. To maintain consistency, one version has to be chosen as the ‘correct’ version. Conflict resolution provides a method to consistently and deterministically select which version of the document to use.
Couchbase’s conflict resolution is set during the bucket creation and can’t be changed later. Currently, two types of conflict resolutions are supported: Timestamp and Sequence Number.
After every mutation in a document we increase a counter called revision number, so whenever there is a conflict between two documents, the one with the highest revision number will take precedence on both clusters.
Timestamp-based Conflict Resolution (TCR) is the most commonly supported conflict resolution mechanism in databases, TCR resolves conflicts by selecting the document with the most recent timestamp. To be able to perform this effectively it is essential that the time stamps created by each server are closely aligned.
However, TCR might increase data loss, as it ignores how many times a document has been updated, and if one of the server’s clock is fast/slow you will end up with a messed up conflict resolution. That is why the default option in Couchbase is “Sequence Number”.
By default, all documents within a target bucket will be replicated, but you can still choose which you actually would like to replicate to other clusters by filtering them using regular expressions.
Currently, filtering can only be made by key, so If you want to replicate just documents with the type “hotel” or “flight” for example, I would recommend you to prefix your key with the field you would like to filter:
and then in the XDCR configuration, add the following regular expression:
Note that you can test your regex by adding some keys in the Test Key field.
If you need a Disaster & Recovery plan or just would like to bring your data closer to the user, Cross Data Center Replication (XDCR) is a feature you should consider using, It is simple to set up, requires almost no maintenance and has been heavily tested in several high load use cases like Amadeus, eBay and Viber.