Cloud-Native Certificates – De Facto Standards For Kubernetes

Couchbase uses TLS to ensure communication across the network is secure, preventing malicious third parties from eavesdropping or tampering with requests, for instance client requests, intercluster requests (node-to-node encryption) and intracluster requests (cross data center replication – XDCR). The latest release of cloud-native Couchbase Autonomous Operator (CAO) 2.3 provides first-class support for the de facto kubernetes.io/tls secret type for storing certificates and the associated key.

What is TLS?

TLS (transport layer security) is the most common standard for securing communication between two parties over a network. It covers authentication, encryption and integrity. Its most common use is for securing an HTTP connection. You’ve probably come across it when visiting an HTTPS website. The S stands for secure! It’s represented by a little padlock in the address bar of some web browsers.

If you view the website’s certificate, you can view fields like: who it was issued by, and when its valid start and end dates are. In a simple world, the issuer is known as a certificate authority (CA).

A CA is a trusted organization whose role is to issue digital certificates. Your computer’s operating system comes with a list of CAs already installed. However, it is also possible to add new CAs that you trust, either because a previous CA is now out of date and needs updating or because you want to self-sign something.

Certificate chaining

By trusting the CA, you trust all the certificates they have signed. Realistically, it’s far too dangerous to place the CA directly on the line like that, so it’s more common for a CA to have signed an intermediate certificate and delegate separate security domains to the intermediate certificates. This intermediate certificate also can then go on to create signed certificates. These final certificates tend to be used by servers to present to your browser for HTTPS connections. 

This consecutive signing forms what is known as a certificate chain. Your browser receives a certificate and goes up the chain until it reaches a CA. If the CA is in your list of trusted CAs, the handshake continues, and you get a secured connection. Suppose the CA doesn’t appear in the trust store. In that case, you get an error in your browser analogous to “invalid certificate authority“. 

To check the validity of a server’s certificate, the certificate’s signature can be decoded using the CA’s public key (which is available in their certificate) and validated. Success in this process proves that they did indeed sign the certificate and not a malicious third party.

This does mean that anyone with the private key would be able to masquerade as the original CA, which is why intermediate certificates tend to be used. Instead of all issued certificates being made invalid by a compromised CA, only part of the chain is. 

To apply for a signed certificate, an end-user creates a private key and a certificate signing request (CSR). The CSR contains the private key’s complementary public key which is embedded in the final signed certificate. Following the same logic, this private key proves that the server is using a certificate they actually own since information is digitally signed and verifiable using the public key from the certificate.

TLS, Kubernetes, and Couchbase Cloud-Native CAO

Kubernetes provides a standard for storing these certificates and private keys with a kubernetes.io/tls spec. By establishing a standard, it means that all systems will generate and consume TLS certificates and keys in a consistent format, allowing for better interoperability. With the latest release of CAO 2.3, users are recommended to use secrets conforming to this spec.

Previously, in CAO 2.1, TLS secrets were provided with the pkey.key and chain.pem fields, this is an artifact of hardcoded paths in Couchbase server:

The downside of this format was that it did not provide very good interoperability with third-party certificate management systems.

Then CAO 2.2 was released with support for cert-manager. Support was achieved by creating a translation layer which renamed the files and also rewrote the key from the PKCS#8 format to the required PKCS#1 format if required, thus extending the TLS support Couchbase Server provides.

The TLS secrets provided by cert-manager are a slight extension of the native kubernetes.io/tls spec.

This extended spec utilised an extra ca.crt field to provide the root CA certificate responsible for signing the respective TLS certificate stored in the tls.crt field.

Providing better conformance is achieved by separating out the ca.crt field into a separate CA secret. This provides direct integration with a larger scope of third-party TLS management systems, allowing the TLS management systems to handle certificate generation and rotation.

Additionally, servers running Couchbase Server 7.1 and CAO 2.3 are able to use the spec.networking.tls.rootCAs for creating a trust pool. Trust pools allow Couchbase Server to validate certificates against multiple CAs. Couchbase Server can use one CA, whilst validating client certificates against an arbitrary amount of separate CAs. 

This allows for client certificates to be piecemeal updated as needed, instead of requiring a rotation of all client certificates simultaneously. Since the secret storing the CA information is of the Kubernetes TLS standard, the CA secrets can be directly imported by CAO without needing any manual intervention.

• Read CAO 2.3 Concepts in Couchbase documentation for further information.