Today we are announcing the General Availability of Couchbase Autonomous Operator for Kubernetes (CAO) Version 2.2.
This release provides an automated cloud-native database experience with exciting new features and improvements, such as:
- Auto-scaling Couchbase services
- Online storage scaling (scale up your storage without having to restart pods)
- Centralized log forwarding and automated audit logging with Fluent Bit
- Couchbase server-group customization
- Online backup volume resizing
- Custom upgrades
- Automatic resource allocation
- A new tool:
cbsbctl(a tool to effectively reduce the number of steps in the Couchbase Service Broker deployments)
- Helm improvements
- Prometheus improvements
- Backup/Restore improvements
- TLS Settings
- CERT Manager integration
- XDCR admin password rotation
Let’s take a closer look at what’s inside CAO 2.2.
Auto-Scaling Couchbase Services
In the previous release (CAO 2.1) release, we announced the auto-scaling of stateless Couchbase services and promised to soon announce the auto-scaling of stateful Couchbase services. Today, we are announcing support for auto-scaling of all Couchbase services, in CAO 2.2. It is critical to understand what metrics to use to set the thresholds to maintain the consistent performance and productivity of your applications, so we’ve included best practice recommendations with some key metrics for Couchbase services.
Couchbase’s auto-scaling feature monitors your cluster and automatically adjusts capacity to maintain steady and predictable performance, based on pre-defined thresholds for all Couchbase services. As a result, organizations provide a consistent experience to their users with no unexpected costs during peak times (which might occur with unchecked scaling or in the absence of auto-scaling).
This auto-scaling feature of CAO is unique across the database industry. There is no other database software that provides auto-scaling per service which means with a predefined auto-scale policy, you can scale up or down the number of pods. Each pod may run a single Couchbase service or multiple Couchbase services as per your server configuration, providing true on-demand auto-scaling of Couchbase services, and you only have to pay for what’s required at a point in time.
You can set the auto-scaling policy with any Couchbase metric that fits your environment-specific needs. While we don’t limit your auto-scaling capabilities attached to any specific metric, we came up with some recommendations for Couchbase’s Data, Index, and Query services to assure uninterrupted service and consistent performance in all situations. The default
downScaling operation can be stopped by explicitly disabling the
downScale in the HPA config. (We highly recommend not to set the auto-
downScale policy for critical applications in production environments.)
To quickly try it out for yourself, follow these tutorials:
Online Expansion of Persistent Volumes
When you scale services, you also need to scale your storage. But sometimes you only need to scale storage without scaling the database services or changing the cluster size.
Online expansion of persistent volumes in CAO 2.2 allows you to infinitely scale up your storage with zero downtime meaning you don’t have to restart the pods with Kubernetes’ new feature of dynamic PVC resizing. This is more cost-effective for you and your organization. You can keep PVC to the minimum size necessary to operate at the start and expand in the future as per your requirements.
In CAO 2.2, the
enableOnlineVolumeExpansion flag is added. This defaults to false and must be explicitly set to true as follows within the
CouchbaseCluster spec to enable the Online Storage Scaling feature:
In this feature, CAO only supports scaling up the storage, it does not support scaling down the storage because of Kubernetes limitations.
To use this capability, you must configure the underlying StorageClass associated with backup volume to allow volume expansion.
Log Forwarding and Audit Log Management
CAO 2.2 supports log forwarding through the optional deployment of a third-party log processor.
The log processor runs in a sidecar container on each Couchbase pod, which then reads the log files and forwards them to standard console output. For this purpose, Couchbase supplies a default log processor image based on Fluent Bit.
Read more here about Log Forwarding and Audit Log Management. To quickly try it out, follow the tutorial here.
Improved XDCR Connection Support
CAO 2.2 introduces the ability to modify remote cluster identification and authentication settings. This provides the ability to rotate passwords and certificates on the remote cluster, or even replace the remote cluster entirely.
Improved Server Group Support
In this release, we have lifted Server Group immutability restrictions.
Couchbase Server groups and CAO pod scheduling settings can be modified while a cluster is running. This release also allows you to modify availability zones while a cluster is running. All server group migration operations use a shortest-path algorithm in order to minimize disruption.
Improved Security Defaults
CAO 2.2 makes TLS 1.2 the minimum required version by default, and will automatically update the TLS minimum version unless it is explicitly specified with the
couchbaseclusters.spec.networking.tls.tlsMinimumVersion parameter. It also allows you to set
couchbaseclusters.spec.networking.tls.tlsMinimumVersion to TLS 1.0 after upgrading the CRDs and before restarting the Operator.
TLS Certificate and Key Handling
CAO 2.2 provides secret shadowing to support the widely used format for TLS resources such as
tls.crt to provide integration with third-party providers and to maintain backward compatibility with Couchbase Server’s required format for the TLS resources to be called as
chain.pem. As the secret is shadowed, the Operator can reformat private keys, and therefore now supports PKCS #8 formatted private keys.
Refer to the tutorial Using a Certificate Manager for an example.
For existing clusters using the legacy
couchbaseclusters.spec.networking.tls.static source, the Operator works as before: directly mounting and consuming the TLS secret without a shadow secret.
Automatic Resource Allocation
Kubernetes allows pods to reserve and limit compute resources. Resource reservation provides Kubernetes with the ability to fairly schedule pods so that they don’t compete for CPU and memory.
CAO 2.2 now provides the ability to automatically manage pod CPU and memory resource requests with the
couchbaseclusters.spec.autoResourceAllocation field. When in use, pods will have their resource requests automatically populated depending on the services enabled on that pod, and the individual Couchbase service quotas.
Updating quotas will cause the cluster to upgrade as it requests to update pod resources. We have introduced a new dummy memory quota for the query service to allow the management of memory resource requests, even though Couchbase itself does not provide the ability to constrain this service.
Improved Upgrade Behavior
In CAO 2.1, we introduced two types of upgrades: a one-pod-at-a-time rolling upgrade, and a one-shot upgrade.
CAO 2.2 allows rolling upgrades where you can upgrade either a fixed number of pods or a percentage of the cluster size. You can set both values. It will consider both values and will select the one which will result in the fewest number of pods to upgrade at a time.
Rolling upgrades can be configured with the
Online Backup Volume Resizing
CAO 2.2 supports the online resizing of backup volumes. Backup volumes can be resized in one of two ways:
- Backup volumes can be manually resized by directly editing the
- Backup volumes can be automatically resized by CAO. In this mode of operation, the
couchbasebackups.spec.autoScalingfield controls the behavior.
This is cost-effective for you and your organization. You can keep backups to the minimum size necessary to operate and expand in the future as per your requirements.
To use these resizing capabilities, you must configure the underlying
StorageClass associated with backup volume to allow volume expansion.
Improved Backup and Restore
To mitigate performance issues, in CAO 2.2,
couchbasebackuprestores.spec.threads can now be specified to configure the number of concurrent
cbbackupmgr clients to use when backing up or restoring data.
It also introduces filters that allow you to control exactly what you want to restore from a backup. Now you can choose to include or exclude specific buckets, restore data for a particular service, and even restore to a different bucket name when restoring documents. Refer to Additional Restore Options for more information. This release requires you to upgrade your backup image to at least
Couchbase Service Broker (CSB) improvements
In the last release (CAO 2.1), we introduced Couchbase Service Broker (CSB).It’s a cloud-native integration of CAO with the Open Service Broker API. In this release of CAO 2.2, we provide a tool called
cbsbctl to simplify the CSB deployment process and save time with minimum installation steps. We have also introduced the new service class and service plans specific to the CAO version.
Customizable Prometheus Metrics
This version of the Couchbase Prometheus Exporter allows for certain customizations to exported metrics. Currently, it supports the following customizations:
- Change the namespace, subsystem, name, and help text for each metric.
- Enable and disable exporting metrics to Prometheus.
Improvements for Couchbase Helm Charts
With the CAO 2.2 release, Couchbase Helm Charts support creating and updating all resource types. This release has also improved the Helm Chart documentation.