Blog Post

Couchbase Blueprints in Apache Brooklyn

Shane Johnson of Couchbase Published

This is a guest post by our friends at Cloudsoft (link) on the Apache Brooklyn project (link). We're excited to be collaborating with Cloudsoft. The cloud is the new operating system, and projects like Apache Brooklyn enable the enterprise to be more efficient. What if you could deploy a dozen instances of Couchbase Server to Amazon Web Services with the click of a button?


Couchbase blueprints are now available in Apache Brooklyn!

@ZaidM (GitHub) and @Nakomis (GitHub) have been busy on Github working with Couchbase to add support for the extremely scalable, high-performance JSON document store. With their latest commit (link), provisioning a cluster is as easy as writing:

name: My Couchbase Cluster
services:
- type: brooklyn.entity.nosql.couchbase.CouchbaseCluster
  initialSize: 3
 
location:
  jclouds:google-compute-engine:
    identity: your-gce-acct
    credential: your-gce-token

Simply follow the instructions (link), unpack, and run "bin/brooklyn launch", then go to the web console and paste the YAML above (with your credentials).

As soon as you deploy, the topology is shown and you can track provisioning and command execution. It all runs in parallel, due to how Brooklyn manages processing, apart from where there is an explicit dependency — such as needing to know a subset of Couchbase nodes in order for the webapp to connect. And it will run in many locations, from Amazon to OpenStack, by way of Docker or non-cloud pre-existing machines.

After a few minutes, it's completed, and metrics from Couchbase are monitored to ensure it's running well and drive your custom management policies such as DR and autoscaling as we'll show below. The sensors tabs also show the URLs for management and endpoints for clients to connect to.

console-sensors.png

Of course in the real world you need more. That's where the power of the Brooklyn project shines and your Couchbase automation sparkles. Let's illustrate a more interesting YAML which shows how to:

  • customizing Couchbase credentials
  • specify the hardware spec (portable across many clouds)
  • attaching a scaling policy
  • deploying Couchbase alongside some front-end applications

This last bullet point is especially powerful: you can use the same open-source tool chain to deploy and manage the applications around Couchbase as you use for Couchbase itself. This enables integration testing at scale, fully automated, and version-controlled blueprints for entire stacks.

Let's use as an example a Couchbase load-generator webapp developed by @Neykov (link).

We'll define a policy which seeks to keep operations-per-node in the range of 500-1000. This is nothing for Couchbase on such beefy hardware, of course, but it's a range we can comfortably demonstrate! Once we've got Couchbase deployed and our web app deployed, you can create the bucket in Couchbase through it's GUI (using the credentials in the YAML below), and then start the two load generators. If you need more load generators, use the "effectors" tab to "resize" the load-gen cluster, and as you start to drive 3000+ ops/sec through Couchbase, you'll observe it scaling up as per the policy.

Here's the bigger, real-world YAML code in all its glory, here running in Softlayer:

name: My Couchbase with Elastic Load Generator

services:

- type: brooklyn.entity.nosql.couchbase.CouchbaseCluster
  id: cb-cluster
  adminUsername: Administrator
  adminPassword: Password
  initialSize: 3
  brooklyn.config:
    provisioning.properties:
      minRam: 16384
      minCores: 4
  brooklyn.policies:
  - type: brooklyn.policy.autoscaling.AutoScalerPolicy
    brooklyn.config:
      metric: $brooklyn:sensor("brooklyn.entity.nosql.couchbase.CouchbaseCluster",
        "couchbase.stats.cluster.per.node.ops")
      metricLowerBound: 500
      metricUpperBound: 1000
      minPoolSize: 3
      maxPoolSize: 8

- type: brooklyn.entity.webapp.ControlledDynamicWebAppCluster
  name: Web Couchbase Load Gen Cluster
  war: https://github.com/neykov/web-load-gen/raw/master/load-gen.war
  brooklyn.config:
    provisioning.properties:
      minCores: 4
    java.sysprops:
      brooklyn.example.couchbase.nodes: $brooklyn:formatString("'%s'",
        component("cb-cluster").attributeWhenReady("couchbase.cluster.node.addresses"))
  initialSize: 2

  location:
    jclouds:softlayer:
      identity: your-sl-acct
      credential: your-sl-api-access-key

About Cloudsoft

Cloudsoft is the leading open source application management company. Cloudsoft AMP is the commercially supported version of Apache Brooklyn, a framework for modeling, monitoring and managing applications through autonomic blueprints currently undergoing incubation at the Apache Software Foundation. Leverages Apache jclouds (link). AMP delivers multi-cloud policy-based application governance and compliance. AMP has been adopted by enterprises, ISVs and systems integrators such as IBM Cloud Services.