Couchbase Docker container can easily run on a variety of orchestration platforms:

Docker container using Apache Mesos and Marathon explained how to setup Mesos, Marathon, and run a simple Docker image. The setup was quite involving and a bit
flaky. It required to download and Install Mesos Master and Slave, ZooKeeper, Docker Engine, and Marathon. In some cases, the correct repo need to be added first. These components need to talk to each and so must be configured
accordingly. Even if you get past that setup, how do you monitor the entire infrastructure as one entity?
Meet DC/OS – Datacenter Operating System. Its a distributed operating system using Apache Mesos
as its kernel.
dcos-kernel
DC/OS can installed
in a variety of ways:

  • Local using Vagrant
  • Cloud using Amazon/CloudFormation, Microsoft Azure and Packet
  • On-prem using CentOS or CoreOS

This blog will show how to setup a DC/OS cluster using CloudFormation templates on Amazon and run a Couchbase Docker container.

Launch DC/OS cluster

Launch DC/OS cluster:
dcos-couchbase-1

Take the defaults:
dcos-couchbase-2

Give the template a name, select a previously create KeyPair, change the number of slaves:
dcos-couchbase-3

Take the defaults:
dcos-couchbase-4

Verify the configuration:
dcos-couchbase-5

Click on “I acknowledge that …” and on Create to start the template creation. CloudFormation Stack Status  page comes up:
dcos-couchbase-6

Make sure to choose the appropriate region. After ~10-15 mins, the status changes:
dcos-couchbase-7

Wait for the status to change from CREATE_IN_PROGRESS to CREATE_COMPLETE.

Download and Configure DC/OS CLI

DC/OS CLI can be used to manage your cluster nodes, install DC/OS packages, inspect the cluster state, and administer the DC/OS service subcommands. Install DC/OS CLI on
your local machine.

On your CloudFormation Stack Status  page, select the created stack, Outputs tab, and copy the address of
Mesos master.
dcos-couchbase-8

Configure the DC/OS CLI to use this cluster:

Authenticate:

Enter the URL in the browser, proceed to the unsafe URL:
dcos-couchbase-9

Copy the token to your clipboard:
dcos-couchbase-10

Paste the authentication token in the terminal window:

Mesos and Marathon UI

Mesos UI is available using the address of the Mesos master:
dcos-couchbase

Click on Services to see Marathon service already installed:
dcos-couchbase-12

Click on marathon to see the list of tasks:
dcos-couchbase-13

As expected for a freshly created cluster, no tasks have been assigned yet. Click on Nodes to see the nodes:
dcos-couchbase-14

Install and Configure Marathon Load Balancer

DC/OS slave nodes are not directly exposed on the Internet. An “external” load balancer can be configured to expose the tasks running on the slaves. Marathon-lb, short for Marathon
Load Balancer, is a load balancer available as a Mesos service. It is based on HAProxy that provides proxying and load balancing for TCP and HTTP based applications, with features such as SSL support, HTTP compression, health checking and more.
Marathon-lb subscribes to Marathon’s event bus and updates the HAProxy configuration in real time.

Marathon service UI will show the LB task running:
dcos-couchbase-15

AWS Load Balancer allows port 80 and 443 by default. We’ll run a Couchbase server that will be exposed at port 8091.

In CloudFormation Stack Status  page, copy the value from Values column of PublicSlaveDnsAddress:
dcos-couchbase-16

In AWS Console, select Load Balancers, add a new firewall rule to allow port 8091 on TCP:
dcos-couchbase-17

Run Couchbase Server Docker container on DC/OS

Run Couchbase Server Docker container on DC/OS using the following configuration file:

This configuration file uses arungupta/couchbase image that configures the Couchbase Server using pre-defined Couchbase REST API.
This image is Couchbase Docker Image. cpus and mem attributes define the processing memory needed to run this task. Give the command
to run Couchbase in DC/OS cluster:

Use the previously copied for PublicSlaveDnsAddress and access the Couchbase Web Console at http://:8091. In our case, the URL is: http://couchbase-publicsl-vjzmwpa38k6d-429093455.us-west-1.elb.amazonaws.com:8091/index.html.
This shows up the login page as:
dcos-couchbase-18

Enter the login credentials as Administrator and password:
dcos-couchbase-19

Click on Sign In to see:
 dcos-couchbase

Learn more about Couchbase Web Console.

Marathon UI is updated to show all the running services:
dcos-couchbase-21

Couchbase Docker image log can be seen in Log Viewer:
 dcos-couchbase-22

And the standard output view:
dcos-couchbase-23

Mesos dashboard is updated to show the resources that are consumed:
dcos-couchbase-24-

Finally, the complete stack can be deleted from StackFormation template page:
dcos-couchbase-25

Further reading:

Enjoy!

Now, you’ve seen Couchbase on Docker Swarm, Couchbase on Kubernetes, Couchbase on OpenShift 3.
This blog showed how to run a Couchbase Docker image on Mesos and DC/OS.

Where else would you like Couchbase container to run?

Author

Posted by Arun Gupta, VP, Developer Advocacy, Couchbase

Arun Gupta is the vice president of developer advocacy at Couchbase. He has built and led developer communities for 10+ years at Sun, Oracle, and Red Hat. He has deep expertise in leading cross-functional teams to develop and execute strategy, planning and execution of content, marketing campaigns, and programs. Prior to that he led engineering teams at Sun and is a founding member of the Java EE team. Gupta has authored more than 2,000 blog posts on technology. He has extensive speaking experience in more than 40 countries on myriad topics and is a JavaOne Rock Star for three years in a row. Gupta also founded the Devoxx4Kids chapter in the US and continues to promote technology education among children. An author of several books on technology, an avid runner, a globe trotter, a Java Champion, a JUG leader, NetBeans Dream Team member, and a Docker Captain, he is easily accessible at @arungupta.

3 Comments

  1. Hello Arun Gupta,

    First of all, thanks for all your contribution, it has really helped us working with Couchbase.

    Actually I have a question: we are trying to run an instance of Couchbase on DCOS. Everything is working fine except we are not able to access this instance through the client SDK “cbc n1ql…” or the PHP library (eg. CouchbaseN1qlQuery class). REST API is working fine.

    We’re constantly getting LCB_ECONNREFUSED or LCB_NETWORK_ERROR depending on the action attempted.

    We installed the PHP library using https://developer.couchbase.com/documentation/server/4.6/sdk/php/start-using-sdk.html.
    Here is the port routing we put in the marathon json :


    "docker": {
    "image": "registry.marathon.l4lb.thisdcos.directory/couchbase/server:4.6.1",
    "network": "BRIDGE",
    "portMappings": [
    {
    "containerPort": 8091,
    "hostPort": 0,
    "servicePort": 10114,
    "protocol": "tcp",
    "labels": {
    "VIP_0": "/projects/ccu/couchbase/couchbaseccu:8091"
    }

    },
    {
    "containerPort": 8092,
    "hostPort": 0,
    "servicePort": 10115,
    "protocol": "tcp",
    "labels": {
    "VIP_0": "/projects/ccu/couchbase/couchbaseccu:8092"
    }

    },
    {
    "containerPort": 8093,
    "hostPort": 0,
    "servicePort": 10116,
    "protocol": "tcp",
    "labels": {
    "VIP_0": "/projects/ccu/couchbase/couchbaseccu:8093"
    }
    },
    {
    "containerPort": 8094,
    "hostPort": 0,
    "servicePort": 10117,
    "protocol": "tcp",
    "labels": {
    "VIP_0": "/projects/ccu/couchbase/couchbaseccu:8094"
    }
    },
    {
    "containerPort": 11210,
    "hostPort": 0,
    "servicePort": 10118,
    "protocol": "tcp",
    "labels": {
    "VIP_0": "/projects/ccu/couchbase/couchbaseccu:11210"
    }
    }

    1. Any help would be really appreciated,
      Thanks

      1. cbc n1ql -U couchbase://couchbasecouchbaseccu.marathon.l4lb.thisdcos.directory/bucket_name 'CREATE INDEX profile_given_name...' outputs
        772ms [I3bcc1f16] {7396} [TRACE] (http-io - L:249) POST http://172.17.0.28:8093/query/service. Body=192 bytes
        772ms [I3bcc1f16] {7396} [DEBUG] (lcbio_mgr - L:448) (HE=0xaf2520) Creating new connection because none are available in the pool
        773ms [I3bcc1f16] {7396} [TRACE] (lcbio_mgr - L:359) (HE=0xaf2520) New pool entry: I=0xaf2940
        773ms [I3bcc1f16] {7396} [INFO] (connection - L:474) (SOCK=000000005708fc0d) Starting. Timeout=75000000us
        773ms [I3bcc1f16] {7396} [DEBUG] (connection - L:238) (SOCK=000000005708fc0d) Created new socket with FD=7
        773ms [I3bcc1f16] {7396} [TRACE] (connection - L:339) (SOCK=000000005708fc0d) Scheduling I/O watcher for asynchronous connection completion.
        773ms [I3bcc1f16] {7396} [TRACE] (connection - L:354) (SOCK=000000005708fc0d) connect() failed. errno=111 [Connection refused]
        773ms [I3bcc1f16] {7396} [ERROR] (connection - L:156) (SOCK=000000005708fc0d) Failed to establish connection: LCB_ECONNREFUSED (0x2C), os errno=111
        773ms [I3bcc1f16] {7396} [DEBUG] (lcbio_mgr - L:318) (HE=0xaf2520) Received result for I=0xaf2940,C=(nil); E=0x2c
        773ms [I3bcc1f16] {7396} [ERROR] (http-io - L:226) Connection to failed with Err=0x2c
        773ms [I3bcc1f16] {7396} [WARN] (http-io - L:85) Not retrying. No nodes available
        773ms [I3bcc1f16] {7396} [TRACE] (confmon - L:275) Refreshing current cluster map

Leave a reply