Blog Post

CouchTalk Calling

J. Chris Anderson of Couchbase Published

tldr;

The storage part of CouchTalk is easy, just jpeg and wav resources keyed by id. What's interesting is the application model that assumes sequential keys, allowing the front end to index into the data without issuing any queries. To detect the last key we just watch browser events on the img tag and stop on 404.

Talk What?

Everyone knows push-to-talk messaging is hot, and the audio/video features in HTML5 bring it within our grasp from JavaScript. As a web dev, I’m always interested in pushing the edge of what’s practical with today’s browsers. If you haven’t played with CouchTalk yet, go do it now (requires Google Chrome).

Design Considerations

The main aspect of Couchbase Server I wanted to highlight with the CouchTalk app is speed. Push-to-talk is especially performance sensitive — add a few tenths of a second latency and an app can go from fun to sluggish. Couchbase Server’s managed memory architecture means that the performance you see today is primarily limited by the end-to-end Internet latency, with response times at the application server mostly based on the data sizes. Small thumbnails are served in a few milliseconds, where large images can take longer as they are limited by the connection speeds.

Implementation (Architecture and Data Model)

The application server is extremely simple, just an HTTP server for reading and writing images and audio clips, and a web endpoint to hand out new message ids to clients.

The HTTP server requests can be load balanced across the node.js servers in whatever way, so scaling that aspect of the application is a matter of following Couchbase Server sizing practices, and adding more node.js servers when needed.

PubNub is used to notify room members of new changes. The PubNub messages do not contain actual images or audio clips. Instead the messages send document IDs, which the client can use to request the media.

The data model is driven by an INCR key pattern, so for each message in the room, the client declares their intent to save a message, and the server gives them a key to save it under. They keys are formatted `room-sequence` where the sequence is an incrementing integer. Clients already know which room they are in, so once they know a current sequence, they can backfill recent messages by generating their keys.

When a client asks for the next message ID it just increments the key and sends the result back to the client. Increment operations are atomic so you can use them to construct a keyspace without any collisions.

    db.incr(“ct-"+room, {initial: 0}, function(err, result){
      cb(err, ["snap",room,result.value].join('-'))
    })

This key pattern is a great way to manage related data in Couchbase Server. Another example would be: if you know the number of comments on a blog post you can load all the comments. Or in the case of CouchTalk: there’s no need to track how many snapshots are part of a clip, as long as you know how long the clip is and how frequently new snapshots are taken.

When images or audio are saved to the server, we use the “add” command instead of “set” to ensure that images and audio are immutable, once they’ve been set they won't be modified.

    db.add(req.params.snapshot_id, new Buffer(data, "base64"), opts, function(err, result) {})

Note that we are directly saving a binary buffer into the database. Thanks Couchbase!