It seems these days that almost every game studio has been working on networked games where players can interact and co-operate with their friends and other players around the world. Considering my previous experience building such servers and that Couchbase fits the bill as a backing store for a system like this, I thought perhaps this may be an excellent topic to write about! I will be writing this in multiple parts with each part implementing one specific aspect of the game server, additionally, I will be doing the same tutorial using our PHP client library to show that off as well.

Project Layout

To start off, we need to set up a few basic things to let us send and receive HTTP requests as well as to connect to our Couchbase cluster. If you're not sure how to do this yet, please take a look at my previous blog post where I explain with a bit more detail than I will below. We are going to start out our project with a typical Node.js directory structure with a few extra folders to help organize the game server code.

/lib/
/lib/models/
/lib/app.js
/lib/database.js
/package.json

As per the normal Node.js structure we start with our 'lib' folder to hold all of our source files, with a lib/server.js to act as the main file and finally our package.json to describe the projects dependancies and other meta data. We additionally add a database.js which will centrally manage our database connection to prevent us from having to instantiate a new Connection for each request as well as the /lib/models/ folder which we will use to hold our various database model source code.

The Basics

Here is some content for your package.json. We give our project a name, point to its main Javascript file and then define a couple of prerequisite modules we will need later on. Once you have saved this file, executing npm install in your project root directory should install the referenced dependancies.

{
  “main”: “./lib/app”,
  “license” : “Apache2”,
  “name”: “gameapi-couchbase”,
  “dependencies”: {
    “couchbase”: “~1.0.0”,
    “express”: “~3.4.0”,
    “uuid”: “~1.4.1”
  },
  “devDependencies”: {
  },
  “version”: “0.0.1”
}

Our next step is to set up the core of our game server. This is placed into our /lib/app.js. I will go through the sections of this file block by block, and provide an explanation of what it is doing for each one.

First we need to import the modules we are going to need in this file. Right now we only need the express module for HTTP routing and parsing, but later in this tutorial we will be adding more to it.

 
var express = require('express');

Next, lets setup express, we additionally attach express' bodyParser sub-module so that we can parse JSON POST and PUT bodies. This will help later when our game clients need to pass us blocks of JSON data.

 
var app = express();
app.use(express.bodyParser());

For the sole purpose of demonstration, lets add a simple route to our HTTP server to handle requests to our server's root.

 
app.get('/', function(req, res, next) {
  res.send({minions: 'Bow before me for I am ROOT!'});
});

Finally lets get our HTTP server listening on port 3000.

 
app.listen(3000, function () {
  console.log('Listening on port 3000');
});

Here is a rough idea of what your app.js should look like so far:

 
var express = require('express');

var app = express();
app.use(express.bodyParser());

app.get('/', function(req, res, next) {
  res.send({minions: 'Bow before me for I am ROOT!'});
});

app.listen(3000, function () {
  console.log('Listening on port 3000');
});

For the last bit of our project basics, lets set up our database connection. The code is fairly straightforward, we import the couchbase module and subsequently export a new connection to our locally hosted server and the bucket 'gameapi' through a property of the module named mainBucket.

 
var couchbase = require('couchbase');

// Connect to our Couchbase server

module.exports.mainBucket = new couchbase.Connection({bucket:'gameapi'}, function(){});

At this point if you open a terminal in your project root and execute node lib/app.js, you should see the message “Listening on port 3000”. You can also now point your browser to http://localhost:3000 and see our work so far in action.

It is at this point that I suggest you install an application that will allow you to craft specific HTTP requests, I personally love the POSTman extension for Google Chrome. This will be important later when you want to test endpoints that are not simple GET requests!

Account Creation – Account Model

Now that we have our basic server running, lets starting working on the 'game' portion of our game server. We are going to start by implementing the account creation endpoint which will be accessible by doing a POST request to the /users URI. To start this process, we are going to first build a model for our endpoint handler to deal with to abstract some of the details of our database implementation. These models are where the bulk of our interactions with Couchbase Server will happen.

Lets first start by creating a new file in our /lib/models folder called 'accountmodel.js'. Once you've got your accountmodel.js file ready and opened, lets start by importing some of the modules we will need.

var uuid = require('uuid');
var couchbase = require('couchbase');
var db = require('./../database').mainBucket;

As you can see, there are 4 modules that we are going to need right now. We will be using the uuid module to generate UUID's for our database objects. I have seen a lot of people using sequence counters implemented using Couchbase's incr/decr system, but I much prefer the UUID method I will be using here as it prevents the neccesity of doing an additional database operation. Next we import the couchbase module which we will use to access various constants that we will need (errors mainly). And finally we import the database module and grab the connection to our gameapi bucket we created earlier.

Next we define a simple helper function which will help strip away any database-level propertys our model needs which are not important to the rest of the server. Right now the 'type' property is the only property that we will be stripping. This property will be used by the gameapi to identify what kind of object a particular item in our bucket is for when doing map-reduces later on.

function cleanUserObj(obj) {
  delete obj.type;
  return obj;
}

Now we define our AccountModel class.

function AccountModel() {
}

And export the class to other files which import this one. I suggest you keep this statement always at the bottom of your file to make it easier to find when you are trying to identify what a particular file exports.

 
module.exports = AccountModel;
 

Now that our Model boilerplate is done, we can build our create function which will allow us to create user object. I will be breaking this function down into smaller chunks to simplify explaining them.

Lets start with the definition of the function itself.

AccountModel.create = function(user, callback) {
};

Next, lets create an object which will be inserted into our Couchbase bucket. We specify a type for the object, which as mentioned above will be used later. We generate the user a UID which will help us refer to our user throughout. Finally, we copy the users details that were passed into the create function. You may notice that we do not preform any validation on the data being passed to our model, this is because for the most part our request handling code will have a better idea of what to accept or not accept, and our model is just responsible to get the data stored. Last of all, we generate a key to use to refer to this document, we use the document type and the users UID for this purpose.

 
var userDoc = {
  type: 'user',
  uid: uuid.v4(),
  name: user.name,
  username: user.username,
  password: user.password
};
 
var userDocName = 'user-' + userDoc.uid;

In order to allow ourselves to find this user in the future by their username (it's probably not a good idea to get your users to remember their UIDs!), we will create a 'referential document', that is, a document with a key based on the user's username that points back to our user's document (using their UID). This also has the added benefit of preventing multiple users from having the same username.

var refDoc = {
  type: 'username',
  uid: userDoc.uid
};
var refDocName = 'username-' + userDoc.username;
 

Finally, we need to insert these documents into our Couchbase bucket. First, we insert the referential document and handle the keyAlreadyExists error specifically by returning a message advising the user that the username is taken and simply passing along any other errors (we should probably wrap our Couchbase errors at the Model level, but that is not important at this point in the series). The fact that we insert the referential documents first here is important because ++TODO++ Why is it important again? –TODO–. Next we insert the user document itself and finally we invoke the callback that was passed to us. We do first sanitize the returned object using the function we created earlier to make sure that none of our database-level properties are leaked to other layers of our application. You might also note that we are passing a 'cas' value through our callback. This is going to be important later when we need to perform optimistic locking on our Account object.

 
db.add(refDocName, refDoc, function(err) {
  if (err && err.code === couchbase.errors.keyAlreadyExists) {
    return callback('The username specified already exists');
  } else if (err) {
    return callback(err);
  }

  db.add(userDocName, userDoc, function(err, result) {
    if (err) {
      return callback(err);
    }

    callback(null, cleanUserObj(userDoc), result.cas);
  });
});

Here is what your accountmodel.js file should look like so far:

var uuid = require('uuid');
var couchbase = require('couchbase');
var db = require('./../database').mainBucket;

function cleanUserObj(obj) {
  delete obj.type;
  return obj;
}

function AccountModel() {
}

AccountModel.create = function(user, callback) {
  var userDoc = {
    type: 'user',
    uid: uuid.v4(),
    name: user.name,
    username: user.username,
    password: user.password
  };
  var userDocName = 'user-' + userDoc.uid;

  var refDoc = {
    type: 'username',
    uid: userDoc.uid
  };
  var refDocName = 'username-' + userDoc.username;

  db.add(refDocName, refDoc, function(err) {
    if (err && err.code === couchbase.errors.keyAlreadyExists) {
      return callback('The username specified already exists');
    } else if (err) {
      return callback(err);
    }

    db.add(userDocName, userDoc, function(err, result) {
      if (err) {
        return callback(err);
      }

      callback(null, cleanUserObj(userDoc), result.cas);
    });
  });
};

module.exports = AccountModel;

Account Creation – Request Handling

Now that we've completed the create function in our account model, we can now write an express route to handle requests to create accounts and pass these requests through to our function. First we need to define a route.

app.post('/users', function(req, res, next) {
  // Next bits go in here!
});

And… perform some validation to ensure the neccessary data was passed to the endpoint.

if (!req.body.name) {
  return res.send(400, 'Must specify a name');
}
if (!req.body.username) {
  return res.send(400, 'Must specify a username');
}
if (!req.body.password) {
  return res.send(400, 'Must specify a password');
}

Once the data has been *cough* “validated” */cough*, we can generate the SHA1 hash for the user's password (never store a user's passwords in plain-text!) and then execute the create function that we built earlier on in this post. You may also notice that I remove the user's password from the user object before passing it back to the client. This is again for security as we want to limit the transmission of the user's password (in any format) as much as possible.

 
var newUser = req.body;
newUser.password = crypt.sha1(newUser.password);

accountModel.create(req.body, function(err, user) {
  if (err) {
    return next(err);
  }

  delete user.password;
  res.send(user);
});

To summarize, your entire account creation route should look like this:

app.post('/users', function(req, res, next) {
  if (!req.body.name) {
    return res.send(400, 'Must specify a name');
  }
  if (!req.body.username) {
    return res.send(400, 'Must specify a username');
  }
  if (!req.body.password) {
    return res.send(400, 'Must specify a password');
  }

  var newUser = req.body;
  newUser.password = crypt.sha1(newUser.password);

  accountModel.create(newUser, function(err, user) {
    if (err) {
      return next(err);
    }

    delete user.password;
    res.send(user);
  });
});

Finale

Well, we have finally reached the end of part 1 of our tutorial. We have a lot of our basics out of the way, so future parts of the series should be a bit shorter (not promising anything though!). At this point, you should be able to execute a POST request to your /users endpoint and create a new user like so:

 
> POST /users
{
  “name”: “Brett Lawson”,
  “username”: “brett19”,
  “password”: “success!”
}
< 200 OK
{
    “uid”: “b836d211-425c-47de-9faf-5d0adc078edc”,
    “name”: “Brett Lawson”,
    “username”: “brett19”
}

Unfortunately, at this point there isn't much you can do with your new found accounts, except perhaps marvel at their existence in our database. Hopefully you will stick around for Part 2 where I will introduce sessions and authentication of the users that are now able to register.

The full source for this application is available here: https://github.com/brett19/node-gameapi

Enjoy! Brett

Posted by Brett Lawson, Software Engineer, Couchbase

14 Comments

  1. I\’d encourage anyone doing this to look into using hapi instead of express. https://github.com/spumko/hapi

    1. Hey mayhap, might I inquire why you would suggest this? I chose express as it is a well known, stable library and had some features which were lacking in restify and other similar libraries.
      Cheers, Brett

      1. It was created by Eran, worked on Oath1 and was the lead on Oauth2.
        There are issues with express that he has solved. You should probably just watch this video

        1. Thanks for that mayhap. Quite an interesting video 🙂 Unfortunately, my blog series already started, so would be dangerous to consider changing at this point!

          1. Glad you liked it! Maybe next time 🙂

  2. Hi! Very helpful and beginner friendly tutorial.What I noticed: the bucket gameapi does not exist, you\’re using the crypt module before it is discussed and you\’re passing the accountModel.create req.body not newUser. I look forward to part 2 of the series! Thanks

  3. Shouldn\’t you be passing newUser (which has the password encrypted) to accountModel.create() instead of the original unmodified req.body that would contain the password in plaintext?

    1. Also I wonder if it wouldn\’t be better to hash passwords at the model level since it could technically be considered a \”data preparation\” step necessary for proper data storage. In other words, you would *never* want to store a user object with a password that hasn\’t been hashed first or at least encrypted.

      So in accountmodel.js, I would suggest setting the userDoc.password property to require(\’crypto\’).createHash(\’sha1\’).update(user.password).digest(\’base64\’) and then just get rid of the newUser variable altogether in app.js, passing req.body to accountModel.create() as shown in the tutorial.

      You also need to initialize accountmodel.js before calling the create() method on it as follows:

      var AccountModel = require(\’./models/accountmodel.js\’);

      Finally, for this tutorial, I would recommend indicating that the Couchbase connection should be in database.js as reading through that section naturally implied app.js since that was the last file the reader was working out of and wasn\’t signaled to pop into database.js to \”set up their database connection.\”

      1. The reason I handled the encryption of the password in the app was because I was trying to design the models such that what you store is what you can retreive, additionally, if you swapped out my custom models for a real Node.js ODM such as ottoman (https://github.com/couchbasela…, you wouldn\’t have the ability to automatically encrypt the password in the model anyways. In regards to the first issue, you are right, I have update the tutorial.
        Cheers, Brett

  4. Saransh Mohapatra February 6, 2014 at 9:06 pm

    A very trivial question….But why can\’t we store the userDocName : \’user-username\’ since username is supposed to be unique. This would save us extra data writes and also reads. Or is this for some reason that I am missing out?

    1. Hey Saransh,
      In my original implementation, and potentially in this one if I were to expand it. There is more than a single method for authenticating a user to their account. You may support username/password, but also potentially Facebook or Google Play accounts. In this case you need separate lookup keys for each, but to still point to the same generic account type.
      Cheers, Brett

  5. I have a simple question. I\’m new to Couchbase and to all NoSQL databases so maybe it\’s a stupid question.

    I want to check username and email against the database, so that there aren\’t two equal emails or usernames. You already did it for the username, with a reference document. Would you do it the same way for the email? If you have 100k users wouldn\’t that bloat the database?
    What would be the best approach? reference documents? get all users and check their usernames and emails? Map reduce?.

    1. Hey Jose,
      Using reference documents is the easiest way to implement any sort of unique lookup table. You could use map/reduce, however due to the fact that indexing happens asynchronously, its possible you would end up with duplicate users. From a space perspective, using a map/reduce view or using referential documents will likely use similar amounts of space.
      Cheers, Brett

  6. […] Blog of the Week: Game Servers and Couchbase with Node.js – Part 1 […]

Leave a reply