Blog Post

Game Servers and Couchbase with Node.js - Part 3

Brett Lawson of Couchbase Published

In this part of the series, we will be setting up a game data storage system to allow you to store player game state over the course of their enjoyment of your game. To do this, we are going to create some /state and /states endpoints which will represent individual blocks of state data. We will allow multiple named state blocks to allow the game to divide the state data into separately updatable blocks to avoid needing to write many state blocks when only one part has changed.

If you have yet to read Part 1 and Part 2 of this series, I suggest you do, as this part and any future ones build on those!

Quick Aside - Session Renewal

Something that should have been in my previous blog post, which is important, is renewing the users session whenever they access it. Without this, the session is guaranteed to expire after 60 minutes, regardless of whether the player is still playing. This is obviously not our intention, so lets fix that!

First we need to add a new function to our SessionModel, so open up your sessionmodel.js and lets add the following block. It is a fairly straightforward function; it takes a session id and preforms a touch operation against it to reset the expiry time to 3600 again (starting from the time of execution of the touch, not when the key was originally inserted).

SessionModel.touch = function(sid, callback) {
  var sessDocName = 'sess-' + sid;

  db.touch(sessDocName, {expiry: 3600}, function(err, result) {
    callback(err);
  });
};

Now that we have our model function for updating the session, lets find a good place to call. Our authUser method seems like a good fit as it is executed on any endpoint, which requires the user to be authenticated. Lets do that now. Here is our new authUser function with our touch call added.

function authUser(req, res, next) {
  req.uid = null;
  if (req.headers.authorization) {
    var authInfo = req.headers.authorization.split(' ');
    if (authInfo[0] === 'Bearer') {
      var sid = authInfo[1];
      sessionModel.get(sid, function(err, uid) {
        if (err) {
          next('Your session id is invalid');
        } else {
          sessionModel.touch(sid, function(){});
          req.uid = uid;
          next();
        }
      });
    } else {
      next('Must be authorized to access this endpoint');
    }
  } else {
    next('Must be authorized to access this endpoint');
  }
}

As part of this session expiry fix, you may need to pull your couchnode version directly from GitHub due to a bug with our touch implementation that was fixed prior to the posting of this blog, but after our latest cycle release.

Game States - The Model

Now that we've cleared up our little issue from our previous part, lets move on to implementing our game state saving! As I said above, we are going to be allowing the games to store data in multiple state blocks to reduce network traffic. From a storage perspective, we will be storing all of these state blocks in a single Couchbase document, and this document will be lazily created upon the first request to save information for the user. Up until the point that there is something saved, we will emulate an empty state list for the user, as you will see shortly.

To begin, lets set up our standard model file layout in model/statemodel.js. We import our needed modules and set up a model with no methods called StateModel.

var db = require('./../database').mainBucket;
var couchbase = require('couchbase');

function StateModel() {
}

module.exports = StateModel;

Now that we have the basics for our model, lets start implementing some of the methods that will be needed. Lets start with a model to allow us save a new state block. This function will handle both the creation and updating of a state block. This makes the client-side logic much simpler as we do not need to worry about whether the state block already exists at an API level. We will be using a form of optimistic locking where a version number will be stored with each state block. Whenever a state block is updated, you will need to pass the existing version number that is on the server before the server will accept the new data. This is to prevent multiple copies of the game running simultaneously from trampling each other's data. This is also the first place that we will use Couchbase's optimistic locking to ensure that we are not making simultaneous changes to our states object from two endpoint calls.

Lets start with our save function prototype.

StateModel.save = function(uid, name, preVer, data, callback) {
};

Our first real step is to build a name for our state storage document, and then request this document from Couchbase to check if one exists yet.

var stateDocName = 'user-' + uid + '-state';
db.get(stateDocName, function(err, result) {
  // Code below goes in here!
});

Now we check to see if there were any errors requesting an existing states document. If we encounter an error, we verify that it was not a 'not found' error. If the document was not found, we ignore this error and continue, this is due to the lazy-creation nature of our states document.

if (err) {
  if (err.code !== couchbase.errors.keyNotFound) {
    return callback(err);
  }
}

Next we move our existing state document (or a new document if one was not found), into a separate variable for easier access, and so we can handle both existing documents and new document in the same manner.

var stateDoc = {
  type: 'state',
  uid: uid,
  states: {}
};
if (result.value) {
  stateDoc = result.value;
}

Now we do the same thing for our state block itself. You will notice we ensure our stateBlock var references the actual state documents' states array. Another point worth mentioning is that our default state block version is 0. This means that when executing a save for the first time, it is expected that the client will specify the version 0 to clarify that it is aware that this will be a new state block.

var stateBlock = {
  version: 0,
  data: null
};
if (stateDoc.states[name]) {
  stateBlock = stateDoc.states[name];
} else {
  stateDoc.states[name] = stateBlock;
}

Next we need to verify that the version specified by the caller still matches what is stored in our cluster. If this is not the case, another user must have made changes since the last time the client retrieved save data. It is expected that if there is a version mismatch, the client will retrieve the new data, perform any necessary merging and then reattempt their update.

if (stateBlock.version !== preVer) {
  return callback('Your version does not match the server version.');
} else {
  stateBlock.version++;
  stateBlock.data = data;
}

As I mentioned at the beginning of this section, we will be also using the optimistic locking built-in to Couchbase to ensure that our state document writes are preformed in order. Due to the fact that we preform our get previous, then do our version comparison and finally do the write again here, there is a chance another call to our state saving endpoint has altered the object since our original get, but before our set, optimistic locking using cas values prevents this. To learn more about cas values, please check out the Couchbase manual on cas values.

var setOptions = {};
if (result.value) {
  setOptions.cas = result.cas;
}

Last of all for this particular method, we preform our set, any errors that occur are propagated to the caller (this should probably be wrapped at the model level, as mentioned before), and the callback invoked with the state block we've stored.

db.set(stateDocName, stateDoc, setOptions, function(err, result) {
  if (err) {
    return callback(err);
  }

  callback(null, stateBlock);
});

Finally, here is our entire save method. It is quite long, but hopefully relatively understandable!

StateModel.save = function(uid, name, preVer, data, callback) {
  var stateDocName = 'user-' + uid + '-state';
  db.get(stateDocName, function(err, result) {
    if (err) {
      if (err.code !== couchbase.errors.keyNotFound) {
        return callback(err);
      }
    }

    var stateDoc = {
      type: 'state',
      uid: uid,
      states: {}
    };
    if (result.value) {
      stateDoc = result.value;
    }

    var stateBlock = {
      version: 0,
      data: null
    };
    if (stateDoc.states[name]) {
      stateBlock = stateDoc.states[name];
    } else {
      stateDoc.states[name] = stateBlock;
    }

    if (stateBlock.version !== preVer) {
      return callback('Your version does not match the server version.');
    } else {
      stateBlock.version++;
      stateBlock.data = data;
    }

    var setOptions = {};
    if (result.value) {
      setOptions.cas = result.cas;
    }

    db.set(stateDocName, stateDoc, setOptions, function(err, result) {
      if (err) {
        return callback(err);
      }

      callback(null, stateBlock);
    });
  });
};

The next method we will include is a findByUserId. This method will allow us to build an endpoint that returns all state blocks for any particular user. This is mainly a client-side optimization to allow fetching of all state blocks at once rather than preforming multiple requests. The function is extremely straightforward. Using the same document name as our save function, we attempt to load the state document from our cluster, if it exists, we return the list of states within this block, if the document is missing, we return an empty list to the user. Any other errors are forwarded to the caller.

StateModel.findByUserId = function(uid, callback) {
  var stateDocName = 'user-' + uid + '-state';
  db.get(stateDocName, function(err, result) {
    if (err) {
      if (err.code === couchbase.errors.keyNotFound) {
        return callback(null, {});
      } else {
        return callback(err);
      }
    }
    var stateDoc = result.value;

    callback(null, stateDoc.states);
  });
};

The last model function we need to build is our method for accessing one single state block for a user. This function is nearly identical to our findByUserId function, except we additionally drill down to a specific state block by name rather than returning the entire list.

StateModel.get = function(uid, name, callback) {
  var stateDocName = 'user-' + uid + '-state';

  db.get(stateDocName, function(err, result) {
    if (err) {
      return callback(err);
    }
    var stateDoc = result.value;

    if (!stateDoc.states[name]) {
      return callback('No state block with this name exists.');
    }

    callback(null, stateDoc.states[name]);
  });
};

Game States - Request Handling

Now that we have our model all wrapped up and ready to use, lets start building the request handlers for our 3 endpoints! We are going to be building an endpoint for requesting all of the states for one user, an endpoint for requesting a particular state block and finally an endpoint for updating a specific state block.

Before we can build our request handlers, we first need to add a reference to the statemodel.js file that we created earlier!

var stateModel = require('./models/statemodel');

Now, lets start with our save endpoint. As with the previous parts, our request handlers are extremely simple and simply forward the pertinent parts of our request into the model. We expect the state block name as part of the URI, the version number as part of our query and finally the actual state block data in the body of the request.

app.put('/state/:name', authUser, function(req, res, next) {
  stateModel.save(req.uid, req.params.name, parseInt(req.query.preVer, 10),
      req.body, function(err, state) {
    if (err) {
      return next(err);
    }

    res.send(state);
  });
});

Next we need the ability to retrieve a state block that we have previously stored.

app.get('/state/:name', authUser, function(req, res, next) {
  stateModel.get(req.uid, req.params.name, function(err, state) {
    if (err) {
      return next(err);
    }

    res.send(state);
  });
});

Last but not least, the endpoint for requesting all of the state blocks stored for a particular user. As I said before, this is primarily for optimizing the game loading sequence where we generally need to retrieve all of the state blocks.

app.get('/states', authUser, function(req, res, next) {
  stateModel.findByUserId(req.uid, function(err, states) {
    if (err) {
      return next(err);
    }

    res.send(states);
  });
});

Finito!

Now that we have built our model, and implemented the necessary request handlers, you should now be able to start your app as in previous parts and perform some requests against our game server to see our hard work paying off!

> POST /state/test?preVer=0
Header(Authorization): Bearer 0e9dd36c-5e2c-4f0e-9c2c-bffeea72d4f7
{
  "name": "We Rock!",
  "level": "13"
}
< 200 OK
{
    "version": 1,
    "data": {
        "name": "We Rock!",
        "level": "13"
    }
}


> GET /state/test
Header(Authorization): Bearer 0e9dd36c-5e2c-4f0e-9c2c-bffeea72d4f7
< 200 OK
{
    "version": 1,
    "data": {
        "name": "We Rock!",
        "level": "13"
    }
}


> GET /states
Header(Authorization): Bearer 0e9dd36c-5e2c-4f0e-9c2c-bffeea72d4f7
< 200 OK
{
    "test": {
        "version": 1,
        "data": {
            "name": "We Rock!",
            "level": "13"
        }
    }
}

Success!

The full source for this application is available here: https://github.com/brett19/node-gameapi

Enjoy! Brett