Blog Post

NorthScale and Open Source contributions

Published

Though a number of core developers in the memcached project know, the casual memcached user may not be aware that over the last few months NorthScale released a bevy of memcached client and server contributions. NorthScale is continuing to work with the other project contributors and leaders to get these contributions in shape for inclusion into the various core projects.

NorthScale was, in part, born of contribution to memcached.  Of course, there is always that bit about investors, funding and market opportunity.  Where the rubber meets the road though, the confidence in the developers at NorthScale to meet those needs comes from past contribution to memcached and in a track-record of contributions to Brad Fitzpatrick's original creation which have increased stability, functionality and utility for a number of users over the years.  The simple strategy being implemented by NorthScale?  Continue the contribution by working with other developers and users on getting the things they need to use and make sense of both the clients and servers to memcached.  We also work to bring memcached to new classes of users through packaging/support. Just to enumerate a few of the things posted over the last few months:

  • A new "hot key" analysis feature. I listed this one first because I think it's one of the most useful new things for existing users.  As long time users know, memcached has a stat for nearly everything.  One area for improvement was continuous monitoring of what the hot data in the cache is.  There had been some network packet capture tools for this, but nothing built into the server.  You can see this in the server shipped by NorthScale on the analytics screen, but it's generic in nature.  Sean Lynch did nearly all of this work.
  • Update memcached codebase to support Microsoft Windows. As he's previously blogged about, Patrick Galbraith built upon previous work to bring the memcached up to date on Windows.  Where previous ports could stop shortly past the "work for me" stage, the goal here was to get it to the well tested (as Trond shows in his post about Windows) stage.  All of this was done with the intent of contributing the work back to the core.
  • Huge updates of the memcached storage engine work, targeted at the next memcached release but available now. This includes things like Dustin and Trond's previously open sourced bucket_engine and Trond's sample engines.  In fact, through the memcached mailing list KaiGai Kohei of NEC has even built upon the engine work and provided some new functionality on top of Trond's work.
  • Updates to various clients: spymemcached, libmemcached and Fauna (the ruby memcached gem). I did some work on updating and testing the SASL implementation in spymemcached, Trond and Patrick did some work on the SASL implementation in libmemcached and Sean/Trond added SASL support and some bugfixes to Fauna.

All of this is in various stages of publishing, review and contribution to the projects from which they came.  While it takes more effort, it's just second nature for us to build toward clean contributions that benefit the overall community.  There are some great ideas out there, and we hugely value good interactions with the community.