Couchbase Blog

August 10, 2010

Memcached, go-derper, Black Hat and an Amazon Web Services (AWS) Security Bulletin

If you are a user of memcached and have deployed instances on Amazon EC2, you may have received a message from Amazon over the weekend (we received one on 8/7/2010) indicating you may have a “Possible Insecure Memcached Configuration.” Here’s the body of the message we received:

We've sent you this email to let you know that we have observed that you may be running memcached in an insecure configuration. Specifically, we have noticed that you have at least one security group that allows the whole internet to have access to the port most commonly used by memcached (11211).

There has been a lot of recent attention by the security community about the lack of access controls on memcached and recently some exploits have been published. This has highlighted the importance of running with strict access controls. While we are not aware of any unauthorized access to your Amazon EC2 instances, we do believe you should have your technical team look at this immediately.

We suggest that you audit your security group settings and restrict access to only the instances and IP addresses that need access. Most users only authorize other Amazon EC2 instances to access their memcached server. If you need to access your memcached server from outside of Amazon EC2, you can also authorize just trusted addresses to access your security group.

If you need additional assistance, you can reach our Premium Support team by sending email to aws-security-support@amazon.com.

Regards,
The Amazon Web Services Team

Great email and service from the AWS team, and the suggested fix is spot on.

This posting is meant to provide some background on the issue and the alluded to “recent attention” the issue has received. The issue is relevant to all users of memcached, not just those deploying on Amazon EC2.

The vulnerability
The genesis of this bulletin was almost certainly the result of the development of go-derper by the team at sensepost, highlighted at the blackhat USA 2010 conference on July 30, 2010.

The highlighted vulnerability can be summarized as: if you deploy memcached on a server, leave the TCP port on which memcached is configured to listen (11211, by default) exposed to the Internet, leave the memcached ASCII protocol enabled, AND you are not using SASL authentication with the memcached binary protocol, then there is a trivial way for Bad Guys to retrieve and replace most of the contents of your cache. go-derper.rb is a simple Ruby application, built by sensepost, that can be used to exploit the vulnerability.

Eliminating the vulnerability
Let’s examine the vulnerability, clause-by-clause, and highlight what can be done to eliminate it, starting at the top:

Read more »

August 7, 2010

Memcached security

Memcached security is a hot topic since the sensepost guys released go-derper at blackhat.

The presentation was pretty good and informative, but it seems like the hype around it has left a bunch of people confused. Although much of this was covered in the presentation, it needs to be restated as much as possible.

Read more »

August 2, 2010

Membase Server – We’re Making Great Progress

It seems like just yesterday we posted the bits for Beta 1 for Membase Server, but in fact it was over a month ago and since then we’ve demo’d Membase at a number of events and have had literally hundreds of conversations with users, customers, partners and anyone else interested in NoSQL solutions. It’s been a whirlwind (in a good way!) of activity and I wanted to personally thank everyone who’s been involved and provided feedback. Being the beta program manager, I wanted to touch specifically on the last month as it relates to the beta program at large:

Read more »

July 27, 2010

membase at OSCON 2010

Last week was very busy.  We at NorthScale had the release of beta 2 of membase followed by membase's presence at a second conference.  Though we'd already launched the project, OSCON was a great platform to get into further detail about membase itself, the project behind it, what's in the roadmap and how other folks can get involved.

Read more »

July 21, 2010

Another membase milestone reached – beta 2 ships!

Another membase milestone was reached today – beta 2 was released and is available for download! Several cool features have been added, including support for datasets whose size exceeds the size of aggregate cluster main memory (i.e. supporting disk > RAM); very sexy, and useful, real-time and historical stat displays; and support for deploying moxi, the membase proxy, on a client-side machine. Looking back over the last three weeks, community reaction to membase has exceeded our collective expectations. We knew we were addressing an unmet need, but it is always a good feeling to hear it confirmed. We’ve had hundreds of downloads of membase beta 1 over the last three weeks and the feedback has been overwhelmingly positive: - “Membase appears to be the reliable, sharding and persistent memcached-alike we’ve all
    been waiting for…” - “Membase is fast! like memcached fast. very low latency under load and good throughput…” - “Oh this is so hot, so very, very, hot…” But while it is nice to hear the good stuff, I tend to prefer hearing about the things people don’t like or the things users having trouble with.

Read more »

July 14, 2010

Hello from Membase-land!

Greetings again! This will be a quick one...just wanted to let you know that Membase is coming along swimmingly and we've been getting some great feedback on the beta from users around the world.

The plan is to release a Beta 2 drop in the next few days which will add support for having your disk storage be greater than your available RAM. It will also introduce a standalone, client-side proxy (better for performance).

Read more »

July 12, 2010

moxi and vbuckets

Lots of great enhancements have gone into membase and memcached recently, and I'm especially excited with the new vbucket capability -- see: http://dustin.github.com/2010/06/29/memcached-vbuckets.html.  Say hello to the ability to explicitly migrate and replicate keys/values between servers, without downtime, while still keeping to memcached's uber performance.

And, moxi (the memcached/membase proxy) is keeping pace with the new vbucket improvements.  You can find the latest moxi open-source development work happening on the 'vbucket' branch here: http://github.com/northscale/moxi/tree/vbucket.

Read more »

July 10, 2010

What exactly is membase?

It has been just over couple weeks since the launch of membase.org, along with NorthScale's partners at Zynga and NHN.  In that time, we've been steadily increasing the postings on the wiki and responding to questions on the mailing list, the XMPP Chat and the IRC channel.  When questions come up, they tend to be about about how membase compares to other Open Source projects, what kind of client one would use or what the pieces are when deployed.

Read more »

July 6, 2010

Membase Beta Refreshed…Ubuntu Support and More!

Greetings all! As the Membase beta program manager, I want to let everyone know that we’ve put out an update to the Membase beta. The most notable addition is official support for Ubuntu, version 9.04 and higher. There are also a number of bug fixes, and release notes are available. Download the refresh here. On a related note, we had the first of our weekly beta webinar calls last week. These will be every Thursday until the end of the beta, and all are welcome to join. I’ll be performing demos, talking about bugs and “gotchas” and answering any and all questions. You will automatically be invited to join the call when you download the beta.

Read more »

June 28, 2010

Scaling memcached with vbuckets

For years, people have used memcached to scale large sites. Originally, there was a simple modulo selection hash algorithm that was used. It still is used quite a bit actually and it’s quite easy to understand (although, it’s shown regularly that some people don’t truly understand it when applied to their full system). The algorithm is basically this:

Read more »