I attended the Strata Conference last week and although I did not catch the talks, the buzz was definitely in the air on the show floor. "Machine Learning," "I am a Data Scientist" and "we are showing off a Hadoop connector" seemed like the most common pick up lines. Our Helicopter giveaways was amongst the most popular. I Wish I had one of them, but, as a Couchbase employee I was not eligible.
Our own James Phillips spoke at a well attended session on Connecting Mobile devices to the cloud. We demonstrated a Hadoop connector as well on the show floor. The idea was that Couchbase would be used as a System of Record for making decisions that often need to be made in sub-milli-seconds. However, these decisions are made based on summarized information available from processing copious amount of data available regarding personal preferences, location data, event data (did the showing of the advertisement lead to a view, a click or a buy) and so on.
In the interactions I had with attendees, being able to use a high performant data layer for real-time decisions coupled with a decision layer for helping make these real-time decisions seemed to resonate well. From a technical perspective, we used sqoop to import the data as Key, Value pairs into Hadoop at which point a variety of tools supported by the Hadoop ecosystem become available. We used pig for analyzing the data and running ad-hoc queries such as how many of the actual ads. were clicked, what were the most popular items and so on.
Based on the quality of interaction, there is no doubt that Big Data is prime time. I look forward to the next installment of the Strata conference in Oct. 2012 which is being billed as a merger of StrataConf and Hadoop World.