BigData - NoSQL Hadoop - Couchbase Tugdual “Tug” Grall Technical Evangelist email: tug@couchbase.com twitter: @tgrall

About me Web Tugdual “Tug” Grall @tgrall Couchbase http://blog.grallandco.com tgrall NantesJUG co-founder Pet Project : http://www.resultri.com Tugdual “Tug” Grall Couchbase Technical Evangelist eXo CTO Oracle Developer/Product Manager Mainly Java/SOA Developer in consulting firms

$30B Database Market Being Disrupted 95% <50%? Relational Technology Relational Technology Other Relational Technology Relational Technology NoSQL Technology The database industry is about $30B today and is dominated by companies like Oracle, IBM, and Microsoft Relational technology has dominated the industry for the last 40 years and is the technology underpinning for 95% of the industry today. We believe the database industry is being disrupted. In 10 – 15 years we believe relational technology will make up a much smaller percentage of the industry. It’s too early to tell whether it will be 50%, 40%, or 30% percent but it seems clear to me it will be much small than 95% We believe most of the future operational database growth will be NoSQL 2012 2027 All new database growth will be NoSQL

Operational vs. Analytic Databases Get insights from data Real-time, Interactive Databases Fast access to data NoSQL There are two types of databases. Each is focused on a very different problem. Analytic databases were referred to in the past as OLAP databases. They are focused on looking through every record in a huge database to answer a question or gain an insight about the data contained in it. These analyses are batch processes that access every piece of data in the database, are very “read” heavy, and produce results in seconds, minutes, or sometimes days. For analytic databases, “real time” means an analysis takes a few seconds to run. Real-time interactive databases are often referred to as operational databases. They store a lot of data but usually much less than an analytic database. They must provide access to individual records in a database in milliseconds so that users of an application get good response time. Since the requirements of each database is very different, the architectures and capabilities of each are very different as well. When I refer to NoSQL in my presentation, I am referring to real-time, interactive databases. This is the type of NoSQL database Couchbase provides. Couchbase MongoDB Cassandra Hbase Cloudera Hortonworks Mapr

What Is Biggest Data Management Problem Driving Use of NoSQL in Coming Year? 49% 35% 29% 16% 12% 11% Lack of flexibility/ rigid schemas Inability to scale out data Performance challenges Cost All of these Other Source: Couchbase Survey, December 2011, n = 1351.

Hadoop & NoSQL

What is Sqoop? Sqoop is a tool designed to transfer data between Hadoop and relational databases. You can use Sqoop to import data from a relational database management system (RDBMS) such as MySQL or Oracle into the Hadoop Distributed File System (HDFS), transform the data in Hadoop MapReduce, and then export the data back into an RDBMS. sqoop.apache.org

What is Sqoop? Traditional ETL T Data Application Data

What is Sqoop? A different paradigm Application Data Data

What is Sqoop? A very scalable different paradigm Data Application

What is Sqoop? Where did the Transform go? Application Data T T T T T

Sqoop Details Sqoop Default connection is via JDBCLots of custom connectorsCouchbase, VoltDB, VerticaTeradata, NetezzaOracle, MySQL, Postgres

Ad and offer targeting 40 milliseconds to respond with the decision. profiles, real time campaign statistics 3 2 1 profiles, campaigns events

Moving Parts

Content and Recommendation Targeting

Content Driven Site: Moving Parts

Couchbase

Couchbase Server Core Principles Easy Scalability Consistent High Performance Grow cluster without application changes, without downtime with a single click Consistent sub-millisecond read and write response times with consistent high throughput Always On 24x365 Flexible Data Model No downtime for software upgrades, hardware maintenance, etc. JSON document model with no fixed schema.

Couchbase Handles Real World Scale

Q&A