No SQL is not about SQL

No SQL is a Zoo.. Key-Value Stores Wide Column Stores Document Stores Graph Databases

Why not Traditional RDBMs? Offer incredibly useful guarantees and have been battleworn and tested.

Referential Integrity

ACID Transactions

Update Inventory BEGIN TRANSACTION Update Products Set Inventory = Inventory Where ProductID = If (Select Inventory from Products Where ProductId = ) < 0 ROLLBACK TRANSACTION COMMIT TRANSACTION t

And SQL.. SQL is a powerful expressive DSL (Domain Specific Language) that many, many people understand.

So Why No SQL?

Web Scale

Web scale can be done in SQL

How? Vertical Part / Logical Sharding (Instagram) Caching (28 terabytes Facebook, 2008) SQL + No SQL Think about your Architect Want to learn more? Spend time on

But a reasonable question is.. How much time should we be devoting to managing scaling problems versus adding business value to these systems?

So what are we giving up?

A vailability C onsistency P artition tolerant MongoDB MySQL SQL Server Oracle RDBMs HBase (Hadoop) Google BigTable Dynamo Couch Cassandra Voldemort Redis SimpleDB CAP

FriendsWhoCook.com A social network of friends who enjoy cooking great food. -Add my Recipes -Add my friends -Show my friends -Like / Comment on my Friends Recipes -Search recipes of my friends, their friends, and so on by.

Problem 1: Store Recipes

Fairly Simple Object class Recipe { Image Photo List Comments List Ingredients List Likes Category RecipeCategory }

Becomes a complex RDBMess

Object-Relational Impedance Mismatch

No SQL: Document Store Data element is a document Documents grouped into collections Often store in JSON Works great with Domain Driven Design Schema-less

Document Store Examples MongoDB (PC) CouchDB (PA) RavenDB (PA)

DEMO: MongoDB

Demo: CouchDB

Problem 2: Model the Social Graph

Friends in RDBMS For a more sophisticated view of modeling graphs in an RDBMs:

Get my Friends int SELECT FirstDegreeProfile.ID, FirstDegreeProfile.FirstName, FirstDegreeProfile.LastName FROMProfile AS FirstDegreeProfile JOIN Friendship ON FirstDegreeProfile.ID = Friendship.FriendID WHERE Friendship.ProfileID

Friends and their friends int = 1 Select FirstDegreeFriendship.FriendId as MyFriendId, SecondDegreeProfile.ID as SecondDegreeId, SecondDegreeProfile.FirstName as SecondDegreeFirstName, SecondDegreeProfile.LastName as SecondDegreeLastName from Profile as SecondDegreeProfile Join Friendship as SecondDegreeFriendship ON SecondDegreeProfile.ID = SecondDegreeFriendship.FriendID join Friendship as FirstDegreeFriendship ON SecondDegreeFriendship.ProfileID = FirstDegreeFriendship.FriendID Where FirstDegreeFriendship.ProfileId /* Note: A much better solution would use a recursive CTE to compute transitive closure */

Graph Databases Optimized for graphs data Check out Neo4J

Problem 3: Schemaless / Big Data Facebook's Network: Credit Traud & Frost, UNC-Chapel Hill

How do we ask these questions? After we changed the like button icon for half of our users, did we get more or less likes from that sample? Of users who click on our ads, what pages did they spend the most time on? Which hidden patterns might make us competitive that we arent even aware of? Want to get far ahead of the pack? Read The Lean Startup by Eric Ries

Is this Actionable?

How about this?

Wide Column A Bigtable is a sparse, distributed, persistent multidimensional sorted map Source:

MapReduce Map(k,v) [(k 1, v 1 ), (k 2, v 2 ), (k 1, v 3 ), (k 3, v 4 )] Map(k, v) (list of intermediate key / value pairs) Internal Step: Takes list of intermediate key value pairs and converts to a key / list of values. Reduce(k, [v 1, v 2, v 3 …]) (k, n 1 ), (k, n 2 )

One Down Side… We have to have smart people write MapReduce programs and the problems need to be expressible as Map Reduce.. General solutions are BIG money.

Final thought: Big Data is BIG = ?

Things to Read Bigtable: A Distributed Storage System for Structured Data Dynamo: Amazons Highly Available Key-value Store MapReduce: Simplified Data Processing on Large Clusters The Google File System Towards Robust Distributed Systems gTable

Creative Commons Acknowledgments and Thanks! Bobwitlox rosipaw