CS422 Principles of Database Systems Introduction to NoSQL Chengyu Sun California State University, Los Angeles

The Need for NoSQL Big Data Semi-structured Data Large-scale Parallel Processing

Is CSNS Big? Database 112 tables 1.6 million records 260 MB (including indexes) Files 327, GB Data collected on 11/29/2015

Some Data Are Definitely Big Google processed 24 PB data per day in 2009 Facebook had 1.5 PB photos and 60 billion images in 2009 As of 7/1/2015, the Internet Archive Wayback Machine contains 23 PB data and grows at a rate of TB per week

How Big Is Big? Currently any data set over a few TB is considered big data Data size large enough to span several storage units Traditional RDBMS starts to show signs of stress

Semi-Structured Data Data that has some structure but does not conform strictly to a schema Data may be irregular or incomplete Structure may change rapidly or unpredictably

Semi-Structured Data Example: HTML Pages Many HTML pages have a structure: header, footer, menu, title, content … But many don’t And those that do implement the structure in all kinds of different ways HTML5 introduces many new tags New data, e.g. Same data now under different tags, e.g. vs.

The “Web Scale” Google (2012) 3.3 billion search per day Twitter (2013) 500 million tweets per day Facebook (4/2015) 936 million daily active users

Scalability The ability of a system to increase throughput with addition of resources to address load increases Vertical Scaling – faster CPUs, more memory, bigger hard drives … Horizontal Scaling – add more nodes to server clusters

Why Not RDBMS? Its strengths are also its weakness StrengthWeakness Schema Clearly defines data and relationship; ensures data quality and integrity. Not suitable for semi- structured data. ACID Guarantees the correctness of the operations and the durability of the data. Makes it very difficult to scale SQL One language for all data and all RDBMS. Impedes rapid application development (RAD) due to the mismatch between SQL and the application languages.

NoSQL No SQL, Not Relational, Not Only SQL … A term that describes a class of data storage and manipulation technologies and products that do not follow the RDBMS principles and focus on large datasets, performance, scalability, and agility.

Types of NoSQL Databases Key-Value Stores Document Databases Column Family Stores Graph Databases

Key-Value Stores Simple, fast, scalable ProductUsed By RedisTwitter, GitHub, Snapchat, Craigslist DynamoEA, New York Times, HTC CassandraFacebook, Twitter, Reddit VoldemortLinkedin

Document Databases A document in a document database consists of a loosely structured set of key-value pairs. ProductUsed By MongoDBFacebook, Craigslist, Adobe CouchDBApple, BBC

Document Example { ‘first_name’: ‘John’, ‘last_name’: ‘Doe’, ‘age’: 20, ‘address’: { ‘street’: ‘123 Main’ ‘city’: ‘Los Angeles’ “state”: ‘CA’ }

Column Family Stores Data is stored in a column-oriented way as opposed to the row-oriented format in RDBMS ProductUsed By BigTableGoogle HBaseFacebook, Yahoo, Hulu and others HypertableBaidu, Rediff

Column and Column Family Columns: first_name, last_name, gender, occupation, zip_code Column families name : first_name, last_name profile : gender, occupation location : zip_code Column families typically need to be pre- defined while new columns can be added at any time

Units of Data row-key: 1 first_name: John last_name: Doe gender: male zip_code: row-key: 2 first_name: Jane zip_code: 10002

Data Storage row-key: 1 first_name: John last_name: Doe row-key: 2 first_name: Jane row-key: 1 zip_code: row-key: 2 zip_code: row-key: 1 gender: male namelocationprofile

Graph Databases Stores vertices (i.e. entities) and edges (i.e. relationships between vertices) Optimized for graph storage and processing ProductUsed By Neo4jInfoJobs, Addidas

NoSQL Database Example: MongoDB

MongoDB Server DB Database Collection

MongoDB Shell > mongo A command line client that provides an interactive JavaScript interface to MongoDB

Basic MongoDB Shell Commands help show dbs use Switch to database won’t be created until some data is inserted into it show collections db.dropDatabase()

Some Collection Methods db..insert() db..update() db..save() db..find() db..remove()

Basic CRUD Operations Create a database test1 Create two documents (i.e. objects or records) John and Jane Save the two documents to a collection users Query the collection

Using find() find( query, projection ) Both query and projection are documents in the form of { field1:, field2: … } Query examples: {firstName: {$eq:“John”}} {age: {$gt:20, $lte:30}}

Programming Language Support Drivers for various server-side programming language – rivers/ rivers/ REST web service for client-side JavaScript Example: mongo.html

Readings Professional NoSQL by Shashank Tiwari MongoDB Manual -