NoSQL Databases Cloudant & Redis Nikolay Tomitov www.softacad.bg Technical Trainer SoftAcad Training Center www.softacad.bg

Schema-free Databases Documental Databases Relational Databases www.softacad.bg

JSON Data Format www.softacad.bg

Meet Cloudant Cloudant == managed installation of Apache CouchDB in the cloud www.softacad.bg

Global Data Network Cloudant scales within & between data centers Global networks of servers Built-in replication and sync Push the data closer to the user & application Disaster recovery www.softacad.bg

Nice Features Eliminates data layer-related complexity, delay, cost, and risk Performance & scale with no worries Build epic apps, easily Enables advanced app features and analytics Full-text search, off-line computing, replication, incremental MapReduce, code and data distribution... Accelerates release cycles (reduces time to value) Start for free, pay as you grow www.softacad.bg

Developing with Cloudant Provision Signup is free, and access is immediate Define No schema, usually just JSON Access RESTful API View Define data views via Javascript Search Integrated, Lucene-like full-text search Analyze Incremental, chainable MapReduce Operate Dashboards to monitor, compact, replicate Grow Automatic

CouchDB Inside Data is stored and returned in JSON format Queried via HTTP RESTful API Index building language: Javascript Simple and intuitive interface www.softacad.bg

REST Interface – DEMO Create: Read: Update: Delete: HTTP PUT /db/test HTTP GET /db/test Update: Delete: HTTP DELETE /db/test www.softacad.bg

Querying the DB Views: the way to arrange data to answer our questions Method to build views: Incremental MapReduce using JavaScript www.softacad.bg

DEMO – Simple MapReduce www.softacad.bg

Example www.softacad.bg

How to Find sum(checkouts) ? www.softacad.bg

Simple Map: emit(checkout) www.softacad.bg

Simple Reduce: sum(checkouts) www.softacad.bg

Simple Reduce: sum(checkouts) www.softacad.bg

Views can be Distributed www.softacad.bg

Practice Practice Map Reduce at: http://blog.mudynamics.com/wp- content/uploads/2009/04/icouch.html http://couchdb.apache.org/downloads.html www.softacad.bg

Meet Redis “Redis is an open source, advanced key-value store It is often referred to as a data structure server Since keys can contain strings, hashes, lists, sets and sorted sets www.softacad.bg

Features First It’s really fast Non-blocking I/O, single threaded 100,000+ read/writes a second It complements your existing storage layer It is very feasible for small apps But..trade performance for durability www.softacad.bg

Keys Short keys perform better Could not contain whitespaces Common convention is : obj- type:id:field user:77:pass = abcd SHA1(data) makes a useful key too

Supported Data Types strings (up to 1GB) Lists of strings Sets of strings Sorted sets Hashes Pubsub channels

Commands on All Keys EXISTS[key] DEL[key] TYPE[key] RENAMENX[key] EXPIRE [key] EXPIREAT[key] TTL[key]

Commands on Strings SET[key value] MGET[key1 key2 key3..] GET [key] INCR/DECR [key] INCRBY/DECRBY[key]

Common Uses Store session data + creation time Hit counts key =MD5(url) Incr hits-by-key:$key APPEND can be used for logging Redis as shared memory location Progress bars, polling

Durability More durability -> less performance Default behaviour fork-and-save a snapshot to disk every… 15 mins if one key has changed 5 mins if 10 keys have changed 60 secs if 10000 keys have changed SAVE/BGSAVE commands trigger a save

DEMO (JAVA)

Check It Out Try it : Redis Admin UI http://try.redis-db.com http://www.servicestack.net/mythz_blog/?p=381 www.softacad.bg

Amazon SimpleDB

SimpleDB: Overview Non–relational data store Designed to be highly available and flexible Available only on Amazon WS

SimpleDB: Functionality Low touch – almost no administration needed Highly available – automatic replication and fail over Flexible – not bound by a schema Simple to use – supports only the most common operations Designed for integration with other Amazon Web Services Secure RESTful interface

SimpleDB: Pricing Free tier – 1 GB/month, 25 SimpleDB machine hours Only outgoing transfers are charged Data transfers in the same region are free Monthly rates between $0.12 and $0.05

SimpleDB: Pros Optimized reads and writes Easily integrated with other AWS Uses the HTTP protocol for communication Ideal for storing non – structured or semi – structured data

SimpleDB: Cons Limited functionality compared to relational and other non – relational products Complex queries need a lot of coding No fulltext indexing All queries have the N + 1 problem

Demo Amazon Simple DB

Google Datastore (GDS)

GDS: Overview Schemaless object datastore Designed to scale Hierarchical Available only on Google App Engine

GDS: Functionality No administration needed Flexible – not bound by a schema Designed for no downtime Automatic replication Queries must run against predefined indexes Rich query engine Atomic transactions APIs available for all the languages supported by GAE

GDS: Pricing Free quota: 1GB, 200 indexes Monthly rates: $0.24/gigabyte; between $0.01 and $0.10 for 100k database operations

GDS: Pros Fast Hierarchical queries boost speed Atomic transactions supported Large and active community The Java API include implementations of JDO and JPA

GDS: Cons Limited functionality Compared to relational databases and other non–relational data stores Indexes require additional configuration

Google App Engine Datastore Demo Google App Engine Datastore

MongoDB

Overview Non – relational, schema free database Highly scalable Engineered for maximum speed Available as a stand alone product A number of hosted MongoDB solutions available MongoLab MongoHQ Mongo Machine

MongoDB: Functionality Almost administration free High availability – replication and sharding A rich set of features Complex queries Aggregate functions Map/Reduce operations Indexes Stored procedures BLOB storage (GridFS) Database drivers available for a large list of programming languages

MongoDB: Hosted Solutions Easy setup (No billing info needed) Intuitive interface 240MB free Deploy on Amazon EC2, Rackspace or Joynet MongoDB Version 1.8 Easy setup (Credit card needed) Intuitive interface 16MB free Deploy only on Amazon EC2 Different MongoDB Version according to price plan MongoLab MongoHQ

MongoDB: Pros Fast Rich feature set Mature product Large and active community

MongoDB: Cons No relational database style transactions No data validation

Demo MongoDB on MongoLab

Homework Register for a free Cloudant account at https://cloudant.com. Write a simple "Dictionary" application in Java, C#, PHP, or JavaScript to perform the following in Cloudant: Add a dictionary entry (word + translation) List all words and their translations Find the translation of given word The UI of the application is up to you (it could be Web-based, GUI or console-based). You may download a Cloudant client library from http://wiki.apache.org/couchdb/Related_Projects

Homework (2) Implement the previous task ("Dictionary") with Redis Register for a free "Redis To Go" account at: https://redistogo.com Download the client libraries for your favorite programming language from http://redis.io/clients Implement the previous task ("Dictionary") with MongoDB (register at MongoLab or MongoHQ) Optionally (if you want) implement the previous task ("Dictionary") with Google Datastore or Amazon SimpleDB