Introduction à Couchbase Server 2.0 Tugdual Grall

{“about” : “me”} Tugdual “Tug” Grall Couchbase Technical Evangelist eXo CTO Oracle Developer/Product Manager Mainly Java/SOA Developer in consulting firms Tgrall NantesJUG co-founder Pet Project :

Agenda 1.Introduction to Couchbase Server 2.Cluster Management 3.Operation and Document Design 4.Introduction to Views and Queries 5.Cross Data Center Replication 6.Q&A Couchbase Developer Day Paris is this Friday!

Couchbase Server 2.0

Couchbase Server NoSQL Document Database

Couchbase Open Source Project Leading NoSQL database project focused on distributed database technology and surrounding ecosystem Supports both key-value and document-oriented use cases All components are available under the Apache 2.0 Public License Obtained as packaged software in both enterprise and community editions. Couchbase Open Source Project

Easy Scalability Consistent High Performance Always On 24x365 Grow cluster without application changes, without downtime with a single click Consistent sub-millisecond read and write response times with consistent high throughput No downtime for software upgrades, hardware maintenance, etc. Flexible Data Model JSON document model with no fixed schema. Couchbase Server

New in 2.0 JSON supportIndexing and Querying Cross data center replicationIncremental Map Reduce

Heartbeat Process monitor Global singleton supervisor Configuration manager on each node Rebalance orchestrator Node health monitor one per cluster vBucket state and replication manager http REST management API/Web UI HTTP 8091 Erlang port mapper 4369 Distributed Erlang Erlang/OTP storage interface Couchbase EP Engine Memcapable 2.0 Moxi Memcapable 1.0 Memcached New Persistence Layer 8092 Query API Query Engine Data ManagerCluster Manager Couchbase Server 2.0 Architecture

New Persistence Layer storage interface Couchbase EP Engine Memcapable 2.0 Moxi Memcapable 1.0 Object-level Cache Disk Persistence 8092 Query API Query Engine HTTP 8091 Erlang port mapper 4369 Distributed Erlang Heartbeat Process monitor Global singleton supervisor Configuration manager on each node Rebalance orchestrator Node health monitor one per cluster vBucket state and replication manager http REST management API/Web UI Erlang/OTP Server/Cluster Management & Communication (Erlang) RAM Cache, Indexing & Persistence Management (C & V8) The Unreasonable Effectiveness of C by Damien Katz Couchbase Server 2.0 Architecture

Single node - Couchbase Write Operation Managed Cache Disk Queue Disk Replication Queue App Server Couchbase Server Node Doc 1 To other node

GET Doc Single node - Couchbase Read Operation Disk Queue Replication Queue App Server Doc 1 Managed Cache Disk To other node Couchbase Server Node

COUCHBASE SERVER CLUSTER Basic Operation Docs distributed evenly across servers Each server stores both active and replica docs Only one server active at a time Client library provides app with simple interface to database Cluster map provides map to which server doc is on App never needs to know App reads, writes, updates docs Multiple app servers can access same document at same time User Configured Replica Count = 1 READ/WRITE/UPDATE ACTIVE Doc 5Doc 2Doc SERVER 1 ACTIVE Doc 4Doc 7Doc SERVER 2 Doc 8 ACTIVE Doc 1Doc 2Doc REPLICA Doc 4 Doc 1 Doc 8 Doc REPLICA Doc 6 Doc 3 Doc 2 Doc REPLICA Doc 7 Doc 9 Doc 5 Doc SERVER 3 Doc 6 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Doc 9

Add Nodes to Cluster Two servers added One-click operation Docs automatically rebalanced across cluster Even distribution of docs Minimum doc movement Cluster map updated App database calls now distributed over larger number of servers REPLICA ACTIVE Doc 5Doc 2Doc Doc 4Doc 1Doc SERVER 1 REPLICA ACTIVE Doc 4Doc 7Doc Doc 6Doc 3Doc SERVER 2 REPLICA ACTIVE Doc 1Doc 2Doc Doc 7Doc 9Doc SERVER 3SERVER 4SERVER 5 REPLICA ACTIVE REPLICA ACTIVE DocDoc 8DocDoc 9DocDoc 2DocDoc 8DocDoc 5DocDoc 6 READ/WRITE/UPDATE APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 COUCHBASE SERVER CLUSTER User Configured Replica Count = 1

Fail Over Node REPLICA ACTIVE Doc 5Doc 2Doc Doc 4Doc 1Doc SERVER 1 REPLICA ACTIVE Doc 4Doc 7Doc Doc 6Doc 3Doc SERVER 2 REPLICA ACTIVE Doc 1Doc 2Doc Doc 7Doc 9Doc SERVER 3SERVER 4SERVER 5 REPLICA ACTIVE REPLICA ACTIVE Doc 9Doc 8DocDoc 6Doc Doc 5DocDoc 2Doc 8Doc App servers accessing docs Requests to Server 3 fail Cluster detects server failed Promotes replicas of docs to active Updates cluster map Requests for docs now go to appropriate server Typically rebalance would follow DocDoc 1Doc 3 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 User Configured Replica Count = 1 COUCHBASE SERVER CLUSTER

Demo Time

Operation & Document Design

Basic Store & Retrieve Operations get(key) Retrieve a document set(key, value) Store a document or replace if it exists add(key, value) Store a document and fail if it exists replace(key, value) Replace a document and fail if it does not exist

Lots of other Operations Atomic Counters (increment, decrement) Append/Prepend CAS (Compare and Set) ­Optimistic Locking “Get with Lock” ­Write Lock on Objects Bulk Operations ­Saves network overhead Stats View operations

Couchbase 2.0 Bonus Points: JSON JSON is a lightweight format to represent document structure in a language-independent manner. If JSON documents are stored, the View engine can be used. Allows to build secondary indexes on your datasets. Makes it possible to ask questions like “Give me all user documents by lastname”.

Flexible Data Model No need to worry about the database when changing your application Records can have different structures, there is no fixed schema Allows painless data model changes for rapid application development { “ID”: 1, “FIRST”: “Dipti”, “LAST”: “Borkar”, “ZIP”: “94040”, “CITY”: “MV”, “STATE”: “CA” } JSON

Aggregate View of Data

Store and Retrieve Aggregates Easier to Distribute Data More Flexibility Reduced Latency order::1001 { uid: ji22jd, customer: Ann, line_items: [ { sku: , quan: 3, unit_price: 48.0 }, { sku: , quan: 1, unit_price: 39.0 }, { sku: , quan: 1, unit_price: 51.0 } ], payment: { type: Amex, expiry: 04/2001, last5: } }

Demo Time

Official SDKs Community SDKs Python Ruby Go Clojure

Indexing and Querying

Indexing and Querying – The basics Define materialized views on JSON documents and then query across the data set Using views you can define Primary indexes Simple secondary indexes (most common use case) Complex secondary, tertiary and composite indexes Aggregations (reduction) Indexes are eventually indexed Queries are eventually consistent Built using Map/Reduce technology Map and Reduce functions are written in Javascript

COUCHBASE SERVER CLUSTER Indexing and Querying User Configured Replica Count = 1 ACTIVE Doc 5Doc 2Doc SERVER 1 REPLICA Doc 4 Doc 1 Doc 8 Doc APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Doc 9 Indexing work is distributed amongst nodes Large data set possible Parallelize the effort Each node has index for data stored on it Queries combine the results from required nodes ACTIVE Doc 5Doc 2Doc SERVER 2 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc 9 ACTIVE Doc 5Doc 2Doc SERVER 3 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc 9 Query

Demo Time

Cross Data Center Replication (XDCR)

Cross Data Center Replication – The basics Replicate your Couchbase data across clusters Clusters may be spread across geos Configured on a per-bucket (per-database) basis Supports unidirectional and bidirectional operation Application can read and write from both clusters ­Active – Active replication Replication throughput scales out linearly Different from intra-cluster replication

Cross Data Center Replication (XDCR) COUCHBASE SERVER CLUSTER US ACTIVE DocDoc 2 SERVER 1 Doc 9 SERVER 2 SERVER 3 RAM Doc ACTIVE Doc RAM ACTIVE Doc RAM DISK Doc DISK Doc DISK COUCHBASE SERVER CLUSTER EMEA ACTIVE DocDoc 2 SERVER 1 Doc 9 SERVER 2 SERVER 3 RAM Doc ACTIVE Doc RAM ACTIVE Doc RAM DISK Doc DISK Doc DISK

Couchbase Handles Real World Scale

Q & A

Couchbase DeveloperDays Paris ­This Friday ! ­Presentations & Labs ­Information ­Use the promo code TUG

Thank Get Couchbase Server at