2. Introduction à Couchbase Server 2.0 Tugdual Grall TechnicalEvangelist @tgrall

3. {“about” : “me”} Tugdual “Tug” Grall Couchbase Technical Evangelist eXo CTO Oracle Developer/Product Manager Mainly Java/SOA Developer in consulting firms Web @tgrall http://blog.grallandco.com Tgrall NantesJUG co-founder Pet Project : http://www.resultri.com

4. Agenda 1.Introduction to Couchbase Server 2.Cluster Management 3.Operation and Document Design 4.Introduction to Views and Queries 5.Cross Data Center Replication 6.Q&A Couchbase Developer Day Paris is this Friday! http://couchbase.com/events

5. Couchbase Server 2.0

6. Couchbase Server NoSQL Document Database

7. Couchbase Open Source Project Leading NoSQL database project focused on distributed database technology and surrounding ecosystem Supports both key-value and document-oriented use cases All components are available under the Apache 2.0 Public License Obtained as packaged software in both enterprise and community editions. Couchbase Open Source Project

8. Easy Scalability Consistent High Performance Always On 24x365 Grow cluster without application changes, without downtime with a single click Consistent sub-millisecond read and write response times with consistent high throughput No downtime for software upgrades, hardware maintenance, etc. Flexible Data Model JSON document model with no fixed schema. Couchbase Server

9. New in 2.0 JSON supportIndexing and Querying Cross data center replicationIncremental Map Reduce

10. Heartbeat Process monitor Global singleton supervisor Configuration manager on each node Rebalance orchestrator Node health monitor one per cluster vBucket state and replication manager http REST management API/Web UI HTTP 8091 Erlang port mapper 4369 Distributed Erlang 21100 - 21199 Erlang/OTP storage interface Couchbase EP Engine 11210 Memcapable 2.0 Moxi 11211 Memcapable 1.0 Memcached New Persistence Layer 8092 Query API Query Engine Data ManagerCluster Manager Couchbase Server 2.0 Architecture

11. New Persistence Layer storage interface Couchbase EP Engine 11210 Memcapable 2.0 Moxi 11211 Memcapable 1.0 Object-level Cache Disk Persistence 8092 Query API Query Engine HTTP 8091 Erlang port mapper 4369 Distributed Erlang 21100 - 21199 Heartbeat Process monitor Global singleton supervisor Configuration manager on each node Rebalance orchestrator Node health monitor one per cluster vBucket state and replication manager http REST management API/Web UI Erlang/OTP Server/Cluster Management & Communication (Erlang) RAM Cache, Indexing & Persistence Management (C & V8) The Unreasonable Effectiveness of C by Damien Katz Couchbase Server 2.0 Architecture

12. 3 3 3 3 2 2 Single node - Couchbase Write Operation Managed Cache Disk Queue Disk Replication Queue App Server Couchbase Server Node Doc 1 To other node

13. GET Doc 1 3 3 3 3 2 2 Single node - Couchbase Read Operation Disk Queue Replication Queue App Server Doc 1 Managed Cache Disk To other node Couchbase Server Node

14. COUCHBASE SERVER CLUSTER Basic Operation Docs distributed evenly across servers Each server stores both active and replica docs Only one server active at a time Client library provides app with simple interface to database Cluster map provides map to which server doc is on App never needs to know App reads, writes, updates docs Multiple app servers can access same document at same time User Configured Replica Count = 1 READ/WRITE/UPDATE ACTIVE Doc 5Doc 2Doc SERVER 1 ACTIVE Doc 4Doc 7Doc SERVER 2 Doc 8 ACTIVE Doc 1Doc 2Doc REPLICA Doc 4 Doc 1 Doc 8 Doc REPLICA Doc 6 Doc 3 Doc 2 Doc REPLICA Doc 7 Doc 9 Doc 5 Doc SERVER 3 Doc 6 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Doc 9

15. Add Nodes to Cluster Two servers added One-click operation Docs automatically rebalanced across cluster Even distribution of docs Minimum doc movement Cluster map updated App database calls now distributed over larger number of servers REPLICA ACTIVE Doc 5Doc 2Doc Doc 4Doc 1Doc SERVER 1 REPLICA ACTIVE Doc 4Doc 7Doc Doc 6Doc 3Doc SERVER 2 REPLICA ACTIVE Doc 1Doc 2Doc Doc 7Doc 9Doc SERVER 3SERVER 4SERVER 5 REPLICA ACTIVE REPLICA ACTIVE DocDoc 8DocDoc 9DocDoc 2DocDoc 8DocDoc 5DocDoc 6 READ/WRITE/UPDATE APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 COUCHBASE SERVER CLUSTER User Configured Replica Count = 1

16. Fail Over Node REPLICA ACTIVE Doc 5Doc 2Doc Doc 4Doc 1Doc SERVER 1 REPLICA ACTIVE Doc 4Doc 7Doc Doc 6Doc 3Doc SERVER 2 REPLICA ACTIVE Doc 1Doc 2Doc Doc 7Doc 9Doc SERVER 3SERVER 4SERVER 5 REPLICA ACTIVE REPLICA ACTIVE Doc 9Doc 8DocDoc 6Doc Doc 5DocDoc 2Doc 8Doc App servers accessing docs Requests to Server 3 fail Cluster detects server failed Promotes replicas of docs to active Updates cluster map Requests for docs now go to appropriate server Typically rebalance would follow DocDoc 1Doc 3 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 User Configured Replica Count = 1 COUCHBASE SERVER CLUSTER

17. Demo Time

18. Operation & Document Design

19. Basic Store & Retrieve Operations get(key) Retrieve a document set(key, value) Store a document or replace if it exists add(key, value) Store a document and fail if it exists replace(key, value) Replace a document and fail if it does not exist

20. Lots of other Operations Atomic Counters (increment, decrement) Append/Prepend CAS (Compare and Set) ­Optimistic Locking “Get with Lock” ­Write Lock on Objects Bulk Operations ­Saves network overhead Stats View operations

21. Couchbase 2.0 Bonus Points: JSON JSON is a lightweight format to represent document structure in a language-independent manner. If JSON documents are stored, the View engine can be used. Allows to build secondary indexes on your datasets. Makes it possible to ask questions like “Give me all user documents by lastname”.

22. Flexible Data Model No need to worry about the database when changing your application Records can have different structures, there is no fixed schema Allows painless data model changes for rapid application development { “ID”: 1, “FIRST”: “Dipti”, “LAST”: “Borkar”, “ZIP”: “94040”, “CITY”: “MV”, “STATE”: “CA” } JSON

23. Aggregate View of Data http://martinfowler.com/bliki/AggregateOrientedDatabase.html

24. Store and Retrieve Aggregates Easier to Distribute Data More Flexibility Reduced Latency order::1001 { uid: ji22jd, customer: Ann, line_items: [ { sku: 0321293533, quan: 3, unit_price: 48.0 }, { sku: 0321601912, quan: 1, unit_price: 39.0 }, { sku: 0131495054, quan: 1, unit_price: 51.0 } ], payment: { type: Amex, expiry: 04/2001, last5: 12345 } }

25. Demo Time

26. Official SDKs Community SDKs www.couchbase.com/develop Python Ruby Go Clojure

27. Indexing and Querying

28. Indexing and Querying – The basics Define materialized views on JSON documents and then query across the data set Using views you can define Primary indexes Simple secondary indexes (most common use case) Complex secondary, tertiary and composite indexes Aggregations (reduction) Indexes are eventually indexed Queries are eventually consistent Built using Map/Reduce technology Map and Reduce functions are written in Javascript

29. COUCHBASE SERVER CLUSTER Indexing and Querying User Configured Replica Count = 1 ACTIVE Doc 5Doc 2Doc SERVER 1 REPLICA Doc 4 Doc 1 Doc 8 Doc APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Doc 9 Indexing work is distributed amongst nodes Large data set possible Parallelize the effort Each node has index for data stored on it Queries combine the results from required nodes ACTIVE Doc 5Doc 2Doc SERVER 2 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc 9 ACTIVE Doc 5Doc 2Doc SERVER 3 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc 9 Query

30. Demo Time

31. Cross Data Center Replication (XDCR)

32. Cross Data Center Replication – The basics Replicate your Couchbase data across clusters Clusters may be spread across geos Configured on a per-bucket (per-database) basis Supports unidirectional and bidirectional operation Application can read and write from both clusters ­Active – Active replication Replication throughput scales out linearly Different from intra-cluster replication

33. Cross Data Center Replication (XDCR) COUCHBASE SERVER CLUSTER US ACTIVE DocDoc 2 SERVER 1 Doc 9 SERVER 2 SERVER 3 RAM Doc ACTIVE Doc RAM ACTIVE Doc RAM DISK Doc DISK Doc DISK COUCHBASE SERVER CLUSTER EMEA ACTIVE DocDoc 2 SERVER 1 Doc 9 SERVER 2 SERVER 3 RAM Doc ACTIVE Doc RAM ACTIVE Doc RAM DISK Doc DISK Doc DISK

34. Couchbase Handles Real World Scale

35. Q & A

36. Couchbase DeveloperDays Paris ­This Friday ! ­Presentations & Labs ­Information http://www.couchbase.com/events ­Use the promo code TUG

37. Thank you! tug@couchbase.com @tgrall Get Couchbase Server at http://www.couchbase.com/download