1. CMU SCS Carnegie Mellon Univ. Dept. of Computer Science 15-415/615 - DB Applications C. Faloutsos – A. Pavlo Lecture#26: Database Systems

2. CMU SCS 4 2 2 1 1 1 1 System Votes Faloutsos/PavloCMU SCS 15-415/6152

3. CMU SCS Two-Phase Commit Faloutsos/PavloCMU SCS 15-415/6153 Node 1Node 2 Application Server Commit Request Node 3 OK Phase1: Prepare Phase2: Commit Participant Coordinator

4. CMU SCS Paxos 4

5. CMU SCS Paxos Consensus protocol where a coordinator proposes an outcome (e.g., commit or abort) and then the participants vote on whether that outcome should succeed. Does not block if a majority of participants are available and has provably minimal message delays in the best case. –First correct protocol that was provably resilient in the face asynchronous networks Faloutsos/PavloCMU SCS 15-415/6155

6. CMU SCS Paxos Faloutsos/PavloCMU SCS 15-415/6156 Node 1Node 2 Application Server Commit Request Node 3 Proposer Node 4 Acceptor Propose Commit Agree Accept

7. CMU SCS Paxos Faloutsos/PavloCMU SCS 15-415/6157 Node 1Node 2 Application Server Commit Request Node 3 Proposer Node 4 Acceptor Propose Commit Agree Accept X

8. CMU SCS Paxos Proposer Acceptors Propose(n) Agree(n) Propose(n+1) Commit(n) Reject(n, n+1) Commit(n+1) Agree(n+1) Accept(n+1)

9. CMU SCS 2PC vs. Paxos 2PC is a degenerate case of Paxos. –Single coordinator. –Only works if everybody is up. Use leases to determine who is allowed to propose new updates to avoid continuous rejection. Faloutsos/PavloCMU SCS 15-415/6159

10. CMU SCS Google Spanner Google’s geo-replicated DBMS (>2011) Schematized, semi-relational data model. Concurrency Control: –2PL + T/O (Pessimistic) –Externally consistent global write-transactions with synchronous replication. –Lock-free read-only transactions. 10

11. CMU SCS Google Spanner 11 CREATE TABLE users { uid INT NOT NULL, email VARCHAR, PRIMARY KEY (uid) }; CREATE TABLE albums { uid INT NOT NULL, aid INT NOT NULL, name VARCHAR, PRIMARY KEY (uid, aid) } INTERLEAVE IN PARENT users ON DELETE CASCADE; CREATE TABLE users { uid INT NOT NULL, email VARCHAR, PRIMARY KEY (uid) }; CREATE TABLE albums { uid INT NOT NULL, aid INT NOT NULL, name VARCHAR, PRIMARY KEY (uid, aid) } INTERLEAVE IN PARENT users ON DELETE CASCADE; users(1001) albums(1001, 9990) albums(1001, 9991) users(1002) albums(1002, 6631) albums(1002, 6634)

12. CMU SCS Google Spanner Ensures ordering through globally unique timestamps generated from atomic clocks and GPS devices. Database is broken up into tablets: –Use Paxos to elect leader in tablet group. –Use 2PC for txns that span tablets. TrueTime API 12

13. CMU SCS Google Spanner Node 1Node 2 NETWORK Application Server Set A=2, B=9 Application Server A=1 Set A=0, B=7 B=8 T1T2 13 Paxos or 2PC

14. CMU SCS Google F1 OCC engine built on top of Spanner. –Read phase followed by a write phase –In the read phase, F1 returns the last modified timestamp with each row. No locks. –The timestamp for a row is stored in a hidden lock column. The client library returns these timestamps to the F1 server –If the timestamps differ from the current timestamps at the time of commit the transaction is aborted 14

15. CMU SCS Redis Remote Dictionary Server (2009) Key-value data store: –Values can be strings, hashes, lists, sets and sorted sets. –Single-threaded execution engine. In-memory storage: –Snapshots + WAL for persistence. CMU SCS 15-415/61515

16. CMU SCS Redis 16 STRING page:index.html → … view_count → 12345 SET users_logged_in → { 1, 2, 3, 4, 5 } latest_post_ids → { 111, 112, 119, … } LIST HASH user:999:session → time => 1430086922 username => tupac SORTED SET current_user_scores → odb ~ 11 tupac ~ 12 biggie ~ 19 eazye ~ 20 Keys Values

17. CMU SCS

18. Redis Asynchronous master-slave replication: –Master sends oplog to downstream replicas. –Do not wait for acknowledgements. Newer version can ensure that at least some replicas are available before accepting writes but still not check whether they received those writes. CMU SCS 15-415/61518

19. CMU SCS Redis Supports some notion of transactions: –Operations are batched together and executed serially on server side. –Allows for compare-and-swap. Does not support rollback! CMU SCS 15-415/61519

20. CMU SCS MongoDB Document Data Model –Think JSON, XML, Python dicts –Not Microsoft Word documents Different terminology: –Document → Tuple –Collection → Table/Relation 20

21. CMU SCS MongoDB JSON-only query API Single-document atomicity. –No server-side joins: –“Pre-join” collections by embedding related documents inside of each other. No cost-based query planner / optimizer. 21

22. CMU SCS MongoDB A customer has orders and each order has order items. 22 Customers Orders Order Items R2(orderId, custId, …) R1(custId, name, …) R3(itemId, orderId, …)

23. CMU SCS MongoDB A customer has orders and each order has order items. 23 Customers Orders Order Items Customer Order Order Item ⋮ { "custId" : 1234, "custName" : "Christos", "orders" : [ { "orderId" : 10001, "orderItems" : [ { "itemId" : "XXXX", "price" : 19.99 }, { "itemId" : "YYYY", "price" : 29.99 } ] }, { "orderId" : 10050, "orderItems" : [ { "itemId" : “ZZZZ", "price" : 49.99 } ] } ] }

24. CMU SCS MongoDB Heterogeneous distributed components. –Centralized query router. Master-slave replication. Auto-sharding: –Define ‘partitioning’ attributes for each collection (hash or range). –When a shard gets too big, the DBMS automatically splits the shard and rebalances. 24

25. CMU SCS MongoDB Originally used mmap storage manager –No buffer pool. –Let the OS decide when to flush pages. –Single lock per database. Version 3 (2015) now supports pluggable storage managers. –WiredTiger from BerkeleyDB alumni. –Fine-grained locking. 25