CMU SCS Carnegie Mellon Univ. Dept. of Computer Science /615 - DB Applications C. Faloutsos – A. Pavlo Lecture#26: Database Systems

CMU SCS System Votes Faloutsos/PavloCMU SCS /6152

CMU SCS Two-Phase Commit Faloutsos/PavloCMU SCS /6153 Node 1Node 2 Application Server Commit Request Node 3 OK Phase1: Prepare Phase2: Commit Participant Coordinator

CMU SCS Paxos 4

CMU SCS Paxos Consensus protocol where a coordinator proposes an outcome (e.g., commit or abort) and then the participants vote on whether that outcome should succeed. Does not block if a majority of participants are available and has provably minimal message delays in the best case. –First correct protocol that was provably resilient in the face asynchronous networks Faloutsos/PavloCMU SCS /6155

CMU SCS Paxos Faloutsos/PavloCMU SCS /6156 Node 1Node 2 Application Server Commit Request Node 3 Proposer Node 4 Acceptor Propose Commit Agree Accept

CMU SCS Paxos Faloutsos/PavloCMU SCS /6157 Node 1Node 2 Application Server Commit Request Node 3 Proposer Node 4 Acceptor Propose Commit Agree Accept X

CMU SCS Paxos Proposer Acceptors Propose(n) Agree(n) Propose(n+1) Commit(n) Reject(n, n+1) Commit(n+1) Agree(n+1) Accept(n+1)

CMU SCS 2PC vs. Paxos 2PC is a degenerate case of Paxos. –Single coordinator. –Only works if everybody is up. Use leases to determine who is allowed to propose new updates to avoid continuous rejection. Faloutsos/PavloCMU SCS /6159

CMU SCS Google Spanner Google’s geo-replicated DBMS (>2011) Schematized, semi-relational data model. Concurrency Control: –2PL + T/O (Pessimistic) –Externally consistent global write-transactions with synchronous replication. –Lock-free read-only transactions. 10

CMU SCS Google Spanner 11 CREATE TABLE users { uid INT NOT NULL, VARCHAR, PRIMARY KEY (uid) }; CREATE TABLE albums { uid INT NOT NULL, aid INT NOT NULL, name VARCHAR, PRIMARY KEY (uid, aid) } INTERLEAVE IN PARENT users ON DELETE CASCADE; CREATE TABLE users { uid INT NOT NULL, VARCHAR, PRIMARY KEY (uid) }; CREATE TABLE albums { uid INT NOT NULL, aid INT NOT NULL, name VARCHAR, PRIMARY KEY (uid, aid) } INTERLEAVE IN PARENT users ON DELETE CASCADE; users(1001) albums(1001, 9990) albums(1001, 9991) users(1002) albums(1002, 6631) albums(1002, 6634)

CMU SCS Google Spanner Ensures ordering through globally unique timestamps generated from atomic clocks and GPS devices. Database is broken up into tablets: –Use Paxos to elect leader in tablet group. –Use 2PC for txns that span tablets. TrueTime API 12

CMU SCS Google Spanner Node 1Node 2 NETWORK Application Server Set A=2, B=9 Application Server A=1 Set A=0, B=7 B=8 T1T2 13 Paxos or 2PC

CMU SCS Google F1 OCC engine built on top of Spanner. –Read phase followed by a write phase –In the read phase, F1 returns the last modified timestamp with each row. No locks. –The timestamp for a row is stored in a hidden lock column. The client library returns these timestamps to the F1 server –If the timestamps differ from the current timestamps at the time of commit the transaction is aborted 14

CMU SCS Redis Remote Dictionary Server (2009) Key-value data store: –Values can be strings, hashes, lists, sets and sorted sets. –Single-threaded execution engine. In-memory storage: –Snapshots + WAL for persistence. CMU SCS /61515

CMU SCS Redis 16 STRING page:index.html → … view_count → SET users_logged_in → { 1, 2, 3, 4, 5 } latest_post_ids → { 111, 112, 119, … } LIST HASH user:999:session → time => username => tupac SORTED SET current_user_scores → odb ~ 11 tupac ~ 12 biggie ~ 19 eazye ~ 20 Keys Values

CMU SCS

Redis Asynchronous master-slave replication: –Master sends oplog to downstream replicas. –Do not wait for acknowledgements. Newer version can ensure that at least some replicas are available before accepting writes but still not check whether they received those writes. CMU SCS /61518

CMU SCS Redis Supports some notion of transactions: –Operations are batched together and executed serially on server side. –Allows for compare-and-swap. Does not support rollback! CMU SCS /61519

CMU SCS MongoDB Document Data Model –Think JSON, XML, Python dicts –Not Microsoft Word documents Different terminology: –Document → Tuple –Collection → Table/Relation 20

CMU SCS MongoDB JSON-only query API Single-document atomicity. –No server-side joins: –“Pre-join” collections by embedding related documents inside of each other. No cost-based query planner / optimizer. 21

CMU SCS MongoDB A customer has orders and each order has order items. 22 Customers Orders Order Items R2(orderId, custId, …) R1(custId, name, …) R3(itemId, orderId, …)

CMU SCS MongoDB A customer has orders and each order has order items. 23 Customers Orders Order Items Customer Order Order Item ⋮ { "custId" : 1234, "custName" : "Christos", "orders" : [ { "orderId" : 10001, "orderItems" : [ { "itemId" : "XXXX", "price" : }, { "itemId" : "YYYY", "price" : } ] }, { "orderId" : 10050, "orderItems" : [ { "itemId" : “ZZZZ", "price" : } ] } ] }

CMU SCS MongoDB Heterogeneous distributed components. –Centralized query router. Master-slave replication. Auto-sharding: –Define ‘partitioning’ attributes for each collection (hash or range). –When a shard gets too big, the DBMS automatically splits the shard and rebalances. 24

CMU SCS MongoDB Originally used mmap storage manager –No buffer pool. –Let the OS decide when to flush pages. –Single lock per database. Version 3 (2015) now supports pluggable storage managers. –WiredTiger from BerkeleyDB alumni. –Fine-grained locking. 25