1. Formal Modeling and Analysis of RAMP Transaction Systems Si Liu, Peter Csaba Ölveczky, Muntasir Raihan Rahman, Jatin Ganhotra, Indranil Gupta, and José Meseguer University of Illinois at Urbana-Champaign and University of Oslo SAC – Pisa, April 2016

2. Background Huge data in cloud systems - databases must be partitioned - Facebook: 25 terabytes of data per day Full consistency of distributed typically impossible - trade-offs: consistency level vs. efficiency Promising approach: Read Atomic Multi-Partition transactions (RAMP) by Bailis et al (SIGMOD 2014) - atomic visibility: all or none of transactions updates are observable - no fractured reads: X friend of Y, but Y not friend of X Read Atomicity - baseline consistency model by Cerone et al (CONCUR 2015) - needed for most sensible consistency levels: Causality, Parallel Snapshot Isolation, Snapshot Isolation, Serialisability 2

3. Motivation RAMP not formalized and model checked - only “hand proofs” of correctness provided Distributed data stores for cloud computing challenging domain for formal methods - very little work (Amazon use of TLA+; UIUC work on Cassandra and Megastore) RAMP paper only mentions variants of RAMP - what properties do they satisfy? - what are the consequences of using/not using certain “building blocks” for distributed data stores? Ultimate long-term goal - library of formal executable building blocks - mix and match to build data stores with desired consistency and availability trade-offs 3

4. Read Atomic Multi-Partition Transactions Algorithms for ensuring atomic visibility in partitioned databases – either all of a transaction’s updates are observed, or none are Useful for e.g., multi-key updates, maintaining foreign key constraints, secondary indexes and materialized views 4

5. Our Approach 1.Developed executable formal models of RAMP and its suggested variants in rewriting logic - first detailed description of proposed variants - first formal model of RAMP 2.Formalized all desired RAMP correctness properties as reachability problems 3.Used Maude model checker to verify correctness properties of RAMP (up to certain bounds) 4.Analyzed 7 proposed variants of RAMP 5.Analyzed the dependencies of RAMP of its building blocks - is two-phase commit part really needed? (Yes!) 5

6. RAMP Execution & Algorithm 5

7. RAMP Variants/Extensions/Optimizations Formalized RAMP - without 2PC - with faster commit - with one-phase writes Example – RAMP with one-phase writes 7 “We have considered two-phase writes, but, if a user does not wish to read her writes (thereby sacrificing session guarantees outlined in the Appendix), the client can return after issuing its PREPARE round (without sacrificing durability). The client can subsequently execute the COMMIT phase asynchronously, …”

8. The distributed state of RAMP model is a “multiset” of Partitions, Clients and Messages - Example: Receiving Prepared Messages (Line 20 – 21 in RAMP’s pseudo code) crl [receive-prepared-2pc] : msg prepared(ID) from O’ to O => (if NS’ == empty then startCommit(OI,SQN,O) else none fi) if NS’ := delete(ID,NS). RAMP Model in Maude 7

9. We have formalized the following optimizations of RAMP - RAMP without 2PC - RAMP with faster commit - RAMP with one-phase writes -Example (RAMP with one-phase writes): Receiving Prepared Messages crl [receive-prepared-1pw] : msg prepared(ID) from O’ to O => if NS =/= empty. rl [receive-prepared-1pw-next-txn] : msg prepared(ID) from O’ to O < O : Client | pendingPrep : ID, pendingOps : OI, latest : VL, sqn : SQN, txn : TR TRS, result : RS > => < O : Client | pendingOps : rw(TR), pendingPrep : w(TR), 1stGets : r(TR), latest : vl(TR, VL), result : RS rs(TR), sqn : SQN + 1 > startCommit(OI, SQN, O) genOps(TR, SQN + 1, O). Formalizing Variants of RAMP in Maude 8

10. Validating Correctness Properties Model checking starts with single initial configuration - limited coverage Developed general techniques in Maude for model checking for all possible initial configurations up to certain bounds - #clients - #ops - #data items init(parameters) rewrites in one step to each possible initial configuration - start model checking with init(parameters) - init(4,2,(x,y)) generates 2764 initial states 10

11. Read Atomic Isolation: searching for a reachable final state where one of the clients’ committed results has a fractured read Validating Correctness Properties – Example 10

12. Validating Correctness Properties - Conclusion Analyzed all 4 key properties - Read Atomic Isolation - Companions Present - Synch Independence - Read-Your-Writes Analyzed 7 versions of RAMP against those properties - RAMP-Fast/Small without 2PC/with one-phase writes/with fast commit Results agree with the proved and conjectured results in the original paper: All versions satisfy the properties, except that - RAMP without 2PC does not satisfy read atomicity, companions present and read-your-writes - RAMP with one-phase writes does not satisfy read-your-writes 12

13. Summary First formal model of RAMP Developed and formalized different variants/optimizations of RAMP just hinted at in RAMP paper One of first works on formalizing distributed transactional data stores for cloud computing Model checking verification of all desired properties of RAMP (up to certain bounds) Analyzed proposed variants of RAMP Developed general Maude techniques for model checking for all initial system configurations up to certain bounds Studied the effect of splicing in and out different “building blocks” 13

14. Ongoing and Future Work Symmetry reduction - 6 initial states for 6 ops (compared to 2764 initial states for 4 ops) Use statistical model checking techniques to evaluate the performance of the different variations of RAMP Understanding the effect of various building blocks of distributed transactional protocols w.r.t. consistency 14

15. 15

16. How we go about it We use: 1. Maude - Modeling framework for distributed systems - Supports rewriting logic specification and programming - Efficiently executable 2. Reachability Analysis - By using Maude’s search command to search for reachable states from the initial states that match the search pattern and satisfy the search condition (search [n] t 0 =>* pattern such that condition.)