1. Modeling and Analyzing Periodic Distributed Computations Anurag Agarwal Vijay Garg (garg@ece.utexas.edu)garg@ece.utexas.edu Vinit Ogale The University of Texas at Austin SSS2010

2. Motivation Many distributed computations are infinite o E.g., various reactive systems, servers Correctness Specifications o Safety: Nothing "bad" will ever happen o Liveness: Something "good" will happen eventually Specified using Temporal Logic [Pnueli 77] Runtime verification of properties How can one detect violation of liveness properties?

3. Difficulties Can observe only a finite execution How to exhibit the offending execution? How to detect if a global state is reachable in an infinite computation?

4. Recurrent Global State: Liveness violation Two identical states with P1 being hungry P1 does not eat in the intermediate computation

5. Overview Predicate Detection in Partial Order Recurrent global states Modeling infinite computations as d-diagrams Vector clocks in d-diagrams Predicate detection in d-diagrams Related work Conclusions and Future Work

6. Predicate Detection Predicate: A property expressed using variables on processes. e.g., more than one process is in critical section Predicate detection: Determining if an execution trace satisfies the predicate 6 Predicate detection trace predicate Yes No Program

7. Trace Model: Total Order Total order: interleaving of events in a trace –Temporal Rover [Drusinsky 03], –Java-MaC [Kim, Kannan, Lee, Sokolsky, and Viswanathan 04], –JPaX [Havelund and Rosu 04] –PET [Gunter, Kurshan, Peled 00] Low computational complexity 7

8. Partial Order Traces Predicate Detection –Exponential time algorithm for general predicate [Cooper and Marzullo 91] –NP-complete for simple boolean expressions (2-CNF) [Mittal and Garg 01] –Efficient algorithms for linear predicates [Chase, Garg 95], relational predicates [Tomlinson, Garg 93], Temporal Logic [Ogale Garg 07] 8 {} {e 1 } {f 1 } {e 1, f 1 } {e 2, e 1, f 1 } {e 2, e 1, f 2, f 1 } {e 1, f 2, f 1 } e1e1 e2e2 f1f1 f2f2 P1P1 P2P2 {e2,e1}{e2,e1} Computation Corresponding computation lattice

9. Recurrent global states (Consistent) Global states which occur more than once

10. D-diagram A d-diagram is a finite representation of an infinite periodic distributed computation ; (V,R,F,B) V: set of vertices R: recurrent vertices (infinite instances) F: forward edges for all i:e i -> f i B: shift edges for all i: e i -> f i + 1

11. Examples Set of natural numbers under natural order Set of natural numbers with no order

12. Unrolling the d-diagram A directed graph can be generated by "unrolling" a d-diagram ie. creating infinite instances of the recurrent vertices and generating the appropriate edges between them.

13. Finite Width Posets Lemma: A directed graph G defined by a d- diagram has finite width iff for every recurrent vertex there exists a cycle that includes a shift- edge.

14. Shift-of-a-cut A consistent cut in the graph can be "shifted" forwards or backwards with respect to some recurrent vertices to generate another consistent cut.

15. Overview Predicate Detection in Partial Order Recurrent global states Modeling infinite computations as d-diagrams Vector clocks in d-diagrams Predicate detection in d-diagrams Related work Conclusions and Future Work

16. Vector Clocks in a Distributed System e happened before f iff V(e) < V(f) [Fidge 89, Mattern 89] P1P1 (1,0,0)(2,1,0)(3,1,0) P2P2 (0,1,0)(0,2,0) P3P3 (0,0,1)(0,0,2)(2,1,3) How do we timestamp infinite sets of events?

17. Vector Clocks for d-diagrams J(e) = least consistent cut that includes e Theorem: For a recurrent vertex e, J(e) is guaranteed to stabilize after shift-diameter of the d-diagram. J(e i+1 ) can be derived from J(e i ) by shifting the cut. Shift-diameter of a d-diagram: Maximum number of shift-edges in the shortest path between any two vertices Lemma: On a d-diagram of a computation, shift-diameter is at most 2N.

18. Vector Clock for d-diagrams PV(e) = (V(e 1 ), V(e 2 ),..., V(e n ); I(e)) where V(e i ) = vector timestamp of e i I(e) = V(e i+1 ) - V(e i ) n = shift diameter Given p-timestamp, V(e n+j ) = V(e n ) + j * I(e) I(a) = [2,2] PV(a) = ([1,0], [3,0], [5,2]; [2,2]

19. Detecting global predicates Theorem: Sufficient to detect predicates on a finite part of the computation obtained by unrolling the d-diagram some number of times. Hence o If a predicate is never true in finite part, then it'll never be true in the infinite computation o If a predicate becomes true on recurrent events, then it'll be true infinitely often during the computation. Stopping Rule: the number of unrollings required is less than N (number of processes)

20. Recurrent Global State Detection Step 1: Ensure that the computation can be replayed (Deterministic Replay) [LM 87] Step 2: Compute a global state G (Global Snapshot Algorithm) [CL85]. Let the vector clock be Y. Step 3: Replay the computation detecting the first global state H that matches G (Conjunctive Predicate Detection [GW92] with vector clock Z) Step 4: Return (G,H) if Y != Z

21. Related Work Global Predicate Detection on happened-before model (e.g. conjunctive, linear, temporal logic predicates) – interpretation over finite traces Petri Nets – modeling and analyzing concurrent systems (versus a single computation) Message Sequence Charts (MSC) – incomparable to d-diagrams (e.g. require a message sent in a MSC to be received in the same MSC node)

22. Conclusions Recurrent global states D-diagram as model of an infinite periodic poset suitable for distributed computation Algorithm to timestamp events in a d-diagram Algorithm to detect global predicates in a d-diagram

23. Future Work Minimum unrolling Detecting general temporal logic formulas on d-diagrams

24. Questions? Recurrent global states Modeling infinite computations as d-diagrams Vector clocks in d-diagrams Predicate detection on d-diagrams

25. Backup Slides

26. Computation Model A distributed computation consists of N sequential processes P1, P2,... PN A directed graph is used to represent the computation with the vertices corresponding to the events and the edges representing the dependencies. o Acyclic graphs can represent finite computations A consistent cut in the distributed graph represents a set of vertices such that if it contains a vertex e, it contains all its incoming neighbors as well o A consistent cut represents a valid global state The frontier of a consistent cut is the set of events who successors from the same process don't exist in the cut. o The frontier can be used to uniquely represent a consistent cut

27. Trace Model: Partial Order Partial order: Lamports happened-before model [Lamport 78] –suitable for concurrent and distributed programs –encodes exponential number of total orders, captures bugs that may not be found with a total order 27

28. Vector Clock for d-diagrams We prove that after sufficient unrolling, the smallest consistent cuts containing different iteration of all recurrent vertices can be obtained simply by shifts. This iteration is called the shift-diameter. We associate a p-timestamp with every recurrent vertex which allows us to generate the timestamp for any iteration of the vertex: PV(e) = (V(e^1), V(e^2),..., V(e^n); I(e)) where V(e^i) = vector timestamp of e^i (the i-th iteration of e); I(e) = V(e^(n+1)) - V(e^n) n = shift diameter Given p-timestamp, V(e^(n+j)) = V(e^n) + j * I(e)

29. Detecting global predicates A predicate is a property defined on the states of the processes as well as channels We show that its sufficient to detect predicates on a finite part of the computation i.e. all possible consistent cuts that can be found in the full computation can be found in a finite subset obtained by rolling the d-diagram sufficient number of times. Hence o If a predicate is never true in finite part, then it'll never be true in the infinite computation o If a predicate becomes true on recurrent events, then it'll be true infinitely often during the computation. We also show that the number of unrollings required are less than N (number of processes)