Producing short counterexamples using “crucial events” Sujatha Kashyap (Univ. of Texas at Austin) Dr. Vijay Garg (IBM Research)

Structure of this talk Motivation and Objectives Related Work Preliminaries: Programs, Traces, Lattices Meet- and Join-Closed Formulae, a Logic CETL ½ CTL Crucial Events, Crucial Paths, and Short Counterexamples Model Checking Using Crucial Events Experimental Results Summary, Q&A CAV 2008

Motivation and objectives State space reduction + short counterexamples CAV 2008

Related work, and where we fit in State space reduction Short counterexamples POR DMC POR + DMC Our approach POR: D. Peled, Combining partial order reductions with on-the-fly model-checking. CAV ’94. DMC: S. Edelkamp, S. Leue, A. Lluch-Lafuente, Directed explicit-state model checking in the validation of communication protocols. Int. J. on STTT 6(4), 2004. POR+DMC: A. Lluch-Lafuente, S. Edelkamp, S. Leue, Partial order reduction in directed model checking. Proc. of the 9th Int. SPIN Workshop on Model Checking of Software, 2002. CAV 2008

Preliminaries Finite-State Program: P = (S, T, s0) S: Finite set of states T µ S £ S: Finite set of deterministic transitions t = ®(s) s0 2 S: Initial state enabled(s): Set of transitions executable from s. t is reachable in P iff: ®0 ®1 ®2 s0 s1 s2 t CAV 2008

Preliminaries (contd.) Full state space graph of P: Directed, rooted, edge-labeled graph: Rooted at s0 Vertex set = set of reachable states of P ®-labeled edge from s to t iff ® 2 enabled(s) and t = ®(s). Path: Sequence of vertices (states) on some path in the full state space graph. s0 s1 s2 s3 … Transition sequence: Sequence of edge labels (transitions) on some path in the full state space graph. ®0 ®1 ®2 … Each occurrence of a transition is called an event. ®2 S3 S2 ®3 ®6 ®1 S5 S4 S1 ®5 ®4 ®0 S0 CAV 2008

Concurrency and independence x=1, y=1 x = 0 x := 1 x = 1 y := 1 x := 1 ¯ ® x=1, y=0 x=0, y=1 y = 0 y := 1 y = 1 ® ¯ x := 1 y := 1 x=0, y=0 Independence relation: I µ (T £ T) (®, ¯) 2 I if, whenever ®, ¯ 2 enabled(s): They neither enable nor disable each other. Executing them in either order results in the same state. Dependent: not independent D = (T £ T) n I It is not always sufficient to explore a single interleaving of independent events E.g., “it is always true that x ¸ y” . In CTL, AG(x ¸ y) CAV 2008

Traces and lattices Trace-equivalent sequences are derived by (repeatedly) commuting adjacent independent transitions. E.g., I = {(a,b) (b,c)} {a, b, c, a, b} a b abcab {a, b, c, b} {a, b, c, a} bacab acbab abcba a b acbba {a, b, c} b c Trace {a, b} {a, c} b a All trace-equivalent transition sequences Start at the same state Contain the same set of events End at the same state c {b} {a} a b {} Lattice CAV 2008

Lattices Directed, acyclic graph Each vertex represents the state reached after executing the corresponding set of events Closed under meet (set intersection) and join (set union) If G, H are vertices, so are (G Å H) and (G [ H) {a, b, c, a, b} a b {a, b, c, b} {a, b, c, a} a b {a, b, c} b c {a, b} {a, c} b a c {b} {a} a b {} Lattice CAV 2008

POR vs. our approach POR: If the property cannot distinguish between different sequences of a trace, then it is sufficient to explore a single sequence. D. Peled, Combining partial order reductions with on-the-fly model checking, in CAV ’94 Patrice Godefroid and Pierre Wolper. A partial approach to model checking, Information and Computation, 1994. Our approach: For a subset of CTL (called CETL), it is always sufficient to explore a single sequence. CAV 2008

Meet-closure, join-closure K = I [ J Meet-closed I J Join-closed G H = I Å J F = G Å H Regular = Meet- and join-closed CAV 2008

Relevant CTL operators ®2 S3 E[p U q] EF q = E [true U q] EG p E[q R p] E[q R p] = E[p U (p Æ q)] Ç EG p EG p = E [false R p] Process-local state formula: Atomic proposition consisting only of local variables from a single process. S2 ®3 ®6 ®1 S5 S4 S1 ®5 ®4 ®0 S0 p is true q is true p Æ q is true CAV 2008

Process-local state formulae are regular. Regular CTL formulae Process-local state formulae are regular. Theorem: If p and q are regular, so are: (p Æ q) E[p U (p Æ q)] EF q = E[ true U (true Æ q)] E[q R p] EG p = E[ false R p] CETL ½ CTL: Process-local state formulae are in CETL. If p and q are in CETL, so are p Æ q, E[p U (p Æ q)] , and E[q R p]. CAV 2008

Crucial events, crucial paths Executing the events in crucial(G, Á, ¾) is necessary and sufficient to lead to a Á-satisfying state in ¾. State space reduction Crucial paths form short counterexamples. K = {®, ¯, γ} γ ¯ ® ¯ ® : satisfies Á G Á is meet-closed crucial(G, Á, ¾) = K \ G CAV 2008

Model Checking CETL Using Crucial Events CAV 2008

Reduced state space search Full state space search: explore enabled(s) Reduced state space search: explore ample(s, Á) µ enabled(s) Baseline algorithm: ALMC A local, recursive, DFS-based CTL model checking algorithm. Reference: Vergauwen, B., Lewi, J., A linear local model checking algorithm for CTL, in CONCUR ’93. CAV 2008

ample(s, Á, ¾) for Á = E[p U (p Æ q)] Theorem: s ² Á in ¾ iff there exists a crucial path for (p Æ q) in ¾ that is a witness for s ² Á. Theorem: Sufficient ample set: ample(s, Á, ¾) = {®}, where ® 2 crucial(s, q, ¾) ®(s) ² p ¼6 ¼5 ¼4 ¸3 ¼3 ¼2 ¸2 = ¸3 Å ¼5 ¼1 ¸1 = ¸2 Å ¼4 s ² E[p U (p Æ q)] : satisfies p : satisfies p Æ q CAV 2008

ample(s, Á, ¾) for Á = E[q R p] E[q R p] = E[p U (p Æ q)] Ç EG(p) Theorem: Sufficient ample set for EG(p): ample(s, EG(p), ¾) = {®}, where ®(s) ² p ample(s, Á, ¾) = {®}, where ® 2 crucial(s, q, ¾) s ² EG(p) CAV 2008

Á = E[p U (p Æ q)] or E[q R p] ample(s, Á) Á = E[p U (p Æ q)] or E[q R p] Condition (C1) Along every path starting from s in the full state space graph, a transition that is dependent on a transition from ample(s, Á) cannot be executed without a transition from ample(s, Á) occurring first. [Peled ‘94] Theorem [Peled ’94]: If ample(s, Á) satisfies (C1), then it contains an event for each maximal trace starting from s. Universally crucial event: ® 2 ucrucial(s, Á) iff for every maximal trace ¾ starting from s, ® 2 crucial(s, Á, ¾) Condition (C2) If ample(s, Á) ≠ enabled(s), then for each ® 2 ample(s, Á): ® 2 ucrucial(s, q) ®(s) ² p [Peled ’94]: D. Peled, Combining partial order reductions with on-the-fly model checking, in CAV ’94. CAV 2008

Theorem: Exploring ample sets satisfying (C1) and (C2) is sufficient for model checking CETL. CAV 2008

Identifying universally crucial events Open problem for general CETL formulae. Can be recursively computed for special cases: When Á is a process-local state formula When Á = Á1 Æ Á2 When Á = E[Á1 U (Á1 Æ Á2)] or Á = E[ Á2 R Á1] and : Á1 is meet-closed CAV 2008

Experimental Results

Implementation details SPICED: Simple PROMELA Interpreter with Crucial Event Detection Based on SPIN http://spinroot.com/spin/ BEEM database: BEnchmarks for Explicit Model Checkers http://anna.fi.muni.cz/models/ Contains PROMELA models with errors injected, and property specifications for verification. CETL could express 77% of the properties in the BEEM database. Experimental results from 75 different variations (different problem sizes, location of errors) of 15 different models from the BEEM database. Compared against SPIN with POR. CAV 2008

Histogram of trail reduction Trail Reduction Factor = (Length of SPIN + POR trail) / (Length of SPICED trail) CAV 2008

Speedup = (Time taken by SPIN + POR) / (Time taken by SPICED) Histogram of speedup Speedup = (Time taken by SPIN + POR) / (Time taken by SPICED) CAV 2008

Histogram of relative memory consumption Relative memory consumption = (MB taken by SPIN + POR) / (MB taken by SPICED) CAV 2008

State space reduction in the absence of errors Reduction factor = Number of states in full graph / Number of states in reduced graph CAV 2008

Summary Meet- and join-closure can be exploited for state space reduction, and the production of short counterexamples. Several CTL operators preserve meet- and join-closure. CETL ½ CTL is a logic comprising only of meet- and join-closed formulae. An efficient model checking algorithm for CETL was presented, exploiting lattice theoretic characteristics. Experimental results were presented. CAV 2008

Q & A