1. Producing short counterexamples using “crucial events”
Sujatha Kashyap (Univ. of Texas at Austin)
Dr. Vijay Garg (IBM Research)

2. Structure of this talk Motivation and Objectives Related Work
Preliminaries: Programs, Traces, Lattices
Meet- and Join-Closed Formulae, a Logic CETL ½ CTL
Crucial Events, Crucial Paths, and Short Counterexamples
Model Checking Using Crucial Events
Experimental Results
Summary, Q&A
CAV 2008

3. Motivation and objectives
State space reduction + short counterexamples
CAV 2008

4. Related work, and where we fit in
State space reduction
Short counterexamples
POR
DMC
POR +
DMC
Our
approach
POR: D. Peled, Combining partial order reductions with on-the-fly model-checking. CAV ’94.
DMC: S. Edelkamp, S. Leue, A. Lluch-Lafuente, Directed explicit-state model checking in the validation of communication protocols. Int. J. on STTT 6(4), 2004.
POR+DMC: A. Lluch-Lafuente, S. Edelkamp, S. Leue, Partial order reduction in directed model checking. Proc. of the 9th Int. SPIN Workshop on Model Checking of Software, 2002.
CAV 2008

5. Preliminaries Finite-State Program: P = (S, T, s0)
S: Finite set of states
T µ S £ S: Finite set of deterministic transitions
t = ®(s)
s0 2 S: Initial state
enabled(s): Set of transitions executable from s.
t is reachable in P iff: ®0 ®1 ®2 s0 s1 s2 t
CAV 2008

6. Preliminaries (contd.)
Full state space graph of P: Directed, rooted, edge-labeled graph:
Rooted at s0
Vertex set = set of reachable states of P
®-labeled edge from s to t iff ® 2 enabled(s) and t = ®(s).
Path: Sequence of vertices (states) on some path in the full state space graph.
s0 s1 s2 s3 …
Transition sequence: Sequence of edge labels (transitions) on some path in the full state space graph.
®0 ®1 ®2 …
Each occurrence of a transition is called an event. ®2 S3 S2 ®3 ®6 ®1 S5 S4 S1 ®5 ®4 ®0 S0
CAV 2008

7. Concurrency and independence
x=1, y=1
x = 0
x := 1
x = 1
y := 1
x := 1
x=1, y=0
x=0, y=1
y = 0
y := 1
y = 1
x := 1
y := 1
x=0, y=0
Independence relation: I µ (T £ T)
(®, ¯) 2 I if, whenever ®, ¯ 2 enabled(s):
They neither enable nor disable each other.
Executing them in either order results in the same state.
Dependent: not independent
D = (T £ T) n I
It is not always sufficient to explore a single interleaving of independent events
E.g., “it is always true that x ¸ y” . In CTL, AG(x ¸ y)
CAV 2008

8. Traces and lattices
Trace-equivalent sequences are derived by (repeatedly) commuting adjacent independent transitions.
E.g., I = {(a,b) (b,c)}
{a, b, c, a, b} a b
abcab
{a, b, c, b}
{a, b, c, a}
bacab
acbab
abcba a b
acbba
{a, b, c} b c
Trace
{a, b}
{a, c} b a
All trace-equivalent transition sequences
Start at the same state
Contain the same set of events
End at the same state c
{b}
{a} a b {}
Lattice
CAV 2008

9. Lattices Directed, acyclic graph
Each vertex represents the state reached after executing the corresponding set of events
Closed under meet (set intersection) and join (set union)
If G, H are vertices, so are (G Å H) and (G [ H)
{a, b, c, a, b} a b
{a, b, c, b}
{a, b, c, a} a b
{a, b, c} b c
{a, b}
{a, c} b a c
{b}
{a} a b {}
Lattice
CAV 2008

10. POR vs. our approach
POR: If the property cannot distinguish between different sequences of a trace, then it is sufficient to explore a single sequence.
D. Peled, Combining partial order reductions with on-the-fly model checking, in CAV ’94
Patrice Godefroid and Pierre Wolper. A partial approach to model checking, Information and Computation, 1994.
Our approach: For a subset of CTL (called CETL), it is always sufficient to explore a single sequence.
CAV 2008

11. Meet-closure, join-closure
K = I [ J
Meet-closed I J
Join-closed G
H = I Å J
F = G Å H
Regular = Meet- and join-closed
CAV 2008

12. Relevant CTL operators®2 S3
E[p U q]
EF q = E [true U q]
EG p
E[q R p]
E[q R p] = E[p U (p Æ q)] Ç EG p
EG p = E [false R p]
Process-local state formula:
Atomic proposition consisting only of local variables from a single process. S2 ®3 ®6 ®1 S5 S4 S1 ®5 ®4 ®0 S0
p is true
q is true
p Æ q is true
CAV 2008

13. Process-local state formulae are regular.
Regular CTL formulae
Process-local state formulae are regular.
Theorem: If p and q are regular, so are:
(p Æ q)
E[p U (p Æ q)]
EF q = E[ true U (true Æ q)]
E[q R p]
EG p = E[ false R p]
CETL ½ CTL:
Process-local state formulae are in CETL.
If p and q are in CETL, so are p Æ q, E[p U (p Æ q)] , and E[q R p].
CAV 2008

14. Crucial events, crucial paths
Executing the events in crucial(G, Á, ¾) is necessary and sufficient to lead to a Á-satisfying state in ¾.
State space reduction
Crucial paths form short counterexamples.
K = {®, ¯, γ} γ
: satisfies Á G
Á is meet-closed
crucial(G, Á, ¾) = K \ G
CAV 2008

15. Model Checking CETL Using Crucial Events
CAV 2008

16. Reduced state space search
Full state space search: explore enabled(s)
Reduced state space search:
explore ample(s, Á) µ enabled(s)
Baseline algorithm: ALMC
A local, recursive, DFS-based CTL model checking algorithm.
Reference: Vergauwen, B., Lewi, J., A linear local model checking algorithm for CTL, in CONCUR ’93.
CAV 2008

17. ample(s, Á, ¾) for Á = E[p U (p Æ q)]
Theorem: s ² Á in ¾ iff there exists a crucial path for (p Æ q) in ¾ that is a witness for s ² Á.
Theorem: Sufficient ample set: ample(s, Á, ¾) = {®}, where
® 2 crucial(s, q, ¾)
®(s) ² p ¼6 ¼5 ¼4 ¸3 ¼3 ¼2
¸2 = ¸3 Å ¼5 ¼1
¸1 = ¸2 Å ¼4
s ² E[p U (p Æ q)]
: satisfies p
: satisfies p Æ q
CAV 2008

18. ample(s, Á, ¾) for Á = E[q R p]
E[q R p] = E[p U (p Æ q)] Ç EG(p)
Theorem: Sufficient ample set for EG(p): ample(s, EG(p), ¾) = {®}, where
®(s) ² p
ample(s, Á, ¾) = {®}, where
® 2 crucial(s, q, ¾)
s ² EG(p)
CAV 2008

19. Á = E[p U (p Æ q)] or E[q R p]
ample(s, Á)
Á = E[p U (p Æ q)] or E[q R p]
Condition (C1)
Along every path starting from s in the full state space graph, a transition that is dependent on a transition from ample(s, Á) cannot be executed without a transition from ample(s, Á) occurring first. [Peled ‘94]
Theorem [Peled ’94]: If ample(s, Á) satisfies (C1), then it contains an event for each maximal trace starting from s.
Universally crucial event: ® 2 ucrucial(s, Á) iff for every maximal trace ¾ starting from s, ® 2 crucial(s, Á, ¾)
Condition (C2)
If ample(s, Á) ≠ enabled(s), then for each ® 2 ample(s, Á):
® 2 ucrucial(s, q)
®(s) ² p
[Peled ’94]: D. Peled, Combining partial order reductions with on-the-fly model checking, in CAV ’94.
CAV 2008

20. Theorem: Exploring ample sets satisfying (C1) and (C2) is sufficient for model checking CETL.
CAV 2008

21. Identifying universally crucial events
Open problem for general CETL formulae.
Can be recursively computed for special cases:
When Á is a process-local state formula
When Á = Á1 Æ Á2
When Á = E[Á1 U (Á1 Æ Á2)] or Á = E[ Á2 R Á1] and : Á1 is meet-closed
CAV 2008

22. Experimental Results

23. Implementation details
SPICED: Simple PROMELA Interpreter with Crucial Event Detection
Based on SPIN
BEEM database: BEnchmarks for Explicit Model Checkers
Contains PROMELA models with errors injected, and property specifications for verification.
CETL could express 77% of the properties in the BEEM database.
Experimental results from 75 different variations (different problem sizes, location of errors) of 15 different models from the BEEM database.
Compared against SPIN with POR.
CAV 2008

24. Histogram of trail reduction
Trail Reduction Factor = (Length of SPIN + POR trail) / (Length of SPICED trail)
CAV 2008

25. Speedup = (Time taken by SPIN + POR) / (Time taken by SPICED)
Histogram of speedup
Speedup = (Time taken by SPIN + POR) / (Time taken by SPICED)
CAV 2008

26. Histogram of relative memory consumption
Relative memory consumption = (MB taken by SPIN + POR) / (MB taken by SPICED)
CAV 2008

27. State space reduction in the absence of errors
Reduction factor = Number of states in full graph / Number of states in reduced graph
CAV 2008

28. Summary
Meet- and join-closure can be exploited for state space reduction, and the production of short counterexamples.
Several CTL operators preserve meet- and join-closure.
CETL ½ CTL is a logic comprising only of meet- and join-closed formulae.
An efficient model checking algorithm for CETL was presented, exploiting lattice theoretic characteristics.
Experimental results were presented.
CAV 2008

29. Q & A