1. Abstraction for Falsification Thomas Ball Orna Kupferman Greta Yorsh Microsoft Research, Redmond, US Hebrew University, Jerusalem, Israel Tel Aviv University, Israel CAV’05

2. Abstraction for Verification Goal: prove properties Sound abstraction for verification –properties of abstract system hold for corresponding concrete system –  : C  A –if abstract state a satisfies property P then all concrete states represented by a satisfy P

3. Abstraction for Verification Goal: prove properties Sound abstraction for verification –properties of abstract system hold for corresponding concrete system –  : C  A –  a  A if a  P then  c  C.  (c)=a  c  P

4. Abstraction for Verification Goal: prove properties Sound abstraction for verification –properties of abstract system hold for corresponding concrete system –  : C  A –  a  A if a  P then  c  C.  (c)=a  c  P Falsification detect errors

5. Abstraction for Verification Goal: prove properties Sound abstraction for verification –errors of the abstract system exist in corresponding concrete system –  : C  A –  a  A if a  P then  c  C.  (c)=a  c  P Falsification falsification detect errors

6. Abstraction for Verification Goal: prove properties Sound abstraction for verification –errors of the abstract system exist in corresponding concrete system –  : C  A –  a  A if a  P then  c  C.  (c)=a  c  P Falsification falsification detect errors  c  C.  (c)=a  c  P

7. Motivation An abstraction that is sound for falsification need not be sound for verification. Existing frameworks for abstraction for verification –Modal Transition System (MTS) –MTS, PKS,KMTS - equivalent in expressive power [ Godefroid,Jagadessan – VMCAI’03 ] –can be too restrictive for falsification

8. Main Results New framework for abstraction –Ternary Modal Transition System (TMTS) –TMTS is stronger than MTS –Semantics of -calculus for TMTS Weak reachability –TMTS with parameterized transitions gives tighter underapproximation –TMTS with assume-guarantee transitions for complete reasoning

9. may Modal Transition Systems underapproximation overapproximation Concrete Abstract a a’   total a a’ must    c.  (c) = a   c’.  (c’) = a’  c  c’ MAY(a,a’)MUST+(a,a’) MUST–(a,a’)      c, c’. c  c’   (c) = a   (c’) = a’   (existential abstraction) must  may underapproximation  c’.  (c’) = a’   c.  (c) = a  c  c’ onto a a’ must   [ T. Ball - FMCO’04 ] must  may must+ and must– are incomparable

10. TMTS strictly more expressive than MTS MTS may and must+ transitions precision preorder is logically characterized by PML  ::= p | AX  |   |    TMTS may, must+ and must– transitions precision preorder is logically characterized by full-PML  ::= p | AX  | AY  |   |    full-PML is strictly more expressive than PML [Pinter,Wolper - PODC’84] [Kupferman,Pnueli - LICS’95]

11. full-PML is strictly more expressive than PML pp p pp p p K1K2 unwind  = EX( (EYp)  (EY  p) ) K1   K2   PML is insensitive to unwinding no PML formula can distinguish between K1 and K2

12. TMTS: what does it buy us? Verifying specifications with past operators Reasoning about specifications in falsification setting –must+ for verification and must- for falsification Tighter weak reachability in abstract system –combine must+ and must- along the path

13. Semantics of -calculus for TMTS  : C  A (C, c 1 )   [ (A, a 1 )   ] - the value of the -calculus formula  in state a 1 of TMTS A

14. [ (A, a)   ] = T –for all concrete state c with  (c) = a, (C, c)   [ (A, a)   ] = T  –there exists a concrete state c with  (c) = a and (C, c)   [ (A, a)   ] = F –for all concrete state c with  (c) = a, (C, c)   [ (A, a)   ] = F  –there exists a concrete state c with  (c) = a and (C, c)   [ (A, a)   ] = M –there exist concrete states c and c’ such that  (c) =  (c’) = a and (C, c)   and (C, c’)   [ (A, a)   ] =  Semantics of -calculus for TMTS

15. Information Lattice TF Truth Lattice  T F 

16. Information Lattice TF Truth Lattice  T T FF M T F  FF TT M

17. [(A,a)   ] = ?[ C   ]   -1 (a) = {c1,c2,c3} {c1,c2,c3} {c1,c2}{c2,c3}{c1,c3} {c1}{c2}{c3} { } CL T = [3,0] F = [0,3]  = [?,?] F  = [?,(>0)] T  = [ (>0),?] M = [(>0),(>0)] AL Abstraction function  : P(CL)  AL Concretization function  : AL  P(CL)  (T) = { {c1,c2,c3} }  (T  ) = { {c1}, {c2}, {c3}, {c1,c2},{c2,c3},{c1,c3}, {c1,c2,c3} }  (M) = { {c1}, {c2}, {c3}, {c1,c2},{c2,c3},{c1,c3} }  (F  ) = { {}, {c1}, {c2}, {c3}, {c1,c2},{c2,c3},{c1,c3} }  (F) = { {} }

18. Truth Lattice [(A,a)   ] = ?[ C   ]   -1 (a) = {c1,c2,c3} {c1,c2,c3} {c1,c2}{c2,c3}{c1,c3} {c1}{c2}{c3} { } CL T = [3,0] F = [0,3]  = [?,?] F  = [?,(>0)] T  = [ (>0),?] M = [(>0),(>0)] AL Abstraction function  : P(CL)  AL Concretization function  : AL  P(CL) Order in the abstract lattice (induced by the concrete order and  ) :  v1, v2  AL v1  v2   (v1)   (v2) Order in the concrete powerset lattice (Hoare order with set inclusion) :  D1, D2  P(CL). D1  D2   d1  D1.  d2  D2. d1  d2

19. Truth Lattice Abstraction function  : P(CL)  AL  ( { d1,..., dk } ) =  {  (d1),...,  (dk) }  : CL  AL  (d) = [  ( |d| ),  ( |  -1 (a)| - |d| ) ]  { T, F, M }  (n)  { n = 0, n = |  -1 (a)|, 0 < n < |  -1 (a)| } –  (n) = n if n = 0 or n= |  -1 (a)| –  (n) = (>0)otherwise Join operator [ t1, f1 ]  [ t2, f2] = [ (t1 == t2) ? t1 : ?, (f1 == f2) ? f1 : ? ]

20. Semantics of  1   2 Semantics of conjunction in the concrete powerset lattice –  D1, D2  P(CL). D1  D2 = D1  D2 –D1  D2 = { d1  d2 |  d1  D1  d2  D2 } Semantics of conjunction in the abstract lattice is conservative –  v1, v2  AL.  (  (v1)   (v1) )  v1  # v2

21. Semantics of -calculus for TMTS [ (A, a)   1   2 ] [ (A, a)  EX  ] [ (A, a)   ]

22. [ (A, a)   1   2 ] = [ (A, a)   1 ]  # [ (A, a)   2 ] 6-valued Semantics of  1   2

23. ## FFF MTT T  FFFFFFF FF FFF FF FF FF FF MFFF ?FF MFF TT FFF FF ?TT  TFFF M?T   FFF FF 

24. ## FFF MTT T  FFFFFFF FF FFF FF FF FF FF MFFF ?FF MFF TT FFF FF ?TT  TFFF MTT T   FFF FF 

25. c2c2 a  1 = T   c1c1  1   2 = ?  2 = T  11 22 22 11  6-valued Semantics of  1   2 Example

26. ## FFF MTT T  FFFFFFF FF FFF FF FF FF FF MFFF ?FF MFF TT FFF FF  TT  TFFF MTT T   FFF FF  6-valued Semantics of  1   2

27. Information Lattice TF Truth Lattice  T T FF M T F  FF TT M

28. ## FFF MTT T  FFFFFFF FF FFF FF FF FF FF MFFF ?FF MFF TT FFF FF  TT  TFFF MTT T   FFF FF  6-valued Semantics of  1   2

29. ## FFF MTT T  FFFFFFF FF FFF FF FF FF FF MFFF FF FF MFF TT FFF FF  TT  TFFF MTT T   FFF FF 

30. [ (A, a)  EX  ] = Semantics of EX  F if for all a’, if may(a,a’) then [(A, a’)   ] = F T if exists a’ s.t. must+(a,a’) and [(A,a’)   ] = T T  if exists a’ s.t. must–(a,a’) and [(A,a’)   ]  T   otherwise

31. c’  a  EX  = T  a’ must–   = T   c [ (A, a)  EX  ] = T  exists a’ s.t. must–(a,a’) and [(A,a’)   ] = T  exists c’ such that  (c’)=a’ and c’   for all c’ with  (c’)=a’ there is c with  (c)=a such that c  c’ if [ (A, a)  EX  ] = T  then there exists c with  (c) = a and c  EX      EX 

32. Semantics of  The semantics of PML operators is monotonic –Least fixpoint operator can be computed by iterations from F is the usual way: –[(A,a)  Z.  (Z) ] = [ (A, a)   *(F) ]

33. The 6-valued semantics is at least as precise as the standard 3-valued semantics of -calculus for MTS [(A,a)   ] =  –3-valued abstraction refinement of must+ transitions [Shoham,Grumberg – CAV’03] adapt for must- Hypermust transitions –[Larsen,Xinxin-LICS‘90] [Shoham,Grumberg – CAV’04] –adapt for must– –MTS with hypermust+ is incomparable with TMTS  EX(x>6)  T  EX(x>6)  F  EX(x>6) = T  Semantics of -calculus for TMTS  EX(x>6) = ? must – x = 7x = 10 may x > 6 x:=x–3  789...  789

34. Semantics of -calculus for TMTS The 6-valued semantics is at least as precise as the standard 3-valued semantics of -calculus for MTS [(A,a)   ] =  –3-valued abstraction refinement of must+ transitions [Shoham,Grumberg – CAV’03] adapt for must- Hypermust transitions –[Larsen,Xinxin-LICS‘90] [Shoham,Grumberg – CAV’04] –adapt for must– –MTS with hypermust+ is incomparable with TMTS

35. Weak Reachability a’ is weakly-reachable from a  c, c’.  (c)=a   (c’)=a’  c  * c’ c  c’  a’ a initial state error state error trace Related to testing

36. L1: TF L0: FTL0: FF L2: TFL3: FTL2: FF L4: FTL4: FF L4: TF x<6x>7 (x=6)  (x=7) may must– L0: if x<6 then L1: x:= x + 3 L2: if x > 7 then L3: x :=x – 3 L4: Predicates: (x 7) Example

37. L1: TF L0: FTL0: FF L2: TFL3: FTL2: FF L4: FTL4: FF L4: TF x<6x>7 (x=6)  (x=7) may must– L0: if x<6 then L1: x:= x + 3 L2: if x > 7 then L3: x :=x – 3 L4: Predicates: (x 7) Example x = 5

38. Underapproximation of Weak Reachability if [must+]*(a,a’) then a’ is weakly reachable from a Arbitrary combinations of must+ and must– transitions do not preserve weak reachability Find a tighter underapproximation of weak-reachability

39. L1: TF L0: FTL0: FF L2: TFL3: FTL2: FF L4: FTL4: FF L4: TF x<6x>7 (x=6)  (x=7) may must– L0: if x<6 then L1: x:= x + 3 L2: if x > 7 then L3: x :=x – 3 L4: Predicates: (x 7) Example must – ? must + ? x = 9  x = 6  x = 5  x = 2 

40. Underapproximation of Weak Reachability if [must+]*(a,a’) then a’ is weakly reachable from a Arbitrary combinations of must+ and must– transitions do not preserve weak reachability Find a tighter underapproximation of weak-reachability

41. Observations a 3 is weakly reachable from a 1 if there exists a 2 such that must–(a 1,a 2 ) and must+(a 2,a 3 ) Onto nature of must– is preserved by [must-]* Total nature of must+ is preserved by [must+]* a3a3 must+ a1a1 a2a2 must–    [T.Ball – FMCO’04]

42. Underapproximation If there exists a 1, a 2, a 3 such that [must–]*(a 1,a 2 ) and [must+]*(a 2,a 3 ) then a 3 is weakly-reachable from a 1   a3a3 [must+]* a1a1 a2a2 [must–]* [T.Ball – FMCO’04]

43. L1: TF L0: FTL0: FF L2: TFL3: FTL2: FF L4: FTL4: FF L4: TF x<6x>7 (x=6)  (x=7) may must– L0: if x<6 then L1: x:= x + 3 L2: if x > 7 then L3: x :=x – 3 L4: Predicates: (x 7) Example

44.  a a’   ( total from a? ) MUST+ ? ( onto a’ ?) MUST– ? NO NO MAY Parameterized Transitions

45.  a a’   must+(  ) total from   c.  (c) = a  c     c’.  (c’) = a’  c  c’ MUST+(  )   Parameterized Transitions  a a’   must–(  ) MUST–(  )  c’.  (c’) = a’  c’     c.  (c) = a  c  c’ onto    if  is TRUE then must+(  ) is must+ and must–(  ) is must–

46. Observation a 3 is weakly reachable from a 1 if there exists a 2 such that –must–(  1 )(a 1,a 2 ) – must+(  2 ) (a 2,a 3 ) –  1   2  a 2 is satisfiable a3a3 must+(  2 ) a1a1 a2a2 must–(  1 )    11 22

47. Observation a 3 is weakly reachable from a 1 if there exists a 2 such that –must–(  1 )(a 1,a 2 ) – must+(  2 ) (a 2,a 3 ) –  1   2  a 2 is satisfiable Strongest parameters  1 and  2 a3a3 a1a1 a2a2 must–(  1 )    11 22 must+(  2 )

48.  a a’   s MUST+ ( WP(s,a’) ) Strongest Parameters Generated automatically as part of the construction of TMTS  c.  (c) = a  c     c’.  (c’) = a’  c  c’ if must+(  ) then a  (   WP(s,a’))  a a’   s MUST– ( SP (s,a) )  c’.  (c’) = a’  c’     c.  (c) = a  c  c’ if must–(  ) then a  (   SP(s,a))

49. L1: TF L0: FTL0: FF L2: TFL3: FTL2: FF L4: FTL4: FF L4: TF x<6x>7 (x=6)  (x=7) may must– L0: if x<6 then L1: x:= x + 3 L2: if x > 7 then L3: x :=x – 3 L4: Predicates: (x 7) Example SP(x:=x+3, x<6) = x < 9 WP(x:=x-3, x<6) = x < 9

50. L1: TF L0: FTL0: FF L2: TFL3: FTL2: FF L4: FTL4: FF L4: TF x<6x>7 (x=6)  (x=7) must– L0: if x<6 then L1: x:= x + 3 L2: if x > 7 then L3: x :=x – 3 L4: Predicates: (x 7) Example SP(x:=x+3, x<6) = x < 9 WP(x:=x-3, x<6) = x < 9 must–(x<9) must+(x<9)  must– (x < 9)  must+ (x < 9)

51. Tighter Underapproximation If there exists a 1,...,a 5 s.t. [must–]*(a 1,a 2 ) must–(  1 )(a 2,a 3 ) must+(  2 ) (a 3,a 4 ) [must+]*(a 4,a 5 )  1   2  a 3 is satisfiable then a 5 is weakly-reachable from a 1 a4a4 a2a2 a3a3    11 22   a5a5 a1a1 must+(  2 ) must–(  1 ) [must+]* [must–]*

52. Complete Reasoning –a’ is reachable by a certain sequence of abstract transitions from a –a’ is weakly-reachable from a Assume-guarantee transitions –another type of parameterized transitions: must+

53.  a a’   must+  c.  (c) = a  c     c’.  (c’) = a’  c’   ’  c  c’ MUST+ MUST+   Assume-Guarantee Transitions ’’ Which  and  ’ predicates do we need?  ’’ a a’      c’.  (c’) = a’  c’   ’   c.  (c) = a  c    c  c’ MUST– MUST– must–

54. The idea...    33 33    3   3 is satisfiable a4a4 a2a2 a3a3 a5a5 a1a1 s1s1 s2s2 s3s3 s4s4 must–  1 = a 1  2 = SP(s 1,  1 )  a 2  3 = SP(s 2,  2 )  a 3 must+  3 = WP(s 3,  4 )  a 3  4 = WP(s 4,  5 )  a 4  5 = a 5

55. Assume-guarantee transitions Complete Reasoning about Weak Reachability –a’ is reachable by a certain sequence of assume-guarantee transitions from a –a’ is weakly-reachable from a Finding right parameters ~ computing loop invariants

56. Weak Reachability: Summary [must–] *[must+]*must–(  1 )must+(  2 ) [must–] *[must+]* Previous work [T.Ball – FMCO’04]: Parameterized transitions Assume-guarantee transitions –complete reasoning

57. Applications Falsification of properties in CTL, LTL Abstraction-guided test generation –tighter underapproximation of weakly- reachable states improves coverage of the generated tests –example of QuickSort’s partition function

58. Predicate-Complete Testing (PCT) [T. Ball, FMCO’04] Abstract system defined by predicate abstraction Coverage: abstract state a is covered when test execution reaches some concrete state represented by a Coverage criteria ?

59. [T. Ball, FMCO’04] Abstract system defined by predicate abstraction Coverage criterion: |L| / |U| all possible states Predicate-Complete Testing (PCT) Upper bound U [may]* Reachable states Lower bound L initial states weakly-reachable states

60. Predicate-Complete Testing (PCT) [T. Ball, FMCO’04] Abstract system defined by predicate abstraction Coverage criterion: |L| / |U| Abstraction-guided test-generation strategy Tighter underapproximation of weakly-reachable states improves coverage of the generated tests

61. Example: QuickSort’s Partition Function void partition(int a[], int n) { assume(n>2); int p := a[0]; int lo := 1; int hi := n-1; L0: while (lo <= hi) { L2: while (a[lo] <= p) { L3: lo := lo + 1; } L5: while (a[hi] > p) { L6: hi := hi – 1; } if (lo < hi) { L9: swap(a,lo,hi); } LC: ; } L6:TTFT L6:FFFT LC:FFFF L3:TTTFL3:TTTT L3:FTTF L9:TTFF L3:FFTFL6:FTFT Predicates: (lo p)  1 = SP( lo:=lo+1,TTTF )  2 = WP( lo:=lo+1, FFTF)  1   2  “FTTF” = (lo=hi)  (a[lo]  p)  (a[lo-1]<p)  (a[lo+1]<p) must–(  1 ) must+(  2 ) 532

62. Example: QuickSort’s Partition Function void partition(int a[], int n) { assume(n>2); int p := a[0]; int lo := 1; int hi := n-1; L0: while (lo <= hi) { L2: while (a[lo] <= p) { L3: lo := lo + 1; } L5: while (a[hi] > p) { L6: hi := hi – 1; } if (lo < hi) { L9: swap(a,lo,hi); } LC: ; } L6:TTFT L6:FFFT LC:FFFF L3:TTTFL3:TTTT L3:FTTF L9:TTFF L3:FFTFL6:FTFT ( lo <= hi ) must–(  1 ) must+(  2 ) 532 lo p = 5 hi BOF! ! Predicates: (lo p)

63. Example: QuickSort’s Partition Function void partition(int a[], int n) { assume(n>2); int p := a[0]; int lo := 1; int hi := n-1; L0: while (lo <= hi) { L2: while (a[lo] <= p) { L3: lo := lo + 1; } L5: while (a[hi] > p) { L6: hi := hi – 1; } if (lo < hi) { L9: swap(a,lo,hi); } LC: ; } L6:TTFT L6:FFFT LC:FFFF L3:TTTFL3:TTTT L3:FTTF L9:TTFF L3:FFTFL6:FTFT Predicates: (lo p)  3 = SP( lo:=lo+1,TTTT )  4 = WP( hi:=hi-1, FFFT)  3   4  “FTFT” is unsatisfiable must–(  3 ) must+(  4 ) The path is infeasible ! must–(  3 ) is must– must+(  4 ) is must+

64. Summary Ternary Modal Transition System (TMTS) –onto and total must transitions –full-PML logical characterizes precision preorder on TMTS 6-valued semantics of -calculus for TMTS Tighten underapproximation of weak reachability with parameterized transitions –completeness result using assume-guarantee transitions