1. Lazy Annotation for Program Testing and Verification (Supplementary Materials) Speaker: Chen-Hsuan Adonis Lin Advisor: Jie-Hong Roland Jiang December 3, 2010 1

2. Outline How to compute Interpolants of program sequence Concolic Approach (without learning) Dart: Directed Automated Random Testing December 3, 2010 2

3. Strongest and Weakest Interpolants If I and I′ are both interpolants for (F,G), then so are I ∧ I′ and I ∨ I′ Let F ∧ G be unsatisfiable. The strongest interpolant for (F, G), denoted SI (F, G), is the unique interpolant for (F, G) that implies any other interpolant. The weakest interpolant for (F,G), denoted WI(F,G), is the unique interpolant that is implied by any other interpolant SI (F, G) implies WI (F, G) December 3, 2010 3

4. Interpolants of Sequences We want to handle program paths, therefore a generalization of interpolant is needed. Given a sequence of formulas Γ = A 1,A 2,…,A n, we say that Ā 0, Ā 1,…, Ā n is in an interpolant for Γ when: Ā 0 = TRUE and Ā n = FALSE, For all 1≤i≤n, Ā i-1 ∧ A i implies Ā i, and For all 1≤i≤n, Ā i is in L(A 1,…,A i ) ∩ L(A i+1,…,A n ) If Γ is quantifier-free we can derive a quantifier- free interpolant for Γ (from the refutation of Γ ) December 3, 2010 4

5. Interpolants for Sequences (con’t) An intuition: So this is a structured refutation of A 1, …, A k (Ā i ∧ A i+1 ) implies Ā i+1 December 3, 2010 5 A1A1 A2A2 A3A3 AkAk... Ā1Ā1 Ā2Ā2 Ā3Ā3 Ā k-1... TrueFalse 

6. Iterative Computation of Interpolants Given a formula F = F 1 ∧... ∧ F n, determine whether F is unsatisfiable, and if so, find interpolants for the pairs (F..i, F i+1.. ), i ∈ {1,...,n}, where F..i := F 1 ∧... ∧ F i and F i+1.. :=F i+1 ∧... ∧ F n Each formula F i models a program instruction A formula F = F 1 ∧... ∧ F n models a trace through a program In order to check if the trace is feasible or spurious, one can check if F is satisfiable or unsatisfiable December 3, 2010 6

7. Iterative Computation of Interpolants (con’t) Definition (Tracking Property) Let F 1 ∧... ∧ F n be unsatisfiable, and let K i be interpolants for (F..i, F i+1.. ). We say that the family {K i } satisfies the tracking property if ( K i ∧ F i+1 |= K i+1 ) Proposition: Let F 1 ∧ F 2 ∧... ∧ F n be unsatisfiable. Let {I i } and {J i } be families of predicates defined according to the following procedures: I 0 := true, I i+1 := any interpolant for (I i ∧ F i+1, F i+2.. ), where i=0,..., n−1 J n := false, J i−1 := any interpolant for (F..i−1, ¬(F i → J i )), where i=n,..., 1 {I i } and {J i } are interpolants for (F..i,F i+1.. ) and satisfy the tracking property December 3, 2010 7

8. Iterative Computation of Interpolants (con’t) Ｉ nterpolants satisfying the tracking property “explain” the infeasibility of a trace by providing Hoare annotations Evaluate strongest interpolants (I i ) {true} X := true {X} Y := X {X ∧ Y } assume(¬Y ∧ Z) {false}   Evaluate weakest interpolants (J i ) {true} X := true {X ∨ ¬Z} Y := X {Y ∨ ¬Z} assume(¬Y ∧ Z) {false}  By definition, I i |= J i ; Ex: (X ∧ Y) |= (Y ∨ ¬Z) December 3, 2010 8

9. Iterative Computation of Interpolants (con’t) Evaluate strongest interpolants (I i ) {true} X := true {X} Y := X {X ∧ Y } assume(¬Y ∧ Z) {false Evaluate weakest interpolants (J i ) {true} X := true {X ∨ ¬Z} Y := X {Y ∨ ¬Z} assume(¬Y ∧ Z) {false} Intuitively, the strongest interpolants at node n records all facts that are established by the path leading up to n Ex: the strongest interpolant at node 2 is {X ∧ Y} Intuitively, the weakest interpolant at n represents the disjunction of all conditions that make the trace infeasible if they hold at n Ex: the weakest interpolant at node 2 is {Y ∨ ¬Z} December 3, 2010 9

10. Outline How to compute Interpolants of program sequence Concolic Approach (without learning) Dart: Directed Automated Random Testing December 3, 2010 10

11. Motivation of software testing Today, QA is mostly testing “50% of my company employees are testers, and the rest spends 50% of their time testing!” -- Bill Gates 1995 December 3, 2010 11

12. Concolic Approach Combine concrete and symbolic execution for unit testing (Concrete + Symbolic = Concolic) DART: Directed Automated Random Testing Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation Authors Patrice Godefroid (Bell Labs) Nils Klarlund (Bell Labs) Koushik Sen (CS, UIUC) December 3, 2010 12

13. Example (C code) int double(int x) { return 2 * x; } void test_me(int x, int y) { int z = double(x); if (z==y) { if (y == x+10) abort(); /* error */ } (1) Interface extraction: parameters of toplevel function external variables return values of external functions main(){ int tmp1 = randomInt(); int tmp2 = randomInt(); test_me(tmp1,tmp2); } (2) Generation of test driver for random testing: Problem: probability of reaching abort() is extremely low! December 3, 2010 13

14. DART: Directed Search main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if (z==y) { if (y == x+10) abort(); /* error */ } Concrete Execution Symbolic Execution Path Constraint x = 36, y = 99 create symbolic variables x, y December 3, 2010 14

15. DART: Directed Search main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if (z==y) { if (y == x+10) abort(); /* error */ } Concrete Execution Symbolic Execution Path Constraint create symbolic variables x, y x = 36, y = 99, z = 72 z = 2 * x December 3, 2010 15

16. DART: Directed Search main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if (z==y) { if (y == x+10) abort(); /* error */ } Concrete Execution Symbolic Execution Path Constraint create symbolic variables x, y x = 36, y = 99, z = 72 z = 2 * x 2 * x != y Solve: 2 * x == y Solution: x = 1, y = 2 December 3, 2010 16

17. DART: Directed Search main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if (z==y) { if (y == x+10) abort(); /* error */ } Concrete Execution Symbolic Execution Path Constraint x = 1, y = 2 create symbolic variables x, y December 3, 2010 17

18. DART: Directed Search main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if (z==y) { if (y == x+10) abort(); /* error */ } Concrete Execution Symbolic Execution Path Constraint create symbolic variables x, y x = 1, y = 2, z = 2 z = 2 * x December 3, 2010 18

19. DART: Directed Search main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if (z==y) { if (y == x+10) abort(); /* error */ } Concrete Execution Symbolic Execution Path Constraint create symbolic variables x, y x = 1, y = 2, z = 2 z = 2 * x 2 * x == y December 3, 2010 19

20. DART: Directed Search main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if (z==y) { if (y == x+10) abort(); /* error */ } Concrete Execution Symbolic Execution Path Constraint create symbolic variables x, y 2 * x == y x = 1, y = 2, z = 2 z = 2 * x y != x + 10 Solve: (2 * x == y) Æ (y == x +10) Solution: x = 10, y = 20 December 3, 2010 20

21. DART: Directed Search main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if (z==y) { if (y == x+10) abort(); /* error */ } Concrete Execution Symbolic Execution Path Constraint x = 10, y = 20 create symbolic variables x, y December 3, 2010 21

22. DART: Directed Search main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if (z==y) { if (y == x+10) abort(); /* error */ } Concrete Execution Symbolic Execution Path Constraint create symbolic variables x, y x = 10, y = 20, z = 20 z = 2 * x December 3, 2010 22

23. DART: Directed Search main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if (z==y) { if (y == x+10) abort(); /* error */ } Concrete Execution Symbolic Execution Path Constraint create symbolic variables x, y x = 10, y = 20, z = 20 z = 2 * x 2 * x == y December 3, 2010 23

24. DART: Directed Search main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int double(int x) {return 2 * x; } void test_me(int x, int y) { int z = double(x); if (z==y) { if (y == x+10) abort(); /* error */ } Concrete Execution Symbolic Execution Path Constraint create symbolic variables x, y 2 * x == y y == x +10 z = 2 * x x = 10, y = 20, z = 20 Program Error December 3, 2010 24

25. Concolic Testing: A Middle Approach + Complex programs + Efficient - Less coverage + No false positive - Simple programs - Not efficient + High coverage - False positive Random Testing Symbolic Testing Concolic Testing + Complex programs +/- Somewhat efficient + High coverage + No false positive December 3, 2010 25

26. Limitations: A Comparative View Concolic: Broad, shallowRandom: Narrow, deep December 3, 2010 26

27. Hybrid Concolic Testing Interleave Random Testing and Concolic Testing to increase coverage Deep, broad, hybrid Search December 3, 2010 27

28. Thanks for your attention December 3, 2010 28