1. Aditya V. Nori, Sriram K. Rajamani Microsoft Research India

2.  An industrial strength program verifier  Idea: Synergize verification and testing  Synergy [ FSE ’06 ], Dash [ ISSTA ‘08 ], SMASH [ POPL ‘10 ] algorithms to perform scalable analysis  Engineered a number of optimizations for scalability  Integrated with Microsoft’s Static Driver Verifier ( SDV ) toolkit and used internally

4.  Overview of Yogi  Overview of optimizations  Evaluation setup  Empirical Results  Summary

5. void foo() { *p = 4; *q = 5; if (condition) error(); } Question Is error() unreachable for all possible inputs? Verification: can prove the absence of bugs, but can result in false errors Testing: finds bugs, but can’t prove their absence

6. no Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes Proof! yes Input: Program P Property ψ

7. no Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes Proof! yes Input: Program P Property ψ void foo(int y) { 0: int x, lock = 0; 1: do { 2: lock = 1; 3: x = y; 4: if (*) { 5: lock = 0; 6: y = y+1; } 7: } while (x != y); 8: if (lock != 1) 9: error(); 10: } y = 1 0 1 2 3 4 5 6 7 8 9 × × × × × × × × × × × × × × 10 × Symbolic execution + Theorem proving

8. no Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes Proof! yes Input: Program P Property ψ void foo(int y) { 0: int x, lock = 0; 1: do { 2: lock = 1; 3: x = y; 4: if (*) { 5: lock = 0; 6: y = y+1; } 7: } while (x != y); 8: if (lock != 1) 9: error(); 10: } 0 1 2 3 4 5 6 7 8:ρ 9 × × × × × × × × × × × × × × 10 × 8:¬ρ ×

9. no Can extend test beyond frontier? Refine abstraction Construct initial abstraction Construct random tests Test succeeded? Bug! Abstraction succeeded? τ = error path in abstraction f = frontier of error path yes no yes Proof! yes Input: Program P Property ψ void foo(int y) { 0: int x, lock = 0; 1: do { 2: lock = 1; 3: x = y; 4: if (*) { 5: lock = 0; 6: y = y+1; } 7: } while (x != y); 8: if (lock != 1) 9: error(); 10: } 0 1 2 3 4:¬s 5:¬s 6:¬r 9 × × × × × × × ×× × × 7:¬q × 8:¬p × 4:s 5:s 6:r 7:q 8:p × 10

10.  Initial abstraction from property predicates  Relevance heuristics for predicate abstraction  Suitable predicates ( SP )  Control dependence predicates ( CD )  Interprocedural analysis  Global modification analysis  Summaries for procedures  Thresholds for tests  Fine tuning environment models

11.  Benchmarks:  30 WDM drivers and 83 properties ( 2490 runs)  Anecdotal belief: most bugs in the tools are usually caught with this test suite  Presentation methodology:  Group optimizations logically such that related optimizations are in the same group  Total time taken, total number of defects found for every possible choice of enabling/disabling each optimization in the group

12. state { enum {Locked = 0, Unlocked = 1} state = Unlocked; } KeAcquireCancelSpinlock.Entry { if (state != Locked) { state = Locked; } else abort; } KeReleaseCancelSpinlock.Entry { if (state == Locked) { state = Unlocked; } else abort; } 0 1 0 1 0 1

13. Abstraction using SLIC predicates Total time (minutes) #defects#timeouts yes 216024177 no 258024186 16%

14. A C B D A C B D C Irrelevant?

15.  Abstract assume statements that are not potentially relevant by skip statements  If Yogi proves that the program satisfies property, we are done.  Otherwise, validate the error trace and refine the abstraction by putting back assume statements, if the error trace is spurious

16. int x; void foo() { bool protect = true; … if (x > 0) protect = false; … if (protect) KeAcquireCancelSpinLock(); for (i = 0; i < 1000; i++) { a[i] = readByte(i); } if (protect) KeReleaseCancelSpinLock(); } A C B D A C B D C

17. int x; void foo() { bool protect = true; … if (x > 0) protect = false; … if (protect) KeAcquireCancelSpinLock(); for (i = 0; i < 1000; i++) { a[i] = readByte(i); } if (protect) KeReleaseCancelSpinLock(); } A C B D A C B D C

18. int x; void foo() { bool protect = true; … if (x > 0) protect = false; … if (protect) KeAcquireCancelSpinLock(); for (i = 0; i < 1000; i++) { a[i] = readByte(i); } if (protect) KeReleaseCancelSpinLock(); }

19. SP heuristic CD heuristic Total time (minutes) #defects#timeouts yes 216024177 yes no258023991 no yes240023887 no 2894235174 10%

20. SP heuristic CD heuristic Total time (minutes) #defects#timeouts yes 216024177 yes no258023991 no yes240023887 no 2894235174 16%

21. SP heuristic CD heuristic Total time (minutes) #defects#timeouts yes 216024177 yes no258023991 no yes240023887 no 2894235174 25%

23. A C B D A C B D C foo(…)

24. Modification analysis SummariesTotal time (minutes) #defects#timeouts yes 216024177 yes no2760239109 no yes3180237134 no 3780236165 32%

25. Modification analysis SummariesTotal time (minutes) #defects#timeouts yes 216024177 yes no2760239109 no yes3180237134 no 3780236165 28%

26. Modification analysis SummariesTotal time (minutes) #defects#timeouts yes 216024177 yes no2760239109 no yes3180237134 no 3780236165 42%

28. Test threshold Total time (minutes) #defects#timeouts 250260023692 500216024177 1000235924088 1500240023989

29. if (DestinationString) { DestinationString->Buffer = SourceString; // DestinationString->Length should be set to the // length of SourceString. The line below is missing // from the original stub SDV function DestinationString->Length = strlen(SourceString); } if (SourceString == NULL) { DestinationString->Length = 0; DestinationString->MaximumLength = 0; } Issue type#issues Integers used as pointers 8 Uninitialized variables 15 Type inconsistencies 9

30.  Described optimizations implemented in Yogi  Evaluated optimizations on the WDM test suite  Empirical data used to decide which optimizations to include in Yogi  We believe that this detailed empirical study of optimizations will enable tool builders to decide which optimizations to include and how to engineer their tools  http://research.microsoft.com/yogi http://research.microsoft.com/yogi