1. Synergy: A New Algorithm for Property Checking
Bhargav S. Gulavani (IIT Bombay)
Yamini Kannan (Microsoft Research India)
Thomas A. Henzinger (EPFL)
Aditya V. Nori (Microsoft Research India)
Sriram K. Rajamani (Microsoft Research India)

2. Problem statement
Check if a program satisfies a given safety property:
API usage rules
Protocols on objects
Interesting programs have infinite state spaces ranging over infinite domains
This problem in general is undecidable

3. Two approaches to property checking
Testing: find inputs and executions that demonstrate effectively violations of a property
Verification: find a proof that all executions of the program satisfy a property

4. Tests: presence of bugs
void foo(int a) { 0: i = 0; 1: c = 0; 2: while (i < 1000) { 3: c = c + i; 4: i = i + 1; } 5: assume (a <= 0); 6: assert (false); ×
(a = -5) 1 × 2 × × × × 3 × × × × 4 × × × × 5 × 6 ×

5. Proofs: absence of bugs
void foo(int y1, int y2) {
0: state = 1;
1: if (y1) {
2: x0 = x0 + 1; }
else {
3: x0 = x0 – 1;
4: if (y2) {
5: x1 = x1 + 1;
6: x1 = x1 – 1;
7: assert (state == 1);
exponential number of tests required
O: state=1
1: state=1
2: state=1
3: state=1
4: state=1
linear proof exists!
5: state=1
6: state=1
7: state=1
Error

6. Key insights
Testing works when errors are easy to find and is inefficient for finding proofs
Verification works when proofs are easy to find and is inefficient for finding errors

7. Questions Can we combine “systematically” testing with verification?
How does one generate/direct test cases?
Can abstraction help?
Given a spurious abstract error trace, how does one perform refinement?
Can testing help?

8. Solution: Synergy
Combines under- and over-approximation reasoning (testing and verification) of programs.
Unifies several disparate existing algorithms in the literature:
Counterexample driven refinement approaches for verification (SLAM, BLAST)
Directed testing approaches (DART)
Partition refinement algorithms (Lee-Yannakakis, Paige-Tarjan)

9. Construct random tests Construct initial proof
Synergy – sketch
Input:
Program P
Property ψ
Construct random tests
Construct initial proof
Test succeeded?
yes
Bug! no
Proof succeeded?
yes
Proof! no
τ = error path in failed proof
f = frontier of error path
Can extend test beyond frontier?
yes no
Refine proof

10. Example void foo(int y) { 1: do { 2: lock(); 3: x = y; 4: if (*) {
Does this program obey the locking rule?
void foo(int y) {
1: do {
2: lock();
3: x = y;
4: if (*) {
5: unlock();
6: y = y + 1; }
7: } while (x != y);
8: unlock();

11. Example void foo(int y) { 1: do { 2: lock(); 3: x = y; 4: if (*) {
5: unlock();
6: y = y + 1; }
7: } while (x != y);
8: unlock();

12. Construct random tests Construct initial proof
Example
Input:
Program P
Property ψ
Construct random tests
Construct initial proof
void foo(int y) {
1: do {
2: lock();
3: x = y;
4: if (*) {
5: unlock();
6: y = y + 1; }
7: } while (x != y);
8: unlock();
Test succeeded?
yes
Bug! no
Proof succeeded?
yes
Proof! no
τ = error path in failed proof
f = frontier of error path
Can extend test beyond frontier?
yes no
Refine proof

13. Construct random tests Construct initial proof
Example
Can extend test beyond frontier?
Refine proof
Construct random tests
Construct initial proof
Input:
Program P
Property ψ
Test succeeded?
Bug!
Proof succeeded?
τ = error path in failed proof
f = frontier of error path
yes no
Proof! ×
y = 1 1 ×
void foo(int y) {
1: do {
2: lock();
3: x = y;
4: if (*) {
5: unlock();
6: y = y + 1; }
7: } while (x != y);
8: unlock(); × 2 × × 3 × × 4 × × 5 7 × × × 6 8 × × 9

14. Construct random tests Construct initial proof
Example
Can extend test beyond frontier?
Refine proof
Construct random tests
Construct initial proof
Input:
Program P
Property ψ
Test succeeded?
Bug!
Proof succeeded?
τ = error path in failed proof
f = frontier of error path
yes no
Proof! ×
y = 1 1 ×
void foo(int y) {
1: do {
2: lock();
3: x = y;
4: if (*) {
5: unlock();
6: y = y + 1; }
7: } while (x != y);
8: unlock(); × 2 × × 3 × × 4 × × 5 7 × × × 6 8 × ×
τ=(0,1,2,3,4,7,8,9) 9
frontier

15. Construct random tests Construct initial proof
Example
Can extend test beyond frontier?
Refine proof
Construct random tests
Construct initial proof
Input:
Program P
Property ψ
Test succeeded?
Bug!
Proof succeeded?
τ = error path in failed proof
f = frontier of error path
yes no
Proof! × 1 ×
void foo(int y) {
1: do {
2: lock();
3: x = y;
4: if (*) {
5: unlock();
6: y = y + 1; }
7: } while (x != y);
8: unlock(); × 2 × × 3 × × 4 × × 5 7 × × × 6
8⋀p
8⋀¬p × × ×
split <pc=8> into two regions wrt p= (lock.state != L) 9

16. Construct random tests Construct initial proof
Example
Can extend test beyond frontier?
Refine proof
Construct random tests
Construct initial proof
Input:
Program P
Property ψ
Test succeeded?
Bug!
Proof succeeded?
τ = error path in failed proof
f = frontier of error path
yes no
Proof! × 1 ×
void foo(int y) {
1: do {
2: lock();
3: x = y;
4: if (*) {
5: unlock();
6: y = y + 1; }
7: } while (x != y);
8: unlock(); × 2 × × 3 × × 4 × × 5 7 × × × 6
8⋀p
8⋀¬p × × ×
τ=(0,1,2,3,4,7,<8,p>,9) 9
frontier

17. Correct, the program is × × void foo(int y) × { 1: do { 2: lock();× 1 ×
void foo(int y) {
1: do {
2: lock();
3: x = y;
4: if (*) {
5: unlock();
6: y = y + 1; }
7: } while (x != y);
8: unlock(); × 2 × × 3 × ×
4⋀¬s
4⋀s × ×
5⋀¬s
5⋀s ×
6⋀¬r
6⋀r ×
7⋀¬q
7⋀q × ×
8⋀p
8⋀¬p 9 ×

18. Construct random tests Construct initial proof
Example
Can extend test beyond frontier?
Refine proof
Construct random tests
Construct initial proof
Input:
Program P
Property ψ
Test succeeded?
Bug!
Proof succeeded?
τ = error path in failed proof
f = frontier of error path
yes no
Proof! 6 5 3 2 4 1
void foo(int a) {
0: i = 0;
1: c = 0;
2: while (i < 1000)
3: c = c + i;
4: i = i + 1; }
5: if (a <= 0)
6: error();

19. Construct random tests Construct initial proof
Example
Can extend test beyond frontier?
Refine proof
Construct random tests
Construct initial proof
Input:
Program P
Property ψ
Test succeeded?
Bug!
Proof succeeded?
τ = error path in failed proof
f = frontier of error path
yes no
Proof! 6 5 3 2 4 1 ×
a = 45 ×
void foo(int a) {
0: i = 0;
1: c = 0;
2: while (i < 1000)
3: c = c + i;
4: i = i + 1; }
5: if (a <= 0)
6: error(); × × × × × × × × × ×

20. Construct random tests Construct initial proof
Example
Can extend test beyond frontier?
Refine proof
Construct random tests
Construct initial proof
Input:
Program P
Property ψ
Test succeeded?
Bug!
Proof succeeded?
τ = error path in failed proof
f = frontier of error path
yes no
Proof! 6 5 3 2 4 1 × ×
void foo(int a) {
0: i = 0;
1: c = 0;
2: while (i < 1000)
3: c = c + i;
4: i = i + 1; }
5: if (a <= 0)
6: error(); × × × × × × × × × ×
τ=(0,1,2,(3,4,2)1000,5,6)
frontier

21. Construct random tests Construct initial proof
Example
Can extend test beyond frontier?
Refine proof
Construct random tests
Construct initial proof
Input:
Program P
Property ψ
Test succeeded?
Bug!
Proof succeeded?
τ = error path in failed proof
f = frontier of error path
yes no
Proof! 6 5 3 2 4 1 × ×
void foo(int a) {
0: i = 0;
1: c = 0;
2: while (i < 1000)
3: c = c + i;
4: i = i + 1; }
5: if (a <= 0)
6: error(); × × × × × × × × × ×
a = -5 ×

22. Soundness and Termination
Theorem: Suppose we run Synergy on any program P= ⟨Σ, σI, →⟩ and property ψ
If Synergy returns (“pass”, ≃), then the abstract program P≃ = ⟨Σ≃, σI≃, →≃⟩ simulates P, and thus is a proof that P does not reach ψ
If Synergy returns (“fail”, t), then t is an error trace
Theorem: If P= ⟨Σ, σI, →⟩ has a finite bisimulation quotient, then Synergy terminates
Theorem: Synergy terminates in strictly more cases than the Lee-Yannakakis algorithm

23. Implementation
Synergy is the core engine of a property checking tool called Yogi
Checks x86 binaries against safety properties
Implemented in F# over MSR program analysis infrastructure

24. Experimental Evaluation
program (correct)
Yogi
SLAM LY
iterations
time (secs)
test1.c 9
3.92 4
1.7 *
test2.c 6
7.88
1.55
test3.c 5
2.19 13
8.03
test4.c 2
2.67 12
3.52 22
8.08
test5.c
1.28 1
0.9
test6.c
1.45
1.27
1.75
test7.c
2.11
1.11
2.06
test8.c
1.19
test9.c 3
1.39
1.42
test10.c
1.52
1.25
test11.c
1.30
5.03
test12.c 7
2.30
10.25
test13.c
3.17
1.31
3.18
test14.c
1.06
3.45
test15
5.98
5.65
test16.c
9.2
test17.c
2.28
test18.c 24
13.41
test19.c
10.84
test20.c
9.42

25. The Future
Only integer domains with linear arithmetic have been considered
Incorporate more expressive domains
New techniques for discovering predicates
Interprocedural analysis
Check properties of device drivers
A more comprehensive analysis of combining over– and under-approximations of programs