1. Localizing Errors in Counterexample Traces
From Symptom to Cause:
Localizing Errors in Counterexample Traces
Mayur Naik, Purdue University
Thomas Ball, Microsoft Research
Sriram K. Rajamani, Microsoft Research

2. Model Checking + Fully automatic: Does not require the user
to provide annotations.
+ Transparent: Produces a source-level error
trace (counterexample).
− An error trace represents a symptom of the error as opposed to its cause.
− State-of-the-art model checkers report only
one error trace.

3. The Problem
How do we localize the cause of the error in an error trace?
How do we produce multiple error traces having distinct causes?
Note: Problem is relevant to other error-detection
techniques as well.

4. What is a “Cause”?
We define a “cause” to be those parts of an error trace not contained in any correct trace.
The program fragments containing the cause are rendered unreachable.
The model checker is invoked again to produce additional error traces.

5. Example main() { AcquireLock(); if (...) ReleaseLock(); else { ... }
return;

6. Error #1: Lock acquired in succession
main() {
AcquireLock();
if (...)
ReleaseLock();
else {
... }
return;

7. Correct Trace Computation
main() {
AcquireLock();
if (...)
ReleaseLock();
else {
... }
return;

8. Error Cause Localization
main() {
AcquireLock();
if (...)
ReleaseLock();
else {
... }
return;

9. Error Recovery main() { AcquireLock(); if (...) ReleaseLock(); else {}
return;
Insert halt
Unreachable from entry of main in future runs of the model checker

10. Error #2: Lock held on exit
main() {
AcquireLock();
if (...)
ReleaseLock();
else {
halt; ... }
...
return;

11. Correct Trace Computation
main() {
AcquireLock();
if (...)
ReleaseLock();
else {
halt; ... }
...
return;

12. Error Cause Localization
main() {
AcquireLock();
if (...)
ReleaseLock();
else {
halt; ... }
...
return;

13. Error Recovery main() { AcquireLock(); if (...) ReleaseLock(); else {
halt; ... }
...
return;
Insert halt
Unreachable from entry of main in future runs of the model checker

14. Final (Error-Free) Program
main() {
AcquireLock();
if (...)
ReleaseLock();
else {
halt; ... }
return;

15. Our Results
A technique that exploits correct traces for error cause localization.
Efficient algorithm for computing correct traces.
Experimental results in the context of the SLAM toolkit.
repair

16. Transitions and Edges 1 AcquireLock(); 2 if (...) 3 ReleaseLock();
else { }
5 AcquireLock();

17. Transitions and Edges <(3, L), (5, U)> 1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
5 AcquireLock();
<(3, L), (5, U)>

18. Transitions and Edges project(<(3, L), (5, U)>) = (3, 5)
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
5 AcquireLock();
project(<(3, L), (5, U)>) = (3, 5)

19. High-Level Algorithm while true do
switch ModelCheck(G, ve) of // ve is of the form assert(e)
case FAILURE(T):
let C = GetCorrectTransitions(G, ve) and
K = project(T) \ project(C) in
if K = Ø then
break
for each (vi, vj) in K do
insert a halt statement between vi and vj
case SUCCESS:

20. Computing Correct Transitions
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
5 AcquireLock();
(1,U)
(2,L)
(3,L)
(4,L)
(5,U)
ve ≡ assert(s==U)
(5,L)
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
Reachable state-space computed by model checker
Transitions in Error Trace (T)

21. Computing Correct Transitions
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
5 AcquireLock();
(1,U)
(2,L)
(3,L)
(4,L)
(5,U)
ve ≡ assert(s==U)
(5,L)
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
Transitions in Error Trace (T)
Correct Transitions (C)

22. Computing Correct Transitions
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
5 AcquireLock();
(1,U)
(2,L)
(3,L)
(4,L)
(5,U)
ve ≡ assert(s==U)
(5,L)
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(3, L), (5, U)>
Transitions in Error Trace (T)
Correct Transitions (C)

23. Computing Correct Transitions
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
5 AcquireLock();
(1,U)
(2,L)
(3,L)
(4,L)
(5,U)
ve ≡ assert(s==U)
(5,L)
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
Transitions in Error Trace (T)
Correct Transitions (C)

24. Computing Correct Transitions
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
5 AcquireLock();
(1,U)
(2,L)
(3,L)
(4,L)
(5,U)
ve ≡ assert(s==U)
(5,L)
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
Transitions in Error Trace (T)
Correct Transitions (C)

25. Example 1: An omission error
(1,U)
(2,L)
(5,L)
(4,L)
(6,L)
(1,U)
(2,L)
(5,U)
(3,L)
(6,U)
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
6 AcquireLock();
...
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
Transitions in Error Trace (T)
Correct Transitions (C)

26. Error Cause Localization
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
6 AcquireLock();
...
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
K = project (T) \ project (C)
Transitions in Error Trace (T)
Correct Transitions (C)

27. Error Cause Localization
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
6 AcquireLock();
...
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
K = project (T) \ project (C)
Transitions in Error Trace (T)
Correct Transitions (C)

28. Error Cause Localization
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
6 AcquireLock();
...
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
K = project (T) \ project (C)
Transitions in Error Trace (T)
Correct Transitions (C)

29. Error Cause Localization
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
6 AcquireLock();
...
(1,U)
(2,L)
(4,L)
(5,L)
(6,L)
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
K = project (T) \ project (C)
= { (2, 4), (4, 5) }
Transitions in Error Trace (T)
Correct Transitions (C)

30. Experimental Results Name of driver LOC mouse packet filter 984
serial mouse port
7441
keyboard packet filter
1067
IEEE 1394 bus driver
5818
keyboard class driver
13161
i8042 port
22168
packet-based DMA
24971
serial port
30905

31. Experimental Results Name of driver LOC Number of edges in error trace
error cause
mouse packet filter
984 73
110 4
serial mouse port
7441 56 1
keyboard packet filter
1067
107
IEEE 1394 bus driver
5818 45 7 44 60 81 3 85
keyboard class driver
13161
158
i8042 port
22168
127
124 5
packet-based DMA
24971 75
serial port
30905
248

32. Experimental Results Name of driver LOC Number of edges in
Error cause localized?
error trace
error cause
mouse packet filter
984 73 No
110 4
Yes
serial mouse port
7441 56 1
keyboard packet filter
1067
107
IEEE 1394 bus driver
5818 45 7 44 60 81 3 85
keyboard class driver
13161
158
i8042 port
22168
127
124 5
packet-based DMA
24971 75
serial port
30905
248

33. Example 2: A variable-value error
(2,(S,S))
(4,(S,S))
(5,(S,F))
(1,(S,S))
(6,(S,F))
Error Trace
main() {
1 int status = S;
2 if (*)
status = foo();
else {
foo();
status = S; }
6 assert(status==x);
(1,(S,S))
(2,(S,S))
(3,(S,S))
(6,(S,S))
(6,(F,F))
Correct Traces
enum { S, F } x = S;
int foo() {
if (*)
x = S;
else
x = F;
return x; }
Program state is of the form (status, x)

34. Example 2: A variable-value error
(2,(S,S))
(4,(S,S))
(5,(S,F))
(1,(S,S))
(6,(S,F))
Error Trace
(6,(S,S))
(3,(S,S))
Correct Traces
(2,(S,S))
(4,(S,S))
(5,(S,S))
(1,(S,S))
(6,(F,F))
main() {
1 int status = S;
2 if (*)
status = foo();
else {
foo();
status = S; }
6 assert(status==x);
enum { S, F } x = S;
int foo() {
if (*)
x = S;
else
x = F;
return x; }
Program state is of the form (status, x)

35. Error Cause Localization
(2,(S,S))
(4,(S,S))
(5,(S,F))
(1,(S,S))
(6,(S,F))
Error Trace
(6,(S,S))
(3,(S,S))
Correct Traces
(2,(S,S))
(4,(S,S))
(5,(S,S))
(1,(S,S))
(6,(F,F))
main() {
1 int status = S;
2 if (*)
status = foo();
else {
foo();
status = S; }
6 assert(status==x);
enum { S, F } x = S;
int foo() {
if (*)
x = S;
else
x = F;
return x; }
K = project (T) \ project (C)
= Ø

36. High-Level Algorithm while true do
switch ModelCheck(G, ve) of // ve is of the form assert(e)
case FAILURE(T):
let C = GetCorrectTransitions(G, ve) and
K = project(T) \ project(C) in
if K = Ø then
break
for each (vi, vj) in K do
insert a halt statement between vi and vj
case SUCCESS:

37. High-Level Algorithm while true do
switch ModelCheck(G, ve) of // ve is of the form assert(e)
case FAILURE(T):
let C = GetCorrectTransitions(G, ve) and
K = project(T \ C) in
if K = Ø then
break
for each (vi, vj) in K do
insert a halt statement between vi and vj
case SUCCESS:

38. Example 2: A variable-value error
(2,(S,S))
(4,(S,S))
(5,(S,F))
(1,(S,S))
(6,(S,F))
Error Trace
(6,(S,S))
(3,(S,S))
Correct Traces
(2,(S,S))
(4,(S,S))
(5,(S,S))
(1,(S,S))
(6,(F,F))
main() {
1 int status = S;
2 if (*)
status = foo();
else {
foo();
status = S; }
6 assert(status==x);
enum { S, F } x = S;
int foo() {
if (*)
x = S;
else
x = F;
return x; }
Program state is of the form (status, x)

39. Error Cause Localization
(6,(S,S))
(3,(S,S))
Correct Traces
(2,(S,S))
(4,(S,S))
(5,(S,S))
(1,(S,S))
(6,(F,F))
main() {
1 int status = S;
2 if (*)
status = foo();
else {
foo();
status = S; }
6 assert(status==x);
(1,(S,S))
(2,(S,S))
(4,(S,S))
(5,(S,F))
(6,(S,F))
Error Trace
enum { S, F } x = S;
int foo() {
if (*)
x = S;
else
x = F;
return x; }
K = project (T \ C)
= { (4, 5), (5, 6) }

40. Example 1: An omission error
(1,U)
(2,L)
(5,L)
(4,L)
(6,L)
(1,U)
(2,L)
(5,U)
(3,L)
(6,U)
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
6 AcquireLock();
...
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
Transitions in Error Trace (T)
Correct Transitions (C)

41. Error Cause Localization
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
6 AcquireLock();
...
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
K = project (T \ C)
Transitions in Error Trace (T)
Correct Transitions (C)

42. Error Cause Localization
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
6 AcquireLock();
...
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
K = project (T \ C)
Transitions in Error Trace (T)
Correct Transitions (C)

43. Error Cause Localization
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
6 AcquireLock();
...
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
K = project (T \ C)
Transitions in Error Trace (T)
Correct Transitions (C)

44. Error Cause Localization
1 AcquireLock();
2 if (...)
ReleaseLock();
else { }
6 AcquireLock();
...
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
K = project (T \ C)
= { (2, 4), (4, 5), (5, 6) }
Transitions in Error Trace (T)
Correct Transitions (C)

45. Limitations
Control-based approach fails to localize the cause when every edge in the error trace is contained in some correct trace.
Transition-based approach localizes the cause to a suffix of the error trace.
Model Imprecision: Infeasible paths can misguide error cause localization using either approach.

46. Related Work Multiple Counterexamples Error Cause Localization
Verisim [Bhargavan et al., TSE ’02]
Error Cause Localization
Explaining counterexamples [Jin et al., TACAS ’02]
Explaining type errors [Wand, POPL ’86; Johnson & Walz, POPL ’86; Beaven & Stansifer, LOPLAS ’93; Duggan & Bent, SCP ’96; Chitil, ICFP ’01; Tip & Dinesh, TOSEM ’01]
Program Slicing [Weiser, TSE ’84]
Algorithmic Debugging [Shapiro, Ph.D. thesis ’82]
Delta Debugging [Zeller, FSE ’99]
Anomaly Detection
Static: Meta-Level Compilation [Hallem et al., PLDI ’02]
Dynamic: Daikon [Ernst et al., TSE ’01], DIDUCE [Hangal & Lam, ICSE ’02]

47. Conclusions
We have presented a technique for localizing the causes of errors in counterexample traces.
A combination of the control-based and transition-based approaches appears promising.
Our technique is quite general and should be applicable to error detection tools based on data-flow analysis as well.