1. BugHint: A Visual Debugger Based on Graph Mining
Jennifer Leopold, Nathan Eloe, and Patrick Taylor

2. Outline Background and related work
- Debugging with beginning programmers
- Visualization in debugging
BugHint
- Algorithm
- Graphic user interface
Experiments
- Human test study and analysis of results
Future work
Questions and comments

3. Debugging with Beginning Programmers
Syntax, editor, compiler, operating system commands, …
Overwhelming!!!
Last thing students want to learn is a sophisticated debugger like GDB

4. Debugging with Beginning Programmers
However, some intro programming classes have successfully incorporated debuggers into the teaching of various constructs
Ex: Easier to see what is going on in a loop

5. Visualization in Debugging
Visualization facilitates understanding of code and data structures
Java debugger jBixbe

6. Our Motivation
Create an easy-to-use debugging tool that would give a beginning programmer a hint about where the bug might be in his/her program

7. BugHint: The Algorithm
Input:
A program (in C or C++)
At least 1 test case that produces correct results
At least 1 test case that produces incorrect results
Output:
The line(s) in the code that are likely causing the bug

8. BugHint: The Algorithm
How the analysis is done: (1) Construct a control flow graph (CFG) of the input program node = basic block of code edge = flow from one block to another
Simple example:
if (x != y) // line 1 {
x = y; // line 2
y = x; // line 3 }
cout << x << y; // line 4 B1 B2 B4
B1 = line 1
B2 = lines 2, 3
B4 = line 4

9. BugHint: The Algorithm
How the analysis is done: (2) Construct a CFG for each input test case
Case No. x y
result 1 5
x = 5, y = 5
correct 2 4
incorrect 3
x = 4, y = 4
Case 1
Case 2
Case 3 B1 B4 B1 B2 B4 B1 B2 B4
B1 = line 1
B2 = lines 2, 3
B4 = line 4

10. BugHint: The Algorithm
How the analysis is done: (3) Remove non-discriminant edges (and nodes)
Case No. x y
result 1 5
x = 5, y = 5
correct 2 4
incorrect 3
x = 4, y = 4
Case 1
Case 2
Case 3 B1 B4 B1 B2 B4 B1 B2 B4
B1 = line 1
B2 = lines 2, 3
B4 = line 4

11. BugHint: The Algorithm
How the analysis is done: (4) Find the largest connected subgraph that exists in the (majority of) the incorrect graphs and doesn’t exist in the (majority of) the correct graphs
Case No. x y
result 1 5
x = 5, y = 5
correct 2 4
incorrect 3
x = 4, y = 4
Case 1
Case 2
Case 3
B1 = line 1
B2 = lines 2, 3
B4 = line 4
empty B2 B2

12. BugHint: The Algorithm
How the analysis is done: This corresponds to the block of code where the bug likely occurs In this example, the swap is not being done correctly
Simple example:
if (x != y) // line 1 {
x = y; // line 2
y = x; // line 3 }
cout << x << y; // line 4 B1 B2 B4
B1 = line 1
B2 = lines 2, 3
B4 = line 4

13. BugHint: The Algorithm
Some cases are more difficult to analyze!
Example:
if (x <= y) // line 1
min = y; // line 2
else
min = y; // line 3
cout << min; // line 4 B1 B2 B3 B4
B1 = line 1
B2 = line 2
B3 = line 3
B4 = line 4
Case 1
Case 2
Case 3
Case No. x y
result 1 5
min = 5
correct 2 4
min = 4 3
incorrect B1 B2 B4 B1 B3 B4 B1 B2 B4

14. BugHint: The Algorithm
Some cases are more difficult to analyze!
Example:
if (x <= y) // line 1
min = y; // line 2
else
min = y; // line 3
cout << min; // line 4 B1 B2 B3 B4
B1 = line 1
B2 = line 2
B3 = line 3
B4 = line 4
Case 1
Case 2
Case 3
Case No. x y
result 1 5
min = 5
correct 2 4
min = 4 3
incorrect B1 B2 B4 B1 B3 B4 B1 B2 B4

15. BugHint: The Algorithm
Relax the rule that the subgraph in the incorrect case(s) does not occur in all correct case(s)
Example:
if (x <= y) // line 1
min = y; // line 2
else
min = y; // line 3
cout << min; // line 4
Conclude that the bug is in B2, which is line 2
(should be min = x;)
Case 1
Case 2
Case 3
Case No. x y
result 1 5
min = 5
correct 2 4
min = 4 3
incorrect B2 B3 B2
B1 = line 1
B2 = line 2
B3 = line 3
B4 = line 4

16. BugHint: The Algorithm
How the analysis is done:
Sometimes it will be the opposite situation (i.e., it will be code that is executed in the correct cases that is not executed in the incorrect cases)
Sometimes no hint can be given because the control path will be the same for any inputs
Ex: int sum = 5, total = 10;
float avg;
avg = sum / total; // gives 0, not 0.5

17. BugHint: The GUI
Clicking on a block in a trace will highlight node and lines in code
Lines that are the bug hint
Can step through a selected test case execution
Trace for a selected test case
Specify test cases
Input values for selected test case
Bug hint

18. Experiments
Wanted to test the hypothesis that BugHint would improve debugging skill
163 students who had just completed CS1
Treatment group: intro to BugHint, pre-training exercises with it
Control group: tips on manual debugging, pre-training exercises using those tips
Both groups then given programs with a bug and asked to fix it; treatment group given a bug hint

19. Experiments
Wanted to test the hypothesis that BugHint would improve debugging skill
Treatment group:
Better able to come up with additional test cases
Better able to add comments to the code they were debugging
Said it was easier to find and fix the bug
Didn’t take significantly less time to find and fix the bug than the control group

20. Future Work Add ability to debug user-defined functions
Prioritize subgraphs that contain statements with multiple operators and/or particular operators (e.g., && and ||)
More usability and usefulness studies, including scalability

21. Questions? Comments? Ideas?