BugHint: A Visual Debugger Based on Graph Mining Jennifer Leopold, Nathan Eloe, and Patrick Taylor

Outline Background and related work - Debugging with beginning programmers - Visualization in debugging BugHint - Algorithm - Graphic user interface Experiments - Human test study and analysis of results Future work Questions and comments

Debugging with Beginning Programmers Syntax, editor, compiler, operating system commands, … Overwhelming!!! Last thing students want to learn is a sophisticated debugger like GDB

Debugging with Beginning Programmers However, some intro programming classes have successfully incorporated debuggers into the teaching of various constructs Ex: Easier to see what is going on in a loop

Visualization in Debugging Visualization facilitates understanding of code and data structures Java debugger jBixbe

Our Motivation Create an easy-to-use debugging tool that would give a beginning programmer a hint about where the bug might be in his/her program

BugHint: The Algorithm Input: A program (in C or C++) At least 1 test case that produces correct results At least 1 test case that produces incorrect results Output: The line(s) in the code that are likely causing the bug

BugHint: The Algorithm How the analysis is done: (1) Construct a control flow graph (CFG) of the input program node = basic block of code edge = flow from one block to another Simple example: if (x != y) // line 1 { x = y; // line 2 y = x; // line 3 } cout << x << y; // line 4 B1 B2 B4 B1 = line 1 B2 = lines 2, 3 B4 = line 4

BugHint: The Algorithm How the analysis is done: (2) Construct a CFG for each input test case Case No. x y result 1 5 x = 5, y = 5 correct 2 4 incorrect 3 x = 4, y = 4 Case 1 Case 2 Case 3 B1 B4 B1 B2 B4 B1 B2 B4 B1 = line 1 B2 = lines 2, 3 B4 = line 4

BugHint: The Algorithm How the analysis is done: (3) Remove non-discriminant edges (and nodes) Case No. x y result 1 5 x = 5, y = 5 correct 2 4 incorrect 3 x = 4, y = 4 Case 1 Case 2 Case 3 B1 B4 B1 B2 B4 B1 B2 B4 B1 = line 1 B2 = lines 2, 3 B4 = line 4

BugHint: The Algorithm How the analysis is done: (4) Find the largest connected subgraph that exists in the (majority of) the incorrect graphs and doesn’t exist in the (majority of) the correct graphs Case No. x y result 1 5 x = 5, y = 5 correct 2 4 incorrect 3 x = 4, y = 4 Case 1 Case 2 Case 3 B1 = line 1 B2 = lines 2, 3 B4 = line 4 empty B2 B2

BugHint: The Algorithm How the analysis is done: This corresponds to the block of code where the bug likely occurs In this example, the swap is not being done correctly Simple example: if (x != y) // line 1 { x = y; // line 2 y = x; // line 3 } cout << x << y; // line 4 B1 B2 B4 B1 = line 1 B2 = lines 2, 3 B4 = line 4

BugHint: The Algorithm Some cases are more difficult to analyze! Example: if (x <= y) // line 1 min = y; // line 2 else min = y; // line 3 cout << min; // line 4 B1 B2 B3 B4 B1 = line 1 B2 = line 2 B3 = line 3 B4 = line 4 Case 1 Case 2 Case 3 Case No. x y result 1 5 min = 5 correct 2 4 min = 4 3 incorrect B1 B2 B4 B1 B3 B4 B1 B2 B4

BugHint: The Algorithm Some cases are more difficult to analyze! Example: if (x <= y) // line 1 min = y; // line 2 else min = y; // line 3 cout << min; // line 4 B1 B2 B3 B4 B1 = line 1 B2 = line 2 B3 = line 3 B4 = line 4 Case 1 Case 2 Case 3 Case No. x y result 1 5 min = 5 correct 2 4 min = 4 3 incorrect B1 B2 B4 B1 B3 B4 B1 B2 B4

BugHint: The Algorithm Relax the rule that the subgraph in the incorrect case(s) does not occur in all correct case(s) Example: if (x <= y) // line 1 min = y; // line 2 else min = y; // line 3 cout << min; // line 4 Conclude that the bug is in B2, which is line 2 (should be min = x;) Case 1 Case 2 Case 3 Case No. x y result 1 5 min = 5 correct 2 4 min = 4 3 incorrect B2 B3 B2 B1 = line 1 B2 = line 2 B3 = line 3 B4 = line 4

BugHint: The Algorithm How the analysis is done: Sometimes it will be the opposite situation (i.e., it will be code that is executed in the correct cases that is not executed in the incorrect cases) Sometimes no hint can be given because the control path will be the same for any inputs Ex: int sum = 5, total = 10; float avg; avg = sum / total; // gives 0, not 0.5

BugHint: The GUI Clicking on a block in a trace will highlight node and lines in code Lines that are the bug hint Can step through a selected test case execution Trace for a selected test case Specify test cases Input values for selected test case Bug hint

Experiments Wanted to test the hypothesis that BugHint would improve debugging skill 163 students who had just completed CS1 Treatment group: intro to BugHint, pre-training exercises with it Control group: tips on manual debugging, pre-training exercises using those tips Both groups then given programs with a bug and asked to fix it; treatment group given a bug hint

Experiments Wanted to test the hypothesis that BugHint would improve debugging skill Treatment group: Better able to come up with additional test cases Better able to add comments to the code they were debugging Said it was easier to find and fix the bug Didn’t take significantly less time to find and fix the bug than the control group

Future Work Add ability to debug user-defined functions Prioritize subgraphs that contain statements with multiple operators and/or particular operators (e.g., && and ||) More usability and usefulness studies, including scalability

Questions? Comments? Ideas?