240-222 Computer Programming Techniques Semester 1, 1998 7. Testing and Debugging Objective of these slides: to talk about the various approaches to testing and debugging code.

Overview: 1. Testing 2. Debugging 3. Types of Errors 4. Compile Time Errors 5. Before Debugging 6. The Debugging Process 7. Debugging Advice 8. Check List 9. Common Bugs 10. Runtime Debugging Aids

1. Testing Testing is a systematic process that attempts to increase your confidence that a program is correct. "Testing can only show the presence of bugs, not prove their absence." Two kinds: 1.1. Black-box Testing 1.2. Glass-box Testing

1.1. Black-box Testing Consider only the input/output behaviour of each function. Devise test data based on specification. easy values (trivial cases) typical values (easily checkable by hand) extreme values (limits in specification) illegal values (check reliability)

Black-box Test Example Consider a program to print a summary of occurrences of words in a text file. Data cases: empty file file with no words file with one word file with one word many times file with same word twice on line small files of random words large text where results known

1.2. Glass-box Testing Examine the internal structure of functions. Excellent method for debugging small functions. Example switch (a) { case 1: x = 3; break; case 2: if (b = 0) x = 2; else x = 4; break; case 3: while (c > 0) process(c); break; }

2. Debugging Debugging is the process of locating and removing errors. It has been estimated that 85% of debugging time is spent locating errors and only 15% spent fixing them.

3. Types of Errors compile-time errors run-time crashes incorrect results

4. Compile Time Errors Use the -Wall option of gcc: gcc -Wall -o example example.c Extra warnings include: not using a locally defined variable not declaring a function

5. Before Debugging To make the debugging process easier: Use existing (tested) code. Develop the program in a modular fashion (first debug functions separately and then debug combination). continued

Develop functions systematically one at a time remember dependencies for ordering Document your code. Program for clarity: “Will I understand this in two weeks time?”

6. The Debugging Process 1. Identify/describe the error. 2. Gather data about program behaviour. 3. Guess/locate what caused the error. 4. Test your guess. 5. If guess was not correct, go to step 2.

7. Debugging Advice Keep a record of debugging. Save a copy of your program before you attempt to fix it. Restore the saved program if the fix does not work. Make sure you are using a listing of the current version of the program. continued

Learn from negative results (eliminate possible error sources). Don't assume that: the computer is at fault the system software is faulty Compare the program with its comments. continued

Find the simplest data that causes the error. Compare data that gives right answers with data that gives wrong answers. Look for a pattern. Find the simplest data that causes the error. Execute your program by hand on the wrong data. continued

Insert diagnostic printf() statements (more on this later). Only use a few printf() statements for debugging at any one time. Find the earliest point at which a variable contains a wrong value or control reaches the wrong part of the program. continued

Copy program; simplify program to simplest form that gives the error. Keep good/long test cases in files for easy reuse. Explain your program to someone else. Use stubs (dummy functions). continued

Stub Example #include <stdio.h> float complex_input(void); void process(float val); int main() { float val; val = complex_input(); process(val); return 0; } continued

float complex_input(void) / float complex_input(void) /* Rather than do complex input, return a typical value. This function is a stub. */ { return 1.11111111; } void process(float val) /* Process input This function is being tested. */ { : /* actual code */ }

8. Check List Are all comments terminated? Have all variables been initialized? Does every loop terminate? Are function parameters correctly defined? Are parentheses where you want them?

9. Common Bugs Array index out of bounds Unintended side effects e.g. assignment to globals, a = b instead of a == b Use of uninitialised variables case fall-through in a switch continued

Pointers (memory leaks) Reversed logic in conditions e.g. x < 5 rather than x > 5 Macros (not included in this course, since they are so tricky to use)

10. Run-time Debugging Aids 10.1. Where to put printf()s? 10.2. Conditional Compilation 10.3 assert 10.4. Source-level Debugging

10.1. Where to put printf()s? At the start of functions print input parameters At the return points of functions print output parameters and return value At the start of each loop printf()s must identify themselves printf()s should be conditional

10.2. Conditional Compilation It is tiresome to keep adding and removing comments around printfs. Conditional compilation: selectively compile portions of the program select which printfs to compile using a flag

Example #include <stdio.h> #define TEST int foo(int); int main() { int n = 1; : #ifdef TEST printf("Reached main(); n = %d\n", n); #endif : foo(n); : } compiled, depending on the presence of TEST continued

compiled, depending on the presence of TEST int foo(int n) { n = n + 2; #ifdef TEST printf("Reached foo(); n = %d\n", n); #endif : }

Easier Switching Remove the #define line, and define TEST as part of the compilation: gcc -DTEST -o examp examp.c This is the same as having #define TEST inside examp.c

Fancier #if #include <stdio.h> #define TESTFLAG 2 int main() { int n1 = 1, n2 = 2, n3 = 3; : #if TESTFLAG == 1 printf("In main(); n1 = %d\n", n1); #elif TESTFLAG == 2 printf("In main(); n2 = %d\n", n2); #else printf("In main(); n3 = %d\n", n3); #endif : foo(n1); }

10.3. assert For testing the truth of assertions about the program. assert calls can include any C expression: assert(x ==y); assert(ptr != NULL); If the evaluation is 0 (false) at runtime, an error message is produced, and execution is aborted.

Example #include <stdio.h> #include <assert.h> int main() { int x = 9, y = 8; assert(x == (y+1)); foo(x, y); assert(x > y); bar(y, x); return 0; } continued

Only use asserts for debugging. If NDEBUG is defined (either using #define, or -D), then the asserts are ignored. Only use asserts for debugging.

10.4. Source-level Debugging Run your program within a debugging environment, where its execution can be examined as it progresses. Popular UNIX tools: gdb dbx sdb, adb

10.4.1. gdb Information on gdb: gdb is the interactive source-level debugger from the GNU Project. Information on gdb: man gdb gdb -help ginfo (not supported on calvin)

gdb Overview a) Main Features b) How to Use gdb c) Strategies for Use d) Basic gdb Commands e) lowercase.c f) Execution g) Using gdb on lowercase

10.4.1.a. Main Features source-level breakpoints source line stepping monitoring variables analysing core dumps

10.4.1.b. How to Use gdb 1) Compile with the -g option of gcc: $ gcc -g -o examp examp.c 2) Execute the object code within gdb: $ gdb examp

10.4.1.c. Strategies for Use For non-terminating programs For programs that give wrong answers

For Programs that Don't Terminate 1) Start the code: run 2) Wait; interrupt execution: ctrl-c 3) Find location: where list <function name> print <variable name> 4) Step through the code: step

For Programs that Give Wrong Answers 1) Think about the error. 2) Put traces and/or breakpoints into the code. 3) Run the code. 4a) Examine trace output. 4b) At each breakpoint, step through the code. continued

5) Print variables and list code often: print p.data print account[] list foo 6) If the code looks okay, continue execution. 7) Delete traces and/or breakpoints that are no longer needed.

10.4.1.d. Basic gdb Commands 1. Help: help help <command> help <command class> continued

2. Break points: break <function> break point at start of function break <line> break point at line watch <variable> break point when variable is changed ctrl-c interrupt execution continued

3. Manipulating traces and break points: info break display the sequence numbers of the active break points delete <seq number> remove break point with that number continued

4. Displaying: list <function> list function list list next 10 lines list <line1>, <line2> list the range of lines print <variable> print the variable's value where display current calls continued

5. Moving on: step execute the next source line next step but proceed through a function call cont continue execution run begin execution of program quit quit gdb

10.4.1.e. lowercase.c /* convert the input to lowercase */ #include <stdio.h> #include <ctype.h> #define SIZE 1024 void lower(char lin[]); int main() { char line[SIZE]; while (gets(line) != NULL) { lower(line); puts(line); } return 0; } continued

void lower(char lin[]) /. convert any uppercase letters to lowercase void lower(char lin[]) /* convert any uppercase letters to lowercase */ { int count = 0; while (lin[count] != '\0') if (isupper(lin[count])) { lin[count] = tolower(lin[count]); count++; } }

10.4.1.f. Execution $ gcc -o lowercase lowercase.c $ lowercase GGGGG ggggg ggg $ Run ‘suspended’; I had to type ctrl-c

10.4.1.g. Using gdb on lowercase $ gcc -g -o lowercase lowercase.c $ gdb lowercase What's the error?

catsix{26}% gcc -g -o lowercase lowercase catsix{26}% gcc -g -o lowercase lowercase.c catsix{27}% gdb lowercase GDB is free ... (gdb) run Starting program: /home/ad/temp/lowercase GGGGG ggggg ggg Program received signal SIGINT, Interrupt. 0x8000573 in lower (lin=0xbffff84c "ggg") at lowercase.c:28 28 while(lin[count] != '\0') Run ‘suspended’; I had to type ctrl-c continued

(gdb) where #0 0x8000573 in lower (lin=0xbffff84c "ggg") at lowercase (gdb) where #0 0x8000573 in lower (lin=0xbffff84c "ggg") at lowercase.c:28 #1 0x800052c in main () at lowercase.c:16 #2 0x80004a4 in ___crt_dummy__ () #3 0x9d in ?? () Cannot access memory at address 0x33. (gdb) list lower 20 } 21 22 23 24 void lower(char lin[]) 25 { 26 int count = 0; 27 28 while(lin[count] != '\0') 29 if (isupper(lin[count])) { continued

(gdb) list 28 23 24 void lower(char lin[]) 25 { 26 int count = 0; 27 28 while(lin[count] != '\0') 29 if (isupper(lin[count])) { 30 lin[count] = tolower(lin[count]); 31 count++; 32 } continued

(gdb) step 29 if (isupper(lin[count])) { (gdb) print count $1 = 0 (gdb) print lin $2 = 0xbffff84c "ggg" (gdb) print lin[0] $3 = 103 'g' (gdb) step 32 } continued

(gdb) step 28 while(lin[count] (gdb) step 28 while(lin[count] != '\0') (gdb) step 29 if (isupper(lin[count])) { (gdb) step 32 } (gdb) print count $4 = 0 continued

(gdb) step 28 while(lin[count] (gdb) step 28 while(lin[count] != '\0') (gdb) step 29 if (isupper(lin[count])) { (gdb) step 32 } (gdb) print count $5 = 0 (gdb) quit The program is running. Quit anyway (and kill it)? (y or n) y catsix{29}%

gdb with Breakpoints catsix{30}% gdb lowercase GDB is free ... (gdb) break lower Breakpoint 1 at 0x8000566: file lowercase.c, line 26. (gdb) run Starting program: /home/ad/temp/lowercase GGGGG continued

Breakpoint 1, lower (lin=0xbffff84c "GGGGG") at lowercase Breakpoint 1, lower (lin=0xbffff84c "GGGGG") at lowercase.c:26 26 int count = 0; (gdb) cont Continuing. ggggg ggg continued

Breakpoint 1, lower (lin=0xbffff84c "ggg") at lowercase Breakpoint 1, lower (lin=0xbffff84c "ggg") at lowercase.c:26 26 int count = 0; (gdb) cont Continuing. Program received signal SIGINT, Interrupt. 0x8000596 in lower (lin=0xbffff84c "ggg") at lowercase.c:29 29 if (isupper(lin[count])) { (gdb) quit The program is running. Quit anyway (and kill it)? (y or n) y catsix{31}% Run ‘suspended’; I had to type ctrl-c