White Box Testing

Agenda White-box vs Black-box Program Flow Controls White-box Test Methods Exercises Complexity Q&A

What is White-box Testing? Looking at the internal structure of a program and deriving test cases based on the logic or control flow. Test cases can be designed to reach every branch in the code and to exercise each condition Typically done during unit testing Also known as: –Structural Testing –Glass-Box Testing

What is Black-box Testing? Looking at the program from an external point of view and deriving test cases based on the specification. The only criteria upon which the program is judged is if it produces the correct output for a given input.

Why Do Both? Black-box Impossible to write a test case for every possible set of inputs and outputs Some of the code may not be reachable without extraordinary measures Specifications are not always complete White-box Does not address the question of whether or not the program matches the specification Does not tell you if all of the functionality has been implemented Does not discover missing program logic

Basic Program Flow Controls IF IF-Then-Else FOR While Do-While Case

IF Diagram

IF-THEN-ELSE Diagram

FOR or WHILE Diagram

DO-WHILE Diagram

CASE Diagram

Example Code Fragment

Example Control Flow Graph Source: The Art of Software Testing – Glenford Myers

Exercise #1 int main (int argc, char *argv[]) { /* Process CTRL-C Interrupts */ signal(SIGINT,catcher); if (validate_command_line(argc)) return(1); if (job_initialize(argv)) return(1); processTestList(); /* Process all testCases in TestList */ displayResults(); fprintf(stdout,"Test Complete\n"); job_termination(); return(0); } /* main */ Can you diagram this code?

Did You Get Something Like This?

Exercise #2 /* Attempt Statusreadback -log SRB data to logFile */ int process_srb(void) { int srb_count = 0; do { srb_count = read_printer(srb_in); } while (!srb_count); fprintf(logFile,"%s\n",srb_in); return(srb_count); } /* process_srb */ /* Write String to Printer via Parallel or Serial port */ void write_printer (char *outputline) { if (strstr(printertype,"PAR") != NULL) { bwrite_parST(printerport,outputline); } else if (strstr(printertype,"SER") != NULL) { bwwrite_serial(printerport,outputline,strlen(outputline)); } else if (strstr(printertype, "FILE") !=NULL) { fprintf(printerFile,"%s",outputline); } } /* write_printer */ Can you diagram this code?

Did You Get Something Like This?

White-box Test Methods Statement Coverage Decision/Branch Coverage Condition Coverage Decision/Condition Coverage Path Coverage

Example Code Fragment If ((A>1) & (B=0)) then Do; X=X/A; END; If ((A==2) | (X>1)) then Do; X=X+1; END; Source: The Art of Software Testing – Glenford Myers

Statement Coverage Exercise all statements at least once How many test cases?  A=2 and B=0 (ace) Source: The Art of Software Testing – Glenford Myers

Decision/Branch Coverage Each decision has a true and a false outcome at least once How many test cases?  A=2 and B=0 (ace)  A=1 and X=1 (abd) Source: The Art of Software Testing – Glenford Myers

Condition Coverage Each condition in a decision takes on all possible outcomes at least once Conditions: A>1, B=0, A=2, X>1 How many test cases?  A=2, B=0, and X=4 (ace)  A=1, B=1, and X=1 (abd) Source: The Art of Software Testing – Glenford Myers

Decision/Condition Coverage Each condition in a decision takes on all possible outcomes at least once, and each decision takes on all possible outcomes at least once How many test cases?  A=2, B=0, and X=4 (ace)  A=1, B=1, and X=1 (abd) What about these?  A=1, B=0, and X=3  A=2, B=1, and X=1 (abe)

Multiple Condition Coverage Exercise all possible combinations of condition outcomes in each decision Conditions: Source: The Art of Software Testing – Glenford Myers A>1, B=0 A>1, B<>0 A<=1, B=0 A 0 A=2, X>1 A=2, X<=1 A<>2, X>1 A<>2, X<=1

Multiple Condition Coverage How many test cases?  A=2, B=0, X=4  A=2, B=1, X=1  A=1, B=0, X=2  A=1, B=1, X=1 Source: The Art of Software Testing – Glenford Myers (ace) (abe) (abd)

Path Coverage Every unique path through the program is executed at least once How many test cases?  A=2, B=0, X=4 (ace)  A=2, B=1, X=1 (abe)  A=3, B=0, X=1 (acd)  A=1, B=1, X=1 (abd) Source: The Art of Software Testing – Glenford Myers

McCabe’s Cyclomatic Complexity Software metric Developed by Tom McCabe (circa 1976) Directly measures the number of linearly independent paths through a program’s source code, taking into account the various decision points Independent of implementation language Source: Wikipedia

Calculating Complexity

How Complex Should Code Be? <10: Simple module, not much risk 10-20: More Complex; moderate risk 20-50: Complex; high risk >50: Untestable; extremely high risk Source: Carnegie Mellon Software Engineering Institute

Complexity Caveats As code is broken into smaller modules to decrease cyclomatic complexity, structural complexity increases Some modules may have high complexity but are very easy to comprehend and easy to test High complexity numbers are only an indicator of something to investigate