“White box” or “glass box” tests

Basic goal of white box testing developer client user Test planning White box testing--”lowest level”: Part of a comprehensive test plan—see fig. 11.1 in text Unit tests User test activities Client test activities Integration tests Structure tests Functional tests Performance tests

General method 1: Path testing or basis path testing basic goal: make sure all paths in the control structure of a unit are tested at least once why? 1. "bad code" is more likely in rarely executed paths 2. what we think is a "rare" path may turn out to be taken a lot 3. syntax errors can occur anywhere

White box testing--example integer a,b,count=0; input a,b; if (a == 0) while (b > 0) {b = b-1; count++;} else if (a > 0) while (b < 0) {b = b+1; count--;} else a = b; output a,b,count; What (minimal) set of tests would make sure that all paths are tested in this code? Are there other test cases that would be advisable?

White box testing requirements what needs to be tested: 1. all independent paths must be followed at least once 2. all logical decisions must have both true and false input 3. all loops must be executed at boundaries and within their bounds 4. all internal data structures must be used

node represents one or more statements Basis path method This method guarantees that every statement will be executed at least once Flow graph: node represents one or more statements edges represent control flow, as in flowcharts Example: flow graphs for control structures: Until Sequential If Case While

Basis path method (continued) Determine number of tests needed: develop a "flow graph" G --make graph for basic control structure --sequential statements with no branches in or out can be merged into one statement compute the "cyclotomic complexity”: V(G) = E - N + 2, where E = #edges, N = #nodes or V(G) = number of planar regions defined by the graph V(G) is an upper bound on the number of "independent" paths through the code which must be followed through an appropriate choice of test cases in order to execute every statement at least once NOTE: this gives the number of paths for the GRAPH; program structure may mean a smaller number of paths is actually needed

Basis path method--example Flow chart: 1 Example: 2 integer a,b,count=0; //1 input a,b; //2 if (a == 0) //3 while (b > 0) //4 {b = b-1; //5 count++;} //6 else if (a > 0) //7 while (b < 0) //8 {b = b+1; //9 count--;} //10 else //11 a = b; //12 output a,b,count; //13 3 4 7 Flow graph: 5 6 8 11 E=11,N=8; E-N+2=5 5 paths 9 12 10 1,2,3 * * 4 7 13 * 11,12 5,6 9,10 8 Tests: a b 0 2: path 1-3,4,5-6,4 0 -1: path 1-3,4,13 1 -1: path 1-3,7,8,9-10,8 1 2: path 1-3,7,8,13 -1 -1: path 1-3,11-12,13 13 * 1 edge, multiple paths

Basis path method—6 steps 2. Construct flow chart: 1 1. Number lines of executable code: integer a,b,count=0; //1 input a,b; //2 if (a == 0) //3 while (b > 0) //4 {b = b-1; //5 count++;} //6 else if (a > 0) //7 while (b < 0) //8 {b = b+1; //9 count--;} //10 else //11 a = b; //12 output a,b,count; //13 2 3 4 7 5 6 8 11 9 12 3. Construct flow graph: 10 11,12 1,2,3 5,6 4 13 7 13 5. Determine tests, paths (paths: see previous slide): a 0 0 1 1 -1 b 2 -1 -1 2 -1 9,10 8 Note; treat a function call (including a recursive call) as ONE statement, don’t expand 4. Compute upper bound on number of paths: E=11,N=8; E-N+2=5; 5 paths 6. Add additional tests (e.g., let b=0)

Basis path method--augmentations Basis path method can also be used with a “graph matrix” to automate the generation of test cases (Beizer, Software Testing Techniques, 1990): Example: use the flow graph to define a graph matrix: Entries can be, for example: --Edges (1 or 0) --Probability of execution --Time to traverse an edge --Memory needed to traverse edge --Resources needed to traverse edge Values can be used to determine what test cases should be run 1 1 2 3 4 5 5 2 3 4

General method 2: based on definition / redefinition of variables: Data flow testing General method 2: based on definition / redefinition of variables: data flow testing: focuses on program variables x---e.g., require that every definition use chain (DU) be covered at least once DU chains:for each program statement S def(S)={x|x defined in S} and use(S)={x|x is used in S} example: if S is the statement a = b + c then def(S) = {a}; use(S) = {b,c} x at S is live at S':there is a path from S to S' in which x is not redefined

def(S) = {b}; def(S') = {c}; def(S'') = {a}; def (S''') = {d} Def{S}, use{S} Example: b = 3; //S c = 4; //S' a = b + c; //S'' d = a + b; //S''' def(S) = {b}; def(S') = {c}; def(S'') = {a}; def (S''') = {d} use(S) = f; use(S') = f; use(S'')= {b,c}; use(S''')={a,b} a at S'' is live at S'''; b at S is live at S',S'', and S'''; c at S' is live at S'';

definition use chain (DU) of x is [x,S,S'] DU chains definition use chain (DU) of x is [x,S,S'] where x is in def(S) and use(S') and def. of x in S is live at S' so in this example: b = 3; //S c = 4; //S' a = b + c; //S'' d = a + b; //S''' DU chains are: a: [a, S'', S'''] b: [b, S,S''] and [b,S,S'''] c: [c, S', S''] testing: must cover every DU chain at least once

another simple example: 0. input j,k; 1. x = j; y = k; DU chains--example another simple example: 0. input j,k; 1. x = j; y = k; 2. if (x > 0) 3. y = y + 2; 4. x = x + 5; 5. endif 6. z = x + y + 4; DU chains: 1-[j,0,1]; 2-[k,0,1]; 3-[x,1,2],4-[x,1,4],5-[x,1,6],6-[x,4,6] 7-[y,1,3],8-[y,1,6],9-[y,3,6] test cases: j = 4, k = 5;--covers 1,2,3,4,5,7,8 and j = -4, k = 5;--covers 6,9

General method 3: state-based testing (text, p. 461): Works well for object-oriented designs Derives tests from component’s associated state machine diagram Each transition in the diagram must be traversed at least once

UnitControl process using a state machine State-based testing (continued) Example: Finite-state machine (FSM) model—what state transitions must be tested? Idle GoingUp req > floor req < floor !(req > floor) !(timer < 10) DoorOpen GoingDn u,d,o, t = 1,0,0,0 u,d,o,t = 0,0,1,0 u,d,o,t = 0,1,0,0 u,d,o,t = 0,0,1,1 u is up, d is down, o is open req == floor !(req<floor) timer < 10 t is timer_start UnitControl process using a state machine From Vahid/Grivargis, Embedded System Design, 2000

Polymorphism testing Polymorphism testing: Must be sure to identify and test all possible bindings, static and dynamic Example: fig. 11-15 Network interface Ethernet WaveLAN UMTS

Additional white box testing some methods to broaden the test coverage: condition testing: add tests for each value of a given condition (e.g., if (a == c and b == d) ) domain testing: for each relational expression (a relational-operator b) generate tests for cases: a < b, a = b, a > b loop testing: simple loops (max # of iterations = n): skip it one pass two passes m passes, m < n n-1, n, n+1 passes (8 <= # passes <= 4n+8)

Additional white box testing (continued) nested loops: tests can grow geometrically (= Q (n2)) some reductions may be used (see p. 458, Pressman) concatenated loops: if loops are not independent, should treat as nested loops unstructured loops: avoid these in your design and code Unstructured loops (to be avoided) Nested loops Concatenated loops