Reference Paulo Alencar, University of Waterloo Frank Tsui, Southern Polytechnic State University

2 Overview  Structural Testing  Introduction – General Concepts  Flow Graph Testing  DD-Paths  Test Coverage Metrics  Basis Path Testing  Guidelines and Observations  Data Flow Testing  Hybrid Methods  Retrospective on Structural Testing

3 Structural Testing Also known as glass box, structural, clear box and open box testing. A software testing technique whereby explicit knowledge of the internal workings of the item being tested are used to select the test data. Unlike black box testing that is using the program specification to examine outputs, white box testing is based on specific knowledge of the source code to define the test cases and to examine outputs.

4 Structural Testing Structural testing methods are very amenable to: –Rigorous definitions Data flow, control flow, objectives, coverage criteria, relation to programming language semantics –Mathematical analysis Graphs, path analysis –Precise measurement Metrics, coverage analysis

7 DD-Paths More formally a DD-Path is a chain obtained from a program graph such that: –Case1: it consists of a single node with indeg=0. –Case2: it consists of a single node with outdeg=0, –Case3: it consists of a single node with indeg ≥ 2 or outdeg ≥ 2 –Case4: it consists of a single node with indeg =1, and outdeg = 1 –Case5: it is a maximal chain of length ≥ 1

Case1: it consists of a single node with indeg=0. Case2: it consists of a single node with outdeg=0, Case3: it consists of a single node with indeg ≥ 2 or outdeg ≥ 2 Case4: it consists of a single node with indeg =1, and outdeg = 1 Case5: it is a maximal chain of length ≥ 1

Path Analysis Why path analysis for test case design? –Provides a systematic methodology of White Box testing. Reproducible Traceable Countable What is path analysis? –Analyzes the number of paths that exist in the system –Facilitates the decision process of how many paths to include in the test Reasons similar to why we document test cases Path, as previously defined in Graph Theory, is a sequence of connected nodes

Linearly Independent Path A path through the system is Linearly Independent** from other paths only if it includes some segment or some edge that is not covered in the other path. S1 S3 S2 C The statements are represented by the rectangular and diamond blocks. - The segments between the blocks are the edges, labeled with numbered circles. Path1 : S1 – C1 – S3 Path2 : S1 – C1 – S2 – S3 OR Path1: edges (1,4) Path2: edges (1,2,3) Path1 and Path2 are linearly independent because each includes some edge that is not included in the other. (note: not necessarily nodes) ** This definition will require more explanation later.

Another Example of Linearly Independent Paths S1 S2 S3 S4 S5 C1 C2 C Path1: edges (1,2,8) Path2: edges (1,5,3,9) Path3: edges (1,5,6,4,10) Path4: edges (1,5,6,7) Note that these are all linearly independent

Statement Coverage Method Count all the linearly independent paths Pick the minimum number of linearly independent paths that will include all the statements (S’s and C’s in the diagram) S1 S3 S2 C Path1 : S1 – C1 – S3 : edges (1, 4) Path2 : S1 – C1 – S2 – S3 : edges (1, 2. 3 ) Are both Path1 and Path2 needed to cover all the statements: (S1,C1,S2,S3) ? What would you say?

13 Statement Coverage Testing Statement coverage based testing aims to devise test cases that collectively exercise all statements in a program. foo(1,1)

Another Example of Statement Coverage S1 S2 S3 S4 S5 C1 C2 C The 4 Linearly Independent Paths Covers: Path1: includes S1-C1-S2-S5 : edges (1,2,8) Path2: includes S1-C1-C2-S3-S5 : edges ( 1,5,3,9) Path3: includes S1-C1-C2-C3-S4-S5 : edges (1,5,6,4,10) Path4: includes S1-C1-C2-C3-S5 : edges (1,5,6,7) For 100% Statement Coverage, all we need are 3 paths : Path1, Path2, and Path3 to cover all the statements (S1,C1,S2,C2,S3,C3,S4,S5) no need for Path !!

Statement Coverage Now What do you think of a Software Company which boasts that they test every statement in the software before release? Try this on some students or other professors and see their reaction.

Branch Coverage Method (Also DD-Paths of our text book) 1.Identify all the decisions 2.Count all the branches from each of the decisions ( out-degree of the decision node) 3.Pick the minimum number of paths that will cover all the branches from the decisions.

Branch Coverage Method S1 S3 S2 C One decision C1 : B1 : Path1 : C1 – S3 B2 : Path2 : C1 – S2 Are both Path1 and Path2 needed to cover Branch1 and Branch2 from C1? Branch 1 Branch 2 2 linearly independent paths cover : B1 : Path1 : C1 – S3: B2 : Path2 : C1 – S2 – S3 :

Another Example of Branch Coverage S1 S2 S3 S4 S5 C1 C2 C The 3 Decisions and the Branches: C1: - B1 : C1- S2 - B2 : C1- C2 C2: - B3 : C2 – S3 - B4 : C2 – C3 C3: - B5 : C3 – S4 - B6 ; C3 – S5 We need: Path1 to cover B1, Path2 to cover B2 and B3, Path3 to cover B4 and B5, Path4 to cover B6 The 4 Linearly Independent Paths Cover : Path1: includes S1-C1-S2-S5 Path2: includes S1-C1-C2-S3-S5 Path3: includes S1-C1-C2-C3-S4-S5 Path4: includes S1-C1-C2-C3-S5

19 foo(1,1) and foo(0,1)

Branch Coverage How do you feel about a Software Company who states that they test every branch in the Software before release? Does All Branch Coverage subsume All Statements Coverage?

McCabe’s Cyclomatic Number Is there a way to know how many linearly independent paths exist? –McCabe’s Cyclomatic number which we used to study program complexity may be applied. There are 3 ways to get the Cyclomatic Complexity number from a flow diagram. # of binary decisions + 1 # of edges - # of nodes + 2 # of closed regions + 1

McCabe’s Cyclomatic Complexity Number Earlier Example S1 S3 S2 C We know there are 2 linearly independent paths from before: Path1 : C1 – S3 Path2 : C1 – S2 – S3 McCabe’s Cyclomatic Number: a) # of binary decisions +1 = 1 +1 = 2 b) # of edges - # of nodes +2 = = 2 c) # of closed regions + 1 = = 2 Closed region

McCabe’s Cyclomatic Complexity Number Another Example S1 S3 S2 C McCabe’s Cyclomatic Number: a) # of binary decisions +1 = 2 +1 = 3 b) # of edges - # of nodes +2 = = 3 c) # of closed regions + 1 = = 3 Closed Region C2 S Closed Region There are 3 Linearly Independent Paths

An example of 2 n total path S1 C1 C2 C3 S2 S3 S4S Since for each binary decision, there are 2 paths and there are 3 in sequence, there are 2 3 = 8 total “logical” paths path1 : S1-C1-S2-C2-C3-S4 path2 : S1-C1-S2-C2-C3-S5 path3 : S1-C1-S2-C2-S3-C3-S4 path4 : S1-C1-S2-C2-S3-C3-S5 path5 : S1-C1-C2-C3-S4 path6 : S1-C1-C2-C3-S5 path7 : S1-C1-C2-S3-C3-S4 path8 : S1-C1-C2-S3-C3-S5 How many Linearly Independent paths are there? Using Cyclomatic number = 3 decisions +1 = 4 One set would be: path1 : includes edges (1,2,4,6,9) path2 : includes edges (1,2,4,6,8) path3 : includes edges (1,2,4,5,7,8) path5 : includes edges (1,3,5,7,8) Note 1: with just 2 paths ( Path1 and Path8) all the statements are covered. Note2: with just 2 paths ( Path1 and Path8) all the branches are covered.

Example with a Loop S1 S2 S3 C Total number of ‘logical’ paths may be “infinite” (very large) because of the loop Linearly Independent Paths = 1 decision +1 = 2 path1 : S1-C1-S3 (segments 1,4 ) path2 : S1-C1-S2-C1-S3 (segments 1,2,3,4 ) “One path” will cover all statements: path2 : S1-C1-S2-C1-S3 (S1,C1,S2,S3) “One path” will cover all branches: path2 : S1-C1-S2-C1-S3 branch1 (C1-S2) and branch 2 (C1-S3) *** Note***: To get one path --- S2 must somehow change C1

Example with a Loop (cont.) S1 S2 S3 C Total number of ‘logical’ paths may be “infinite” (very large) because of the loop Linearly Independent Paths = 1 decision +1 = 2 path1 : S1-C1-S3 (segments 1,4 ) path2 : S1-C1-S2-C1-S3 (segments 1,2,3,4 ) “One path” will cover all branches: path2 : S1-C1-S2-C1-S3 branch1 (C1-S2) and branch 2 (C1-S3) Should we consider the loop boundaries in C1? & also bring in our boundary value testing thoughts?

More on Linearly Independent Paths In discussing dimensionality, we talks about orthogonal vectors (or “basis” of vector space). –(e.g.) Two dimensional space has two orthogonal vector from which all the other vectors in two dimension can be obtained via “linear combination” of these 2 vectors: [1,0] [0,1] [1,0] [0,1] [2,4] e.g. [2,4] = 2[1,0] + 4[0,1]

More on Linearly Independent Paths A set of paths is considered to be a Linearly Independent Set if every path may be constructed as a “linear combination” of paths from the linearly independent set. For example: C1 C2 S1 S We already know: a) there are a total of 2 2 =4 logical paths. b) 1 path (path4) will cover all statements c) 2 paths will cover all branches. d) 2 branches +1 = 3 linearly independent paths. path1 path2 path3 path path 4 = path3 + path1 – path2 = (0,1,1,1,0,0)+(1,0,0,0,1,1)- (1,0,0,1,0,0) = (1,1,1,1,1,1) - (1,0,0,1,0,0) = (0,1,1,0,1,1) We pick: path1, path2 and path3 as the Linearly Independent Set logical paths

Other Sets of Linearly Independent Paths -Consider the set of linearly independent paths: 1, 2 & 4 instead. - Can we get Path3 = (0,1,1,1,0,0)? -Consider path3 = path2 + path4 – path1 = (1,0,0,1,0,0,)+(0,1,1,0,1,1) – (1,0,0,0,1,1) = (0,1,1,1,0,0) path1 path2 path3 path Consider another set of linearly independent paths: 2, 3 & 4 instead. - Can we get Path 1 = (1,0,0,0,1,1))? -Consider path1 = path2 + path4 – path3 = (1,0,0,1,0,0,)+(0,1,1,0,1,1) – (0,1,1,1,0,0) = (1,0,0,0,1,1)

More on Linearly Independent Paths C1 C2 S1 S path1 path2 path3 path Note : Although path1 and path3 are linearly independent, they do NOT form a Linearly Independent Set because no linear combination of path1 and path3 can get, say, path4. We already know: a) there are a total of 2 2 =4 logical paths. b) 1 path (path4) will cover all statements c) 2 paths will cover all branches. d) 2 branches +1 = 3 linearly independent paths.

More on Linearly Independent Paths Because the Linearly Independent Set of paths display the same characteristics as the mathematical concept of basis in n-dimensional vector space, the testing using the Linearly Independent Set of paths is sometimes called the “basis” testing. The main notion is that since the “linear independent set of paths” as a set can span all the paths for the design/code construct, then basis testing covers the “essence” of the whole structure. Then is there a way to find a Linearly Independent Set ?

An Algorithm to Find the Basis Set 1.Select a baseline path, an arbitrary, normal execution path that contains as many decisions as possible. 2.Retrace the baseline path and “flip” each of the decision encountered; each flip creates a new path. Continue until all the decisions are flipped 3.The basis set is composed of all the paths generated from steps 1 and 2 above C1 C2 S1 S pick baseline path P1: C1 –S1- C2 – S2: 2. flip C1 P2 : C1 – C2 – S2 : 3. flip C2 P3: C1 - C2 : Cyclomatic # = 2+ 1 = 3; So there are 3 linearly independent paths 5 Can we get the 4 th path : C1 – S1 – C2 : from the above basis set? How about : (P1 + P3) – P2 ? (P1 + P3) – P2 = ( + ) - = - = = P4

Total Possible Logical Paths can be Big! s1 c1 s2 c3 s3 c2 c4 s There are 5 choices each time we process through this loop. For passing through the loop n times we have 5 n possibilities of logical paths. If we go through the loop just 3 times, we have (5) 3 = 125 possible paths! You also note that for n loops there are possible 3n decisions, and we may have 3*4 = 12 decisions for looping 4 times.

Paths Analysis Summary Interested in total number of all possible combinations of “logical” paths Interested in Linearly Independent paths Interest in Branch coverage or DD-path Interested in Statement coverage Which one do you think is the largest set, next largest, - - -, etc.? Compare this list with page 136 of your text what are we missing?

Definition for DD-path DD (decision-decision) path is a path of nodes in a directed graph. A chain is a path in which: –Initial and terminal nodes are distinct (not just one node) –All interior nodes have in-degree = 1 and out-degree =1 A DD-path is a chain in a program graph such that the following are included in the chain: 1.It consists of a single node with in-degree = 0 (initial node) 2.It consists of a single node with out-degree = 0 (terminal node) 3.It consists of a single node with in-deg => 2 or out-deg => 2 4.It consists of a single node with in-deg = 1 and out-deg = 1 5.It is a maximal chain of length => 1.

Condensation of Code to Table then to Graph 1. Program Triangle 2. Dim a, b,c As Integer 3. Dim IsTriangle As Boolean 4. Output ( “enter a,b, and c integers”) 5. Input (a,b,c) 6. Output (“side 1 is”, a) 7. Output (“side 2 is”, b) 8. Output (”side 3 is”, c) 9. If (a<b+c) AND (b<a+c) And (c<b+a) 10. then IsTriangle = True 11. else IsTriangle = False 12. endif 13. If IsTriangle 14. then if (a=b) AND (b=c) 15. then Output (“equilateral”) 16. else if (a NE b) AND (a NE b) AND (b NE c) 17. then Output ( “Scalene”) 18. else Output (“Isosceles”) 19. endif 20. endif 21. else Output (“not a triangle”) 22. endif 23. end Triangle2 Psuedo-code Def of DD-paths on Page 140 code statement Path/node name DD-path Def. (prev. page) Skip 1- 3 (or w/4) 4 first 1 5 – 8 A 5 9 B 3 10 C 4 11 D 4 12 E 3 13 F 3 14 H 3 15 I 4 16 J 3 17 K 4 18 L 4 19 M 3 20 N 3 21 G 4 22 O 3 23 last 2

Condensation Graph from Table first A B CD E F G H I J K L M N O Last - Statements coverage - 4 paths - Branch (DD-path) coverage - 4 paths - Cyclomatic # = 4+1 = lin. Ind paths - All combinations - 8 paths

Closer Look into Path Testing Look at 2 paths from A to E. The paths of either it is a triangle or not a triangle. –But there are many combinations to get “not triangle” ; so we still need to consider utilizing boundary values and equivalence class. Just one test case to cover the path may not be enough Look at the path that leads from D to L (D= not triangle and L= Isosceles triangle). Is that path possible can you generate a test case for that path? Look at the path from C to G (C = Is a triangle and G = output “not triangle”) is this a possible path? –There may be logical dependencies that prevent us from generating test case to traverse certain paths. Thus we also need to consider employing decision tables from Black Box testing technique.