Chapter 8 Path Testing

Path testing Path testing is a “design structural testing” in that it is based on detailed design & the source code of the program to be tested. The methodology uses the graphical representation of the source code –Thus it is very much “control flow” or “path” oriented This methodology has been available since the mid-1970’s and is an important White Box testing technique

Path Testing Paths derived from some graph construct. When a test case executes, it traverses a path. Huge number of paths implies some simplification is needed. Big Problem: infeasible paths. By itself, path testing can lead to a false sense of security.

Program Graphs Definition: Given a program written in an imperative programming language, its program graph is a directed graph in which nodes are statement fragments, and edges represent flow of control. (A complete statement is a “default” statement fragment.)

Program Graphs of Structured Programming Constructs

Sample Program Graph

A B C DE F G first last Common Objection to Path-Based Testing (Trillions of Paths) If the loop executes up to 18 times, there are 4.77 Trillion paths. Impossible, or at least, infeasible, to test them all. [Schach] … = 4,768,371,582,030 Stephen R. Schach, Software Engineering,, (2nd edition) Richard D. Irwin, Inc. and Aksen Associates, Inc. 1993

Schach’s flowchart

Test Cases for Schach’s “Program” 1.First-A-B-C-F-G-Last 2.First-A-B-C-D-F-G-Last 3.First-A-B-D-F-G-A-B-D-F-G-Last 4.First-A-B-E-F-G-Last 5.First-A-B-E-D-F-G-Last These test cases cover Every node Every edge Normal repeat of the loop Exiting the loop A B C DE F G first last

Transforming Code to Diagram In early chapter (chapter 4) on graph theory, we mentioned condensation graph as a graph that was derived by “condensing” a component (or set of code) into a node as a way to simplify the graphical diagram of source code. Here we will : –assume that we have a condensed graph and discuss path analysis

Path Analysis Why path analysis for test case design? –Provides a systematic methodology of White Box testing. Reproducible Traceable Countable What is path analysis? –Analyzes the number of paths that exist in the system –Facilitates the decision process of how many paths to include in the test Reasons similar to why document test cases Path, as previously defined in Graph Theory, is a sequence of connected nodes

Linearly Independent Path A path through the system is Linearly Independent** from other paths only if it includes some segment or edge that is not covered in the other paths. S1 S3 S2 C The statements are represented by the rectangular and diamond blocks (nodes). - The segments between the blocks are the edges, labeled with numbered circles. Path1 : S1 – C1 – S3 Path2 : S1 – C1 – S2 – S3 OR Path1: edges (1,4) Path2: edges (1,2,3) Path1 and Path2 are linearly independent because each includes some edge that is not included in the others. (note: not necessarily nodes) ** This definition will require more explanation later.

Another Example of Linearly Independent Paths S1 S2 S3 S4 S5 C1 C2 C Path1: edges (1,2,8) Path2: edges (1,5,3,9) Path3: edges (1,5,6,4,10) Path4: edges (1,5,6,7) Note that these are all linearly independent

Statement Coverage Method Count all the linearly independent paths Pick the minimum number of linearly independent paths that will include all the statements (nodes) (S’s and C’s in the diagram) S1 S3 S2 C Path1 : S1 – C1 – S3 : edges (1, 4) Path2 : S1 – C1 – S2 – S3 : edges (1, 2. 3 ) Are both Path1 and Path2 needed to cover all the statements: (S1,C1,S2,S3) ? What would you say?

Another Example of Statement Coverage S1 S2 S3 S4 S5 C1 C2 C The 4 Linearly Independent Paths Covers: Path1: includes S1-C1-S2-S5 : edges (1,2,8) Path2: includes S1-C1-C2-S3-S5 : edges ( 1,5,3,9) Path3: includes S1-C1-C2-C3-S4-S5 : edges (1,5,6,4,10) Path4: includes S1-C1-C2-C3-S5 : edges (1,5,6,7) For 100% Statement Coverage, all we need are 3 paths : Path1, Path2, and Path3 to cover all the statements (S1,C1,S2,C2,S3,C3,S4,S5) no need for Path !!

Statement Coverage Now What do you think of a Software Company which boasts that they run every statement in the software before release? Try this on some students or other professors and see their reaction.

Branch Coverage Method (Also DD-Paths of our text book) 1.Identify all the decisions 2.Count all the branches from each of the decisions ( “out-degree” of the decision node) 3.Pick the minimum number of paths that will cover all the branches from the decisions.

Branch Coverage Method S1 S3 S2 C One decision C1 : B1 : Path1 : C1 – S3 B2 : Path2 : C1 – S2 Are both Path1 and Path2 needed to cover Branch1 and Branch2 from C1? Branch 1Branch 2 2 linearly independent paths cover : B1 : Path1 : S1 - C1 – S3 B2 : Path2 : S1 - C1 – S2 – S3

Another Example of Branch Coverage S1 S2 S3 S4 S5 C1 C2 C The 3 Decisions and the Branches: C1: - B1 : C1- S2 - B2 : C1- C2 C2: - B3 : C2 – S3 - B4 : C2 – C3 C3: - B5 : C3 – S4 - B6 ; C3 – S5 We need: Path1 to cover B1, Path2 to cover B2 and B3, Path3 to cover B4 and B5, Path4 to cover B6 The 4 Linearly Independent Paths Cover : Path1: includes S1-C1-S2-S5 Path2: includes S1-C1-C2-S3-S5 Path3: includes S1-C1-C2-C3-S4-S5 Path4: includes S1-C1-C2-C3-S5

Branch Coverage How do you feel about a Software Company who states that they test every branch in the Software before release? Does All Branch Coverage subsume All Statements Coverage?

DD-Paths A DD-Path (decision-to-decision) is a chain in a program graph such that Case 1: it consists of a single node with indeg = 0, Case 2: it consists of a single node with outdeg = 0, Case 3: it consists of a single node with indeg ≥ 2 or outdeg ≥ 2, Case 4: it consists of a single node with indeg = 1 and outdeg = 1, Case 5: it is a maximal chain of length ≥ 1.

DD-Path Graph Given a program written in an imperative language, its DD-Path graph is the directed graph in which nodes are DD-Paths of its program graph, and edges represent control flow between successor DD-Paths. a form of condensation graph 2-connected components are collapsed into an individual node single node DD-Paths (corresponding to Cases ) preserve the convention that a statement fragment is in exactly one DD-Path

Condensation of Code to Table then to Graph 1. Program Triangle 2. Dim a, b,c As Integer 3. Dim IsTriangle As Boolean 4. Output ( “enter a,b, and c integers”) 5. Input (a,b,c) 6. Output (“side 1 is”, a) 7. Output (“side 2 is”, b) 8. Output (”side 3 is”, c) 9. If (a<b+c) AND (b<a+c) And (c<b+a) 10. then IsTriangle = True 11. else IsTriangle = False 12. endif 13. If IsTriangle 14. then if (a=b) AND (b=c) 15. then Output (“equilateral”) 16. else if (a NE b) AND (a NE b) AND (b NE c) 17. then Output ( “Scalene”) 18. else Output (“Isosceles”) 19. endif 20. endif 21. else Output (“not a triangle”) 22. endif 23. end Triangle2 Psuedo-code Def of DD-paths on Page 140 code statement Path/node name DD-path Def. Skip 1- 3 (or w/4) 4 first 1 5 – 8 A 5 9 B 3 10 C 4 11 D 4 12 E 3 13 F 3 14 H 3 15 I 4 16 J 3 17 K 4 18 L 4 19 M 3 20 N 3 21 G 4 22 O 3 23 last 2

Condensation Graph from Table first A B CD E F G H I J K L M N O Last - Statements coverage - 4 paths - Branch (DD-path) coverage - 4 paths - Cyclomatic # = 4+1 = lin. Ind paths - All combinations - 8 paths

Closer Look into Path Testing Look at 2 paths from A to E. The paths of either it is a triangle or not a triangle. –But there are many combinations to get “not triangle” ; so we still need to consider utilizing boundary values and equivalence class. Just one test case to cover the path may not be enough Look at the path that leads from D to L (D= not triangle and L= Isosceles triangle). Is that path possible can you generate a test case for that path? Look at the path from C to G (C = Is a triangle and G = output “not triangle”) is this a possible path? –There may be logical dependencies that prevent us from generating test case to traverse certain paths. Thus we also need to consider employing decision tables from Black Box testing technique.

Condensation of Code to Table then to Graph 1. Program Triangle 2. Dim a, b,c As Integer 3. Dim IsTriangle As Boolean 4. Output ( “enter a,b, and c integers”) 5. Input (a,b,c) 6. Output (“side 1 is”, a) 7. Output (“side 2 is”, b) 8. Output (”side 3 is”, c) 9. If (a<b+c) AND (b<a+c) And (c<b+a) 10. then IsTriangle = True 11. else IsTriangle = False 12. endif 13. If IsTriangle 14. then if (a=b) AND (b=c) 15. then Output (“equilateral”) 16. else if (a NE b) AND (a NE b) AND (b NE c) 17. then Output ( “Scalene”) 18. else Output (“Isosceles”) 19. endif 20. endif 21. else Output (“not a triangle”) 22. endif 23. end Triangle2 Psuedo-code Def of DD-paths on Page 140 code statement Path/node name DD-path Def. Skip 1- 3 (or w/4) 4 first 1 5 – 8 A 5 9 B 3 10 C 4 11 D 4 12 E 3 13 F 3 14 H 3 15 I 4 16 J 3 17 K 4 18 L 4 19 M 3 20 N 3 21 G 4 22 O 3 23 last 2

McCabe’s Cyclomatic Number Is there a way to know how many linearly independent paths exist? –McCabe’s Cyclomatic number which we used to study program complexity may be applied. There are 3 ways to get the Cyclomatic Complexity number from a flow diagram. # of binary decisions + 1 # of edges - # of nodes + 2 # of closed regions + 1

McCabe’s Cyclomatic Complexity Number Earlier Example S1 S3 S2 C We know there are 2 linearly independent paths from before: Path1 : C1 – S3 Path2 : C1 – S2 – S3 McCabe’s Cyclomatic Number: a) # of binary decisions +1 = 1 +1 = 2 b) # of edges - # of nodes +2 = = 2 c) # of closed regions + 1 = = 2 Closed region

McCabe’s Cyclomatic Complexity Number Another Example S1 S3 S2 C McCabe’s Cyclomatic Number: a) # of binary decisions +1 = 2 +1 = 3 b) # of edges - # of nodes +2 = = 3 c) # of closed regions + 1 = = 3 Closed Region C2 S Closed Region There are 3 Linearly Independent Paths

An example of 2 n total path S1 C1 C2 C3 S2 S3 S4S Since for each binary decision, there are 2 paths and there are 3 in sequence, there are 2 3 = 8 total “logical” paths path1 : S1-C1-S2-C2-C3-S4 path2 : S1-C1-S2-C2-C3-S5 path3 : S1-C1-S2-C2-S3-C3-S4 path4 : S1-C1-S2-C2-S3-C3-S5 path5 : S1-C1-C2-C3-S4 path6 : S1-C1-C2-C3-S5 path7 : S1-C1-C2-S3-C3-S4 path8 : S1-C1-C2-S3-C3-S5 How many Linearly Independent paths are there? Using Cyclomatic number = 3 decisions +1 = 4 One set would be: path1 : includes edges (1,2,4,6,9) path2 : includes edges (1,2,4,6,8) path3 : includes edges (1,2,4,5,7,9) path5 : includes edges (1,3,6,9) Note 1: with just 2 paths ( Path1 and Path8) all the statements are covered. Note2: with just 2 paths ( Path1 and Path8) all the branches are covered.

Example with a Loop S1 S2 S3 C Total number of ‘logical’ paths may be “infinite” (very large) because of the loop ---- so we will only look at this “statically” Linearly Independent Paths = 1 decision +1 = 2 path1 : S1-C1-S3 (segments 1,4 ) path2 : S1-C1-S2-C1-S3 (segments 1,2,3,4 ) “One path” will cover all statements: path2 : S1-C1-S2-C1-S3 (S1,C1,S2,S3) “One path” will cover all branches: path2 : S1-C1-S2-C1-S3 branch1 (C1-S2) and branch 2 (C1-S3) *** Note***: To get one path --- S2 must somehow change C1

Example with a Loop (cont.) S1 S2 S3 C Total number of ‘logical’ paths may be “infinite” (very large) because of the loop Linearly Independent Paths = 1 decision +1 = 2 path1 : S1-C1-S3 (segments 1,4 ) path2 : S1-C1-S2-C1-S3 (segments 1,2,3,4 ) “One path” will cover all branches: path2 : S1-C1-S2-C1-S3 branch1 (C1-S2) and branch 2 (C1-S3) Should we consider the loop boundaries in C1? & also bring in our boundary value testing thoughts?

McCabe Basis Paths in the Triangle Program V(G) = (1) = 5 Basis Path Set B1 p1: 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 16, 18, 19, 20, 22, 23 (mainline) p2: 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 16, 18, 19, 20, 22, 23 (flipped at 9) p3: 4, 5, 6, 7, 8, 9, 11, 12, 13, 21, 22, 23 (flipped at 13) p4: 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 20, 22, 23 (flipped at 14) p5: 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 16, 17, 19, 20, 22, 23 (flipped at 16) There are 8 topologically possible paths. 4 are feasible, and 4 are infeasible. Exercise: Is every basis path feasible?

Condensation with Structured Programming Constructs

Violations of Structured Programming Precepts

More on Linearly Independent Paths In discussing dimensionality, we talks about orthogonal vectors (or “basis” of vector space). –(e.g.) Two dimensional space has two orthogonal vector from which all the other vectors in two dimension can be obtained via “linear combination” of these 2 vectors: [1,0] [0,1] [1,0] [0,1] [2,4] e.g. [2,4] = 2[1,0] + 4[0,1]

More on Linearly Independent Paths A set of paths is considered to be a Linearly Independent Set if every path may be constructed as a “linear combination” of paths from the linearly independent set. For example: C1 C2 S1 S We already know: a) there are a total of 2 2 =4 logical paths. b) 1 path (path4) will cover all statements c) 2 paths will cover all branches. d) 2 branches +1 = 3 linearly independent paths. path1 path2 path3 path path 4 = path3 + path1 – path2 = (0,1,1,1,0,0)+(1,0,0,0,1,1)- (1,0,0,1,0,0) = (1,1,1,1,1,1) - (1,0,0,1,0,0) = (0,1,1,0,1,1) We pick: path1, path2 and path3 as the Linearly Independent Set

Other Sets of Linearly Independent Paths -Consider the set of linearly independent paths: 1, 2 & 4 instead. - Can we get Path3 = (0,1,1,1,0,0)? -Consider path3 = path2 + path4 – path1 = (1,0,0,1,0,0,)+(0,1,1,0,1,1) – (1,0,0,0,1,1) = (0,1,1,1,0,0) path1 path2 path3 path Consider another set of linearly independent paths: 2, 3 & 4 instead. - Can we get Path 1 = (1,0,0,0,1,1))? -Consider path1 = path2 + path4 – path3 = (1,0,0,1,0,0,)+(0,1,1,0,1,1) – (0,1,1,1,0,0) = (1,0,0,0,1,1)

More on Linearly Independent Paths C1 C2 S1 S path1 path2 path3 path Note : Although path1 and path3 are linearly independent, they do NOT form a Linearly Independent Set because no linear combination of path1 and path3 can get, say, path4. We already know: a) there are a total of 2 2 =4 logical paths. b) 1 path (path4) will cover all statements c) 2 paths will cover all branches. d) 2 branches +1 = 3 linearly independent paths.

More on Linearly Independent Paths Because the Linearly Independent Set of paths display the same characteristics as the mathematical concept of basis in n-dimensional vector space, the testing using the Linearly Independent Set of paths is sometimes called the “basis” testing. The main notion is that since the “linear independent set of paths” as a set can span all the paths for the design/code construct, then basis testing covers the “essence” of the whole structure. Then is there a way to find a Linearly Independent Set ?

An Algorithm to Find the Basis Set 1.Select a baseline path, an arbitrary, normal execution path that contains as many decisions as possible. 2.Retrace the baseline path and “flip” each of the decision encountered; each flip creates a new path. Continue until all the decisions are flipped 3.The basis set is composed of all the paths generated from steps 1 and 2 above C1 C2 S1 S pick baseline path P1: C1 –S1- C2 – S2: 2. flip C1 P2 : C1 – C2 – S2 : 3. flip C2 P3: C1 - C2 : Cyclomatic # = 2+ 1 = 3; So there are 3 linearly independent paths 5 Can we get the 4 th path : C1 – S1 – C2 : from the above basis set? How about : (P1 + P3) – P2 ? (P1 + P3) – P2 = ( + ) - = - = = P4

Total “Possible” Logical Paths can be Big! s1 c1 s2 c3 s3 c2 c4 s There are 5 choices each time we process through this loop. For passing through the loop n times we have 5 n possibilities of logical paths. If we go through the loop just 3 times, we have (5) 3 = 125 possible paths! You also note that for n loops there may be 3n decisions. e.g.: We have 3*3 = 9 decisions for looping 3 times.

Paths Analysis Interested in total number of all possible combinations of “logical” paths Interested in Linearly Independent paths Interest in Branch coverage or DD-path Interested in Statement coverage