Software Testing and Maintenance 1 Today’s Agenda Quiz 1 Quick Review Continue on Graph-Based Testing Software Testing and Maintenance 1

Software Testing and Maintenance 2 Quick Review What is a basic block? Software Testing and Maintenance 2

Software Testing and Maintenance 3 Graph-Based Testing Introduction Basic Concepts Control Flow Testing Data Flow Testing Summary Software Testing and Maintenance 3

Software Testing and Maintenance 4 Motivation Graph-based testing first builds a graph model for the program under test, and then tries to cover certain elements in the graph model. Graph is one of the most widely used structures for abstraction. Transportation network, social network, molecular structure, geographic modeling, etc. Graph is a well-defined, well-studied structure Many algorithms have been reported that allow for easy manipulation of graphs. Graph-based testing is a model-based testing approach. Graph is one of the most fundamental data structures in computer science. Software Testing and Maintenance 4

Software Testing and Maintenance 5 Major Steps Step 1: Build a graph model What information to be captured, and how to represent those information? Step 2: Identify test requirements A test requirement is a structural entity in the graph model that must be covered during testing Step 3: Select test paths to cover those requirements Step 4: Derive test data so that those test paths can be executed Software Testing and Maintenance 5

Software Testing and Maintenance 6 Graph Models Control flow graph: Captures information about how the control is transferred in a program. Data flow graph: Augments a CFG with data flow information Dependency graph: Captures the data/control dependencies among program statements Cause-effect graph: Modeling relationships among program input conditions, known as causes, and output conditions, known as effects Software Testing and Maintenance 6

Software Testing and Maintenance 7 Graph-Based Testing Introduction Basic Concepts Control Flow Testing Data Flow Testing Summary Software Testing and Maintenance 7

Software Testing and Maintenance 8 Graph A graph consists of a set of nodes and edges that connect pairs of nodes. Formally, a graph G = <N, N0, Nf, E): N: a set of nodes N0  N: a set of initial nodes Nf  N: a set of final nodes E  N  N: a set of edges In our context, N, N0, and Nf contain at least one node. Software Testing and Maintenance 8

Software Testing and Maintenance 9 Example n0 n1 n2 n3 n0 n1 n2 n3 n4 n5 n6 n7 n8 n9 N = {n0, n1, n2, n3} N0 = {n0} Nf = {n3} E = {(n0, n1), (n0, n2), (n1, n3), (n2, n3)} N = {n0, n1, n2, n3 , n4, n5, n6 , n7, n8, n9} N0 = {n0 , n1, n2} Nf = {n7, n8, n9} E = {(n0, n3), (n0, n4), (n1, n4), (n1, n5), …} Software Testing and Maintenance 9

Software Testing and Maintenance 10 Path, Subpath, Test Path A path is a sequence [n1, n2, …, nM] of nodes, where each pair of adjacent nodes (ni, ni+1) is an edge. The length of a path refers to the number of edges in the path A subpath of a path p is a subsequence of p, possibly p itself. A test path is a path, possibly of length zero, that starts at an initial node, and ends at a final node Represents a path that is executed during a test run Software Testing and Maintenance 10

Software Testing and Maintenance 11 Reachability A node n is syntactically reachable from node n’ if there exists a path from n’ to n. A node n is semantically reachable from node n’ if it is possible to execute a path from n’ to n with some input. reach(n): the set of nodes and edges that can be syntactically reached from node n. Software Testing and Maintenance 11

Software Testing and Maintenance 12 Example n3 n4 n7 n0 n1 n2 n5 n6 n8 n9 p1 = [n0, n3, n7] p2 = [n1, n4, n8, n5, n1] p3 = [n4, n8, n5] reach(n0) = ? reach(n5) = ? Reach(no) = {n3, n7, n4, n8, n5, n1, n9} Reach(n5) = {n1, n4, n7, n8, n5, n9} Software Testing and Maintenance 12

Software Testing and Maintenance 13 SESE Graph n0 n1 n2 n3 n4 n5 n6 Software Testing and Maintenance 13

Software Testing and Maintenance 14 Visit & Tour A test path p is said to visit a node n (or an edge e) if node n (or edge e) is in path p. A test path p is said to tour a path q if q is a subpath of p. Software Testing and Maintenance 14

Software Testing and Maintenance 15 Test Case vs Test Path n0 n1 n2 n3 a < b a > b a = b t1: (a = 0, b = 1) => p1 = [n0, n1, n3, n2] t2: (a = 1, b = 1) => p2 = [n0, n3, n2] t3: (a = 2, b = 1) => p3 = [n0, n2] Software Testing and Maintenance 15

Software Testing and Maintenance 16 Graph-Based Testing Introduction Basic Concepts Control Flow Testing Data Flow Testing Summary Software Testing and Maintenance 16

Software Testing and Maintenance 17 Basic Block A basic block, or simply a block, is a sequence of consecutive statements with a single entry and a single exit point. Control always enters a basic block at its entry point, and exits from its exit point. No entry, halt, or exit inside a basic block If a basic block contains a single statement, then the entry and exit points coincide. Software Testing and Maintenance 17

Software Testing and Maintenance 18 Example begin int x, y, power; float z; input (x, y); if (y < 0) power = -y; else power = y; z = 1; while (power != 0) { z = z * x; power = power – 1; } z = 1/z; output (z); end; Block Lines Entry Exit 1 2, 3, 4, 5 2 5 6 3 8 4 9 10 11, 12 11 12 7 14 15 16 Software Testing and Maintenance 18

Software Testing and Maintenance 19 Function Calls Should a function call be treated like a regular statement or as a separate block of its own? Function calls are often treated as blocks of their own because they cause the control to be transferred away from the currently executing function and hence raise the possibility of abnormal termination of the program. Software Testing and Maintenance 19

Software Testing and Maintenance 20 Control Flow Graph A control flow graph is a graph with two distinguished nodes, start and end. Node start has no incoming edges, and node end has no outgoing edges. Every node can be reached from start, and can reach end. In a CFG, a node is typically a basic block, and an edge indicates the flow of control from one block to another. Software Testing and Maintenance 20

Software Testing and Maintenance 21 Example start end 1 2 3 4 5 6 7 8 9 false true Software Testing and Maintenance 21

Software Testing and Maintenance 22 Node Coverage A test set T satisfies Node Coverage on graph G if and only if for every syntactically reachable node n in N, there is some path p in path(T) such that p visits n. path(T): the set of paths that are exercised by the execution of T In other words, the set TR of test requirements for Node Coverage contains each reachable node in G. Software Testing and Maintenance 22

Software Testing and Maintenance 23 Edge Coverage The TR for Edge Coverage contains each reachable path of length up to 1, inclusive, in a graph G. Note that Edge Coverage subsumes Node Coverage. Software Testing and Maintenance 23

Software Testing and Maintenance 24 Node vs Edge Coverage n0 n1 n2 Software Testing and Maintenance 24

Software Testing and Maintenance 25 Edge-Pair Coverage The TR for Edge-Pair Coverage contains each reachable path of length up to 2, inclusive, in a graph G. This definition can be easily extended to paths of any length, although possibly with diminishing returns. Software Testing and Maintenance 25

Edge-Pair vs Edge Coverage n0 n1 n6 n2 n3 n4 n5 a b c d e f g h {c, f} and {d, e} are not covered. Software Testing and Maintenance 26

Complete Path Coverage The TR for Complete Path Coverage contain all paths in a graph. n0 n1 n3 n4 n2 How many paths do we need to cover in the above graph? Software Testing and Maintenance 27

Software Testing and Maintenance 28 Simple & Prime Path A path is simple if no node appears more than once in the path, with the exception that the first and last nodes may be identical. A path is a prime path if it is a simple path, and it does not appear as a proper subpath of any other simple path. Software Testing and Maintenance 28

Software Testing and Maintenance 29 Prime Path Coverage The TR for Prime Path Coverage contains every prime path in a graph. Software Testing and Maintenance 29

Software Testing and Maintenance 30 Example n0 n1 n3 n4 n2 Prime paths = {[n0, n1, n2], [n0, n1, n3, n4], [n1, n3, n4, n1], [n3, n4, n1, n3], [n4, n1, n3, n4], [n3, n4, n1, n2]} Path (t1) = [n0, n1, n2] Path (t2) = [n0, n1, n3, n4, n1, n3, n4, n1, n2] T = {t1, t2} Software Testing and Maintenance 30

Infeasible Test Requirements The notion of “tour” is rather strict. S0 a b d Sf c Let q = [a, b, d], and p = [S0, a, b, c, d, Sf]. Does path p tour path q? Software Testing and Maintenance 31

Software Testing and Maintenance 32 Sidetrips/Detours Tour: Test path p is said to tour path q if and only if q is a subpath of p. Tour with sidetrips: Test path p is said to tour path q with sidetrips if and only if every edge in q is also in p in the same order. Tour with detours: Test path p is said to tour path q with detours if and only if every node in q is also in p in the same order Software Testing and Maintenance 32

Software Testing and Maintenance 33 Example S0 a b d Sf c 1 2 3 4 5 6 S0 a b d Sf c 1 2 5 3 4 Software Testing and Maintenance 33

Software Testing and Maintenance 34 Best Effort Touring If a test requirement can be met without a sidetrip (or detour), then it should be done so. In other words, sidetrips or detours should be allowed only if necessary. Software Testing and Maintenance 34

Software Testing and Maintenance 35 Computing Prime Paths Step 1: Find all the simple paths Find all simple paths of length 0, extend them to length 1, and then to length 2, and so on Step 2: Select those that are maximal Software Testing and Maintenance 35

Software Testing and Maintenance 36 Example 3 1 2 4 5 6 Software Testing and Maintenance 36

Example – Simple Paths (2) 1. [0] 2. [1] 3. [2] 4. [3] 5. [4] 6. [5] 7. [6]! len = 0 8. [0, 1] 9. [0, 4] 10. [1, 2] 11. [1, 5] 12. [2, 3] 13. [3, 1] 14. [4, 4]* 15. [4, 6]! 16. [5, 6]! len = 1 17. [0, 1, 2] 18. [0, 1, 5] 19. [0, 4, 6]! 20. [1, 2, 3] 21. [1, 5, 6]! 22. [2, 3, 1] 23. [3, 1, 2] 24. [3, 1, 5] len = 2 25. [0, 1, 2, 3]! 26. [0, 1, 5, 6]! 27. [1, 2, 3, 1]* 28. [2, 3, 1, 2]* 29. [2, 3, 1, 5] 30. [3, 1, 2, 3]* 31. [3, 1, 5, 6] len = 3 32. [2, 3, 1, 5, 6]! len = 4 ! Indicates a path cannot be extended because there is no outgoing edge. * Indicates a path cannot be extended because the paths is already a cycle. At each step, we extend a path with every node that can be reached from the final node in the path unless that node is already in the path and not the first node. Software Testing and Maintenance 37

Software Testing and Maintenance 38 Example – Prime Paths 14. [4, 4]* 19. [0, 4, 6]! 25. [0, 1, 2, 3]! 26. [0, 1, 5, 6]! 27. [1, 2, 3, 1]* 28. [2, 3, 1, 2]* 30. [3, 1, 2, 3]* 32. [2, 3, 1, 5, 6]! Software Testing and Maintenance 38

Software Testing and Maintenance 39 Example – Test Paths Start with the longest prime paths and extend them to the start and end nodes of the graph [0, 1, 2, 3, 1, 5, 6] [0, 1, 2, 3, 1, 2, 3, 1, 5, 6] [0, 1, 5, 6] [0, 4, 6] [0, 4, 4, 6] Software Testing and Maintenance 39

Software Testing and Maintenance 40 Graph-Based Testing Introduction Basic Concepts Control Flow Testing Data Flow Testing Summary Software Testing and Maintenance 40

Software Testing and Maintenance 41 Definition/Use A definition is a location where a value for a variable is stored into memory. Assignment, input, parameter passing, etc. A use is a location where a variable’s value is accessed. p-use: a use that occurs in a predicate expression, i.e., an expression used as a condition in a branch statement c-use: a use that occurs in an expression that is used to perform certain computation The idea of data flow testing is to make sure that the values created at one point in a program are created and used correctly. Software Testing and Maintenance 41

Software Testing and Maintenance 42 Data Flow Graph A data flow graph (DFG) captures the flow of data in a program To build a DFG, we first build a CFG and then annotate each node n in the CFG with the following two sets: def(n): the set of variables defined in node n use(n): the set of variables used in node n Software Testing and Maintenance 42

Software Testing and Maintenance 43 Example (1) begin float x, y, z = 0.0; int count; input (x, y, count); do { if (x <= 0) { if (y >= 0) { z = y * z + 1; } else { z = 1/x; y = x * y + z; count = count – 1; while (count > 0) output (z); end Node Lines 1 1, 2, 3, 4 2 5, 6 3 7 4 8, 9, 10 5 11, 12, 13 6 14, 15, 16 17, 18 Software Testing and Maintenance 43

Software Testing and Maintenance 44 Example (2) 1 2 3 5 6 4 7 def={x, y, z, count} def={} use = {x} use = {y} def={z} use = {y, z} def={count, y} use = {count, x, y, z} use = {z} count == 0 x <= 0 x > 0 y >= 0 y < 0 Software Testing and Maintenance 44

Software Testing and Maintenance 45 DU-pair & DU-path A du-pair is a pair of locations (i, j) such that a variable v is defined in i and used in j. Suppose that variable v is defined at node i, and there is a use of v at node j. A path p = (i, n1, n2, …, nk, j) is def-clear w.r.t. v if v is not defined along the subpath n1, n2, …, nk. A definition of a variable v reaches a use of v if there is a def-clear path from the definition to the use w.r.t. v. A du-path for a variable v is a simple path from a definition of v to a use of v that is def-clear w.r.t. v. Software Testing and Maintenance 45

Software Testing and Maintenance 46 Example Consider the previous example: Path p = (1, 2, 5, 6) is def-clear w.r.t variables x, y and count, but is not def-clear w.r.t. variable z. Path q = (6, 2, 5, 6) is def-clear w.r.t variables count and y. Path r = (1, 2, 3, 4) is def-clear w.r.t variables y and z. Software Testing and Maintenance 46

Software Testing and Maintenance 47 Notations Def-path set du(n, v): the set of du-paths w.r.t variable v that start at node n. Def-pair set du(n, n’, v): the set of du-paths w.r.t variable v that start at node n and end at node n’. Note that du(n, v) = n’ du(n, n’, v). Software Testing and Maintenance 47

Software Testing and Maintenance 48 All-Defs Coverage For each def-path set S = du(n, v), the TR for All-Defs Coverage contains at least one path in S. Informally, for each def, we need to tour at least one path to at least one use. Software Testing and Maintenance 48

Software Testing and Maintenance 49 All-Uses Coverage For each def-pair set S = du(n, n’, v), the TR for All-Uses Coverage contains at least one path in S. Informally, it requires us to tour at least one path for every def-use pair. Software Testing and Maintenance 49

All-DU-Paths Coverage For each def-pair set S = du(n, n’, v), the TR for All-DU-Paths Coverage contains every path in S. Informally, this requires to tour every du-path. Software Testing and Maintenance 50

Software Testing and Maintenance 51 Example n0 n1 n2 n3 n4 n5 n6 def(0) = {x} use(4) = {x} use(5) = {x} all-defs 0-1-3-4 all-uses 0-1-3-4 0-1-3-5 all-du-paths 0-1-3-4 0-1-3-5 0-2-3-4 0-2-3-5 Software Testing and Maintenance 51

Software Testing and Maintenance 52 Why data flow? Consider the previous example. Assume that there is a fault in line 14, which is supposed to be y = x + y + z. Does the following test set satisfy edge coverage? Can the test set detect the above fault? x y count t1 -2 2 1 t2 t3 t4 To detect the fault, control must first reach line 14, and then line 8. This means that count has to be greater than 1. Software Testing and Maintenance 52

Software Testing and Maintenance 53 Graph-Based Testing Introduction Basic Concepts Control Flow Testing Data Flow Testing Summary Software Testing and Maintenance 53

Subsumption Hierarchy Complete Path Coverage Prime Path Coverage All-du-paths Coverage Edge-pair Coverage All-Uses Coverage Edge Coverage All-Defs Coverage Node Coverage Software Testing and Maintenance 54

Software Testing and Maintenance 55 Recap Graph provides a good basis for systematic test selection. Control flow testing focuses on the transfer of control, while data flow testing focuses on the definitions of data and their subsequent use. Control flow coverage is defined in terms of nodes, edges, and paths; data flow coverage is defined in terms of def, use, and du-path. Software Testing and Maintenance 55