[Ammann and Offutt, “Introduction to Software Testing,” Ch. 8] Logic Coverage Inactive Clause Coverage CS 4501 / 6501 Software Testing [Ammann and Offutt, “Introduction to Software Testing,” Ch. 8]

General Active Clause Coverage (GACC) Review p = (a ↔ b) row a b p pa pb 1 T 2 F 3 4 pa = T a = T pa = T a = F Can be satisfied by {1,2} x {3,4} If (1,4) is used, p is always true If (2,3) is used, p is always false GACC does not guarantee PC

Correlated Active Clause Coverage (CACC) Review p = (a ↔ b) row a b p pa pb 1 T 2 F 3 4 pa = T a = T pa = T a = F p evaluates to T and F Only (1,3) and (2,4) satisfy CACC

Restricted Active Clause Coverage (RACC) Review p = (a ↔ b) row a b p pa pb 1 T 2 F 3 4 pa = T a = T pa = T a = F p evaluates to T and F, same minor truth value Only (1,3) and (2,4) satisfy RACC

CACC vs. RACC What is the difference between CACC and RACC? RACC imposes more constraints on the truth values of minor clauses RACC requires the the truth values for minor clauses are consistent between the two requirements In practice? There are fewer possible truth table row combinations that satisfy RACC RACC leads to more infeasible test requirements than CACC

RACC Leads to Infeasible Requirements If clauses are independent, there is no problem If clauses are dependent, some combination of clauses become infeasible Consider a system with the constraints: A valve must be open in “operational” mode and closed in all other modes Mode cannot be in both “operational” and “standby” at the same time Three clauses: a = the valve is closed b = the system mode is operational c = the system mode is standby p = valve is closed AND (system is in operational mode OR system is in standby mode) = a  (b  c) Constraints: ¬a ↔ b ¬(b  c)

CACC and RACC – Infeasible TRs p = a  (b  c) Constraints: ¬a ↔ b ¬(b  c) row a b c p Constraint violations 1 T 1,2 2 F 3 4 5 6 7 8 CACC can be satisfied by choosing one of the nine pairs RACC can be satisfied by choosing one of the three pairs CACC: out of {1,2,3} x {5,6,7}, only (3,6) is feasible RACC: out of (1,5), (2,6), (3,7), none are feasible

Inactive Clause Coverage (ICC) ACC ensures that “major” clauses do affect the predicates ICC ensures that “major” clauses do not affect the predicates Also known as “Reinforced Condition/Decision Coverage” (RCDC) Inactive Clause Coverage (ICC) : For each p in P and each major clause ci in Cp, choose minor clauses cj, j != i, so that ci does not determine p. TR has four requirements for each ci: ci evaluates to true with p true, ci evaluates to false with p true, ci evaluates to true with p false, ci evaluates to false with p false

ICC Example p = (¬a  ¬b)  (a  ¬c)  (¬a  c) Major clause: b row a pb pc 1 T 2 3 4 5 6 7 8 p = T, pb = F b = T: rows b = F: rows 2, 5 4, 7 p = F, pb = F b = T: rows b = F: rows 1 3 Do the minor clauses have to have the same values when the major clause is true and false?

General Inactive Clause Coverage (GICC) For each clause ci in predicate p, choose minor clauses such that c does not determine p ci evaluates to true with p true, ci evaluates to false with p true, ci evaluates to true with p false, ci evaluates to false with p false Minor clause may differ

GICC Example p = (¬a  ¬b)  (a  ¬c)  (¬a  c) Major clause: b row a pb pc 1 T 2 3 4 5 6 7 8 p = T, pb = F b = T: rows b = F: rows 2, 5 4, 7 Possible tests: (2,4), (2,7), (5,4), (5,7) p = F, pb = F b = T: rows b = F: rows 1 3 Possible tests: (1,3)

GICC Example (exercise) p = (¬a  ¬b)  (a  ¬c)  (¬a  c) Major clause: a row a b c p pa pb pc 1 T 2 3 4 5 6 7 8 p = T, pa = F a = T: rows a = F: rows 4 8 Possible tests: (4,8) p = F, pa = F a = T: rows a = F: rows - Possible tests: No feasible pairs

GICC Example (exercise) p = (¬a  ¬b)  (a  ¬c)  (¬a  c) Major clause: c row a b c p pa pb pc 1 T 2 3 4 5 6 7 8 p = T, pc = F c = T: rows c = F: rows 7 8 Possible tests: (7,8) p = F, pc = F c = T: rows c = F: rows - Possible tests: No feasible pairs

Restricted Inactive Clause Coverage (RICC) For each clause ci in predicate p, choose minor clauses such that c does not determine p ci evaluates to true with p true, ci evaluates to false with p true, ci evaluates to true with p false, ci evaluates to false with p false Minor clause must be the same

RICC Example p = (¬a  ¬b)  (a  ¬c)  (¬a  c) Major clause: b row a b c p pa pb pc 1 T 2 3 4 5 6 7 8 p = T, pb = F b = T: rows b = F: rows 2, 5 4, 7 Possible tests: (2,4), (2,7), (5,4), (5,7) p = F, pb = F b = T: rows b = F: rows 1 3 Possible tests: (1,3) Minor clause(s): a, c

RICC Example (exercise) p = (¬a  ¬b)  (a  ¬c)  (¬a  c) Major clause: a row a b c p pa pb pc 1 T 2 3 4 5 6 7 8 p = T, pa = F a = T: rows a = F: rows 4 8 Possible tests: (4,8) p = F, pa = F a = T: rows a = F: rows - Possible tests: No feasible pairs

RICC Example (exercise) p = (¬a  ¬b)  (a  ¬c)  (¬a  c) Major clause: c row a b c p pa pb pc 1 T 2 3 4 5 6 7 8 p = T, pc = F c = T: rows c = F: rows 7 8 Possible tests: (7,8) p = F, pc = F c = T: rows c = F: rows - Possible tests: No feasible pairs

Logic Coverage Criteria Subsumption Combinatorial Clause Coverage CoC Restricted Active RACC Restricted Inactive RICC Correlated Active CACC General Inactive GICC General Active GACC CC Predicate Coverage PC

Infeasibility Subsumption is not always given. Sometimes test requirements cannot be satisfied. Example: (a > b  b > c)  (c > a) (a > b) = true, (b > c) = true, (c > a) true … infeasible Infeasible test requirements must be recognized and eliminated Recognizing infeasible test requirements is hard, and in general, undecidable More complex criteria also produce more infeasible test requirements

If RACC test generation fails Best Effort Strategy If RACC test generation fails Combinatorial Clause Coverage CoC Restricted Active RACC Restricted Inactive RICC Correlated Active CACC General Inactive GICC General Active GACC CC Predicate Coverage PC Try CACC

Summary Predicates are often very simple In practice, most have only one clause (<= 3 clauses) With only one clause, PC is enough With 2 or 3 clauses, CoC is practical Advantages of ACC and ICC significant for large predicates CoC is impractical for predicates with many clauses ICC always guarantees Predicate coverage