Ch6: Software Verification

1 White-box testing  Structural testing:  (In)adequacy criteria  Control flow coverage criteria

2 Statement coverage criterion  Informally:  Formally:  Difficult to minimize the number of test cases and still ensure the execution of all statements

3 Statement coverage criterion (contd..) Example read (x); read (y); if x > 0 then write ("1"); else write ("2"); end if; if y > 0 then write ("3"); else write ("4"); end if; {,,, } covers all statements {, } is minimal

4 Statement coverage criterion if x < 0 then x := -x; end if z = :x; This may be rewritten as: if x < 0 then x := -x; else null; end if z := x;

5 Edge coverage criterion  Criterion:  What is a control flow graph, and how to construct it?  Finer than statement coverage criterion

6 Control graph construction rules I/O, assignment, or procedure call GG 1 2 if-then-else G 1 if-then G 1 while loop G 1 G 2 two sequential statements

7 Simplification A sequence of edges can be collapsed into just one edge... nn n n n k-1k12 3 n 1 n k

8 Example: Euclid’s algorithm begin read (x); read (y); while x ≠ y loop if x > y then x := x - y; else y := y - x; end if; end loop; gcd : = x; end;

9 Weakness of edge coverage criterion found := false; counter := 1; while (not found) and counter < number_of_items loop if table (counter) = desired_element then found := true; end if; counter := counter + 1; end loop; if found then write ("the desired element is in the table"); else write ("the desired element is not in the table"); end if; test cases: (1) empty table, (2) table with 3 items, second of which is the item to look for

10 Condition coverage criterion  Condition coverage:  Finer than edge coverage

11 Weakness of condition coverage criterion if x ≠ 0 then y := 5; else z := z - x; end if; if z > 1 then z := z / x; else z := 0; end if; {, } causes the execution of all edges, but fails to expose the risk of a division by zero

12 Path-coverage criterion  Path coverage criteria:  Finer than previous kinds of coverage  Issues with path coverage criteria  Path coverage criterion may be used as a guide to determining a few critical paths.

13 White-box testing (contd..)  Guidelines for testing loops:  Guidelines for IF and CASE statements  Once a criterion has been chosen, actual input values may be chosen.  How do you decide which criterion to use?  Can (and should) different criterion be applied to different modules in the same system?

14 White-box testing (contd..)  The presence of unreachable statements may mean that 100% coverage is not achieved.  In summary, whatever coverage criterion we decide, human intervention is needed to solve problems (reachability, etc.)

15 White-box testing (contd..)  Problems with white-box testing:

16 Black-box testing  What is black-box testing:

17 Black-box testing (contd..) The program receives as input a record describing an invoice. (A detailed description of the format of the record is given.) The invoice must be inserted into a file of invoices that is sorted by date. The invoice must be inserted in the appropriate position: If other invoices exist in the file with the same date, then the invoice should be inserted after the last one. Also, some consistency checks must be performed: The program should verify whether the customer is already in a corresponding file of customers, whether the customer’s data in the two files match, etc. Example specification

18 Black-box testing (contd..) Did you consider these cases? An invoice whose date is the current date An invoice whose date is before the current date (This might be even forbidden by law) This case, in turn, can be split into the two following subcases: An invoice whose date is the same as that some existing invoice An invoice whose date does not exist in any previously recorded invoice Several incorrect invoices, checking different types of inconsistencies

19 Systematic black-box techniques  Syntax-driven testing  Decision table based testing  Cause-effect graph based testing

20 Syntax driven testing  Applicability:  Role of formal specification:

21 Syntax driven testing (contd..) Example Consider testing an interpreter of the following language ::= + | - | ::= * | / | ::= ident | ( )

22 Syntax driven testing (contd..)  How to complete coverage principle:  Automated generation of test cases:  Minimal test set: