Automatic Test Generation Generate a test sequence for a given behavioral description Some coverage metric is typically assumed Advantages of Automation: Less designer time/cost Less dependence on designer experience/ability Disadvantages of Automation: Must rely on behavioral description (not spec) Must rely on coverage metrics

Fault Directed Test Generation A subset of potential faults are chosen for test generation Tests are generated specifically for the chosen set Continue until all potential faults are detected Key Points: The faults are defined by the coverage metric - “a statement may be faulty” for statement coverage 2. The input space must be searched 3. These algorithms are typically “complete” - If a fault can be detected then it will be detected

SAT-Based Test Generation SATISFIABILITY (SAT) is the problem of solving a set of boolean constraints Ex. (A AND B) OR (C AND D) An ATG problem can be solved as a SAT problem if the coverage constraints are boolean if (x AND y) then out=1; (x AND y) else out=2; NOT (x AND y) Both statement constraints are boolean Boolean constraints are common in logic

SAT-Based Problems Some constraints are arithmetic, not naturally boolean if (x > y) then out=1; (x > y) else out=2; NOT (x > y) Arithmetic constraints can be converted to boolean but it is expensive The > operator can be replaced by a gate-level implementation of a comparator A better solution would solve both boolean and arithmetic constraints

Integer Linear Programming (ILP) ILP is basically solving simultaneous equations (ala high school algebra) X + Y = 10 X - Y = 16 2X = 26 X = 13 Equations must be linear, variables must have integral values A better solution would solve both boolean and arithmetic constraints

((A > B) OR (B > C)) AND ((A > 3) OR (A < B)) ILP + SAT Use ILP for arithmetic constraints and SAT for boolean constraints ((A > B) OR (B > C)) AND ((A > 3) OR (A < B)) First abstract the arithmetic and solve the boolean (w OR x) AND (y OR z) SAT solution - w is TRUE and y is TRUE Next, solve each arithmetic clause which is TRUE (A > B), (A > 3) 3 > B, A > 3

ILP + SAT Complexities When solving the SAT, many solutions may exist Which clauses will be selected to be TRUE? ((A > B) OR (B > C)) AND ((A > 3) OR (A < B)) (w OR x) AND (y OR z) SAT solution - w is TRUE and z is TRUE Next, solve each arithmetic clause which is TRUE (A > B), (A < B), No Solution A clause selection heuristic is needed - Shortest clause, clause with least common vars, etc. A backtracking mechanism is needed - When conflict occurs, what decisions are reversed?

Constraint Logic Programming (CLP) Need a constraint solving approach which can handle arithmetic and boolean constraints CLP was developed in the AI field but it can be applied to a wode range of problems CLP constaints are declarative: - They express a relationship between between variables which must hold without specifying a computational procedure to enforce the relationship

Declarative vs. Procedural Example: The greatest common divisor between two numbers X and Y Procedural Solution: Write a program which solves computes the GCD - Describe how to solve the problem, step by step. 1. Divide the larger by the smaller and note the remainder. 2. Divide the remainder (92) into the previous divisor. 3. Repeat steps 1 and 2 until the remainder is 1 or zero. 4. When the remainder is zero the last divisor is the GCD. Declarative Solution: Describe the relationship between X, Y, and other variables - Do not describe how to compute the GCD Z. X / Z is an integer, Y / Z is an integer X / Q is an integer, Y / Q is an integer For all possible Q, Z >=Q

CLP Problem Formulation A CLP problem consists of: * a set of variables X={x1,...,xn}, * for each variable xi, a finite set Di of possible values (its domain), * and a set of constraints restricting the values that the variables can simultaneously take. A solution to a CSP is an assignment of a value from its domain to every variable, in such a way that every constraint is satisfied

Internal Constraint Representation An n-ary constraint is one that involves n variables x > 3 is unary, x > y is binary, x + y > z is 3-ary Unary constraints are solved trivially by restricting a variable’s domain. Ex. x > 0, restrict the value of x to positive numbers Binary constraints can be represented as a constraint network x != y y != z z != x X Y Z x != y x != z y != z