272: Software Engineering Fall 2012 Instructor: Tevfik Bultan Lecture 9: Test Generation from Models

Test Data Generation from Models Object-Role Modeling (ORM): A modeling language for specifying relational data models –Kind of like UML class diagrams or entity-relationship diagrams –Has formal semantics It allows the developer specify a data model with various constraints Used in industry Problem: –Satisfiability: Given an ORM diagram, are there any instances that satisfy the constraints specified in the diagram Test generation: Given a satisfiable ORM diagram, generate instances that satisfy the constraints specified in the diagram

ORM diagrams Ovals show entities or values (basic types) –These are sets of objects or values Rectangles show predicates –These represent the relations among the objects –They correspond to the tables in a relational database

ORM constraints Uniqueness constraint: Occurrence of a value in any column of a predicate is unique Mandatory constraint: Every object of an entity is implicitly assumed to participate in some of the predicates in which the entity has a role Frequency constraint: Specifies how many times a tuple can occur in a predicate Subset, equality constraints: It specifies that the tuples in the roles of the first predicate have to be among the tuples in the roles of the second Subtype constraints: Used to model subtyping among entities Value and cardinality constraints: Specify a range of values for primitive type or a cardinality (which can be a number or a range) for an entity Exclusion constraints: specifies that a particulate value can occur online in one table Ring constraints: Constraints on predicates (symmetric, acyclic, transitive, intransitive, reflexive, irreflexive, asymmetric)

Results For the most general case where constraints on the diagrams are specified using arbitrary queries –Satisfiability is undecidable For a more restricted class of diagrams, satisfiability checking is NP- hard Brute-force checking of satisfiability (bounded-exhaustive testing) does not scale For a class of diagrams that the authors define called ORM-, the satisfiability can be checked in polynomial time

NP-Hardness of ORM satisfiability NP-hardness proof shows that well-known NP-complete boolean satisfiability problem, 3SAT, can be reduced to an instance of ORM model satisfiability A 3-CNF formula has the following form: (c 1,1  c 1,2  c 1,3 )  (c 2,1  c 2,2  c 2,3 )  … where each c i,j represents expressions of the form q i or  q i over a set of boolean variables q 1, q 2, …, q n. We call (c i,1  c i,2  c i,3 ) clause c i. Determining satisfiability of a 3-CNF formula is an NP-complete problem A 3-CNF formula can be translated to a ORM diagram, such that, the 3-CNF formula is satisfiable if and only if the generated ORM diagram is satisfiable

NP-Hardness of ORM satisfiability 3-CNF formula to ORM diagram translation Introduce an object type for each q i Introduce an object type for each clause c i Introduce a value constraint for each of the object types above where their value is restricted to a single value qi or ci For each variable q i let n be the number of times it occurs in the clauses. If n > 0, create a (n+1)-ary predicate q i -true For each expression  q i let m be the number of times it occurs in the clauses. If m > 0, create a (m+1)-ary predicate q i -false Connect type q i to predicates q i -true and q i -false Impose and exclusion constraint on the roles played by q i in q i -true and q i -false (meaning that q i is either true or false, but not both) Connect each clause c i to the predicate q i -true if it contains q i or q i - false if it contains  q i

Brute-force approach As it is possible to translate satisfiability of boolean formulas to satisfiability of ORM diagrams, it is also possible to translate satisfiaibility of ORM diagrams, satisfiability of boolean formulas The authors tried this approach They translated ORM diagrams to input languages of CoBaSa and Alloy Analyzer –These are tools that can check satisfiability queries about bounded models –They use SAT solvers as back-end solvers Experiments with this approach shows that ORM satisfiability checking using a SAT solver is not efficient and does not scale to large diagrams

ORM- Given the theoretical difficulty of the general ORM satisfiability problem and the practical inefficiency of the brute-force approach –the authors define a subset of the ORM language for which satisfiability can be determined efficiently. ORM- allows only a subset of ORM constraints Uniqueness constraints cannot overlap Mandatory constraints are allowed on a single role Frequency constraints cannot overlap Value and cardinality constraints are allowed Subtype constraints are allowed According the authors majority of constraints used in practice are covered by this subset. Note that, subset constraints, exclusion constraints and ring constraints are not allowed.

Testing Satisfiability of ORM- Using a transformation from the ORM- to integer constraints the authors show that the satisfiability of the ORM- can be checked by checking satisfiability of the resulting integer constraints They also show that the satisfiability of the resulting integer constraints can be checked in polynomial time

Converting to integer constraints For each type A introduce a variable a representing its cardinality. For each predicate R, introduce a variable r denoting its cardinality. For each role in R played by A introduce a variable r a representing the number of unique elements from A in this role Produce inequalities: r a ≤ a and r a ≤ r For each cardinality constraint for a type A such as min … max produce inequalities: min ≤ a and a ≤ max –Do the same for the predicates. The default min value is 1 and the default max value is M (which could be set to MaxInt) For each value constraint let ds denote the domain size for the value domain, generate the inequality: a ≤ ds For each mandatory constraint for a role played by type A in predicate R, generate the inequality: a ≤ r a

Converting to integer constraints For each frequency constraint with frequency with range f min, …, f max on the roles played in predicate R by types A, B, …, K, introduce the variable r ab…k representing the number of unique tuples in these roles and produce the following inequalities: r ≤ f max. r ab…k f min. r ab…k ≤ r r ab…k ≤ r a. r b. …. r k r a ≤ r ab…k, r b ≤ r ab…k, …, r k ≤ r ab…k For each subtype constraint between types A and S produce the inquality: a ≤ s

Converting to integer constraints For each predicate R with roles played by types A, B, …, N express the implicit uniqueness constraint over all roles by producing the inequality: r ≤ r ab…k. r lm…p. …. r n where right hand side are variables that correspond to ranges of roles that participate in some frequency constraints, followed by the variables corresponding to roles that are under no frequency constraint For each non-independent type A that plays roles in predicates R, S, …., V introduce the constraint: a ≤ r a + s a + … + v a which captures the implicit disjunctive mandatory constraint that each object of a non-independent type needs to appear in some predicate

Properties of this conversion The set of constraints generation by this conversion are satisfiable if and only if the ORM- diagram itself is satisfiable –Hence this approach is both sound and complete This means that we reduced the satisfiability of the ORM- diagrams to satisfiability of a set of integer constraints Satisfiability of integer constraints is a difficult problem itself. However, the integer constraints generated with this conversion are of a specific type for which the satisfiability can be checked efficiently as the authors show

Checking satisfiability of constraints Rewrite all inequalities of the form c. x ≤ y 1. y 2. …. y n as form x ≤ (y 1. y 2. …. y n ) / c For each variable maintain its current upper bound. In every step substitute the current upper bounds for all variables on the rhs and after arithmetic manipulation obtain upper bound candidates for each variable and choose the smallest one Repeat until either an upper bound crosses a lower bound or all upper bounds remain unchanged The algorithm runs in polynomial time because if there are n variables in the system, each inequality can yield a different upper bound at most n times.

Test Generation Any set of integer values satisfying the generated integer constraints can be used to construct an instance of the ORM- data model These instances can then be used as test cases for testing a database So the overall approach would be: –Given a database, specify an ORM- diagram that characterizes a set of test cases for testing the database –First check the satisfiability of the ORM- diagram by generating integer constraints as described above and then check the satisfiability of the generated integer constraints –If the integer constraints are satisfiable, find integer values that satisfy the constraints and convert those integer values the instances of the ORM- model