1. Computer-aided Selection of Test Models for Model Transformation Testing Sagar Sen Benoit Baudry IRISA/INRIA Campus Universitaire de Beaulieu, Rennes 35000, France

2. Outline What is a Model Transformation ? Why i s it important to Test Transformations ? What are the Challenges involved? Solution to Test Model Selection Problem Conclusion What is Test Model Selection in the Testing? Overview of the Transformation Testing Framework

3. What is a Model Transformation ? 1. A model transformation MT(S,T) is a program on a set of input models S and output models T 2. The set of source models S are specified by meta-model MM S 3. The set of target models T are specified by meta-model MM T 4. A model transformation may have a pre-condition MT pre and a post-condition MT post MM S MM T specifies set S specifies set T MT(S,T) MT pre MT post

4. Applications of Model Transformations 1. Model transformation pipelines such as XML to XML Pipelines (Eg: Xproc) 2. Code Generators (Eg: Simulink Model to Code) 3. UML Class Diagram serialization to RDBMS (Eg: Hibernate 3.0) 4. Simulation Traces (Eg: Simulink, Modelica) 5. UML Model Design Re-factoring (Eg: Rhapsody) 7. Code migration (Cobol to Java/C#, VPLUS UI to XML/J2EE ) 8. Game development pipelines (Eg: XNA Studio) 9. Business Process Models (Eg: Websphere business process model) 10. Code reverse engineering...

5. Why test model transformations ? A Typical Model-driven Development Process Model Motivation

6. Why test model transformations ? A Typical Model-driven Development Process Model Motivation Model Composition

7. Test Input Domain (1) G (Set of all Graphs) specifies UML Class Diagram Ecore Model (OMG Standard for representing Model Domain) ECORE UMLCD Model Instance

8. Test Input Domain (2) Is this fragment of a model a valid Class Diagram ? inherits ClassA : Class ClassB: Class ClassC: Class inherits We need additional constraints that cannot easily be expressed using a diagram. name: String is_persistent:Bool name: String is_persistent:Bool name: String is_persistent:Bool context Class: inv : not self.allParents()->includes(self) OCL Constraint : No Cyclic Inheritance ?

9. Test Input Domain (3) : +OCL Constraints G (Set of all Graphs) specifies UML Class Diagram Ecore Model ECORE UMLCD + OCL UMLCD Model Instance context Class: inv : not self.allParents()->includes(self) OCL Constraints on Ecore Model context ClassModel inv uniqueNameClassifier : self.classifier−>forAl l ( cl1, cl2 | cl1.name=cl2.name implies cl1=cl2 ) inv uniqueNameAssoSrc : self.association->forAll (ass1, ass2 | ass1.name=ass2.name implies (ass1=ass2 or ass1.src != ass2.src)) specifies

10. MM UMLCD MM RDBMS specifies set UMLCD MT prepost Test Input Domain (4) : +MT Pre-condition expressed in OCL on ECore G (Set of all Graphs) ECORE UMLCD + OCL UMLCD + pre(MT) specifies

11. Test Input Domain (5) + Testers Requirements 1. Partitions on input domain based on Partitioning Criteria : Category Partitioning 2. Number of objects of a particular type 3. Partial model Instance 4. Mutation analysis score based on output of model transformation 5. Satisfaction of post-condition G (Set of all Graphs) ECORE UMLCD + OCL UMLCD + pre(MT)+Requirements specifies Test Input Domain+ Constraints =Test Model Knowledge

12. Test Model Knowledge 1. A set of model fragments (partition on the input domain) as sets of constraints or partial instances 2. A set of model requirements expressed as constraints on the meta- model 3. The model transformation pre-condition MT pre Partial InstancePre-condition Object instances of MM S expressed as Constraints on context MM S expressed as Requirements Constraints on context MM S expressed as New test direction Constraints on context MM S Or Object instances as +ve or –ve application conditions expressed as Common Constraint Language 1.Analyzable: Do the constraints conflict? 2.Test Model Instance Generation 3.Sufficiently expressive (first-order logic and beyond) 4.Large libraries of predefined constraints Meta-model ECore+OCL expressed as

13. Test Model Selection : Combining Partial Knowledge and Automation Test Model Generation Model Transformation Testing Environment Test Model Qualification Test Model Knowledge refines used by results used by updates

14. Common Constraint Language : Test Model Generation and Analysis Prolog Alloy 1.A partial instance must be Specified 2. Uses first-order horn clause Logic 3. Cannot specify quantifier based constraints 4. Large set of libraries since Prolog is around for more than two decades. 5. Search is based on Back-tracking in a Warren Abstract Machine 6. Constraint syntax is close to first-order logic 1.Anything from the general depth to partial model can be specified 2. Uses first-order relational logic 3. One can specify quantifier based constraints on a set of atoms 4. Smaller constraint library and interface options with programming languages. 5. Relational model is transformed to boolean logic and solved using an efficient SAT solver. 6. Constraint specification close to object orientation and object constraint language. We tried both for model generation!

15. Constraint Logic Programming for Model Completion (1) Finite State Machine Meta-model: Ecore Partial Model Instance in Concrete Syntax with 3 State and Transition Objects Expressed as a Graph in the Himesis Graph Library ModelisInitial = [HF SMStateObject1isInitial; HFSMStateObject2isInitial; HFSMStateObject3isInitial]; ModelisInitial :: [0::1]; ic global : occurrences(1;ModelisInitial;1) CLP Prolog Constraints on Partial Model Instance

16. Constraint Logic Programming for Model Completion (2) :-lib(ic). :-lib(ic_global). solve_Model(Model):- Model=[ Transition0_event, Transition0_exists, Transition1_event, Transition1_exists, Transition2_event, Transition2_exists, State0_currentState, State0_isFinal, State0_isInitial, State0_label, State1_currentState, State1_isFinal, State1_isInitial, State1_label, State2_currentState, State2_isFinal, State2_isInitial, State2_label ], Transition0_event::[1,2,3,4,5], Transition0_exists::[0..1], Transition1_event::[1,2,3,4,5], Transition1_exists::[0..1], Transition2_event::[1,2,3,4,5], Transition2_exists::[0..1], State0_currentState::[0..1], State0_isFinal::[0..1], State0_isInitial::[0..1], State0_label::[0..100], State1_currentState::[0..1], State1_isFinal::[0..1], State1_isInitial::[0..1], State1_label::[0..100], State2_currentState::[0..1], State2_isFinal::[0..1], State2_isInitial::[0..1], State2_label::[0..100],. ModelisInitial = [State1_isInitial; State2_isInitial; State3_isInitial]; ModelisInitial :: [0::1]; ic global : occurrences(1;ModelisInitial;1) labeling(Model). Constraint Logic Program in Python (SWI-Prolog) back-tracking CLP

17. Disadvantage of using Prolog 1. Constraints are expressed on model instances rather than on meta-model elements. 2. A constraint template needs to be designed that can be applied to any partial model instance by applying constraint of all relevant properties by traversing the partial model. 3. General requirements such as depth or number of type of objects in the desired model instance cannot be easily specified in the language (They have to generated) 4. Only first-order horn clause logic is supported. 5. Its quantifier free. Constraints such as all x:Class | x.is_persistent=true cannot be expressed. 6. The constraint expression is close to theoretical logic and not more natural like OCL. 7. Traditional back-tracking is slower than state-of-the art SAT solvers used for hardware verification. Solution: First-order Relational Logic implemented in Alloy with KodKod Relational Model Finder

18. First-order Relational Logic for Model Generation using Alloy(1) module tmp/simpleUMLCD open util/natural as Natural open util/boolean as Bool sig ClassModel { classifier: set Classifier, association: set Association } abstract sig Classifier { name : one Natural } sig PrimitiveDataType extends Classifier {} sig Class extends Classifier { is_persistent: lone Bool, parent : lone Class, attrs : some Attribute } sig Association { name: Natural, dest: one Class, src: one Class } UML Class Diagram Meta-model in Ecore Alloy Model for Ecore Meta-model Similar tool: UML2Alloy

19. First-order Relational Logic for Model Generation using Alloy(2) fact noCyclicInheritance { no c: Class | c in c.^parent } fact uniqueAttributeNames { all c:Class | all a1: c.attrs, a2 : c.attrs | a1.name==a2.name => a1 = a2 } fact allDifferentAttributeObjects { all c1:Class, c2:Class | all a1:c1.attrs, a2:c2.attrs | a1==a2 => c1=c2 } fact uniqueClassifierName { all c1:Classifier, c2:Classifier | c1.name==c2.name => c1=c2 } fact uniqeNameAssocSrc { all a1:Association, a2:Association | a1.name == a2.name => (a1 = a2 or a1.src != a2.src) } Alloy Facts for OCL Constraints

20. First-order Relational Logic for Model Generation using Alloy(3) fact someInheritance { some Class.parent } fact atleastOneAttribute { all c:Class | #c.attrs > 3 } Alloy Facts for Requirements Alloy Run Statement for Generating Objects of different types and depth pred example( ) {} run example for 10 //Depth of 10 run examplea with exactly 1 ClassModel, 2 Class, 3 Association, 20 Attribute Alloy Check Statement for Checking Assertions on Meta-model assert atleastOneAttribute { all c:Class | #c.attrs > 3 } check atleastOneAttribute for 10 //Finds a counterexample if the meta-model does not satisfy this requirement for a depth of upto 10

21. First-order Relational Logic for Model Generation using Alloy(4) : Solving the Model Instance Alloy Model Boolean CNF Formula Model Instance Generated SAT Solver

22. Conclusion 1. Model Transformation Testing involves the discovery of test input models and black-box testing is the most evident way to go. 2. We present a way to use Partial Model Knowledge, Constrained Input Domain Knowledge expressed as a Meta-model, Tester's requirements, and Model Transformation pre-condition all expressed in different languages to a common constraint language. 3. We experiment with Prolog and with Alloy and conclude that Alloy is the way to go. 4. We intend to use the power of first-order relational logic to develop meta-models and constraints on them to refine the knowledge base both manually and automatically. 5. We intend to build a connection with the imperative programming world where Ecore models can be used in model transformations while we use declarative programming (first-order relational logic) to generate test models from the Ecore.