1. On Combining Multi-formalism Knowledge to Select Models for Model Transformation Testing Sagar Sen (1 st year PhD student), Benoit Baudry, Jean-Marie Mottu {ssen,bbaudry,jmottu}@irisa.fr IRISA, Campus universitaire de beaulieu, Rennes 35042, France 11st IEEE ICST, April 9-11, 2008

2. Outline The Problem : Model Transformation Testing The Challenges The Solution Results and Discussion Conclusion 1st IEEE ICST, April 9-11, 2008

3. The Problem (1): A Typical MDE Scenario Motivation 1: Fault Propagation Model Composition High-level Model refactoring Code Bugs in Transformations ! Small Bugs in Transformation => Lots of Bugs in Generated Code ! 1st IEEE ICST, April 9-11, 2008

4. The Problem (1): Summary 1st IEEE ICST, April 9-11, 2008 Pre i MT i Post i Input DomainOutput Domain Input meta-model Output meta-model Test model knowledge 1.Model fragments 2.Test Model Objectives specifies specifies test models Question: How to select test models using knowledge from Input meta-model + Pre i + Test Model Knowledge ?

5. Outline The Problem : Model Transformation Testing The Challenges The Solution Results and Discussion Conclusion 1st IEEE ICST, April 9-11, 2008

6. The Challenges(1): To select a model (a complex graph) from a constrained infinite domain Unbounded number of Transitions 1st IEEE ICST, April 9-11, 2008 6 : State isInitial = true isFinal = false label = 1 : State isInitial = false isFinal = false label = 2 : State isInitial = false isFinal = true label = 3 : Transition event = evt1 outgoingTransition source : Transition event = evt2 source outgoingTransition incomingTransition target incomingTransition target : Transition event = evt3 outgoingTransition source incomingTransition target 1 3 evt1 2 evt2 evt3

7. The Challenges(2): To combine the input domain with test model knowledge to automatically select test models 1st IEEE ICST, April 9-11, 2008 Meta-model Model Transformation Pre-condition Test Model Knowledge 1.Test Model Objectives 2.Model Fragments Test Model Knowledge 1.Test Model Objectives 2.Model Fragments Graphs Test Models specifies

8. Outline The Problem : Model Transformation Testing The Challenges The Solution Results and Discussion Conclusion 1st IEEE ICST, April 9-11, 2008

9. The Solution(1): Combining Knowledge to Common Constraint Language 1st IEEE ICST, April 9-11, 2008 ECORE Model OCL Constraints OCL Constraints OCL Constraint OCL Constraint Requirements/Na tural Language Objects and Properties Meta-model Pre-condition Test Model Objectives Model Fragments Common Constraint Language: Alloy (First order relational logic) Common Constraint Language: Alloy (First order relational logic) expressed as transformed to

10. The Solution(2): Why Alloy? Why not Prolog for instance? 1st IEEE ICST, April 9-11, 2008 10 1.Ability to specify quantified constraints such as some, all, one, lone on a set of objects (not available in Prolog) 2.Ability to synthesize objects that conform to meta-level constraints. In Prolog we are limited to solving for values of variables in existing or pre-created objects. 3.Uses high-speed state-of-the-art SAT solvers compared to back- tracking solvers used by Prolog 4.Language syntax similar to Object Constraint Language which is high-level and closer to natural language. As opposed to first- order logic based predicates in Prolog. 5.Alloy supports Transitive Closure

11. The Solution(3): Transforming (abstracting) an ECORE model to Alloy 1st IEEE ICST, April 9-11, 2008 Lets take the Classifier concept from the UML Class Diagram meta-model Transformed to an Alloy Signature abstract sig Classifier { name : Int } String is abstracted to an integer for simplicity Lets take the Class concept concept that inherits from Classifier Transformed to an Alloy Signature sig Class extends Classifier { is_persistent: one Bool, parent : lone Class, attrs : some Attribute }

12. The Solution(4): Transforming multiplicity constraints to Alloy 1st IEEE ICST, April 9-11, 2008 Containment is transformed to an Alloy fact fact Association_containers { (all o : Association |o in ClassModel.association) } Composite is transformed to an Alloy fact fact ClassModel_association_composite { all o1 : ClassModel, o2 : ClassModel |all p1 : o1.association, p2 : o2.association |p1 = p2 implies o1 = o2 } Other facts are created for multiplicity, opposite, ID, unique properties.

13. The Solution(5): Transforming OCL Constraints (from meta-model and pre-condition) 1st IEEE ICST, April 9-11, 2008 No Cyclic Inheritance inherits ClassA : Class ClassB: Class ClassC: Class inherits name: String is_persistent:Bool name: String is_persistent:Bool name: String is_persistent:Bool ? OCL Version: context Class: inv : not self.allParents()->includes(self) Alloy Version: fact noCyclicInheritance { no c: Class | c in c.^parent } manually to an Alloy fact

14. The Solution(6): Transforming Test Model Objectives Consider the objective expressed in natural language: There has to be some attributes which are non-persistent Alloy Predicate: pred testers_requirement2 { some a:Attribute | a.type.is_persistent=False } manually transformed to an Alloy predicate 1st IEEE ICST, April 9-11, 2008

15. The Solution(7): Transforming Model Fragments pred modelFragment1 { some a1:Association,a2:Association|#a1.src=0 and #a2.src=1 } Consider the model fragment (partition of the input domain) obtained using MMCC (Frank Fleurey et al. 2007) on the UML Class Diagram meta-model Association(source=0) and Association(source =1) At least one Association object with no source and one Association object with one source automatically transformed to an Alloy predicate 1st IEEE ICST, April 9-11, 2008

16. The Solution(8): Current Limitations 1st IEEE ICST, April 9-11, 2008 What’s Cool ? 1. ECORE to Alloy transformation: Automated 2. Alloy XML to ECORE XMI with conformance: Automated 3. Transforming Model Fragments to Alloy Facts: Automated What’s not yet? 1. OCL constraints to Alloy : Manual 2. Test Model Objectives to Alloy: Manual 3. Limited mathematical functions in Alloy to model constraints How? 1. Identifying a subset of OCL transformable to Alloy 2. Separating arithmetic and design Similar tools: UML2Alloy, UML2CSP

17. Outline The Problem : Model Transformation Testing The Challenges The Solution Results and Discussion Conclusion 1st IEEE ICST, April 9-11, 2008

18. Results and Discussion(1): Output of Transformation : An Alloy Model 1st IEEE ICST, April 9-11, 2008 module tmp/simpleUMLCD sig ClassModel { classifier: set Classifier, association: set Association } abstract sig Classifier { name : one Int }. Classes transformed to Alloy Signatures //There is exactly one ClassModel object fact oneClassModel { one ClassModel } //All Classifier objects are contained in a ClassModel fact classifierContainment { all c:Classifier | c in ClassModel.classifier } //All Association objects are contained in a ClassModel fact associationContainment { all a:Association | a in ClassModel.association } OCL transformed to Alloy Facts

19. Results and Discussion(2): Output of Transformation : An Alloy Model 1st IEEE ICST, April 9-11, 2008 pred testers_requirement6 { some c:Class|c.is_persistent=True } pred testers_requirement7 { some c:Class |some a:c.attrs| a.name=c.name and a.type=Class and a.is_primary=True } pred testers_requirement8 { some cA:Class | some aA :cA.attrs, pA:cA.parent.attrs | aA.name==pA.name } Test Model Knowledge transformed to Alloy Predicates

20. Results and Discussion(3): Selection of Test Models 1st IEEE ICST, April 9-11, 2008 The run command: run test_requirement1 for 1 ClassModel,5 int, exactly 5 Class, exactly 20 Attribute, exactly 4 PrimitiveDataType, exactly 5 Association Output: Alloy model instance that satisfies meta-model + pre-condition + test_requirement1 and has the specified size STEPS 1.Selecting the exact number of objects for each Class/Concept 2.Or, selecting a scope for each Class/Concept 3.Selecting a scope for Integers 4.Synthesizing a model using the run command in Alloy

21. 1st IEEE ICST, April 9-11, 2008 CARTIER: OVERALL FRAMEWORK C C Source of Knowledge Cartier Framework: Overview

22. 1st IEEE ICST, April 9-11, 2008 CARTIER: OVERALL FRAMEWORK C C Transformed to Alloy -> CNF -> Solved using a SAT Solver Cartier Framework: Overview

23. 1st IEEE ICST, April 9-11, 2008 CARTIER: OVERALL FRAMEWORK C C Transformed to ECORE XMI Which conforms to meta-model Transformed to ECORE XMI Which conforms to meta-model Cartier Framework: Overview

24. 1st IEEE ICST, April 9-11, 2008 CARTIER: OVERALL FRAMEWORK C C Model Transformation Execution Cartier Framework: Overview

25. 1st IEEE ICST, April 9-11, 2008 CARTIER: OVERALL FRAMEWORK C C Output

26. Results and Discussion(6): Example Output: MM Only 1st IEEE ICST, April 9-11, 2008 No Primary Attribute Knowledge Level MM

27. Results and Discussion(7): Example Output: MM Only + Pre 1st IEEE ICST, April 9-11, 2008 All classes have at least one primary attribute MM+Pre Knowledge Level MM

28. Results and Discussion(8): Example Output: MM + Pre + TMO 1st IEEE ICST, April 9-11, 2008 MM+Pre Knowledge Level MM MM+Pre + TMO There is at least one attribute with the same type of the container Class and all Classes have at least one primary attribute

29. Results and Discussion(9): Example Output: MM + Pre + MF 1st IEEE ICST, April 9-11, 2008 MM+Pre Knowledge Level MM MM+Pre + MF and all Classes have at least one primary attribute There are some Classes with is_persistent= False and some with is_persistent=True

30. Outline The Problem : Model Transformation Testing The Challenges The Solution Results and Discussion Conclusion 1st IEEE ICST, April 9-11, 2008

31. Conclusion 1st IEEE ICST, April 9-11, 2008 1.Testing model transformations is a challenge (like many other problems in life) 2.Gathering test model knowledge can be useful to select test models. 3.We present a methodology to combine various forms of knowledge to a common constraint languag e: Alloy 4.We solve the Alloy model to obtain test models which we automatically transform to XMI (Eclipse Modelling Framework standard) => Academia -> Industry transfer 5.We show that these models can be used to test model transformations Future 6. We want to numerically estimate via mutation analysis the efficiency of test models 7. Design experiments to test model transformations

32. Thank you! 1st IEEE ICST, April 9-11, 2008