MOdel-based GENeration of Tests for Embedded Systems # FP7-ICT Embedded Systems Design Slide 1 FMCO, Graz, Nov. 30 th 2010 Test Cases for CAS - UML/AS (1)  Model(s):  Final  Changed to use 1 orthogonal region to meet the required behaviour  Requirements markup incomplete (little effort)

MOdel-based GENeration of Tests for Embedded Systems # FP7-ICT Embedded Systems Design Slide 2 FMCO, Graz, Nov. 30 th 2010 Test Cases for CAS - UML/AS (2)  15 of the available mutation operators applicable  20 locations  111 mutants (no equivalent ones!)  152 generated TCs (minimal search depth, no optim.)  manual and gen. TCs avaliable in svn repository) Manual Set„Minimal“ generated set Source8 use cases8 mutants I/O Steps70137 Accumulated time 470s800s Found mutants83 / 75%111 / 100%

MOdel-based GENeration of Tests for Embedded Systems # FP7-ICT Embedded Systems Design Slide 3 FMCO, Graz, Nov. 30 th 2010 Test Cases for CAS - UML/AS (3) Test casesMutants

MOdel-based GENeration of Tests for Embedded Systems # FP7-ICT Embedded Systems Design Slide 4 FMCO, Graz, Nov. 30 th 2010 Test Cases for CAS - UML/AS (4):  Strongest generated test case covers 75% of mutants, weakest only 1 mutant  Weakest test case is needed! (short, quiescence)  Longest 4 test cases from minimal set from the 8 mutants cover 96%  Test cases for „guard false“ mutation operator cover over 90 %  1 mutant is found by manual tests but not by guard false mut-op  All mutants from minimal set are located at transitions  Changing code chunks covers removing the containing code snippet

MOdel-based GENeration of Tests for Embedded Systems # FP7-ICT Embedded Systems Design Slide 5 FMCO, Graz, Nov. 30 th 2010 Test Cases for CAS - UML/AS (5):  Open issues:  Generate test cases for missing mutations (when available)  Apply requirements tracing (tools incomplete)  Apply test cases to Simulink model and HIL Supply XSLT conversion style sheet to FFA  Review/evaluate test cases  Inter-Track verifications: Coverage of mutants by white box tests Use Action System as an oracle for white box testing

MOdel-based GENeration of Tests for Embedded Systems # FP7-ICT Embedded Systems Design Slide 6 FMCO, Graz, Nov. 30 th 2010 Test Cases for RELAB - UML/AS (1):  Model:  Final  Split into several model variants by enabling only a subset of the inputs Startup phase No driver ISOBUS errors Errors and Joystick X axis move (1 Position) Equivalence classes for joystick positions yielding different display images Equivalence classes with borders

MOdel-based GENeration of Tests for Embedded Systems # FP7-ICT Embedded Systems Design Slide 7 FMCO, Graz, Nov. 30 th 2010 Test Cases for RELAB - UML/AS (2):  Mutants applied:  Available subset is applied  13 mutations operators  233 Mutants  Test cases generated  66 (non-duplicate) test cases for nodriver variant available in SVN repository  TCG is ongoing at TUG

MOdel-based GENeration of Tests for Embedded Systems # FP7-ICT Embedded Systems Design Slide 8 FMCO, Graz, Nov. 30 th 2010 Test Cases for RELAB - UML/AS (3):  Open issues:  Complete test case generation for current mutation set  Extend test cases for rest of mutation operators  Apply requirements tracing (tools incomplete)  Apply test cases to PTD Already supplied XSLT conversion style sheet to RELAB  Review/evaluate test cases  Inter-Track verifications: Coverage of mutants by white box tests Use Action System as an oracle for white box testing

MOdel-based GENeration of Tests for Embedded Systems # FP7-ICT Embedded Systems Design Slide 9 FMCO, Graz, Nov. 30 th 2010 Test Cases for ELEKTRA - UML/AS (1):  Model:  Some elements still modeled incomplete  Does not yet import cleanly into VIATRA (OCL errors)  Mutants applied:  None  Test cases generated:  None

MOdel-based GENeration of Tests for Embedded Systems # FP7-ICT Embedded Systems Design Slide 10 FMCO, Graz, Nov. 30 th 2010 Test Cases for ELEKTRA - UML/AS (2):  Open issues:  Complete model  Manually model minimal station in UML  Check for errors in OOAS trafo  Get ELEKTRA model export of meeting station  Configure station import tool  Apply import  Generate mutants  Generate test cases  „Partial Order Reduction“ test cases for application to test environment  Add requirements info to model for tracing

MOdel-based GENeration of Tests for Embedded Systems # FP7-ICT Embedded Systems Design Slide 11 FMCO, Graz, Nov. 30 th 2010 Test Cases for ELPult- UML/AS (1):  Model:  Has errors within OOAS trafo  Mutants applied:  None  Test cases generated:  None

MOdel-based GENeration of Tests for Embedded Systems # FP7-ICT Embedded Systems Design Slide 12 FMCO, Graz, Nov. 30 th 2010 Test Cases for ELPult - UML/AS (2):  Open issues:  Fix model and test OOAS trafo  Get ELEKTRA model export of meeting station  Configure station import tool  Apply import  Generate mutants  Generate test cases  Add requirements info into model for tracing

MOdel-based GENeration of Tests for Embedded Systems # FP7-ICT Embedded Systems Design Slide 13 FMCO, Graz, Nov. 30 th 2010 Mutation Operators  Init values, constants, OCL /AGSL literals:  Constant Value Variation Step  Enum Value Exchange  Set Constant Value to Zero/Max  States:  Remove State Entry/Exit Action  Exchange State Entry/Exit Action Method  Transitions:  Remove Transition  Exchange Transition Source/Target  Exchange Transition Trigger  Triggers  Exchange Trigger Signal  Minimally Change Time Trigger Duration  Substantially Change Time Trigger Duration  Minimize/Maximize Time Trigger Dur.  Modify Change Trigger Expression  Exchange Call Trigger Method  Guards:  Fix Guard Value  Invert Guard  Modify Guard Expression  Effect:  Remove Effect  Exchange Effect Method  Modify Effect Body  OCL:  Fix Expression  Negate Expression  Fix Subexpression  Negate Subexpression  Modify Boolean Operator  Modify Relational Operator  Modify Arithmetic Operator  Modify Set Operator  Modify Quantifier  AGSL:  Remove Statement  Reorder Statement  Fix Parameter/Property  Modify Parameter/Property  Modify Operator  Fix Operand  Modify Operand  Fix Result