1. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Institute for Software Technology – Graz University of Technology 1 Willibald Krenn Survey on test case generation Survey – Test Generation Willibald Krenn, TUG

2. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Slide 2 3rd June 2008 Survey on automated testcase generation Models – In a Nutshell (1)  Model-Based Specification  VDM, Z, B, JML, Spec#, CIRCUS, ALLOY, OCL, PiSPEC  Challenge: Sequence Finding (Build FSM)  Examples in Survey: Z, B  Abstract Data-Types (algebraic)  Raise, OBJ, Larch, CASL, ELOTOS (data part)  Test whether implementation fulfills axioms (Challenge: Equality!)  Examples in Survey: Container for Natural Numbers, Sort  Process Algebras  CCS, CSP, LOTOS  Specify behavior  Currently no example in survey  Test-case generation, e.g. based on LTS

3. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Slide 3 3rd June 2008 Survey on automated testcase generation Models – In a Nutshell (2)  Labeled Transition Systems  LTS, IOSTS  LTS: Transition system; Labels (e.g. input/output symbols) on edges;  Test case generation: e.g. test purpose multiplied with Spec  Kripke Structures, Temporal Logic  Example: LTL properties for a brake  Test case generation: e.g. by model checking (counter example)  EFSM, State Charts  State Charts similar to hybrid automata („do“ keyword)  Example in Survey: UML state diagram for ATM withdrawl  Test: State/Transition Equivalence  Hybrid  Hybrid Automata  Hybrid I/O Transition Systems

4. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Slide 4 3rd June 2008 Survey on automated testcase generation Tools – Model Based Spec.  Random Testing  JET JML annotations test data generation  Directed Testing (White Box)  DART, EXE, CUTE / jCUTE, Pex Implementation gets instrumented Do symbolic and concrete execution From symbolic execution: Calculate input data to get high coverage (branches!)  Multi Strategy  Spec Explorer Model-Program State exploration

5. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Slide 5 3rd June 2008 Survey on automated testcase generation Tools – Testing from LTS  Testing with Purposes  TGV, STG Purpose some LTS Multiply specification with purpose and generate tests  AGATHA Symbolic execution of STS Purpose is a sub-tree  Random  TorX On-The-Fly  Fault based (mutation)  Mutate specification, extract discriminating sequence, use sequence as test purpose

6. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Slide 6 3rd June 2008 Survey on automated testcase generation Tools – Testing with Model Checkers  Coverage Criteria  UPPAAL Testing real-time systems Supports generation of test suites that satisfy certain coverage criteria  FShell (white box) White-Box testing of C programs Uses CBMC Control Flow Graph coverage by employing SAT solver

7. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Slide 7 3rd June 2008 Survey on automated testcase generation Tools – Testing from EFSM, State Charts  Purposes  Autolink Creates TTCN test suites Spec given in SDL, Purpose in MSC  TDE/UML Takes UML behavior models annotated with additional test data. E.g., coverage requirements, constraints Generates test cases for C++/Java Eclipse Plug-In

8. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Slide 8 3rd June 2008 Survey on automated testcase generation Fault Injection Techniques  Provided by SP  Gives an overview of different techniques to inject faults into embedded designs  Fault injection is used as experimental dependability validation method.

9. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Slide 9 3rd June 2008 Survey on automated testcase generation Case Studies  GSM 11-11  B-Testing-Tool used for test case generation  Successful  AGEDIS  TGV, GOTCHA used for test case generation  Most interesting: „Lessons learnt“  ASML EUV machine  Model-based integration and testing method  Addresses costly integration problems  TorX used for test case generation  Successful

10. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Slide 10 3rd June 2008 Survey on automated testcase generation Thank you!

11. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Slide 11 3rd June 2008 Survey on automated testcase generation B - Example

12. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Slide 12 3rd June 2008 Survey on automated testcase generation ADT - Example

13. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Slide 13 3rd June 2008 Survey on automated testcase generation LTS - Example !a ?b !c

14. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Slide 14 3rd June 2008 Survey on automated testcase generation LTL - Example

15. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Slide 15 3rd June 2008 Survey on automated testcase generation State Chart - Example

16. MOdel-based GENeration of Tests for Embedded Systems #216679 FP7-ICT-2007-1-3.3 Embedded Systems Design Slide 16 3rd June 2008 Survey on automated testcase generation Hybrid Automaton - Example