Overview of the ETSI Test Description Language Andreas Ulrich (Rapporteur) Siemens AG andreas.ulrich@siemens.com © ETSI 2012. All rights reserved 2nd MBT User Conference 2012 Tallinn, Estonia September 25–27, 2012

2nd MBTUC 2012, Tallinn, Estonia Overview What can I do with TDL? What are the concepts behind TDL? How does TDL compare to other approaches? How does TDL look like to users? What comes next? 2nd MBTUC 2012, Tallinn, Estonia

2nd MBTUC 2012, Tallinn, Estonia What can I do with TDL? 2nd MBTUC 2012, Tallinn, Estonia

2nd MBTUC 2012, Tallinn, Estonia Purpose of TDL Scope Black-box testing of distributed communicating real-time systems Design/representation of test descriptions before implementing them Using different representation syntaxes Indifferent to application domains Telecommunication / automotive / avionics / automation / others Application scenarios Manual specification of tests from use cases, user stories etc. Intermediate representation of automatically generated tests Representing output from MBT tools Generation of Executable tests Test documentation Visualization of test logs 2nd MBTUC 2012, Tallinn, Estonia

An Application Scenario Example for TDL Test Purpose Spec (ExTRA) System Model (State Chart) Automatic Test Case Generation Manual Derivation In Syntax #1 Test Description (TDL) Test Documentation (MS Word) Generation In Syntax #2 Automatic Code Generation Executable Tests (TTCN-3) 2nd MBTUC 2012, Tallinn, Estonia

2nd MBTUC 2012, Tallinn, Estonia TDL Specifies… Interactions between SUT/tester at observable interfaces (only) System inputs/outputs: messages, calls, signals Expected flow of interactions Sequential, alternative, concurrent, recurrent behavior, etc. Mock-up (default) behavior, others Timing constraints System response time (system in/out, out/out constraints) Tester reaction time (system out/in constraints) Abstract test data Relevant values Constraints on test data, i.e. ranges or intervals Parameterization of test descriptions 2nd MBTUC 2012, Tallinn, Estonia

What are the concepts behind TDL? 2nd MBTUC 2012, Tallinn, Estonia

Ingredients of a Software Language Abstract Syntax (abstract syntax graph, meta-model) Concrete Syntax (syntax tree, BNF grammar) Semantics (transitional or operational sem.) 2nd MBTUC 2012, Tallinn, Estonia

2nd MBTUC 2012, Tallinn, Estonia Coverage by TDL Abstract Syntax Standardized, incl. extension capabilities Concrete Syntax At least one standardized representation Semantics Standardized 2nd MBTUC 2012, Tallinn, Estonia

2nd MBTUC 2012, Tallinn, Estonia Using TDL Static analysis Consistency checks, visualization Common TDL meta-model Test concepts expressed in a single abstract syntax Test generation Transformation Abstract test descriptions Various user-defined syntaxes, e.g. graphical, tabular, textual Executable tests Various test environments and languages, e.g. TTCN-3, C# 2nd MBTUC 2012, Tallinn, Estonia

TDL is Adjustable by User TDL concrete syntax may cover only parts of the meta-model TDL meta-model can be extended by a user if need arises User extensions of the TDL meta-model can be subjected to further TDL standardization and maintenance Extends Partially implements User-defined concrete syntax TDL meta-model w/ user extensions 2nd MBTUC 2012, Tallinn, Estonia

How does TDL look like to users? 2nd MBTUC 2012, Tallinn, Estonia

Examples in Possible TDL Concrete Syntax Sequential test with multiple SUT ports Structured test with default behavior Note: Examples only, concrete syntax is definable by user! 2nd MBTUC 2012, Tallinn, Estonia

Examples in Possible TDL Concrete Syntax A concurrent test in graphical and textual notation testdescription joinTD() { UCAROL ! join(Carol, room1); par { { BOB ? joinPDU(Carol, ?); BOB ! joinPDU(Bob, room1) }, { ALICE ? joinPDU(Carol, ?); ALICE ! joinPDU(Alice, room1) } }; UCAROL ? joined() } A test scenario model Note: Examples only, concrete syntax is definable by user! 2nd MBTUC 2012, Tallinn, Estonia

How does TDL compare to other approaches? 2nd MBTUC 2012, Tallinn, Estonia

TDL vs. UML Testing Profile UTP Focus on test descriptions of tester/SUT interactions Various concrete syntaxes Common meta-model with semantics fosters reuse of tests and tools Amendable to user’s needs – extension, reduction of scope Wide scope inherited from UML, covers all kinds of test modeling Graphical language Tool-specific solutions, models hardly transferrable Captures not all needs, further profiles needed, e.g. MARTE The semantic fuzz of UML 2nd MBTUC 2012, Tallinn, Estonia

2nd MBTUC 2012, Tallinn, Estonia TDL vs. TTCN-3 TDL TTCN-3 Abstract test specification level – low level of detail Captures requirements on a tester w.r.t. SUT behavior Leaves room for various tester code generation strategies Lower test execution level – high level of detail Detailed tester implementation, incl. tester internal behavior Captures the user intension of an executable tester 2nd MBTUC 2012, Tallinn, Estonia

2nd MBTUC 2012, Tallinn, Estonia Business Value of TDL Support to test design before rushing to test code Better quality of tests Higher productivity in testing Common meta-model for all kind of tests descriptions Enabled development of specialized tools for Generation of executable test code using different strategies Test documentation and visualization Analysis of test descriptions Offers a way to exchange test descriptions Support for customized test languages (concrete TDL syntax) Graphical, Textual, Tabular representation of tests 2nd MBTUC 2012, Tallinn, Estonia

2nd MBTUC 2012, Tallinn, Estonia What comes next? 2nd MBTUC 2012, Tallinn, Estonia

2nd MBTUC 2012, Tallinn, Estonia Status & Outlook Achievements so far Identification of the purpose and business value of TDL Collection of related work Discussion on TDL features Next steps First draft of TDL covering core features Definition of the TDL meta-model Validation of TDL in a reference implementation  Make a stable draft of the ETSI standard on TDL available in 2013! Do you want to join? 2nd MBTUC 2012, Tallinn, Estonia

2nd MBTUC 2012, Tallinn, Estonia Acknowledgments So far, the TDL initiative has attracted quickly 30+ persons and different organisations from industry, tool providers, and research from inside and outside of ETSI. The rapporteur of this activity is thankful for the many stimulating and controversial discussions by various people to lay out the cornerstones of TDL! Siemens AG acknowledges partial funding of this activity from the ARTEMIS Joint Undertaking, grant agreement no. 269335 and the German BMBF. 2nd MBTUC 2012, Tallinn, Estonia

Thank you! Your questions?