1. STF 454 “DESIGN OF TDL” – STATUS REPORT Last change: 2013-09-05 © ETSI 2011. All rights reserved

2. Document History 2013-09-30: Input to joint STF/SG meeting and MTS#60 2013-05-13: Input to MTS#59 2013-04-03: In preparation of the 3 rd session 2013-03-16: Outcome from Budapest meeting 2013-02-25: Reviewed during SG#2; agenda 2 nd session 2013-02-22: Document created © ETSI 2011. All rights reserved 2

3. TDL Documentation Latest TDL standard document https://docbox.etsi.org/STF/STF454_MTS_TDL/Public/ETSI_ES_TDL.pdf Overview on TDL https://docbox.etsi.org/STF/STF454_MTS_TDL/Public/STF454-TDL- Overview.pptx https://docbox.etsi.org/STF/STF454_MTS_TDL/Public/STF454-TDL- Overview.pptx Current TDL meta-model Requires Eclipse Modeling Distribution and Papyrus UML editor https://docbox.etsi.org/STF/STF454_MTS_TDL/Public/TDL-MM.zip © ETSI 2011. All rights reserved 3

4. REPORT ON SESSIONS/MEETINGS 4 © ETSI 2011. All rights reserved

5. Outcome from 1 st Session (week 7) Goal: Gain common understanding in the team Discussion of TDL use cases and users of TDL Clarifying the scope of TDL for Conformance (functional) testing Interoperability testing Discussed examples of test specs 3GPP UE conf tests IMS iop tests Any input to TDL must be given by convincing examples Walkthrough of TDL feature list (not completed) Done: Structure – Interactions – Behavior Open: Data – Time – Misc. © ETSI 2011. All rights reserved 5

6. Outcome from 2 nd Session (week 10) Goal: Establishing the foundations of TDL Closer look at examples IMS iop testing, Ericsson IMS, IPv6 iop testing 3GPP conf testing, OMA conf testing Generated tests from MBT tools (other STF) HLTD requirements (pre-TDL work) Test spec languages CCDL Consolidation of gathered TDL features Further discussion on TDL features – with action items Result and next steps Initial feature list mostly completed (can be always extended) Further discussions needed, addressed in Action Items © ETSI 2011. All rights reserved 6

7. Outcome from 3 rd Session (week 15) Goal: Laying the foundation of the TDL standard First and complete walk-through and discussion of the meta- model covering Structure of a TDL model Packaging concept Test architecture Behavioural expressions Data values Time concepts Miscellaneous concepts Fixing the structure of the TDL document © ETSI 2011. All rights reserved 7

8. Milestone A: Early Draft on TDL Submitted on 29-April-2013 (week 18) Approved on 16-May-2013 at MTS#59 Contents Structure and ToC of TDL document Rough description of the meta model (abstract syntax) © ETSI 2011. All rights reserved 8

9. Outcome from 4 th Session (week 29) Goal: Close gaps in the TDL meta-model; add tool support Addressing issues raised during the joint STF/SG meeting Review of the meta-model from the early draft Detailing data types and data Detailing time concepts Setup of a proper tool environment to design the meta-model Eclipse Modeling Distribution, Kepler edition Papyrus UML editor plug-in Extensions by MFW to support embedded documentation Results Design tool for the meta-model Next iteration of the TDL meta-model © ETSI 2011. All rights reserved 9

10. Outcome from 5 th Session (week 36) Goal: Stabilize the TDL meta-model Internal review of the current meta-model to fix inconsistencies and address shortcomings Reaching consensus within the STF on features and their semantics Introducing documentation generation from the meta-model project within Eclipse Preparation of the next stable draft Results Stable meta-model for next draft Validation of meta-model with available tools (EMF editor, xText) © ETSI 2011. All rights reserved 10

11. Milestone B: Stable Draft on TDL Submitted on 18-Sep-2013 (week 38) Approval at MTS#60 planned Contents All remarks from SG addressed, but one  Definition of sub- configurations postponed to future TDL version Detailed description of the meta-model All packages All classes contained within All of their attributes Introduction and motivation updated Examples on concrete syntaxes for TDL submitted as separate contributions to MTS#60, e.g. proposal for 3GPP © ETSI 2011. All rights reserved 11

12. Final Session Plan, 1 st Period fromtoweekGAFKPMAUMFWCTIOtherNotes 21.0125.014 Prep.meet. 28.0101.025 04.0208.026 11.0215.027 E2,5E E E E E S#1: Wed-Thu full days, Fri 1/2 day 18.0222.028 H1,0 E1,5 25.0201.039 04.0308.0310 C2,5C C C H E1,0 S#2: Budapest, Wed, Thu, Fri morning 11.0315.0311 18.0322.0312 H1,0H H H H0,5E1,0 G2M, Wed 25.0329.0313 H2,0 Easter: 29.3.--1.4. 01.0405.0414 08.0412.0415 E4,0E E E E E2,8 S#3: ETSI, Tue, Wed, Thu, Fri 15.0419.0416 early draft ready 22.0426.0417 H2,0H H H 29.0403.0518 06.0510.0519 13.0517.0520 MTS#59, 14.-15.5. 20.0524.0521 H2,0H H H H0,5H2,0 G2M, Wed, Thu 27.0531.0522 H1,0H H4,0 E2,0 © ETSI 2011. All rights reserved 12

13. Session Plan, 2 nd Period © ETSI 2011. All rights reserved 13 fromtoweekGAFKPMAUMFWCTIOtherNotes 15.0719.0729 C4,0C C C C2,0E S#4 Munich, Tue-Fri 22.0726.0730 29.0702.0831 05.0809.0832 phone conf 12.0816.0833 19.0823.0834 26.0830.0835 phone conf on Mon or Tue 02.0906.0936 E4,0E E E E E2,0 S#5, ETSI, Tue-Fri 09.0913.0937 stable draft ready 16.0920.0938 23.0927.0939 ?? 30.0904.1040 ?? MTS-TDL, 30.9., MTS#60, 1./2.10. 07.1011.1041 14.1018.1042 21.1025.1043 22.-24.10. UCAAT 28.1001.1144 04.1108.1145 C4,0C C C C E2,0 S#6, Berlin, Tue-Fri 11.1115.1146 18.1122.1147 25.1129.1148 02.1206.1249 E4,0E E E E E2,0 S#7, ETSI, Tue-Fri 09.1213.1250 16.1220.1251 final draft ready

14. STF REPORTING 14 © ETSI 2011. All rights reserved

15. STF Technical Risks © ETSI 2011. All rights reserved 15 RiskSeverityLikelihoodMitigation Scope and contents of TDL draft moving or changing due to “green field” development HighLowMove insufficiently discussed topics to a follow-up STF on TDLv2. Validity and consistency of developed meta-model cannot be guaranteed Medium Work out TDL concrete syntax and its mapping to the MM; enforce validation activities outside the STF. Introduced tool support for meta-model design and doc generation is faulty, which causes heavy manual rework of the document LowMediumFall-back to document-centric work to produce the next draft; put more efforts in the tool support. Low end-user acceptance due to ill-designed TDL concrete syntax MediumLowKeep close contact especially with ETSI end-users; keep MM extensible; plan for a follow-up activity

16. Very Tight STF Resources Hiring of 5 experts opposed to 4 as suggested in ToR dilutes overall resources Already 30% of voluntary work Initial ramp-up to get all experts on same ground required Parallelisation of work difficult to achieve Mainly for the analysis and writing part But not for the design In total 174+5 person days for contracted experts 129.5 PD spent up to milestone B Only 49.5 PD contingency to achieve next milestone C © ETSI 2011. All rights reserved 16

17. Next Steps MTS#60 meeting Discussion of submitted stable draft Agreement on STF continuation after 2013 Goals for milestone C (end of Dec 2013) Validation of MM (voluntary efforts) Further consolidation of meta-model Better readability of standard Experiments with concrete syntax and examples © ETSI 2011. All rights reserved 17

18. PROPOSAL OF A NEW STF ON TDL2 Collection of ideas 18 © ETSI 2011. All rights reserved

19. TDL Roadmap 2014 TDL1 Foundation, manual creation of test description, representation of output of MBT, for documentation purpose 2015 TDL2 Automation support for generating concrete tests from abstract test descriptions 2016 TDL3 Extension of TDL to be used for the generation of test descriptions (input to MBT) © ETSI 2011. All rights reserved 19

20. Goal and Scope of TDL2 Goals Supporting ETSI and further industrial use cases Rational Long term view on ETSI standardization process Compare with TDL use case list Objectives Extended TDL MM for use with test automation Default standardized concrete syntaxes Providing full coverage of the extended MM Formats: visual (graphical), textual Capability to omit MM features © ETSI 2011. All rights reserved 20

21. Future Work, Results from TDL2 Split current WI into parts: Revised WI: Abstract syntax (MM) + semantics New WI: Concrete textual & graphical default syntax (human readable) + mapping to MM New WI: Transfer syntax (machine processable, logical & optional layout information) + mapping to MM STF output Revision of existing MM New WIs for default syntaxes (1 WI per syntax) © ETSI 2011. All rights reserved 21

22. TDL2 timeline Start: Feb 2014 Task 1: Extension of TDL MM Task 2: Graphical concrete syntax Task 3: Transfer syntax Potential STF extension: Jul 2014 Task 4: ETSI textual concrete syntax End: Dec 2014 WI: updated MM description + semantics WI: graphical concrete syntax + MM mapping WI: transfer syntax + MM mapping © ETSI 2011. All rights reserved 22

23. Features for TDL2 MM Test Configuration Specification of sub test configurations (subsets, renaming etc.) Types and Data, Time Expressions over user data types (e.g. guards) and time constraints Support for variables and data functions Behavior Extension of Interaction towards Concurrent Interaction (multiple interactions received at a single port concurrently, aka ‘co-region’ in MSC) New features Support for generation features to derive test descriptions from higher level specifications (e.g. similar to High-Level MSCs) UML profile for TDL2? © ETSI 2011. All rights reserved 23

24. TDL FEATURE DISCUSSION 24 © ETSI 2011. All rights reserved

25. Communication STF – SG Ensure overall direction of TDL design is OK Resolve deadlocks in STF internal discussions by providing guidance and advice Develop a roadmap for TDL TDL feature list with mapping to TDL use cases STF will make proposals Feedback expected from SG © ETSI 2011. All rights reserved 25

26. TDL Use Cases Purpose: Promote TDL features based on their potential usage © ETSI 2011. All rights reserved 26 UCShort descriptionExample ATDL for documentation (incl. informal parts)3GPP test specs BTDL for generation of tests that can be made executable (i.e. all parts are formal) Automatic mapping of a TDL spec to partial TTCN-3 code CTDL for representation of generated tests (i.e. output from MBT tools) Test cases generated from system models DTDL for representation of test logsTest execution log of a TTCN-3 tool ETDL for test generation (i.e. input to MBT tools)Test models as activity diagrams FTDL for performance testingOn-the-fly testing from a TDL spec GTDL for interoperability testingUse case models, from which tests are derived Focus in early draft

27. Internal TDL Feature Discussion For use in STF, communication with SG Proposed structure (template) TDL feature name Covered TDL use cases Motivation Description (abstract syntax, meta-model) Semantics Proposed textual concrete syntax Proposed graphical concrete syntax Examples of use © ETSI 2011. All rights reserved 27

28. TDL Feature Description in Standard Structure of TDL feature description TDL feature name (title of sub-clause) Overview (covered in early draft) Free description of the feature Abstract syntax Representation of the feature and its elements in the meta-model Semantics Preferably formal description of the semantics of the feature Constraints Constraints on the feature that can be statically analyzed Classifier description Description of all elements contained in the meta-classes Note 1: Concrete syntax moved to annex! © ETSI 2011. All rights reserved 28

29. General Design Guidelines Global interaction flow Interaction flow of a test description is totally ordered, unless flow operators are used to express explicitly partial ordering between interactions Global time TDL assumes the existence of a discrete global clock that is used to label all tester/SUT interactions with a global timestamp Global data Data in tester/SUT interactions is globally visible within an interaction flow © ETSI 2011. All rights reserved 29