1. STF 454 “DESIGN OF TDL” – STATUS REPORT Last change: 2014-01-20 © ETSI 2011. All rights reserved

2. Document History 2014-01-20: Updated input to MTS#61 after SG#7 2014-01-15: Input to MTS#61 2013-09-30: Input to joint STF/SG meeting and MTS#60 2013-05-13: Input to MTS#59 2013-04-03: In preparation of the 3 rd session 2013-03-16: Outcome from Budapest meeting 2013-02-25: Reviewed during SG#2; agenda 2 nd session 2013-02-22: Document created © ETSI 2011. All rights reserved 2

3. TDL Documentation Latest TDL standard document https://docbox.etsi.org/STF/STF454_MTS_TDL/Public/ETSI_ES_TDL.pdf Overview on TDL https://docbox.etsi.org/STF/STF454_MTS_TDL/Public/STF454-TDL- Overview.pptx https://docbox.etsi.org/STF/STF454_MTS_TDL/Public/STF454-TDL- Overview.pptx Current TDL meta-model Requires Eclipse Modeling Distribution and Papyrus UML editor https://docbox.etsi.org/STF/STF454_MTS_TDL/Public/TDL-MM.zip © ETSI 2011. All rights reserved 3

4. REPORT ON SESSIONS/MEETINGS 4 © ETSI 2011. All rights reserved

5. Outcome from 1 st Session (week 7) Goal: Gain common understanding in the team Discussion of TDL use cases and users of TDL Clarifying the scope of TDL for Conformance (functional) testing Interoperability testing Discussed examples of test specs 3GPP UE conf tests IMS iop tests Any input to TDL must be given by convincing examples Walkthrough of TDL feature list (not completed) Done: Structure – Interactions – Behavior Open: Data – Time – Misc. © ETSI 2011. All rights reserved 5

6. Outcome from 2 nd Session (week 10) Goal: Establishing the foundations of TDL Closer look at examples IMS iop testing, Ericsson IMS, IPv6 iop testing 3GPP conf testing, OMA conf testing Generated tests from MBT tools (other STF) HLTD requirements (pre-TDL work) Test spec languages CCDL Consolidation of gathered TDL features Further discussion on TDL features – with action items Result and next steps Initial feature list mostly completed (can be always extended) Further discussions needed, addressed in Action Items © ETSI 2011. All rights reserved 6

7. Outcome from 3 rd Session (week 15) Goal: Laying the foundation of the TDL standard First and complete walk-through and discussion of the meta- model covering Structure of a TDL model Packaging concept Test architecture Behavioural expressions Data values Time concepts Miscellaneous concepts Fixing the structure of the TDL document © ETSI 2011. All rights reserved 7

8. Milestone A: Early Draft on TDL Submitted on 29-April-2013 (week 18) Approved at MTS#59 Contents Structure and ToC of TDL document Rough description of the meta model (abstract syntax) © ETSI 2011. All rights reserved 8

9. Outcome from 4 th Session (week 29) Goal: Close gaps in the TDL meta-model; add tool support Addressing issues raised during the joint STF/SG meeting Review of the meta-model from the early draft Detailing data types and data Detailing time concepts Setup of a proper tool environment to design the meta-model Eclipse Modeling Distribution, Kepler edition Papyrus UML editor plug-in Extensions by MFW to support embedded documentation Results Design tool for the meta-model Next iteration of the TDL meta-model © ETSI 2011. All rights reserved 9

10. Outcome from 5 th Session (week 36) Goal: Stabilize the TDL meta-model Internal review of the current meta-model to fix inconsistencies and address shortcomings Reaching consensus within the STF on features and their semantics Introducing documentation generation from the meta-model project within Eclipse Preparation of the next stable draft Results Stable meta-model for next draft Validation of meta-model with available tools (EMF editor, xText) © ETSI 2011. All rights reserved 10

11. Milestone B: Stable Draft on TDL Submitted on 18-Sep-2013 (week 38) Approved at MTS#60 Contents All remarks from SG addressed, but one  Definition of sub- configurations postponed to future TDL version Detailed description of the meta-model All packages All classes contained within All of their attributes Introduction and motivation updated Examples on concrete syntaxes for TDL submitted as separate contributions to MTS#60, e.g. proposal for 3GPP © ETSI 2011. All rights reserved 11

12. Outcome from 6 th Session (week 45) Goal: Addressing feedback from SG#6/MTS#60 meeting Review of all parts of the MM and discussion according to feedback from SG Discussion of results from MM validation performed by STF members Editing of the MM description Results Updated MM Improved documentation © ETSI 2011. All rights reserved 12

13. Outcome from 7 th Session (week 49) Goal: Finalization of TDL document Further improvements of the MM, mainly parameterization Review and improvement of the MM description Discussion on TDL concrete syntax and examples from 3GPP and IMS Results Further consolidation of MM Improved readability of MM and its description Text ready for delivering the Final Draft © ETSI 2011. All rights reserved 13

14. Milestone C: Final Draft on TDL Submitted on 20-Dec-2013 (week 51) Approval at MTS#61 planned Contents Validated(!) description of the meta-model Improved introduction (Clause 4) Informative annex on an example of a concrete textual syntax and it application Meta-model provided as a technical document (Papyrus UML project) © ETSI 2011. All rights reserved 14

15. Next Steps Joint SG/STF meeting, 2014-01-17 – Done! Discussion on submitted final draft Outcome Need to set up a Mantis project to track change requests Improved documentation for Argument Specification requested Discussion on precedence of document vs. technical MM New feature requests by Ericsson (handled in TDL phase 2) Improved data flow specification (in/out parameters, variables) Sub-typing or other forms to support TTCN-3 template concept in TDL MTS#61 meeting, 2014-01-29/30 Approval of final draft Discussion on TDL phase 2 (2014) © ETSI 2011. All rights reserved 15

16. Final Session Plan, 1 st Period fromtoweekGAFKPMAUMFWCTIOtherNotes 21.0125.014 Prep.meet. 28.0101.025 04.0208.026 11.0215.027 E2,5E E E E E S#1: Wed-Thu full days, Fri 1/2 day 18.0222.028 H1,0 E1,5 25.0201.039 04.0308.0310 C2,5C C C H E1,0 S#2: Budapest, Wed, Thu, Fri morning 11.0315.0311 18.0322.0312 H1,0H H H H0,5E1,0 G2M, Wed 25.0329.0313 H2,0 Easter: 29.3.--1.4. 01.0405.0414 08.0412.0415 E4,0E E E E E2,8 S#3: ETSI, Tue, Wed, Thu, Fri 15.0419.0416 early draft ready 22.0426.0417 H2,0H H H 29.0403.0518 06.0510.0519 13.0517.0520 MTS#59, 14.-15.5. 20.0524.0521 H2,0H H H H0,5H2,0 G2M, Wed, Thu 27.0531.0522 H1,0H H4,0 E2,0 © ETSI 2011. All rights reserved 16

17. Session Plan, 2 nd Period © ETSI 2011. All rights reserved 17 fromtoweekGAFKPMAUMFWCTIOtherNotes 15.0719.0729 C4,0C C C C2,0E S#4 Munich, Tue-Fri 22.0726.0730 29.0702.0831 05.0809.0832 phone conf 12.0816.0833 19.0823.0834 26.0830.0835 phone conf on Mon or Tue 02.0906.0936 E4,0E E E E E2,0 S#5, ETSI, Tue-Fri 09.0913.0937 stable draft ready 16.0920.0938 23.0927.0939 ?? 30.0904.1040 ?? MTS-TDL, 30.9., MTS#60, 1./2.10. 07.1011.1041 14.1018.1042 21.1025.1043 22.-24.10. UCAAT 28.1001.1144 04.1108.1145 C4,0C C C C E2,0 S#6, Berlin, Tue-Fri 11.1115.1146 18.1122.1147 25.1129.1148 02.1206.1249 E4,0E E E E E2,0 S#7, ETSI, Tue-Fri 09.1213.1250 16.1220.1251 final draft ready

18. STF REPORTING 18 © ETSI 2011. All rights reserved

19. STF Technical Risks © ETSI 2011. All rights reserved 19 RiskSeverityLikelihoodMitigation Scope and contents of TDL draft moving or changing due to “green field” development HighLowMove insufficiently discussed topics to a follow-up STF on TDLv2. Validity and consistency of developed meta-model cannot be guaranteed MediumLowWork out TDL concrete syntax and its mapping to the MM; enforce validation activities outside the STF. Introduced tool support for meta-model design and doc generation is faulty, which causes heavy manual rework of the document LowHappenedFall-back to document-centric work to produce the next draft; put more efforts in the tool support. Low end-user acceptance due to ill-designed TDL concrete syntax Medium Keep close contact especially with ETSI end-users; keep MM extensible; plan for a follow-up activity

20. Very Tight STF Resources Equal distribution of budget over 5 experts (opposed to the suggested 4) dilutes overall resources Voluntary work shot up to 30% Initial ramp-up to get all experts on same ground required Parallelisation of work difficult to achieve Mainly for the analysis and writing part But not for the design In total 179 person days for contracted experts 129.5 PD spent up to milestone B Only 49.5 PD to achieve milestone C High voluntary efforts for MM validation © ETSI 2011. All rights reserved 20

21. DISCUSSION OF TDL PHASE 2 Review of the related ToR 21 © ETSI 2011. All rights reserved

22. TDL Roadmap 2013 TDL1 Foundation, manual creation of test descriptions, representation of test cases, documentation of tests 2014 TDL2 Automation support for generating concrete tests Concrete syntaxes for end-users and tool interop 2015 TDL3 Features for the generation of abstract tests Support of further use cases, e.g. performance tests © ETSI 2011. All rights reserved 22

23. Goal and Objectives of TDL2 Goal Supporting ETSI and industrial users in using TDL Objectives Extended TDL MM for supporting test automation Default concrete syntaxes Graphical syntax for end-users Textual exchange syntax for tool interoperability Analysis on the needs for a textual syntax to support ETSI use cases © ETSI 2011. All rights reserved 23

24. Deliverables from TDL2 Del. Work Item code Standard number Working title Scope D1RES/ES 203 119-1 V1.2.1 Test Description Language; Meta-Model and Semantics Scope: common concepts, meta-model, semantics D2DES/ ES 203 119-2 V1.1.1 Test Description Language; Graphical Syntax Scope: TDL graphical concrete syntax for end users D3DES/ ES 203 119-3 V1.1.1 Test Description Language; Exchange Format Scope: TDL exchange format for tool interoperability © ETSI 2011. All rights reserved 24 Optionally ES part 4 on TDL textual concrete syntax (no WI created yet)

25. TDL2 organization of work Start: Feb 2014 Task 0: Project management Task 1: Extension of TDL MM (02—12/2014) Task 2: Graphical concrete syntax (02—05—12/2014) Task 3: Exchange syntax (06—12/2014) Task 4a: Analysis on ETSI concrete syntax (02—05/2014) Potential STF extension: Jun 2014 (decision at MTS#62) Task 4b: ETSI concrete syntax (06—12/2014) End: Dec 2014 WI: updated MM description + semantics WI: concrete syntax + MM mapping WI: exchange syntax + MM mapping © ETSI 2011. All rights reserved 25

26. TDL2 milestones M0: 02/2014 Start of work of Tasks 0, 1, 2, 4 M1: 05/2014 (T1) Early draft: updated MM (T2) Early draft: graphical syntax (T3) Start of Task 3 (T4) Decision paper on textual syntax M2: 09/2014 (T1) Stable draft: updated MM (T2) Stable draft: graphical syntax (T3) Early draft: exchange syntax M3: 12/2014 (T1) Final draft: updated MM (T2) Final draft: graphical syntax (T3) Final draft: exchange syntax © ETSI 2011. All rights reserved 26  Discussion at MTS#62 (05/2014)  Discussion at MTS#63 (10/2014)  Approval at MTS#64 (02/2015)

27. TDL task efforts and budget Efforts for experts Task 0: Management  6 PD (STF leader) Task 1: Extended MM  60 PD Task 2: Graphical syntax  84 PD Task 3: Exchange syntax  48 PD Task 4a: Textual syntax  10 PD Expert voluntary contribution of 20% Additionally, 25 PD support from CTI (voluntary!) Budget: 99,600 € + 4000 € travel costs All requested resources granted, no cut! © ETSI 2011. All rights reserved 27

28. T1: Features of extended TDL MM Test Configuration Specification of sub test configurations (subsets, renaming etc.) Types and Data, Time Expressions over user data types (e.g. guards) and time constraints Improved data flow description: variables, data functions, in/out parameters Sub-typing or other forms to support TTCN-3 template concept in TDL Support for arrays of data types and (possibly) component and gate types Behaviour Refinement of interactions for different interface types Extensions of interaction flow operations, e.g. MSC co-regions Abstraction levels in TDL A) Test description level  descriptive specification of tests B) Test implementation level  abstract description of concrete (TTCN-3) tests Suggestion: Use of a default syntax (Annex B?) to help feature discussion © ETSI 2011. All rights reserved 28

29. T2/3: Features of TDL concrete syntaxes T2: Graphical syntax Foundation: OMG Diagram Definition (DD) architecture Certainly inspired by UML T3: Exchange syntax Foundation: OMG Test Information Interchange Format (TestIF) Human-readable syntax vs. XML-based syntax? Mapping rules for all syntactical elements to MM elements need to be defined for both cases © ETSI 2011. All rights reserved 29

30. T4: Analysis for a TDL textual syntax Understanding ETSI’s need on a textual syntax Solicitation of end-user requirements on a potential TDL textual or tabular language from various technical bodies Running projects Upcoming projects Collecting and categorising these end-user requirements Result: decision paper for MTS#62 © ETSI 2011. All rights reserved 30

31. BACK-UP 31 © ETSI 2011. All rights reserved

32. Communication STF – SG Ensure overall direction of TDL design is OK Resolve deadlocks in STF internal discussions by providing guidance and advice Develop a roadmap for TDL TDL feature list with mapping to TDL use cases STF will make proposals Feedback expected from SG © ETSI 2011. All rights reserved 32

33. TDL Use Cases Purpose: Promote TDL features based on their potential usage © ETSI 2011. All rights reserved 33 UCShort descriptionExample ATDL for documentation (incl. informal parts)3GPP test specs BTDL for generation of tests that can be made executable (i.e. all parts are formal) Automatic mapping of a TDL spec to partial TTCN-3 code CTDL for representation of generated tests (i.e. output from MBT tools) Test cases generated from system models DTDL for representation of test logsTest execution log of a TTCN-3 tool ETDL for test generation (i.e. input to MBT tools)Test models as activity diagrams FTDL for performance testingOn-the-fly testing from a TDL spec GTDL for interoperability testingUse case models, from which tests are derived

34. Internal TDL Feature Discussion For use in STF, communication with SG Proposed structure (template) TDL feature name Covered TDL use cases Motivation Description (abstract syntax, meta-model) Semantics Proposed textual concrete syntax Proposed graphical concrete syntax Examples of use © ETSI 2011. All rights reserved 34

35. TDL Feature Description in Standard Structure of TDL feature description TDL feature name (title of sub-clause) Overview (covered in early draft) Free description of the feature Abstract syntax Representation of the feature and its elements in the meta-model Semantics Preferably formal description of the semantics of the feature Constraints Constraints on the feature that can be statically analyzed Classifier description Description of all elements contained in the meta-classes Note 1: Concrete syntax moved to annex! © ETSI 2011. All rights reserved 35