STF 454 “Design of TDL” – Status Report Last change: 2013-10-02 © ETSI 2011. All rights reserved

Document History 2013-09-30: Input to joint STF/SG meeting and MTS#60 2013-05-13: Input to MTS#59 2013-04-03: In preparation of the 3rd session 2013-03-16: Outcome from Budapest meeting 2013-02-25: Reviewed during SG#2; agenda 2nd session 2013-02-22: Document created © ETSI 2011. All rights reserved

TDL Documentation Latest TDL standard document Overview on TDL https://docbox.etsi.org/STF/STF454_MTS_TDL/Public/ETSI_ES_TDL.pdf Overview on TDL https://docbox.etsi.org/STF/STF454_MTS_TDL/Public/STF454-TDL-Overview.pptx Current TDL meta-model Requires Eclipse Modeling Distribution and Papyrus UML editor https://docbox.etsi.org/STF/STF454_MTS_TDL/Public/TDL-MM.zip © ETSI 2011. All rights reserved

Report on Sessions/Meetings © ETSI 2011. All rights reserved

Outcome from 1st Session (week 7) Goal: Gain common understanding in the team Discussion of TDL use cases and users of TDL Clarifying the scope of TDL for Conformance (functional) testing Interoperability testing Discussed examples of test specs 3GPP UE conf tests IMS iop tests Any input to TDL must be given by convincing examples Walkthrough of TDL feature list (not completed) Done: Structure – Interactions – Behavior Open: Data – Time – Misc. © ETSI 2011. All rights reserved

Outcome from 2nd Session (week 10) Goal: Establishing the foundations of TDL Closer look at examples IMS iop testing, Ericsson IMS, IPv6 iop testing 3GPP conf testing, OMA conf testing Generated tests from MBT tools (other STF) HLTD requirements (pre-TDL work) Test spec languages CCDL Consolidation of gathered TDL features Further discussion on TDL features – with action items Result and next steps Initial feature list mostly completed (can be always extended) Further discussions needed, addressed in Action Items © ETSI 2011. All rights reserved

Outcome from 3rd Session (week 15) Goal: Laying the foundation of the TDL standard First and complete walk-through and discussion of the meta-model covering Structure of a TDL model Packaging concept Test architecture Behavioural expressions Data values Time concepts Miscellaneous concepts Fixing the structure of the TDL document © ETSI 2011. All rights reserved

Milestone A: Early Draft on TDL Submitted on 29-April-2013 (week 18) Approved on 16-May-2013 at MTS#59 Contents Structure and ToC of TDL document Rough description of the meta model (abstract syntax) © ETSI 2011. All rights reserved

Outcome from 4th Session (week 29) Goal: Close gaps in the TDL meta-model; add tool support Addressing issues raised during the joint STF/SG meeting Review of the meta-model from the early draft Detailing data types and data Detailing time concepts Setup of a proper tool environment to design the meta-model Eclipse Modeling Distribution, Kepler edition Papyrus UML editor plug-in Extensions by MFW to support embedded documentation Results Design tool for the meta-model Next iteration of the TDL meta-model © ETSI 2011. All rights reserved

Outcome from 5th Session (week 36) Goal: Stabilize the TDL meta-model Internal review of the current meta-model to fix inconsistencies and address shortcomings Reaching consensus within the STF on features and their semantics Introducing documentation generation from the meta-model project within Eclipse Preparation of the next stable draft Results Stable meta-model for next draft Validation of meta-model with available tools (EMF editor, xText) © ETSI 2011. All rights reserved

Milestone B: Stable Draft on TDL Submitted on 18-Sep-2013 (week 38) Approval at MTS#60 planned Contents All remarks from SG addressed, but one  Definition of sub-configurations postponed to future TDL version Detailed description of the meta-model All packages All classes contained within All of their attributes Introduction and motivation updated Examples on concrete syntaxes for TDL submitted as separate contributions to MTS#60, e.g. proposal for 3GPP © ETSI 2011. All rights reserved

Final Session Plan, 1st Period from to week GA FK PM AU MFW CTI Other Notes 21.01 25.01 4   Prep.meet. 28.01 01.02 5 04.02 08.02 6 11.02 15.02 7 E 2,5 S#1: Wed-Thu full days, Fri 1/2 day 18.02 22.02 8 H 1,0 1,5 25.02 01.03 9 04.03 08.03 10 C S#2: Budapest, Wed, Thu, Fri morning 11.03 15.03 11 18.03 22.03 12 0,5 G2M, Wed 25.03 29.03 13 2,0 Easter: 29.3.--1.4. 01.04 05.04 14 08.04 12.04 15 4,0 2,8 S#3: ETSI, Tue, Wed, Thu, Fri 15.04 19.04 16 early draft ready 22.04 26.04 17 29.04 03.05 18 06.05 10.05 19 13.05 17.05 20 MTS#59, 14.-15.5. 20.05 24.05 21 G2M, Wed, Thu 27.05 31.05 22 © ETSI 2011. All rights reserved

Session Plan, 2nd Period from to week GA FK PM AU MFW CTI Other Notes 15.07 19.07 29 C 4,0 2,0 E   S#4 Munich, Tue-Fri 22.07 26.07 30 29.07 02.08 31 05.08 09.08 32 phone conf 12.08 16.08 33 19.08 23.08 34 26.08 30.08 35 phone conf on Mon or Tue 02.09 06.09 36 S#5, ETSI, Tue-Fri 09.09 13.09 37 stable draft ready 16.09 20.09 38 23.09 27.09 39 ? 30.09 04.10 40 MTS-TDL, 30.9., MTS#60, 1./2.10. 07.10 11.10 41 14.10 18.10 42 21.10 25.10 43 22.-24.10. UCAAT 28.10 01.11 44 04.11 08.11 45 S#6, Berlin, Tue-Fri 11.11 15.11 46 18.11 22.11 47 25.11 29.11 48 02.12 06.12 49 S#7, ETSI, Tue-Fri 09.12 13.12 50 16.12 20.12 51 final draft ready © ETSI 2011. All rights reserved

STF Reporting © ETSI 2011. All rights reserved

STF Technical Risks Risk Severity Likelihood Mitigation Scope and contents of TDL draft moving or changing due to “green field” development High Low Move insufficiently discussed topics to a follow-up STF on TDLv2. Validity and consistency of developed meta-model cannot be guaranteed Medium Work out TDL concrete syntax and its mapping to the MM; enforce validation activities outside the STF. Introduced tool support for meta-model design and doc generation is faulty, which causes heavy manual rework of the document Fall-back to document-centric work to produce the next draft; put more efforts in the tool support. Low end-user acceptance due to ill-designed TDL concrete syntax Keep close contact especially with ETSI end-users; keep MM extensible; plan for a follow-up activity © ETSI 2011. All rights reserved

Very Tight STF Resources Hiring of 5 experts opposed to 4 as suggested in ToR dilutes overall resources Already 30% of voluntary work Initial ramp-up to get all experts on same ground required Parallelisation of work difficult to achieve Mainly for the analysis and writing part But not for the design In total 174+5 person days for contracted experts 129.5 PD spent up to milestone B Only 49.5 PD contingency to achieve next milestone C © ETSI 2011. All rights reserved

Next Steps MTS#60 meeting Goals for milestone C (end of Dec 2013) Discussion of submitted stable draft Agreement on STF continuation after 2013 Goals for milestone C (end of Dec 2013) Validation of MM (voluntary efforts) Further consolidation of meta-model Better readability of standard Experiments with concrete syntax and examples © ETSI 2011. All rights reserved

Proposal of a new STF on TDL2 Collection of ideas © ETSI 2011. All rights reserved

TDL Roadmap 2014 TDL1 Foundation, manual creation of test descriptions, representation of test cases, documentation of tests 2015 TDL2 Automation support for generating concrete tests from abstract test descriptions 2016 TDL3 Extension of TDL to provide support for MBT (test generation), support for performance tests © ETSI 2011. All rights reserved

Goal and Scope of TDL2 Goals Rational Objectives Supporting ETSI and further industrial use cases Rational Long term view on ETSI standardization process Objectives Extended TDL MM for use with test automation Default concrete syntaxes Providing full coverage of the extended MM Formats: graphical + (optional) textual/tabular for end-users Transfer syntax Format: textual, machine processable © ETSI 2011. All rights reserved

Future Work, Results from TDL2 Split current WI into parts: Revised WI: Abstract syntax (MM) + semantics New WI: Concrete default syntax + MM mapping New WI: Transfer syntax (logical information, extensible for layout info and backward-compatible) + MM mapping STF output RES on existing MM – multi-part standard ES part 2 on default graphical syntax (full coverage) ES part 3 on on transfer syntax Optional: ES part 4 on new WI on ETSI concrete syntax © ETSI 2011. All rights reserved

TDL2 organization of work Start: Feb 2014 Task 0: Project management Task 1: Extension of TDL MM (02—12/2014) Task 2: Graphical concrete syntax (02—05—12/2014) Task 3: Transfer syntax (06—12/2014) Task 4a: Analysis on ETSI concrete syntax (02—05/2014) Potential STF extension: Jun 2014 (decision at MTS#62) Task 4b: ETSI concrete syntax (06—12/2014) End: Dec 2014 WI: updated MM description + semantics WI: concrete syntax + MM mapping WI: transfer syntax + MM mapping © ETSI 2011. All rights reserved

TDL2 milestones M0: 02/2014 M1: 05/2014 M2: 09/2014 M3: 12/2014 Start of work of Tasks 0, 1, 2, 4 M1: 05/2014 Early draft: updated MM Early draft: graphical syntax Start of Task 3 M2: 09/2014 (T1) Stable draft: updated MM (T2) Stable draft: graphical syntax (T3) Early draft: transfer syntax M3: 12/2014 (T1) Final draft: updated MM (T2) Final draft: graphical syntax (T3) Final draft: transfer syntax End of work  Discussion at MTS#62 (05/2014)  Discussion at MTS#63 (10/2014)  Approval at MTS#64 (02/2015) © ETSI 2011. All rights reserved

TDL2 efforts Assumptions Total Calculation Task 0: Management 4 experts (old:5) working in 2 teams in parallel 25% workload per expert over the year  200 PD in total 6-7 sessions (old: 7) Total Proportions T1 : T2 : T3 = 1 : 2 : 1 Calculation Month*PD*Expert Task 0: Management 6 PD (2 PD per milestone) Task 1: Extended MM 6*5*2 + Task 2: Graphical syntax Task 3: Transfer syntax 4*5*2 Task 4a: ETSI syntax 10 PD © ETSI 2011. All rights reserved

Features for extended TDL MM Test Configuration Specification of sub test configurations (subsets, renaming etc.) Types and Data, Time Expressions over user data types (e.g. guards) and time constraints Support for variables and data functions Support for arrays of data types and (possibly) component and gate types Behavior Extension of Interaction towards Concurrent Interaction (multiple interactions received at a single port concurrently, aka ‘co-region’ in MSC) New features (only if time permits, not part of the ToR) Support for generation features to derive test descriptions from higher level specifications (e.g. similar to High-Level MSCs) UML profile for TDL2? © ETSI 2011. All rights reserved

TDL Feature Discussion © ETSI 2011. All rights reserved

Communication STF – SG Ensure overall direction of TDL design is OK Resolve deadlocks in STF internal discussions by providing guidance and advice Develop a roadmap for TDL TDL feature list with mapping to TDL use cases STF will make proposals Feedback expected from SG © ETSI 2011. All rights reserved

TDL Use Cases Purpose: Promote TDL features based on their potential usage Focus in early draft UC Short description Example A TDL for documentation (incl. informal parts) 3GPP test specs B TDL for generation of tests that can be made executable (i.e. all parts are formal) Automatic mapping of a TDL spec to partial TTCN-3 code C TDL for representation of generated tests (i.e. output from MBT tools) Test cases generated from system models D TDL for representation of test logs Test execution log of a TTCN-3 tool E TDL for test generation (i.e. input to MBT tools) Test models as activity diagrams F TDL for performance testing On-the-fly testing from a TDL spec G TDL for interoperability testing Use case models, from which tests are derived © ETSI 2011. All rights reserved

Internal TDL Feature Discussion For use in STF, communication with SG Proposed structure (template) TDL feature name Covered TDL use cases Motivation Description (abstract syntax, meta-model) Semantics Proposed textual concrete syntax Proposed graphical concrete syntax Examples of use © ETSI 2011. All rights reserved

TDL Feature Description in Standard Structure of TDL feature description TDL feature name (title of sub-clause) Overview (covered in early draft) Free description of the feature Abstract syntax Representation of the feature and its elements in the meta-model Semantics Preferably formal description of the semantics of the feature Constraints Constraints on the feature that can be statically analyzed Classifier description Description of all elements contained in the meta-classes Note 1: Concrete syntax moved to annex! © ETSI 2011. All rights reserved

General Design Guidelines Global interaction flow Interaction flow of a test description is totally ordered, unless flow operators are used to express explicitly partial ordering between interactions Global time TDL assumes the existence of a discrete global clock that is used to label all tester/SUT interactions with a global timestamp Global data Data in tester/SUT interactions is globally visible within an interaction flow © ETSI 2011. All rights reserved