Automated Interoperability Testing ETSI STF 370 Status Stephan Schulz STF Leader © ETSI 2009. All rights reserved MTS#49, Rome, Italy

Project Background Project funded by EC and ETSI Main stakeholders Objective is to extend existing ETSI interoperability testing concepts for the purpose of automation and in context of distributed systems Main stakeholders ETSI TC Methods for Testing and Specification (owner of ETSI WIs) ETSI TC IMS Network Testing ETSI TC GRID B2B community (mainly around HL7) TETRA Association WiMax Forum (NWIIOT) Further signalled interest IPv6 community ITS community Testing labs

Project Overview Planned duration Jan 2009 to Jun 2010 12 experts with various background led by ETSI CTI WP1: Methodology and Framework Output is ETSI Guide WP2: IMS case study based on use of TTCN-3 Application of WP1 in context of IMS core network IOT Basis is ETSI TC INT test specification for 3rd ETSI IMS Plugtest Output is ETSI Technical Report (ATS document), TTCN-3 ATS, codecs & adapters, ETSI TR on Plugtest experience IMS IOT tool and auto IOT concepts will be validated at IMS Plugtest! WP3: Dissemination White paper & training material Presentation to TETRA Forum and at T3UC 2009 + T3UC Asia 2009

WP1 Status First draft of methodology produced by looking at various areas IOT in IMS, WiMax, IPv6, HL7, ROHC, IPTV, WiMedia, SIP VoIP, etc Draft sent out for review and comments Draft expected to further evolve during project Methodology extends ETSI’s generic approach to IOT (EG 202 237) Adds aspects of automation and test system implementation Main points for now Independent of technology to be tested Collection of key terminology Separate verdicts for end-to-end and conformance assessment Limitations, feasibility, and degree of automation Controllability of Equipment Under Test (EUT) interfaces Definition of generic means of interoperability testing Definition of test system development (independent of TTCN-3)

Generic Auto IOT Test Architecture Means of Interoperability Testing EUT 1 Interface Monitor Test Coordinator Configuration interface Application Support Nodes Monitoring interface SUT Test Oracle EUT 2 EUT N Interconnecting Network Stimulating interface Equipment User

WP2 Status Case study on IMS core network IOT ongoing Stable draft of IMS architecture documentation Draft sent out for review and comments TTCN-3 ATS specification phase completed Completed implementation TTCN-3 framework Library based test design Handling of EUT information in IOT event context En/disabling of interface checks upon need Separate management of CON and IOP verdicts Real-time vs. offline IOT test execution Essentially completed specification of stable TTCN-3 IMS IOP tests Current TTCN-3 implementation status 93 % done (but 14 tests marked for now as “skip” as not stable) Number of issues reported to TC INT on IMS IOP test specification Number of tests have been validated

WP2 Status (contd.) TCI SIP and SDP codec implementation completed Based on open source IRISA t3dev codec development kit Excludes handling of XML (only required by 2 tests) Validated with Elvior tool TCI DNS codec implementation “not started” In principle already available Competing proposals for DNS TTCN-3 type system (TT vs. IRISA) Best type system compromise in the middle – but then no codec  For now issue on hold (SIP checks take precedence for now) TCI ISUP codec implementation not started Very complex type structure and only very basic checks in tests Unclear test adapter tool support since not Ethernet based No resource for this activity planned in project plan Required resource for codec implementation not available

WP2 Status (contd.) Test adapter implementation completed Based on open source IRISA t3dev codec development kit Protocol independent, extensible C++ implementation including GUI based upper tester, lower tester, and PCAP capture process Designed test adapter protocol and implemented C++ codecs for test adapter protocol Only been able to integrate with Elvior tool Cygwin dependency of t3devkit has been very difficult to handle with tools “old” TTWB adapter updated to fit new adapter interface

Road ahead Use of STF370 IMS test system as part of ETSI TC INT IMS Plugtest in mid Oct Using Elvior and TT WB based test system implementations Two experts to attend for test system support in context of WP2 to make updates to TTCN-3 code, codec, and adapters Two experts to attend for Plugtest observation in context of WP3 Write experience report after event (WI) Updates of methodology and IMS architecture documents First part of WP3 STF sessions planned to start in Nov Drafting of White paper & Training material (WI)

Topics for discussion with MTS STF 370 comments and questions on EG 202237 (49TD10) ISUP adaptation support Adjusting the schedule of dissemination WIs reflecting the project timelines

Related documents WI DTR/MTS-00116: “Methods for Testing and Specification (MTS); Automated Interoperability Testing; Specific Architectures” Uploaded as 49TD09r1 WI DEG/MTS-00119: Methods for Testing and Specification (MTS); Automated Interoperability Testing; Methodology and Framework Uploaded as 49TD12r1 TTCN-3 code and design documents will be made available after the IMS Plugtest (for next meeting)

Thank you Questions? Comments? stephan.schulz@etsi.org