ETSI Testing Projects ICT-OSA/Parlay Workshop Brazil, March 2006 Anthony Wiles Manager ETSI Protocol and Testing Competence Centre

ETSI Testing Projects - ICT Workshop, Brazil March 2006 The ETSI PTCC Protocol and Testing Competence Centre In-house team of experts providing direct support to ETSI Technical Bodies (for over 12 years) Development of protocols and profiles and the application of modern specification techniques (35%) ‘Standards Engineering’ Validation of standards (5%) E.g., testbed activities (IPv6, SIP/H.323) Development of test specifications (60%) Conformance tests Interoperability tests Network Integration tests Others (PHY, performance …) http://www.etsi.org/ptcc ETSI Testing Projects - ICT Workshop, Brazil March 2006

Typical ETSI Test Specifications Cellular: GSM and 3G (UMTS) terminals Now includes IMS WiFi: HiperMAN, HiperACCESS, WiMax VoIP: H.323, SIP, SIGTRAN Service Creation: OSA/Parlay (API, IDL) IPv6: Core, Security, Mobility, v4-v6 Cordless phones: DECT Radio communications: TETRA, DMR Access terminals: FSK, SMS Broadband: ISDN, DSL Smartcards: Readers, cards, security modules Intelligent Transport Systems (ITS): DSRC Future: More Security, more ITS, GRID ... ETSI Testing Projects - ICT Workshop, Brazil March 2006

IPv6 Test Specifications STF 276 Joint ETSI – eEurope project Supported by the ETSI membership Mainly vendors Require IPv6 test specifications using ETSI methods and test languages Designed to be part of 3GPP and TISPAN NGN IPv6 testing strategy (where applicable) European Commission: Mandate for IPv6 device interoperability Amplify Europe’s voice in IPv6 testing Liaising with the IPv6 Logo Program For more info visit: www.ipt.etsi.org ETSI Testing Projects - ICT Workshop, Brazil March 2006

ETSI Testing Projects - ICT Workshop, Brazil March 2006 Two Phases (2005 – 2007) Phase 1: Core IPv6 Total value approx. 0,5M€ Jan – Dec 2005 Phase2: IPSec, Mobile IPv6, IPv4-6 Transitioning Total value approx. 1,0M€ Jan 2006 – May 2007 Provided as Funding Provision of experts Hosting costs Travel All technical work done at ETSI by STF276 Currently 11 participants in the STF ETSI Testing Projects - ICT Workshop, Brazil March 2006

ETSI Testing Projects - ICT Workshop, Brazil March 2006 Outputs IPv6 Testing Framework Published (update due end 2006) IPv6 Requirements Catalogue On line as part of IPT Library Test specifications Interoperability test specifications Conformance test specifications Library of TTCN-3 test code freely available On-line, extensible and reusable All tests are validated (probably 30% of the effort) On ETSI IPv6 Testbed By member companies For more info visit: www.ipt.etsi.org ETSI Testing Projects - ICT Workshop, Brazil March 2006

ETSI Testing Projects - ICT Workshop, Brazil March 2006 Results Phase 1 Over 1100 Core IPv6 Requirements identified Including 3GPP 300 Core IPv6 conformance tests fully validated 120 Core IPv6 Interoperability Tests All these test specifications including TTCN-3 code are free and open source ETSI Testing Projects - ICT Workshop, Brazil March 2006

ETSI Testing Projects - ICT Workshop, Brazil March 2006 STF276@Work ETSI Testing Projects - ICT Workshop, Brazil March 2006

3GPP UMTS (terminal) Test Specifications (STF 160) Nobody can make money without interoperable handsets (UE)! UE Interoperability towards access technologies is the key issue for mobility Conformant UE = Interoperability (of UE in 3G networks) Good experience from GSM Solution: conformance testing Very detailed test specifications Based on the 3GPP core specs Before commercial products enter the market Voluntary declaration, but certified One of the most important UE quality benchmarks ETSI Testing Projects - ICT Workshop, Brazil March 2006

Methodology for 3G Testing Test Report logging and analysis testing SS Mobile Under Test compilation Executable Test Suite (e.g., C++) implementation Core specs 25.331, 25.321, 2 24.008, 44.018, 44.060 PICS (capability applicability) 34.123-2, 51.010-2 Test Purposes,34.123-1 Descript (prose), 34.108, 51.010-1 Test Cases (TTCN) 34.123-3, 51.010-5 ETSI Testing Projects - ICT Workshop, Brazil March 2006

3G Test Industry Commitment is Vital The TTCN test cases are implemented in the System Simulators (SS) on different platforms Must secure the commitment of the SS vendors SS vendors organise verification and regression testing Each and every test case is validated Bug-free tests have a high real value Important to ensure that a TC will execute correctly even after updates Early identification of clients who deploy the tests Prioritising TCs Scheduling the deployment Certifications ETSI Testing Projects - ICT Workshop, Brazil March 2006

ETSI Testing Projects - ICT Workshop, Brazil March 2006 Funding Test specs developed at ETSI by STF160 Duration of several years Shared costs to ensure the funding 3GPP funding from Organisation Partners Main source, approx 1M€ (61 mm) /year) Voluntary contributions from the marketing partners Range from 65k€ - 150k€ (5 – 12 mm) /year Man-power (free-of-charge) Range from 15 - 18 mm /year ETSI Testing Projects - ICT Workshop, Brazil March 2006

Typical 3GPP UMTS UE Tests Protocol and signalling tests via radio interface Largest part of the testing process Full conformance tests USIM functional tests Identification, PIN, account charging, network priority, phonebook UICC tests (SIM electrical test) power on/off, 1.8v/3v detection, presence detection bit/character transmitting duration, command processing and error handling Acoustic tests Loudness rating, idle noise, echo … Radio Frequency & Radio Resource Management tests Characteristics and performance tests of transmitter & receiver ETSI Testing Projects - ICT Workshop, Brazil March 2006

ETSI Testing Projects - ICT Workshop, Brazil March 2006 Users of 3GPP Tests Global Certification Forum (GCF) PCS Type Certification Review Board (PTCRB ) GCF + PTCRB are the major clients for W-CDMA (FDD) & GSM/GPRS test specs TD-SCDMA Industry Alliance (TDIA) TDIA is the major clients for TD-SCDMA (LCR TDD) test specs GCF WI-10 selected 400 TCs as high priority Divided in 4 packages Progressing sequentially one by one (milestone 80%) UMTS mobile certification will be started if 80% of each package are validated ETSI Testing Projects - ICT Workshop, Brazil March 2006

ETSI Testing Projects - ICT Workshop, Brazil March 2006 Approval Process Test cases are approved by e-mail No quality compromise Two week consultation period TCs are approvedone by one Each test must run on at least one SS platform against two independent commercial mobiles Log files and radio conditions as support information for approval Reason for this TTCN TC, SS and UE are developed at the same time None of them is perfect ETSI Testing Projects - ICT Workshop, Brazil March 2006

ETSI Testing Projects - ICT Workshop, Brazil March 2006 Results 400 TCs in 8 ATSs, 25 MB code Platform independent Running on > 4 platforms We have reached the GCF 80% validated milestone 3G mobile certification starting at beginning of 2005 The light is green for mass 3G mobiles entering the markets 4 types of 3G mobiles are ready to go Begun the real 3G test business 4 validated 3G SS platforms available in the markets 4 test houses in Europe opening 3G test services 3G TTCN experts becoming hot in the labour market We know the price of each 3G coded TC 10 k€ /TC 3G investment for 4 years: 4 mil.€ for 400 TCs All TTCN code can be downloaded free of charge Code is continuously maintained and delivered Every three weeks for 3G members Every three months for everybody ETSI Testing Projects - ICT Workshop, Brazil March 2006

VoIP-related Test Specifications STF 269, STF 270 and others SIP (RFC 3261) Over 400 conformance test cases Fully validated (executables available on commercial tools) STF 270 is doing the maintenance H.225 About 300 conformance test cases No ongoing maintenance SIGTRAN Test purposes for M3UA, M2UA, SCTP TTCN-3 available commercially (not ETSI) IMS SIP (together with SDP) 3GPP RAN5 (STF 160) TTCN-3 will be used Work started March 2006 Based on the generic SIP test suite (see above) augmented to concentrate on 3GPP SIP specific requirements ETSI Testing Projects - ICT Workshop, Brazil March 2006

ETSI – WiMAX Test Specifications STF 252 WiMAX Forum Board meeting in January 2005 Agreed to harmonize the WiMAX/HiperMAN base specifications Agreed to harmonize the WiMAX/HiperMAN test specifications Agreed to exploit ETSI’s testing expertise Agreed to accelerate the work using the ETSI STFs Agreed that validation of the test specifications needed complex interplay of all parties involved WiMAX Forum, ETSI BRAN, equipment vendors, ETSI STF, test tool developers, test labs, etc. ETSI WiMAX Co-operation agreement signed in 2005 ETSI Testing Projects - ICT Workshop, Brazil March 2006

ETSI Testing Projects - ICT Workshop, Brazil March 2006 Results Wave 1: Joint test specifications approved and published Completed Dec. 2005 PICS – Protocol Implementation Conformance Statement TSS&TP – Test Suite Structure and Test Purposes ATS – Approx 60 test cases implemented, validated and used in certification First WiMAX products have been certified Wave 2: New drafts reaching approval PICS – new updated version with corrections TSS&TP – new draft with close to 1400 Test Purposes ATS – new draft with close to 300 additional test cases developed and entering validation ETSI Testing Projects - ICT Workshop, Brazil March 2006

ETSI Testing Projects - ICT Workshop, Brazil March 2006 IPT Library STF 295 IP Testing Library Developed by ETSI TC MTS All MTS IP and IP-related conformance and IOP tests PICS, TPs, Test Suites etc. Due to be published April 2006 SIP, IPv6, H.323, H.248 Includes IPv6 Requirements Catalogue Linked to TPs and TTCN-3 code Powerful search and selection possibilities E.g., allows test suite tailored to specific search criteria ETSI Testing Projects - ICT Workshop, Brazil March 2006

Screen Shot of IPT Library ETSI Testing Projects - ICT Workshop, Brazil March 2006

PlugtestsTM ETSI Interoperability Service Created in 1999, specialized in the organization of test events Open to all organizations Experts from ETSI and other technical communities Skilled ETSI Staff managing test events In 6 years, 66 Events 3500 engineers 1400 Companies Usefulness & Service Satisfaction ranked at 4.35 / 5 ETSI Testing Projects - ICT Workshop, Brazil March 2006

ETSI Testing Projects - ICT Workshop, Brazil March 2006 Why? Provides useful feedback to the standardization process Accelerates time to market via quick product debugging Improves both quality and features of implementations Helps to ensure end users’ satisfaction Reduces deployment costs Allows networking with the market key players COST & TIME SAVING! ETSI Testing Projects - ICT Workshop, Brazil March 2006

ETSI Testing Projects - ICT Workshop, Brazil March 2006 Upcoming events in 2006 Q1 – 2006 SIM/Handset Lawful Interception 3rd WiMAX Plugfest Digital Identity Q2 – 2006 IPv6 Asia 4th SQTE EU round NGN Mobile Application NGN@Home Q3 – 2006 OSA Parlay UWB SIGTRAN Q4 – 2006 Human Factors (HF) IPv6 French Summit & Interop (FR) Powerline RFID ITS ecall ETSI Testing Projects - ICT Workshop, Brazil March 2006

Anthony Wiles ETSI Protocol and Testing Competence Centre Thank You! Phone +33 (0)4 92 94 43 26 e-mail ptcchelp@etsi.org Website www.etsi.org/ptcc Anthony Wiles ETSI Protocol and Testing Competence Centre