1. DOCUMENT #:GSC15-PLEN-51 FOR:Presentation SOURCE:ITU AGENDA ITEM:Plenary 6.10 CONTACT(S):Reinhard.Scholl@itu.int Conformance and Interoperability Testing Reinhard Scholl ITU Global Standards Collaboration (GSC) GSC-15

2. Highlight of Current Activities (1/3)  ITU-T SG 11: Lead study group on test specifications  ITU-T SG 15: Has extensive testing ecosystem  ITU-T SG 16: its IPTV standards (H-series) was put to test in ITU-T’s first interop event (20-23 July) 3 IPTV testing specs approved; others further developed  ITU-T SG 17: Lead study group on languages and description techniques Also: Q14/17 on Testing methodology, specification language and Generic test suites for CIT 2

3. Highlight of Current Activities (2/3)  ITU-T JCA-CIT: Joint Coordination Activity on Conformance and Interoperability Testing Works with all Study Groups to:  address the topics relevant to conformance and interoperability testing  enhance the quality of ITU-T Recommendations Eliminate errors, ambiguities unverified behaviours Include sufficient detail for implementation and testing Include interfaces needed for interconnection and testing  develop methodologies and frameworks for testing 3

4. Highlight of Current Activities (3/3)  Regional ITU consultations on conformance assessment and interoperability Quito, 8 July 2010 Nairobi, 30-31 July 2010 Sydney, 16-17 Sep. 2010  Pilot conformity database developed  ITU Interop started IPTV (July in Geneva; Sep. in Singapore; Dec. in India) Home networking 4

5. Strategic Direction  WTSA-08 Res. 76 on conformance & interop  …supported & complemented by WTDC-10 Res. 47 (ITU-D)  ITU Council-09 mandated a pilot conformity assessment database interoperability events human resources capacity building assistance in the establishment of test facilities in developing countries a business plan for long-term implementation 5

6. Challenges  Regional ITU consultations on conformance assessment and interoperability are a useful discussion platform, see e.g. Presentations at Kenya workshop, in particularPresentations at Kenya workshop  TSB’s overview presentation on status of implementation of WTSA-08 Res. 76overview presentation 6

7. Next Steps/Actions  Continue implementing steps outlined under “Strategic directions”  Study groups to continue working on test specs  Keep momentum of ITU’s IPTV Interop event Expand to other ITU standards, in collaboration with partners as appropriate  Populate conformance database 7

8. Supplementary Slides 8

9. ITU-T SG 11’s Questions on Testing  ITU-T SG 11: Signalling, protocols and test specs Q 8/11 Protocol Test Specifications for NGN Q 8/11 Q 9/11 Monitoring parameters for NGN protocols Q 9/11 Q 10/11 Service test specification for NGN Q 10/11 Q 11/11 QoS tests specification for NGN Q 11/11 Q 12/11 NID and USN test specification Q 12/11

10. ITU-T SG 11 Approved Standards  Q.3900 Methods of testing and model network architecture for NGN technical means testing as applied to public telecommunication networks Q.3900  Q.3901 Distribution of tests and services for NGN technical means testing in the model and operator networks Q.3901  Q.3902 Parameters to be monitored in the process of operation when implementing NGN technical means in public telecommunication networks Q.3902  Q.3903 Formalized presentation of testing results Q.3903  Q.3904 The scenarios, list and types of tests for TM local and NUT testing for IMS on the model networks Q.3904  Q.3910 Monitoring parameters set for NGN protocols Q.3910  Q.3911 Monitoring parameters set for voice services in NGN Q.3911 10

11. 11  ITU-T SG 15: “Optical transport networks and access network infrastructures”  Tight collaboration: Access (DSL, PON):  Broadband Forum Optical transport, framing, cross connecting  Optical Internetworking Forum (OIF)  Metro Ethernet Forum (MEF) Home networking  HomeGrid Forum  Broadband Forum Nairobi, Kenya, 30-31 July 2010 ITU-T SG 15’s Extensive Testing Ecosystem

12.  Develop conformance testing methodology and framework  Develop interoperability testing methodology and framework  Assist standards developers for applying conformance and interoperability testing methodology to their specific needs  Progress work in the area of Testing and Test Control Notation, version 3 (TTCN-3) ITU-T SG 17’s Testing languages, methodologies and framework

13. ITU-T SG 17’s approved standards Conformance Testing Methodology and Framework  X.290 - General Concepts  X.291 - Abstract Test Suite Specification  X.292 - Tree and Tabular Combined Notation  X.293 - Test Realization  X.294 - Requirements on Test Laboratories and Clients  X.295 - Protocol Profile Test Specification  X.296 - Implementation Conformance Statements  X Suppl. 4 - Supplement on generic approach to interoperability testing  X Suppl. 5 - Supplement on interoperability testing framework and methodology Testing and Test Control Notation version 3 (TTCN-3)  Z.161: TTCN-3 core language  Z.162: TTCN-3 tabular presentation format (TFT)  Z.163: TTCN-3 graphical presentation format (GFT)  Z.164: TTCN-3 operational semantics  Z.165: TTCN-3 runtime interface (TRI)  Z.166: TTCN-3 control interface (TCI)  Z.167: TTCN-3 mapping from ASN.1  Z.168: TTCN-3 mapping from CORBA IDL  Z.169: TTCN-3 mapping from XML data definition  Z.170: TTCN-3 documentation comment specification

14. 14 Implementation of the ITU Conformity Programme ITU-T Recs & test suites Supplier’s conformity route decision Test lab (certification body responsibility) 1 st 2 nd or 3 rd party accredited lab (ISO/IEC 17025) ITU Conformity Database Test results (ITU-T X.290) 1st party Evaluation Supplier’s Conformity Declaration (ISO/IEC 17050) Test results (ITU-T X.290) Accredited (ISO/IEC Guide 65) Certification body Evaluation Conformity Certificate issued by Certification Body Conformity Assessment / Certification ITU CIP services Supplier ’ s Request to ITU Route 1 Route 2

15. 15 Supplier’s Declaration of Conformity – SDoC 1/2

16. 16 Supplier’s Declaration of Conformity – SDoC- 2/2

17.  Putting to the test: ITU-T’s IPTV suite of standards: H.721 et al.  Geneva, 20-23 July: testing, followed by showcasing  Participating companies highly satisfied: 4.5 rating (out of 5)  Event highlight on YouTube, ITU website: http://youtube.com/watch?v=CpJmI_5aQrA http://youtube.com/watch?v=CpJmI_5aQrA  ITU press release: http://itu.int/net/pressoffice/press_releases/2010/31.aspx http://itu.int/net/pressoffice/press_releases/2010/31.aspx  Photos on Flickr: http://flickr.com/photos/itupictures http://flickr.com/photos/itupictures ITU’s 1 st Interop Event (1/2)

18. Test report will be fed into the standards development process H.721 Conformity Check List H.762 Conformity Check List H.770 Conformity Check List H.750 Conformity Check List ITU’s 1 st Interop Event (2/2) Testing specs: HSTP.CONF-H721 HSTP.CONF-H762 HSTP.CONF-H770