ITU-T Study Group 12 End-to-end transmission performance of networks and terminals Jean-Yves Monfort, FT

Contents Terms of reference Management Team SG Organization Statistics Highlights of achievements / Projects / FGs Workshops Future work Conclusions

Terms of Reference Responsible for guidance on the end-to-end transmission performance of networks, terminals and their interactions, in relation to the perceived quality and acceptance by users of text, speech, and image applications. This work includes the related transmission implications of all networks (e.g., those based on PDH, SDH, ATM and IP) and all telecommunications terminals (e.g., handset, hands-free, headset, mobile, audiovisual, and interactive voice response). Lead SG for: Quality of Service and Performance

Management Team Mr Jean-Yves Monfort Mr Charles A. Dvorak Chair Mr Jean-Yves Monfort France Vice-Chairs: Mr Charles A. Dvorak (QoS Lead SG Coordinator) United States Mr Klemens P.F. Adler Germany WP Chairs: Mr Rodolfo Ceruti Italy Mr Paul Coverdale Canada TSB Mrs Judit Katona-Kiss COUNTRY: should be the letter acronym, e.g. USA, I (for Italy), J (Japan), etc, as used in the ITU list of participants Delete UNUSED rows in the PowerPoint table

Study Group Structure WP1 Telephonometry and terminals Responsible for test Methodologies and quality parameters for voice terminal/speech processing equipments WP2 Performance, planning and assessment Responsible for Subjective and objective quality assesment, for quality models and Transmission planning WP3 QoS over IP Responsible for the definition and characterisation of QoS parameters for multimedia over IP.

Statistics (I) 10 Rapporteurs meetings held 283 contributions received 6 SG meetings held Max/Min/Average SG participants : 85/49/71

Statistics (II) 35 Recommendations approved 18 New/ 17 Revised 12 Questions assigned by WTSA-2000 4 + 1 New and Revised Questions during study period 15 Questions proposed for next period

Highlights of achievements (I) Quality Assessment Models This activity was mainly applied to speech quality assessment. Three different and complementary approaches are implemented in SG 12 : - Opinion Models "E-Model"-Rec G.107- revised several times during the Study Period, in particular to take into account IP packet loss. Two methodologies to derive Impairment factors for the E-Model from subjective or objective test results : P.833 and P.834 - Psychophysical Models, P.862 (known as "PESQ") with reference input signal Rec. P.563 for assessing speech quality without reference input signal. - Models associated to "INMD" (In Service Non Intrusive Measurement methods). P.561 (measurement principles) and P.562 (Analysis and interpretation) revised to take into account the new network interfaces using packet mode transmission. New draft Recommendation P.VTQ under development, will take into account both static and dynamic calibration. - The scope has been expanded to multimedia assessment in the next Study Period. Multimedia Quality Assessment Models

Highlights of achievements (II) Multimedia QoS over IP This activity, initiated at the beginning of the Study period, has produced fundamental Recommendations. G.1010 "End-user multimedia QoS categories". This Recommendation gives a classification, from the user point of view, of the services, based on parameters as delay, delay variation and loss. This Recommendation has been already referenced in several other Recommendations and Standards. G.1020 “Performance parameter definitions for quality of speech and other voice-band applications utilizing IP networks”. This Recommendation provides guidance on performance parameters for IP networks and end terminals that impact speech quality and other voice-band applications. Annex A defines VoIP Gateway-specific reference points and performance parameters. A new P.1000-series has been created. P.1010 “Fundamental Voice Transmission Objectives for VoIP Terminals and Gateways”.

Highlights of achievements (III) Subjective assessment Methodology A Recommendation in this field is the result of extensive tests in several laboratories, and on several langages. Subjective test results give references for Quality Model validations New methodologies are developped to assess speech processing techniques P.851 "Subjective Quality Evaluation of Telephone Services based on Spoken Dialogue Systems". A new activity in ITU-T (a bridge between Human factors and Technology). P.880 "Continuous Evaluation of Time-Varying Speech Quality". Result of several studies conducted in research laboratories, this new methodology gives a new way of assessing subjectively time varying quality (Mobile, IP,etc.). P.835 for speech communication systems that include noise suppression algorithm"; P.840 for evaluating circuit multiplication Equipment".

Highlights of achievements (IV) Telephonometry, Terminals and Signal Processing Quality ITU-T SG 12 is a major player. Recommendations on artificial ears, mouths, speech signals… telephonometric methods and speech terminal characteristics periodically updated. - An inter-laboratory test has been carried out to define the test methods headsets P.380 - Annexes to P.501 offer a high quality common data base (speech, noise,…), free of rights - Recommendations on speech terminals and speech processing equipments : P.330 for Acoustic Echo Cancellers and Noise Suppressors. P.313 for cordless and mobile Digital Terminals; P.300 for Audio group audio terminals (GATs)

Highlights of achievements (V) Transmission planning, Quality parameters Several fundamental Recommendations have been updated several times during the Study Period, to take into account the technical evolutions of networks : G.101 “The transmission Plan”, based on a new approach. G.113 “Transmission impairments due to speech processing” and mainly the Appendix I on Impairment factors (used by E-Model), G.131 “Talker echo and its control”, G.114 “One-way Transmission Time”, now recommended to evaluate the effects on speech quality of delays below 400 ms using the E-model (G.107)

Highlights of achievements (VI) Definitions, Guidelines and frameworks One major Recommendation on QoS is G.1000 " Communications Quality of Service: A framework and definitions " (see next slide on Lead SG activities for QoS). Several Recommendations have defined specific parameters or terminologies ( MOS based on how they are obtained, Psycho-acoustic parameters) Several documents (e.g. G.100.1, G.108.2, Appendix II to Recommendation G.108) have been developed to help the members to understand correctly the way to implement the rules and the Recommendations. Collaboration with SG 16 on speech coding assessment SG 12 has worked closely with SG 16 to assess the quality of ITU-T 4 kbit/s speech coding algorithm and ITU-T Wideband Speech Coding Algorithm around 16 kbit/s.

Highlights of achievements (VII) Leading Role for QoS and Performance An initial survey was done of all of the QoS-related activities across all ITU-T Study Groups (“An Assessment of IP-related Quality of Service Work in the ITU-T,” - The new Recommendation G.1000 (a definitional framework) maintains the E.800 definition of Quality of Service, and provides a comprehensive matrix for identifying the performance of the many different communications functions that comprise a service. It includes an explicit delineation of four different views of end-to-end QoS - QoS levels that the user requests (or desires) - QoS (or network performance) the service provider commits to - Levels of service quality or network performance actually delivered - What the user ultimately perceives - This information was also captured in a newly crafted ITU-T Brochure on QoS. - G.1000 focuses on the QoS of IP-based networks, noting that a major gap in standards is a scalable and reliable end-to-end solution for satisfying user requests for Y.1541 QoS classes. This was identified as the single biggest challenge to ITU standards delivering end-to-end IP QoS, so a program of communication and coordination was undertaken. - This role needs to continue for the new Study Period (see next slides)

Workshops (I) QoS and user-perceived transmission quality in evolving networks / 18-19 October 2001 / Dakar, Senegal This workshop was mainly tutorial and organized in the week before SG12 Dakar's meeting, giving a complete view of ITU-T activities on End-to-End Quality.

Workshops (II) End to End Quality of Service. What is it? How do we get it? / 1-3 Oct. 2003 / Geneva The main result of the workshop is an improvement of the collaboration between the different standardization bodies involved in QoS and Performance (inside and outside ITU-T). One of the activities of SG 12 as Lead SG on QoS and Performance Steering Committee and Speakers

Workshops (III) Standardization in Telecommunication for motor vehicles / 24-25 Nov 2003 / Geneva New questions created in SG 12 Creation of an Advisory Panel for Standards Cooperation on Telecommunications related to Motor Vehicles(APSC-TELEMOV) and a web site Following action : a new workshop, March 2005, in conjunction with Geneva Motor Show. Steering Committee and Speakers A view of The exibition

Workshops (IV) Workshop on "Wideband Speech Quality in terminals and networks: Assessment and prediction", 8-9 June 2004, Mainz Organised by ETSI, sponsored by ITU-T,SFA, DEGA Wideband Speech : VoIP will certainly give the success . Wideband speech increases the perceived quality and mainly the intelligibility, specially in noisy environment. E-Model (G.107) and P.862 are now well known and used for narrow-band speech; Substancial progress are expected in the near future to predict and to assess wideband speech quality, and to assess the speech quality whatever the bandwidth. The possibility to define "categories", based on acoustics and signal processing performance.

Future Work (I) The main focus is end to end Quality (of service) for all types of media, which implies all kinds of networks and terminals increased emphasis on QoS interoperability between all core and access networks (mobile and fixed networks). This is one of the major challenges for ITU-T in the next years. The work on quality models will continue (in the three domains: psychophysical, in-service non-intrusive and opinion models), for voice (including wideband) but will be extended to multimedia. This work will be carried out in particular in the new Joint Rapporteurs Group (JRG) on Multimedia Quality Assessment (speech quality experts from SG G and video experts from SG E).

Future Work (II) SG 12 will make a more concerted effort to train new members and participants in the work in progress through developing handbooks, web based tutorials, etc. The coordination role on QoS within SG 12 will continue, with a view to improving cooperation inside ITU-T and with other bodies, regarding QoS signalling, QoS interoperability, possibly by driving a "high level" Project on end-to-end QoS.

Future Work (III) The main topics for the next Study Period will be : Transmission planning, in particular focused on NGN; QoS interoperability Quality assessment models ( Psychophysics models, INMD, opinion models), including multimedia quality assessment models (Coll with SGE) and adaptations to Wide band Speech; Speech Quality in motor vehicles environment; Subjective quality Assessment (Coll. With SG F) Terminals, gateways quality parameters and measurement methods, including activities for wideband Network performance Performance-related handbooks, guide, reference implementations; QoS coordination (as lead Study Group or as a Coordination Project).

Future Work (IV) Lead Study Group on QoS As work is still needed on the End-to-End (E2E) IP QoS Signalling solution, and since interoperability of current QoS approaches is likely to be problematic unless other work is done to facilitate interoperability, it is suggested that this coordination and harmonization role of SG 12 needs to continue. The main tasks will be : Coordination of QoS-related activities in the ITU-T (ongoing), with the focus on identifying gaps in QoS standardization that present interoperability obstacles to achieving E2E QoS. Coordination among various standards activities on QoS and collaboration with other standards bodies (ongoing), with an emphasis on interoperability.

Acknowledgements Thanks to the efficient and innovative Management Team The Rapporteurs and Editors The Contributors TSB Staff France Telecom and Sonatel ….and Pat, Chloé, Karine, Youenn SG 12 Recognition Ceremony

Mr Jean-Yves Monfort Since WTSA 2000, J.Y. Monfort acts as Chairman of ITU-T SG 12. With the Management team of the Study Group, he has organised several Workshops, organised a study group meeting in Dakar and contributed to the role of Lead SG on QoS He also contributed to several ITU-T workshops and other conferences to promote ITU-T activities on QoS. He was previously Vice Chairman of ITU-T SG 12 and served several years as Rapporteur on questions dealing with telephony terminal transmission quality and electro-acoustics.He joint SG 12 activities in 1987. For the future, the aim is to pursue the actions to reinforce the cooperation with other standardization bodies, to ensure a great collaboration between the SGs to maintain ITU-T as a Leader in the field of Performance and QoS, facing the new challenges of NGN, Multimedia and Mobiles.