School of Computing, Communications and Electronics University of Plymouth Dr. Lingfen Sun Voice over IP and Voice Quality Measurement

4/2/2005SoCCE, UoP2 Outline of Talk Introduction VoIP Networks What is QoS or Perceived QoS? How to Measure/Predict Voice Quality? Subjective Objective (intrusive and non-intrusive methods) QoS Prediction and Control Research in Plymouth

4/2/2005SoCCE, UoP3 Introduction – the problem Internet Protocol (IP) networks On a steep slope of innovation – long term carriers of all traffic including voice traffic. IP is now the “universal” communications protocol because it facilitates convergence of networks and the ability to offer multiple services on the same networks. Not designed to carry real-time traffic, such as voice and video, because of their variable characteristics (e.g. delay, delay variation and packet loss). These have adverse effects on voice quality.

4/2/2005SoCCE, UoP4 Introduction – Voice Quality in IP networks User perceived quality is the key QoS metric in VoIP applications - The end-user of a VoIP service expects: voice quality to be as good as in traditional networks, and the service to be as reliable. This is not the case at present. This makes it necessary to be able to predict/measure, and if appropriate, control voice quality in order to deliver the desired QoS.

4/2/2005SoCCE, UoP5 VoIP Network and Perceived QoS Network QoS Perceived QoS is measured from ‘mouth to ear’, i.e. end-to-end and depends on the performance of IP network and terminal/gateway. IP Network SCN Gateway SCN: Switched Communication Networks (PSTN, ISDN, GSM …) Network QoS Perceived QoS IP phone IP softphone

4/2/2005SoCCE, UoP6 VoIP – New Applications IP Network /MPLS IAD DSLAM VoDSL Enterprise LAN Dual-mode handset VoWLAN MGW Mobile network AP IP access network PSTN GW IAD: Integrated Access Device DSLAM: DSL Access Multiplexer MGW: Media Gateway MPLS: Multi-protocol Label Switching

4/2/2005SoCCE, UoP7 VoIP Protocol Stack e.g. Ethernet/SDH IP UDP TCP Audio /video RTP RTCP SIP H.323 Physical layer Network layer Transport layer Application layer

4/2/2005SoCCE, UoP8 What is QoS? The ISO standard defines QoS as a concept for specifying how “good” the offered networking services are. QoS can be characterised by a number of specific parameters. For Multimedia Communication System (MCS), QoS concept can be extended to “User QoS” or “Perceived QoS”. For VoIP, Perceived QoS – user perceived voice quality (e.g. MOS)

4/2/2005SoCCE, UoP9 Factors affect voice quality IP Network Receiver Voice source Encoder Sender Packetizer Jitter buffer Decoder De- packetizer End-to-end perceived voice quality (MOS) packet loss network delay jitter coding distortion codec delay delay buffer-delay buffer-loss codec impairment delay Other impairments: echo, sidetone, background noise … Other factors: language, gender, FEC, packet loss concealment Voice receiver

4/2/2005SoCCE, UoP10 Inter-relationships between the QoS Parameters [1] Network Packet Loss Network Jitter Network Delay Codec Performance Overall Packet Loss Perceived Quality Overall Delay Jitter Buffers Network FactorsApplication Factors QoS Service Level

4/2/2005SoCCE, UoP11 QoS parameters [1] QoS Service Class Codec Performance, VAD, Frames per Packet, Jitter Buffer, Codec Delay, FEC (Redundancy) Max Packet Loss, Max Mean delay, Max Delay Variation SERVICE APPLICATION TRANSPORT

4/2/2005SoCCE, UoP12 Key QoS parameters and how they arise Packet Loss Network packet loss (as a result of congestion or rerouting in the IP network) Late arrival loss (dropped at receiver) Link failures and system errors. End-to-end Delay Network delay (transmission and queuing delay) Buffer delay Codec processing delay Packetizing/depacketizing delay Jitter (delay variation) Caused by queuing delays within the IP network

4/2/2005SoCCE, UoP13 Delay impact on multimedia quality [7] 0% Packet Loss Conversational voice and video Voice/video messaging Streaming audio/video Fax 5% 100 msec1 sec10 sec100 sec InteractiveResponsiveTimelyNon-critical Delay For VoIP applications, delay 400 ms, quality unacceptable for most users.

4/2/2005SoCCE, UoP14 How to Enhance QoS? Application-level QoS mechanisms Packet loss compensation (e.g. FEC, loss concealment) Jitter compensation (e.g. buffer algorithms) Adaptive source coding … Network-level QoS mechanisms How to guarantee IP network performance Diffserv (Differentiated Services) Intserv (Integrated Services) …

4/2/2005SoCCE, UoP15 How to Measure Voice Quality? Why need to measure voice quality? For QoS monitoring and/or control purposes to ensure that the technical and commercial requirements (e.g. SLA) are met. How to measure voice quality? Subjective methods (e.g. MOS) Objective methods (e.g. PESQ or E-model)

4/2/2005SoCCE, UoP16 Subjective or objective measurement Subjective Voice Quality Measurement Subjective listening tests by a group of people Provides a benchmark for objective test methods Expensive and time-consuming Objective Speech Quality Measurement Repeatable, automatic, and predicts subjective score Suitable for online quality measurement/monitoring Can be used for intrusive and Non-intrusive measurements.

4/2/2005SoCCE, UoP17 Voice quality measurement SCN IP Network Gateway SCN: Switched Comm. Networks (PSTN, ISDN, GSM …) Non-intrusive Measurement (parameter-based e.g. E-model) MOSc Gateway Intrusive Measurement (e.g. PESQ) Reference speech Degraded speech Non-intrusive Measurement (signal-based e.g. P.563) (e.g. loss, delay, jitter) MOS-LQ MOS-LQ: MOS-Listening quality MOSc: Conversational MOS score

4/2/2005SoCCE, UoP18 Voice quality measurement (cont.) Voice quality measurement Subjective methods Objective methods Non-intrusive methods Intrusive methods Parameter-based methods Signal - based methods Comparison-based methods Calibration

4/2/2005SoCCE, UoP19 Mean Opinion Score (MOS) The most widely used subjective measure of voice quality. Provides a direct link to voice quality as perceived by the end user. Gives average opinion of quality based on asking people to grade the quality of speech on a five-point scale: Excellent, Good, Fair, Poor and Bad. Slow, time-consuming, expensive, not repeatable and cannot be used to monitor voice quality on-line in a large network. Different Categories of MOS Test (ITU P.800[2]) Absolute Category Rating (ACR): only listen to the degraded speech signals (most commonly used) Degradation Category Rating (DCR): rate annoyance or degradation level between the reference and degraded signal Subjective voice quality measurement

4/2/2005SoCCE, UoP20 MOS Test Based on ACR CategorySpeech Quality 5Excellent 4Good 3Fair 2Poor 1Bad Absolute Category Rating (ACR)

4/2/2005SoCCE, UoP21 MOS Test based on DCR CategoryDegradation level 5Inaudible 4Audible but not annoying 3Slightly annoying 2Annoying 1Very annoying Degradation Category Rating (DCR)

4/2/2005SoCCE, UoP22 Online MOS Test Website os os This is our research on subjective tests. The aim is to provide a more efficient method to carry out subjective tests compared to standard MOS test (e.g. ITU P.800). Standard MOS measurement requires a stringent test requirement (e.g. sound proof room, a large number of subjects, test procedures). Thus, it is very time consuming, expensive, and difficult to organise a test.

4/2/2005SoCCE, UoP23 Objective voice quality measurement Automated measure of speech quality using an appropriate model. Conventional methods, e.g. SNR-based approach, are not appropriate as they fail to reveal quality as perceived by the end user. Emerging methods for voice quality prediction are based on models of human auditory perception or psychologically-derived computational models. Can be intrusive (e.g. ITU P.862, PESQ [3]) or Non- intrusive (e.g. ITU P.563 [4] formerly P.SEAM).

4/2/2005SoCCE, UoP24 Intrusive measurement PESQ (Perceptual Evaluation of Speech Quality), ITU P.862, Feb, 2001 Intrusive (active) test, listening-only quality uses test stimuli, such as speech signal System under test PESQ Reference signal/speech Degraded signal/speech PESQ quality score (MOS)

4/2/2005SoCCE, UoP25 Perceptual Evaluation of Speech Quality Transforms the original and degraded speech signals into a psychophysical representation that approximates human perception. Calculates their perceptual distance and maps this into an objective MOS score.

4/2/2005SoCCE, UoP26 PESQ (perceptual difference) reference speech degraded speech Loss position PSQM PESQ

4/2/2005SoCCE, UoP27 OPTICOM- Opera system Opera system "Digital Ear“ Perceptual Voice/Audio Quality PESQ/PSQM/PEAQ

4/2/2005SoCCE, UoP28 Non-intrusive measurement Non-intrusive (passive) test Output-based (speech signal based) or parameter-based Low accuracy if compared to the intrusive methods Adequate for real-time, online monitoring purposes

4/2/2005SoCCE, UoP29 Non-intrusive Speech Quality Prediction Signal-based (output-based): to predict/measure voice quality directly from degraded speech signal (e.g. from T1/E1). Parameter-based: to predict/measure voice quality directly from IP network impairment parameters (e.g. loss, delay, jitter). PSTN Gateway MOS Parameter- based method IP T1/E1 Signal-based method MOS IP Network Signal-based method MOS

4/2/2005SoCCE, UoP30 Signal based (output-based) Method Assess/predict speech quality non-intrusively from degraded speech signal only Need to extract speech features (e.g. unnaturalness voice, noises, time clipping) Mapping to MOS via quality prediction model ITU P.563 – May 2004 (single-end, signal-based or output-based) From T1/E1 link or end terminal Speech speech feature parameters extract/ analysis Speech quality model MOS Pre-processing

4/2/2005SoCCE, UoP31 Parameter based Method IP packets RTP header / network parameter analysis Parameters (e.g. loss, jitter, delay) Quality prediction models (e.g. NN or non-linear models) MOS Access/predict speech quality from IP network impairments (e.g. loss, delay) and codec etc. Neural network model, non-linear regression model, ITU-T E-model [5] External or built-in approach (be located before/after jitter buffer)

4/2/2005SoCCE, UoP32 E-model (ITU G.107, G.108) Computational model – can be used to compute the “Mouth-to-ear” transmission quality. Overall Transmission Quality Rating given by model is referred to as the R factor. R lies in the range and can be mapped to MOS. Designed for network planning, but may be used for non-intrusive quality monitoring/measurement. Based on the principle that “Psychological factors on the psychological scale are additive”

4/2/2005SoCCE, UoP33 E-model equation Ro: base R value (noise level) Id: impairments that are delayed with respect to speech (e.g. talker/listener echo and absolute delay) Is: impairments that occur simultaneously with speech (e.g. quantization noise, received speech level and sidetone level) Ie: equipment impairment (e.g. codec, packet loss, jitter) A: Advantage factor (e.g. 0 for wireline and 10 for GSM)

4/2/2005SoCCE, UoP34 Loss model - maps loss to Ie Curve is CODEC dependant

4/2/2005SoCCE, UoP35 Delay model End to end delay (ms) R Factor Reduction

4/2/2005SoCCE, UoP36 E-model (a simplified version) Delay model Loss model R  MOS MOS Packet loss rate Codec type Delay (d) IdId IeIe

4/2/2005SoCCE, UoP37 E-model (R factor) and MOS TIA 2000

4/2/2005SoCCE, UoP38 Extended E-model Simplified E-model consider only effects from codec, packet loss (random packet loss) and end-to-end delay. Extended E-model [6] Further consider burst loss effects (e.g. 2-state Gilbert model, 3 or 4 states Markov models) Further consider recency effects. Telchemy (

4/2/2005SoCCE, UoP39 Burst Loss vs. Random Loss Burst packet loss Non-bursty packet loss Packet lost Packet received

4/2/2005SoCCE, UoP40 “Recency” Effect [6] “Good” 4.3MOS “Bad” 1.8 MOS (3dB SNR) MOS 3.82 MOS 3.28 MOS 3.18 Source AT&T T1A1.7/ second call

4/2/2005SoCCE, UoP41 Extended E Model [6] Delay, measured using RTCP Network R Factor IeIe Packet Loss Jitter Codec type Loss Model Jitter Model Codec Model Burst model Recency model User R Factor Delay model

4/2/2005SoCCE, UoP42 VQmon – Embedded Monitoring[6] IP Network Gateway VQmon Agent embedded into VoIP Gateway QoS metrics NMS Telchemy (

4/2/2005SoCCE, UoP43 Voice and Video quality Assessment in Psytechnics Psytechnics – spin off from BT Intrusive model (e.g. PESQ) Non-intrusive model psyVoIP (parameter-based) E-model NiQA (signal-based) CCI (Call Clarity Index)/INMD (In-service Non- intrusive Measurement Device)

4/2/2005SoCCE, UoP44 QoS Prediction and Control - Research in Plymouth Aims and objectives To research and develop novel, generic methods for objective measurement, prediction and control of user-perceived quality. To apply the methods to real world problems in communications, audio and healthcare. Examples Non-intrusive voice quality prediction and measurement for VoIP QoS prediction and control for wireless VoIP Multimedia quality prediction (voice, audio and video)

4/2/2005SoCCE, UoP45 Signal Processing & Multimedia Communications Group Research within the Group is concerned with the development of novel, generic signal and information processing methods and their applications to real world problems. Main application areas: Multimedia communications – quality of service prediction and control Audio – sound synthesis, audio quality assessment Biomedicine – intelligent biosignal analysis, biomedical informatics, decision support.

4/2/2005SoCCE, UoP46 About my PhD project Voice source Voice receiver IP Network Receiver Encoder Sender Packetizer Jitter buffer Decoder De- packetizer Non-intrusive measurement MOS End-to-end perceived voice quality (MOS) To develop novel and efficient method/models for non-intrusive quality prediction, To apply the models for perceptual optimization control( e.g. buffer optimization and adaptive sender-bit-rate QoS control)

4/2/2005SoCCE, UoP47 A New Methodology VoIP Network New model (packet loss, delay, codec …) Predicted MOSc PESQ E-model Measured MOSc delay MOS(PESQ) Reference speech Degraded speech Intrusive method (regression or ANN models) Non-intrusive method Based on intrusive quality measurement (e.g. PESQ) to predict voice quality non- intrusively which avoids subjective tests. A generic method which can be easily applied to audio, image and video.

4/2/2005SoCCE, UoP48 Two Non-intrusive Models Artificial neural network models for predicting listening and conversational voice quality Simplified regression models to predict voice quality

4/2/2005SoCCE, UoP49 Three Applications Voice quality monitoring/prediction for real Internet VoIP traces Perceived voice quality driven jitter buffer optimization Perceived voice quality driven QoS control (combined adaptive sender-bit- rate and priority marking control)

4/2/2005SoCCE, UoP50 References 1. M. Buckley, End-to-end QoS control in VoIP systems, Workshop on QoS and user perceived transmission quality in evolving networks, Oct ITU-T Rec. P.800, Methods for subjective determination of transmission quality, Aug ITU-T Rec. P. 862, Perceptual evaluation of speech quality (PESQ), an objective method for end-to-end speech quality assessment of narrow ‑ band telephone networks and speech codecs, Feb ITU-T Rec. P.563, Single-ended method for objective speech quality assessment in narrow-band telephony applications, May ITU-T Recommendation G.107, The E-model, a computational model for use in transmission planning, A. Clark, Modeling the Effects of Burst Packet Loss and Recency on Subjective Voice Quality, 2 nd IPTel Workshop, 2001, pp.123 – H. Schink, Characterising end to end quality of service in TIPHON systems, IP Networking & Mediacom Workshop, April 2001.

4/2/2005SoCCE, UoP51 References L Sun and E Ifeachor, "New Models for Perceived Voice Quality Prediction and their Applications in Playout Buffer Optimization for VoIP Networks“ Proceedings of IEEE ICC 2004, Paris, France, June 2004, pp Z Qiao, L Sun, N Heilemann and E Ifeachor "A New Method for VoIP Quality of Service Control Based on Combined Adaptive Sender Rate and Priority Marking“ Proceedings of IEEE ICC 2004, Paris, France, June 2004, pp L Sun and E Ifeachor, "New Methods for Voice Quality Evaluation for IP Networks" Proceedings of the 18th International Teletraffic Congress (ITC18), Berlin, Germany, 31 Aug - 5 Sep 2003, pp L Sun and E Ifeachor, "Prediction of Perceived Conversational Speech Quality and Effects of Playout Buffer Algorithms“, Proceedings of IEEE ICC 2003, Anchorage, USA, May 2003, pp L Sun and E Ifeachor, "Perceived Speech Quality Prediction for Voice over IP-based Networks" Proceedings of IEEE ICC 2002, New York, USA, April 2002, pp L Sun, G Wade, B Lines and E Ifeachor, "Impact of Packet Loss Location on Perceived Speech Quality“, Proceedings of 2nd IP- Telephony Workshop (IPTEL '01), New York, April 2001, pp

4/2/2005SoCCE, UoP52 Contact details: SPMC Group website: Professor Emmanuel Ifeachor, Head of Group, Dr. Lingfen Sun Homepage:

4/2/2005SoCCE, UoP53 Thank you!