1. Copyright Telchemy Inc 2001 Embedded Passive Monitoring of Voice over IP Service Quality Alan Clark Telchemy, Inc Atlanta, GA alan@telchemy.com

2. Copyright Telchemy Inc 2001 Embedded Passive Monitoring IP Network Gateway Monitoring Agent embedded into VoIP Gateway QoS metrics

3. Copyright Telchemy Inc 2001 Active vs Passive Testing Active Measurement –Establish looped back test call –Send recorded speech file –Compare sent and received files (PSQM/PESQ) Passive Monitoring –Measure characteristics of live calls –Determine Transmission Quality metric –Estimate Speech Quality metrics

4. Copyright Telchemy Inc 2001 The E Model “Mouth to ear” transmission quality measurement Produces an “R” factor typically in the range 50 (bad) -95 (good) R factor can be related to MOS score, Terminate Early (TME) etc. ITU G.107/ G.108 and ETSI ETR250

5. Copyright Telchemy Inc 2001 E Model R = Ro - Is - Id - Ie + A Base R value - Noise level Impairments that occur simultaneously with speech - received speech level - sidetone level - quantization noise Impairments that are delayed with respect to speech - talker echo - listener echo - round trip delay Equipment Impairment Factor - CODEC - multiplexing effects Advantage factor

6. Copyright Telchemy Inc 2001 E Model Delay, measured using RTCP R Factor E Model IeIe Packet Loss Jitter Codec type Loss Model Jitter Model Codec Model

7. Copyright Telchemy Inc 2001 R Factor vs MOS 4.5 4.0 3.0 R FactorMOS

8. Copyright Telchemy Inc 2001 Impairments can vary with time Interaction of streams in IP network causes many impairments to vary with time Adaptive jitter buffer Call re-routing CODEC type changed during call

9. Copyright Telchemy Inc 2001 Measuring Packet Loss Arriving RTP packets Jitter buffer Late packets discarded Monitor jitter and packet loss after jitter buffer CODEC RTCP statistics report every 5 seconds

10. Copyright Telchemy Inc 2001 Burst vs average loss Burst of packet loss Zero packet loss Non-bursty packet loss Is not the same as

11. Copyright Telchemy Inc 2001 Effects of packet loss burst Transmission Quality High Low 4-5 seconds 15-30 seconds User perceived quality profile

12. Copyright Telchemy Inc 2001 Loss Model - Markov model 3 Lost 1 Rcvd P 13 P 22 P 11 P 31 2 Rcvd P 23 P 32 4 Lost P 14 P 41 Burst state Gap state

13. Copyright Telchemy Inc 2001 Frame loss event driven Burst of packet loss Zero packet loss Lost( )

14. Copyright Telchemy Inc 2001 Loss Model - Markov model 3 Lost 1 Rcvd P 13 P 22 P 11 P 31 2 Rcvd P 23 P 32 4 Lost P 14 P 41 Burst state Gap state Model parameters reconstructed at end of call

15. Copyright Telchemy Inc 2001 Loss Model - mapping loss to I e Curve is CODEC dependant

16. Copyright Telchemy Inc 2001 Determining QoS metrics 1. Determine “good” and “bad” state Ie Factor

17. Copyright Telchemy Inc 2001 Determining QoS metrics t = 5 t = 15 1. Determine “good” and “bad” state Ie Factor 2. Estimate Instantaneous R Factor for each state

18. Copyright Telchemy Inc 2001 Determining QoS metrics t = 5 t = 15 1. Determine “good” and “bad” state Ie Factor 2. Estimate Instantaneous R Factor for each state 3. Determine average Ie

19. Copyright Telchemy Inc 2001 Measuring Delay CODEC Accumulate frame Encode Transmission Jitter buffer Decode RTCP exchange

20. Copyright Telchemy Inc 2001 Delay Model 175 mS “knee” End to end delay (mS) R Factor Reduction

21. Copyright Telchemy Inc 2001 “Recency” effect “Good” 4.3MOS “Bad” 1.8 MOS (3dB SNR) MOS 3.82 MOS 3.28 MOS 3.18 Source AT&T T1A1.7/98-031 60 second call

22. Copyright Telchemy Inc 2001 Estimation of recency effect Delay since last significant burst Effects decay over 30-60 seconds Average for call

23. Copyright Telchemy Inc 2001 Integration with VoIP SMS VQmon SNMP Set, Get, Trap End of call msg (DRQ) CDR Service Mgt System Network Mgt System

24. Copyright Telchemy Inc 2001 H.323/H.225 based reporting H.225 Disengage Request (DRQ) nonStandardData field R end INTEGER (0..100) R av INTEGER (0..100) R min INTEGER (0..100) R max INTEGER (0..100)

25. Copyright Telchemy Inc 2001 Embedded Passive Monitoring RTP stream CODECCODEC IP Network CODECCODEC 1 Measure Packet Loss Distribution, Jitter, Delay, CODEC type 2 Produce a transmission quality rating (R Factor) using extension to the E-Model 3 Map R Factor to an estimated MOS Score