1. Alan Clark Telchemy alan@telchemy.com
Modeling the effects of Burst Packet Loss and Recency on Subjective Voice Quality
Alan Clark
Telchemy
Telchemy – IPtel 2001

2. Embedded Monitoring
Need to monitor QoS to provide feedback on network performance / impact on subjective quality
Desirable to provide monitoring in the form of a lightweight software agent
Focus on time varying impairments – burst packet loss and “recency”
Telchemy – IPtel 2001

3. Embedded Monitoring Gateway Gateway QoS metrics VQmon AgentIP
Network
QoS
metrics
VQmon Agent
embedded into
VoIP Gateway
SMS/NMS
Telchemy – IPtel 2001

4. The E Model “Mouth to ear” transmission quality measurement
Produces an “R” factor typically in the range 50 (bad) -95 (good)
R factor can be related to MOS score, Terminate Early (TME) etc.
ITU G.107/ G.108 and ETSI ETR250
Telchemy – IPtel 2001

5. R Factor vs MOS
R Factor
MOS
4.5
4.0
3.0
Telchemy – IPtel 2001

6. E Model R = Ro - Is - Id - Ie + A Base R value - Noise level
Advantage factor
Impairments that
occur simultaneously
with speech
- received speech level
- sidetone level
- quantization noise
Equipment Impairment
Factor
- CODEC
- multiplexing effects
Impairments that
are delayed with
respect to speech
- talker echo
- listener echo
- round trip delay
Telchemy – IPtel 2001

7. Extended E Model Ie Packet Loss Network R Factor Jitter User R Factor
Burst
model
Jitter
Model
Jitter
User
R Factor
Codec
type
Codec
Model
Recency
model
Delay
model
Delay, measured
using RTCP
Telchemy – IPtel 2001

8. Burst vs average loss Burst of packet loss Zero packet loss
Non-bursty packet loss
Telchemy – IPtel 2001

9. Instantaneous Quality
Source France Telecom
Telchemy – IPtel 2001

10. “Recency” effect MOS 3.82 MOS 3.28 MOS 3.18 60 second call
“Bad” 1.8 MOS
(3dB SNR)
“Good” 4.3MOS
Source AT&T
T1A1.7/98-031
Telchemy – IPtel 2001

11. Jitter and Packet Loss Monitor jitter and packet loss after
jitter buffer
Jitter
buffer
Arriving
RTP
packets
CODEC
Late packets
discarded
Telchemy – IPtel 2001

12. Loss Model - Markov model
Model parameters
reconstructed at end
of call 3
Lost
P32
P23
P31 2
Rcvd
P13
Burst state
P22 1
Rcvd 4
Lost
Gap state
P14
P41
P11
Telchemy – IPtel 2001

13. Loss Model - mapping loss to Ie
Curve is CODEC
dependant
Telchemy – IPtel 2001

14. Determining QoS metrics
1. Determine “good”
and “bad” state
Ie Factor
Telchemy – IPtel 2001

15. Determining QoS metrics
1. Determine “good”
and “bad” state
Ie Factor
2. Estimate
Instantaneous
R Factor for
each state
Telchemy – IPtel 2001

16. Determining QoS metrics
3. Determine average Ie
1. Determine “good”
and “bad” state
Ie Factor
2. Estimate
Instantaneous
R Factor for
each state
Telchemy – IPtel 2001

17. Delay effects 4 3 2 Mean Opinion Score Conversation Verify sequence
of random numbers 1 2 3 4
Delay (seconds)
Source NTT
Telchemy – IPtel 2001

18. Measuring Delay Accumulate frame Encode Jitter Transmission buffer
Decode
CODEC
CODEC
RTCP exchange
Telchemy – IPtel 2001

19. Delay Model R Factor Reduction End to end delay (mS) 175 mS “knee”
Telchemy – IPtel 2001

20. Estimation of recency effect
Effects decay over seconds
Average for call
Delay since last significant burst
Telchemy – IPtel 2001

21. Execution model Call 1 Call 2 Initialize Ports Typically 1.4 assembly
Start
call
End
call
Start
call
End
call
Loss events
Loss events
Typically 1.4 assembly
language instructions
per packet
Telchemy – IPtel 2001

22. Integration with VoIP SMS
Service
Mgt
System
CDR
End of call
msg (DRQ)
VQmon
SNMP
Set, Get, Trap
Network
Mgt
System
Telchemy – IPtel 2001

23. Ranking accuracy – Set 1
Telchemy – IPtel 2001

24. Ranking Accuracy – Set 2
Telchemy – IPtel 2001

25. Ranking Accuracy – Set 4
Telchemy – IPtel 2001

26. Conclusions Computational model meets design goals
Ranking accuracy is comparable to human listener
But need -
Systematic comparison with PSQM/ PESQ
Increased level of subjective testing
Add support for VAD, non-PLC
Improved accuracy requires some information on voice frame content
Telchemy – IPtel 2001

27. Further work areas
Use CODEC generated frame loss event – indicate presence of speech energy
Additional subjective testing
Wider variety of audio sources
Force listeners to focus on call content
Design impairments to isolate recency effect, burst characteristics, masking effects
Telchemy – IPtel 2001