Comparisons of FEC and Codec Robustness on VoIP Quality and Bandwidth Efficiency Wenyu Jiang Henning Schulzrinne Columbia University ICN 2002, Atlanta, GA Aug 29, 2002

Introduction to VoIP The Internet is still best-effort Subject to packet loss and delay jitter Options for repairing packet loss Forward error correction (FEC) Low complexity; bit-exact recovery Packet loss concealment (PLC) Receiver-only; no extra BW overhead More robust (error resilient) codec  better PLC quality, and higher bit-rate Question: use FEC or a more robust codec?

Metric of VoIP Quality Mean Opinion Score (MOS) [ITU P.830] Obtained via human-based listening tests Listening (MOS) vs. conversational (MOSc) Grade Quality 5 Excellent 4 Good 3 Fair 2 Poor 1 Bad

FEC and IP Header Overhead An (n,k) FEC code has (n-k)/k overhead Typical IP/UDP/RTP header is 40 bytes codec media pkt size (T=30ms) rmedia rIP iLBC (4,2) FEC 54 bytes 14.4 kb/s 25.1 kb/s 108 bytes 28.8 kb/s 39.5 kb/s G.729 30 bytes 8 kb/s 18.7 kb/s 60 bytes 16 kb/s 26.7 kb/s G.723.1 24 bytes 6.4 kb/s 17.1 kb/s 48 bytes 12.8 kb/s 23.5 kb/s

Predicting MOS in VoIP The E-model: an alternative to human-based MOS estimation Do need a first-time calibration from an existing human MOS-loss curve In VoIP, the E-model simplifies to two main factors: loss (Ie) and delay (Id) A gross score R is computed and translated to MOS. Loss-to-Ie mapping is codec-dependent and calibrated

Predicting MOS in VoIP, contd Example mappings From loss and delay to their impairment scores and to MOS

Predicting MOS under FEC Compute final loss probability pf after FEC [Frossard 2001] Bursty loss reduces FEC performance Increasing the packet interval T makes FEC more efficient under bursty loss [Jiang 2002] Plug pf into the calibrated loss-to-Ie mapping FEC delay is n*T for an (n,k) code Compute R value and translate to MOS

Quality Evaluation of FEC vs. Codec Robustness Codecs under evaluation iLBC: a recent loss-robust codec proposed at IETF; frame-independent coding G.729: a near toll quality ITU codec G.723.1: an ITU codec with even lower bit-rate, but also slightly lower quality. Utilize MOS curves from IETF presentations for FEC MOS estimation Assume some loss burstiness (conditional loss probability of 30%) Default packet interval T = 30ms

G.729+(5,3) FEC vs. iLBC Ignoring delay effect, a larger T improves FEC efficiency and its quality When considering delay, however, using a 60ms interval is overkill, due to higher FEC delay (5*60 = 300ms)

G.729+(5,2) vs. iLBC+(3,2) When iLBC also uses FEC, and still keeping similar gross bit-rate G.729 still prevails, except for low loss conditions when considering delay

G.729+(7,2) vs. iLBC+(4,2) Too much FEC redundancy (e.g., for G.729)  very long FEC block and delay  not always a good idea iLBC wins in this case, when considering delay

G.729+(3,1) vs. iLBC+(4,2) Using less FEC redundancy may actually help, if the FEC block is shorter Now G.729 performs similar to iLBC

Comparison with G.723.1 MOS(G.723.1) < MOS(iLBC) at zero loss  iLBC dominates more low loss areas compared with G.729, whether delay is considered or not

G.723.1+(3,1) vs. iLBC+(3,2) iLBC is still better for low loss G.723.1 wins for higher loss

G.723.1+(4,1) vs. iLBC+(4,2) iLBC dominates in this case whether delay is considered or not, (4,2) code already suffices for iLBC (4,1) code’s performance essentially “saturates”

The Best of Both Worlds Observations, when considering delay: iLBC is usually preferred in low loss conditions G.729 or G.723.1 + FEC better for high loss Example: max bandwidth 14 kb/s Consider delay impairment (use MOSc)

Max Bandwidth: 21-28 kb/s

Effect of Max Bandwidth on Achievable Quality 14 to 21 kb/s: significant improvement in MOSc From 21 to 28 kb/s: marginal change due to increasing delay impairment by FEC

Conclusions Compared listening and conversational MOS achieved by conventional vs. robust codecs, with same BW constraint iLBC is better under low loss conditions Conventional codec + FEC is better under high loss, but Usefulness of FEC redundancy saturates beyond a certain point considering delay At roughly a max BW of 21 kb/s Reveals max achievable quality with current FEC mechanism

Future Work Implement the MOS prediction and optimization procedure in software Investigate the effect of jitter on conventional vs. robust codecs FEC cannot reduce jitter unless there are many out-of-order packets PLC in a robust codec like iLBC incurs a much lower delay, thus may be preferable to FEC

References W. Jiang and H. Schulzrinne, Comparison and optimization of packet loss repair methods on VoIP perceived quality under bursty loss, NOSSDAV 2002 P. Frossard, FEC performance in multimedia streaming, IEEE Comm Letter 3/2001 ITU-T, Subjective performance assessment of telephone-band and wideband digital codecs, Recommendation P.830 2/1996