Yi Liang Department of Electrical Engineering Stanford University April 19, 2000 Loss Recovery and Adaptive Playout Control for Packet Voice Communications over IP

Yi Liang Department of Electrical Engineering, Stanford University Outline Background and Motivation Network Delay and Jitter Characteristics Voice Concealment Using Time Scale Modification and Wave Substitution Adaptive Playout Delay Adjustment Future Work and More Loss Reconstruction/Concealment Techniques

Yi Liang Department of Electrical Engineering, Stanford University Voice Over IP Increased network bandwidth Universal presence of IP, easier and faster Internet access Availability of supporting hardware IP Gateway PSTN H.323

Yi Liang Department of Electrical Engineering, Stanford University Voice Over IP - Challenges The Internet protocol delivers service on a best-effort basis, with no guarantee of quality of service The transmission mechanism is unreliable and connectionless Transport layer – TCP guarantees reliable transmission by retransmission protocols; but not acceptable for live audio due to end-to- end latency

Yi Liang Department of Electrical Engineering, Stanford University Improving QoS – Efforts and Implementations Problems End-to-End Delay Delay Jitter Packet Loss Efforts Minimize Delay Absorb Jitter Reconstruct/ Conceal Loss Implementations Optimize Algorithms Buffer Management Many Loss Recovery Techniques

Yi Liang Department of Electrical Engineering, Stanford University Delay Jitter Local Host: cass.stanford.edu Packet Size=200 bytes Packet Interval=20 ms

Yi Liang Department of Electrical Engineering, Stanford University Jitter Absorption and Buffer Management Sender Receiver Playout time Packet sequence number kK+1K+3K+4K+2 d a : buffer absorption time or allowed max delay kK+1K+3K+4K+2 K+5 kK+1K+3K+4K+2K+5 Packet dropped

Yi Liang Department of Electrical Engineering, Stanford University Delay and Loss Rate d: delay, a non negative random variable The p.d.f. of d: f(d) The delay distribution varies as time - The need to modify playout speed d a : Absorption Time

Yi Liang Department of Electrical Engineering, Stanford University Time Scale Voice Modification Time Scale Voice Modification Conceal Lost Packet Modify Playback Speed t 2371 t 46 a) Error Concealment b) Playback Speed Modification Acceleration Slowing Down Original

Yi Liang Department of Electrical Engineering, Stanford University Waveform Similarity Overlap Add (WSOLA) Algorithm [Verhelst, Roelands]

Yi Liang Department of Electrical Engineering, Stanford University Search for Similar Waveform by Pattern Matching Search for Synchronized Overlap Segment S k Positions of Search Regions

Yi Liang Department of Electrical Engineering, Stanford University WSOLA Implementation To conceal one lost packet: Introduced delay=2 packet time [Stenger, Younes, Reng, Girod]

Yi Liang Department of Electrical Engineering, Stanford University Delay-optimized Time Scale Modification t 46 a) Conventional Method Delay = 2 packet time Packet length extension = 50% t 5 4 b) Improved Method 3 2 Delay = 1 packet time Packet length extension = 1/3 What if double-packet loss?

Yi Liang Department of Electrical Engineering, Stanford University Waveform Substitution Using Pattern Matching template sliding template L L best match L replaced packet Can only conceal a limited number of lost packet using prior information Advantage: Delay=0 search window [Goodman, Wasem]

Yi Liang Department of Electrical Engineering, Stanford University Conceal Consecutive Lost Packets Using Hybrid Method t Delay is reduced to one packet time; voice quality is good even for double-packet loss Can conceal more consecutive lost packets; but voice quality degrades as the number of consecutive lost packets goes up Waveform Substitution Time-scale Modification

Yi Liang Department of Electrical Engineering, Stanford University Adaptive Playout Delay Adjustment [Ramjee, Moon] Average Playout Delay = 5.47ms Late Loss Rate = 10.82% Average Playout Delay = 5.76ms Late Loss Rate = 5.63% Less Bursty Loss

Yi Liang Department of Electrical Engineering, Stanford University Audio Demos II. Playback Speed Modification III. Adaptive Playout Delay Adjustment Example 1: 20% Loss OriginalScaled OriginalLossPlayout I. How Well WSOLA-Based Concealment Works? Voice Stream Suffered from Simulated Random Packet Loss Concealed Voice Using Delay-optimized Time-scale Modification Introduced Delay = 1 packet time OriginalLossConcealed Example 2: 30% Loss

Yi Liang Department of Electrical Engineering, Stanford University Conclusions

Yi Liang Department of Electrical Engineering, Stanford University Future Work Architecture and Protocols Receiver- based Loss Recovery and Concealment Scalable (layered) Encoding Scheme Error Protection and Data Reconstruction Joint Source and Channel Coding Loss Avoidance at Network Layer Protect Real-time Voice from Data, Reserving the Bandwidth (Token Bus, RSVP, DiffServ) Audio Signal Processing Loss-resilient Real-time Data Transmission VoIP

Yi Liang Department of Electrical Engineering, Stanford University Combating Loss

Yi Liang Department of Electrical Engineering, Stanford University Forward Error Correction Various Encoding Schemes 2341 t Original Stream FEC Encoded Packet Loss Reconstructed Protecting from Burst Loss t

Yi Liang Department of Electrical Engineering, Stanford University References For references and more voice samples: