Adaptive Delay Concealment for Internet Voice Applications with Packet-Based Time-Scale Modification Fang Liu, JongWon Kim, C.-C. Jay Kuo IEEE ICASSP 2001

Outline n Introduction n Adaptive Playout Framework n Time-Scale Modification with SOLA n Adaptive Playout with Time-Scale Modification n Experimental Results n Conclusions

Introduction n Research on Internet audio streaming has focused on error control and delay concealment in the presence of delay jitter and packet loss. n A delay spike happens: –when several consecutive packets arrive at the receiver almost simultaneously. n Packet droppings by network delay jitter results in degradation of speech playout.

Introduction n Recover from network packet loss –redundant FEC ( Forward Error Correction ) –time-domain stretching n Reduce the jitter/loss effect –adaptively adjust the silence length between talkspurts n Could the adaptive playout work? –Stable –the silence detection is effective.

Introduction n If a delay spike happens in the middle of a talk spurt? n Extend the silence interval-based adaptive playout by exploiting the time-scale modification scheme. –Every packet could contributed in adapting to the network delay jitter/spike. n SOLA scheme is adopted. ( synchronized overlap-and-add)

Adaptive Payout Framework The short-time energy En(i) the zero crossing rate Zn(i)

Adaptive Payout Framework n A silence segment –En(i) < /8 and Zn(i) < 30 n A transient segment –En(i) / En(i-1) > 1.6 and En(i) > * 2 –En(i) / En(i-1) * 2 n A general segment –otherwise = γ + (1- γ )En(i)

Timings

Delay Correlation

Time-Scale Modification (SOLA) n SAMD ( short-time average magnitude difference )

Adaptive Playout with Time-scale modification n Goal : –to detect a delay spike as soon as possible when it happens n the first packet of each talk-spurt n all packets within a talk-spurt

Adaptive Playout with Time-scale modification

n For each packet received, we have a i = t i + D prop + v i and calculate ^b i and ^vb i n Calculate q i for the first packet and all other packets of each talk-spurt. n Update q i. In this case, it is assumed that the sender-generated content category, is transmitted to the receiver in-band. Calculate the playout length l i (^P) = q i+1 - q i and the target stretching factor α= l i (^P) / l i (O).

Adaptive Playout with Time-scale modification n At scheduling time q i, if a i > q i, packet i will be dropped. Proceed to packet i+1 and restart the algorithm with i = i+1 n Based on the final playout length, update the playout time. p i = q i + D calc Decode and perform time-scale modification based on α n Proceed to packet i+1 with the same algorithm

Experimental Results n Sampling rate : 8 kHz n 16 bits/sample n inter-packetization interval l i (O) : 20ms n 2000 packets in total (40 seconds of speech)

Experimental Results

Conclusions n Time-domain stretching introduce audio artifacts that is within an acceptable range of quality. n By using content-adaptive stretching, we successfully preserve the pitch and the continuity of the original speech. n 1.5~8% improvement over the reference algorithm.