1 NUS.SOC.CS5248 OOI WEI TSANG Playout Buffer Management

NUS.SOC.CS5248 OOI WEI TSANG 2 You are Here Network Encoder Sender Middlebox Receiver Decoder

NUS.SOC.CS5248 OOI WEI TSANG 3 How to recv and play? open socket while not done if socket is readable read packet from socket remove RTP header decode play back

NUS.SOC.CS5248 OOI WEI TSANG 4 What’s Wrong? packet ordering packet loss next packet arrive in-time? Especially bad for audio applications

5 NUS.SOC.CS5248 OOI WEI TSANG Detour: A Brief Intro to Audio Conferencing

NUS.SOC.CS5248 OOI WEI TSANG 6 Audio Compression Normally uncompressed Telephone quality: 8-bit audio 8Khz 20-30ms per packet

NUS.SOC.CS5248 OOI WEI TSANG 7 Common Technique Silence Detection No need to send if there is no sound at the input Talkspurt Sequence of consecutive audio packets (in between silence)

NUS.SOC.CS5248 OOI WEI TSANG 8 Detecting Talkspurt in RTP Marker bit Not reliable as packet with marker bit could be lost Deduce from timestamp and sequence number

9 NUS.SOC.CS5248 OOI WEI TSANG Delay Jitter

NUS.SOC.CS5248 OOI WEI TSANG 10 What causes Jitter? Network delay = Propagation Delay (fixed) + Queueing Delay (variable) Delay jitter is caused by variable queueing delay

NUS.SOC.CS5248 OOI WEI TSANG 11 Delay Jitter Time Transit Time small jitter large jitter

NUS.SOC.CS5248 OOI WEI TSANG 12 Spike Time Transit Time

13 NUS.SOC.CS5248 OOI WEI TSANG Buffer: The Jitter Absorber

NUS.SOC.CS5248 OOI WEI TSANG 14 If Zero Jitter Time

NUS.SOC.CS5248 OOI WEI TSANG 15 With Jitter Time LOSS

NUS.SOC.CS5248 OOI WEI TSANG 16 With Jitter + Buffer Time Buffer Size Playout Delay

17 NUS.SOC.CS5248 OOI WEI TSANG Q: How to set playout delay?

NUS.SOC.CS5248 OOI WEI TSANG 18 Adapting Playout Delay When jitter is low, reduce delay When jitter is high, increase delay

NUS.SOC.CS5248 OOI WEI TSANG 19 Fundamental Trade-off Latency vs Packet Loss

NUS.SOC.CS5248 OOI WEI TSANG 20 Playout Delay Once you set the playout delay, cannot change! NOT true: can change at beginning of talkspurt

NUS.SOC.CS5248 OOI WEI TSANG 21 Adapting Playout Delay SEND RECV PLAY

22 NUS.SOC.CS5248 OOI WEI TSANG Adaptation Algorithm

NUS.SOC.CS5248 OOI WEI TSANG 23 Variables and Notations T send (i) T play (i) T buffer (i) T arrive (i) T delay (i) T net (i)

NUS.SOC.CS5248 OOI WEI TSANG 24 First Packet in the Talkspurt T delay (i) = E net (i) + 4 V net (i) T play (i) = T send (i) + T delay (i) T send (i) T play (i) T buffer (i) T arrive (i) T delay (i) T net (i)

NUS.SOC.CS5248 OOI WEI TSANG 25 Subsequent Packets T play (j) = T play (i) + T send (j) – T send (i) T send (i) T play (i) T buffer (i) T arrive (i) T delay (i) T net (i)

26 NUS.SOC.CS5248 OOI WEI TSANG How to estimate E net (i)

NUS.SOC.CS5248 OOI WEI TSANG 27 Algorithm 1 (Jacobson’s) E net (i) = aE net (i-1) + (1-a)T net (i)

NUS.SOC.CS5248 OOI WEI TSANG 28 Algorithm 2 if T net (i) > E net (i) E net (i) = bE net (i-1) + (1-b)T net (i) else E net (i) = aE net (i-1) + (1-a)T net (i)

NUS.SOC.CS5248 OOI WEI TSANG 29 Algorithm 3 E net (i) = min { T net (j) } (over all packets in previous talkspurt)

NUS.SOC.CS5248 OOI WEI TSANG 30 Algorithm 4 (Ramjee’s) Ramjee’s Proposal Observation: Algorithm 1-3 take too long to react to spike.

NUS.SOC.CS5248 OOI WEI TSANG 31 Spike

NUS.SOC.CS5248 OOI WEI TSANG 32 Ramjee’s Idea Works in 2 modes SPIKENORMAL

NUS.SOC.CS5248 OOI WEI TSANG 33 Three Questions When to switch from normal to spike mode? When to switch from spike back to normal? How to estimate during spike mode?

NUS.SOC.CS5248 OOI WEI TSANG 34 Normal to Spike if difference in delays is large

NUS.SOC.CS5248 OOI WEI TSANG 35 Normal to Spike if difference in delays is large difference in delays: T net (i) – T net (i-1) large: V net (i)

NUS.SOC.CS5248 OOI WEI TSANG 36 Another View of Spike (ZOOM) T net (i) T arrive (i)

NUS.SOC.CS5248 OOI WEI TSANG 37 Spike to Normal slope = slope/2 + |2T net (i) – T net (i-1) – T net (i-2)|/8 if slope < 64 switch to normal

NUS.SOC.CS5248 OOI WEI TSANG 38 Estimation in Spike Mode E net (i) = E net (i-1) + T net (i) – T net (i-1)

NUS.SOC.CS5248 OOI WEI TSANG 39 First Packet in the Talkspurt T delay (i) = E net (i) + 4 V net (i) T play (i) = T send (i) + T delay (i)

NUS.SOC.CS5248 OOI WEI TSANG 40 First Packet in the Talkspurt T delay (i) = E net (i) + 4 V net (i) T play (i) = T send (i) + T delay (i) T send (i) T prop (i) T play (i) T buffer (i) T arrive (i) T q (i) T delay (i) T net (i)

41 NUS.SOC.CS5248 OOI WEI TSANG How to estimate V net (i)

NUS.SOC.CS5248 OOI WEI TSANG 42 Variation of Delay V net (i) = aV net (i-1) + (1-a)|E net (i) – T net (i)|

43 NUS.SOC.CS5248 OOI WEI TSANG Comparisons of 4 Algorithms

NUS.SOC.CS5248 OOI WEI TSANG 44 Playout Delay vs Loss Rate T delay (i) Loss Rate

45 NUS.SOC.CS5248 OOI WEI TSANG Algorithm 5 (Moon’s)

NUS.SOC.CS5248 OOI WEI TSANG 46 Problems of Existing Algorithms Jacobson’s react too slowly Ramjee’s follow the delay too closely

NUS.SOC.CS5248 OOI WEI TSANG 47 Moon’s Idea Collect statistics on packets that have arrived Find t such that q% of last w packets has T net (i) < t T delay (i) = t

NUS.SOC.CS5248 OOI WEI TSANG 48 Example (w =50, q = 90%) num of packets delay

NUS.SOC.CS5248 OOI WEI TSANG 49 Spike Mode T delay (i) = E net (1) SPIKENORMAL

NUS.SOC.CS5248 OOI WEI TSANG 50 Spike Mode

NUS.SOC.CS5248 OOI WEI TSANG 51 Switching Mode Normal to Spike if (T net (i) > k*T delay (i)) Spike to Normal if (T net (i) < k’ * OLDT delay (i))

NUS.SOC.CS5248 OOI WEI TSANG 52 Loss Rate vs. Delay T delay (i) Loss Rate

53 NUS.SOC.CS5248 OOI WEI TSANG Theoretical Lower Bound

NUS.SOC.CS1102 OOI WEI TSANG 54 Dynamic Programming

NUS.SOC.CS1102 OOI WEI TSANG 55 Fibonacci Numbers 01… 012 … i-2i-1i +

NUS.SOC.CS1102 OOI WEI TSANG 56 Fibonacci Numbers fib(n) x[0] = 0 x[1] = 1 for (i = 2; i < n; i++) x[i] = x[i-1] + x[i-2] return x[i]

NUS.SOC.CS1102 OOI WEI TSANG 57 Binomial Coefficient k n

NUS.SOC.CS1102 OOI WEI TSANG 58 Change-Making Problem  [Weiss] 7.6 For a currency with coin C1, C2,.. Cn (cents), what is the min number of coins needed to make K cents of change?

NUS.SOC.CS1102 OOI WEI TSANG 59 Example  C = {1, 5, 10, 20, 50}  K = 76 cents  Give 4 coins = 76

NUS.SOC.CS1102 OOI WEI TSANG 60 Formulation  To make a change of K cents, either make a change of (K-50) cents, or make a change of (K-20) cents, or make a change of (K-10) cents, or make a change of (K-5) cents, or make a change of (K-1) cents  Number of coins for K = 1 + minimum of all the above choices

NUS.SOC.CS1102 OOI WEI TSANG 61 Dynamic Programming …… min+1 K K-5K-10K-20

62 NUS.SOC.CS5248 OOI WEI TSANG Theoretical Lower Bound

NUS.SOC.CS5248 OOI WEI TSANG 63 Goal Input: A packet trace A number of packet losses What is the minimum average playout delay T delay (.) ?

NUS.SOC.CS5248 OOI WEI TSANG 64 Trace k M talkspurts 1,k2,k3,kj,kn k,k

NUS.SOC.CS5248 OOI WEI TSANG 65 New Notations M: Number of Talkspurt N packet (k) or n k Number of packets in talkspurt k N total Total number of packets

NUS.SOC.CS5248 OOI WEI TSANG 66 Define.. d(k, i) minimum average playout delay for choosing i packets to be played out from k-th talkspurt k.. M 1,k2,k3,kj,kn k,k

NUS.SOC.CS5248 OOI WEI TSANG 67 Define.. D(k, i) minimum average playout delay for choosing i packets to be played out from k-th to M-th talkspurt k.. M

NUS.SOC.CS5248 OOI WEI TSANG 68 Dynamic Programming M 012N

NUS.SOC.CS5248 OOI WEI TSANG 69 Formulation for Base Case if i = 0 then D(k,i) = 0 if k = M if i ≤ N packet (M) then D(k,i) = d(k,i) else D(k,i) = ∞

NUS.SOC.CS5248 OOI WEI TSANG 70 Formulation for Recursive Case k.. M

NUS.SOC.CS5248 OOI WEI TSANG 71 Loss Rate vs. Delay T delay (i) Loss Rate 1%

72 NUS.SOC.CS5248 OOI WEI TSANG Video Playout

NUS.SOC.CS5248 OOI WEI TSANG 73 Two Methods I-Policy Display at fixed playout delay Drop late frames E-Policy Display late frame at next period Playout delay increase

NUS.SOC.CS5248 OOI WEI TSANG 74 I-Policy a b c d e f g h ace fh

NUS.SOC.CS5248 OOI WEI TSANG 75 E-Policy a b c d e f g h abdfgceh

NUS.SOC.CS5248 OOI WEI TSANG 76 I-Policy a b c d e f g h abefhg

NUS.SOC.CS5248 OOI WEI TSANG 77 E-Policy a b c d e f g h abcdfe

NUS.SOC.CS5248 OOI WEI TSANG 78 Pro and Cons I-Policy: Fixed latency E-Policy: No Loss Frame or Gap

NUS.SOC.CS5248 OOI WEI TSANG 79 Idea if queue length > L for T seconds drop incoming frames long queue -> small T short queue -> large T

NUS.SOC.CS5248 OOI WEI TSANG 80 Queue Monitoring queue length threshold waiting time

NUS.SOC.CS5248 OOI WEI TSANG 81 Queue Monitoring queue length threshold waiting time

NUS.SOC.CS5248 OOI WEI TSANG 82 Queue Monitoring queue length threshold waiting time

NUS.SOC.CS5248 OOI WEI TSANG 83 Queue Monitoring queue length threshold waiting time

NUS.SOC.CS5248 OOI WEI TSANG 84 Queue Monitoring queue length threshold waiting time

NUS.SOC.CS5248 OOI WEI TSANG 85 Summary Preliminary work by Stone and Jeffay on Video Playout Ramjee’s and Moon’s adaptive audio playout algorithm

86 NUS.SOC.CS5248 OOI WEI TSANG Possible Survey and Project Topic

NUS.SOC.CS5248 OOI WEI TSANG 87 Adapting Playback Besides adapting buffering time, we can adapt playback time Estimating and eliminating clock drift