1. 1 NUS.SOC.CS5248 OOI WEI TSANG Playout Buffer Management

2. NUS.SOC.CS5248 OOI WEI TSANG 2 You are Here Network Encoder Sender Middlebox Receiver Decoder

3. NUS.SOC.CS5248 OOI WEI TSANG 3 How to recv and play? open socket while not done if socket is readable read packet from socket remove RTP header decode play back

4. NUS.SOC.CS5248 OOI WEI TSANG 4 What’s Wrong? packet ordering packet loss next packet arrive in-time? Especially bad for audio applications

5. 5 NUS.SOC.CS5248 OOI WEI TSANG Detour: A Brief Intro to Audio Conferencing

6. NUS.SOC.CS5248 OOI WEI TSANG 6 Audio Compression Normally uncompressed Telephone quality: 8-bit audio 8Khz 20-30ms per packet

7. NUS.SOC.CS5248 OOI WEI TSANG 7 Common Technique Silence Detection No need to send if there is no sound at the input Talkspurt Sequence of consecutive audio packets (in between silence)

8. NUS.SOC.CS5248 OOI WEI TSANG 8 Detecting Talkspurt in RTP Marker bit Not reliable as packet with marker bit could be lost Deduce from timestamp and sequence number

9. 9 NUS.SOC.CS5248 OOI WEI TSANG Delay Jitter

10. NUS.SOC.CS5248 OOI WEI TSANG 10 What causes Jitter? Network delay = Propagation Delay (fixed) + Queueing Delay (variable) Delay jitter is caused by variable queueing delay

11. NUS.SOC.CS5248 OOI WEI TSANG 11 Delay Jitter Time Transit Time small jitter large jitter

12. NUS.SOC.CS5248 OOI WEI TSANG 12 Spike Time Transit Time

13. 13 NUS.SOC.CS5248 OOI WEI TSANG Buffer: The Jitter Absorber

14. NUS.SOC.CS5248 OOI WEI TSANG 14 If Zero Jitter Time

15. NUS.SOC.CS5248 OOI WEI TSANG 15 With Jitter Time LOSS

16. NUS.SOC.CS5248 OOI WEI TSANG 16 With Jitter + Buffer Time Buffer Size Playout Delay

17. 17 NUS.SOC.CS5248 OOI WEI TSANG Q: How to set playout delay?

18. NUS.SOC.CS5248 OOI WEI TSANG 18 Adapting Playout Delay When jitter is low, reduce delay When jitter is high, increase delay

19. NUS.SOC.CS5248 OOI WEI TSANG 19 Fundamental Trade-off Latency vs Packet Loss

20. NUS.SOC.CS5248 OOI WEI TSANG 20 Playout Delay Once you set the playout delay, cannot change! NOT true: can change at beginning of talkspurt

21. NUS.SOC.CS5248 OOI WEI TSANG 21 Adapting Playout Delay SEND RECV PLAY

22. 22 NUS.SOC.CS5248 OOI WEI TSANG Adaptation Algorithm

23. NUS.SOC.CS5248 OOI WEI TSANG 23 Variables and Notations T send (i) T play (i) T buffer (i) T arrive (i) T delay (i) T net (i)

24. NUS.SOC.CS5248 OOI WEI TSANG 24 First Packet in the Talkspurt T delay (i) = E net (i) + 4 V net (i) T play (i) = T send (i) + T delay (i) T send (i) T play (i) T buffer (i) T arrive (i) T delay (i) T net (i)

25. NUS.SOC.CS5248 OOI WEI TSANG 25 Subsequent Packets T play (j) = T play (i) + T send (j) – T send (i) T send (i) T play (i) T buffer (i) T arrive (i) T delay (i) T net (i)

26. 26 NUS.SOC.CS5248 OOI WEI TSANG How to estimate E net (i)

27. NUS.SOC.CS5248 OOI WEI TSANG 27 Algorithm 1 (Jacobson’s) E net (i) = aE net (i-1) + (1-a)T net (i)

28. NUS.SOC.CS5248 OOI WEI TSANG 28 Algorithm 2 if T net (i) > E net (i) E net (i) = bE net (i-1) + (1-b)T net (i) else E net (i) = aE net (i-1) + (1-a)T net (i)

29. NUS.SOC.CS5248 OOI WEI TSANG 29 Algorithm 3 E net (i) = min { T net (j) } (over all packets in previous talkspurt)

30. NUS.SOC.CS5248 OOI WEI TSANG 30 Algorithm 4 (Ramjee’s) Ramjee’s Proposal Observation: Algorithm 1-3 take too long to react to spike.

31. NUS.SOC.CS5248 OOI WEI TSANG 31 Spike

32. NUS.SOC.CS5248 OOI WEI TSANG 32 Ramjee’s Idea Works in 2 modes SPIKENORMAL

33. NUS.SOC.CS5248 OOI WEI TSANG 33 Three Questions When to switch from normal to spike mode? When to switch from spike back to normal? How to estimate during spike mode?

34. NUS.SOC.CS5248 OOI WEI TSANG 34 Normal to Spike if difference in delays is large

35. NUS.SOC.CS5248 OOI WEI TSANG 35 Normal to Spike if difference in delays is large difference in delays: T net (i) – T net (i-1) large: 800 + 2V net (i)

36. NUS.SOC.CS5248 OOI WEI TSANG 36 Another View of Spike (ZOOM) T net (i) T arrive (i)

37. NUS.SOC.CS5248 OOI WEI TSANG 37 Spike to Normal slope = slope/2 + |2T net (i) – T net (i-1) – T net (i-2)|/8 if slope < 64 switch to normal

38. NUS.SOC.CS5248 OOI WEI TSANG 38 Estimation in Spike Mode E net (i) = E net (i-1) + T net (i) – T net (i-1)

39. NUS.SOC.CS5248 OOI WEI TSANG 39 First Packet in the Talkspurt T delay (i) = E net (i) + 4 V net (i) T play (i) = T send (i) + T delay (i)

40. NUS.SOC.CS5248 OOI WEI TSANG 40 First Packet in the Talkspurt T delay (i) = E net (i) + 4 V net (i) T play (i) = T send (i) + T delay (i) T send (i) T prop (i) T play (i) T buffer (i) T arrive (i) T q (i) T delay (i) T net (i)

41. 41 NUS.SOC.CS5248 OOI WEI TSANG How to estimate V net (i)

42. NUS.SOC.CS5248 OOI WEI TSANG 42 Variation of Delay V net (i) = aV net (i-1) + (1-a)|E net (i) – T net (i)|

43. 43 NUS.SOC.CS5248 OOI WEI TSANG Comparisons of 4 Algorithms

44. NUS.SOC.CS5248 OOI WEI TSANG 44 Playout Delay vs Loss Rate T delay (i) Loss Rate

45. 45 NUS.SOC.CS5248 OOI WEI TSANG Algorithm 5 (Moon’s)

46. NUS.SOC.CS5248 OOI WEI TSANG 46 Problems of Existing Algorithms Jacobson’s react too slowly Ramjee’s follow the delay too closely

47. NUS.SOC.CS5248 OOI WEI TSANG 47 Moon’s Idea Collect statistics on packets that have arrived Find t such that q% of last w packets has T net (i) < t T delay (i) = t

48. NUS.SOC.CS5248 OOI WEI TSANG 48 Example (w =50, q = 90%) num of packets delay 1 2 3 4 5 6 7 8 9 10 11 12

49. NUS.SOC.CS5248 OOI WEI TSANG 49 Spike Mode T delay (i) = E net (1) SPIKENORMAL

50. NUS.SOC.CS5248 OOI WEI TSANG 50 Spike Mode

51. NUS.SOC.CS5248 OOI WEI TSANG 51 Switching Mode Normal to Spike if (T net (i) > k*T delay (i)) Spike to Normal if (T net (i) < k’ * OLDT delay (i))

52. NUS.SOC.CS5248 OOI WEI TSANG 52 Loss Rate vs. Delay T delay (i) Loss Rate

53. 53 NUS.SOC.CS5248 OOI WEI TSANG Theoretical Lower Bound

54. NUS.SOC.CS1102 OOI WEI TSANG 54 Dynamic Programming

55. NUS.SOC.CS1102 OOI WEI TSANG 55 Fibonacci Numbers 01… 012 … i-2i-1i +

56. NUS.SOC.CS1102 OOI WEI TSANG 56 Fibonacci Numbers fib(n) x[0] = 0 x[1] = 1 for (i = 2; i < n; i++) x[i] = x[i-1] + x[i-2] return x[i]

57. NUS.SOC.CS1102 OOI WEI TSANG 57 Binomial Coefficient 1 11 11 11 1 1 k n

58. NUS.SOC.CS1102 OOI WEI TSANG 58 Change-Making Problem  [Weiss] 7.6 For a currency with coin C1, C2,.. Cn (cents), what is the min number of coins needed to make K cents of change?

59. NUS.SOC.CS1102 OOI WEI TSANG 59 Example  C = {1, 5, 10, 20, 50}  K = 76 cents  Give 4 coins 50 + 20 + 5 + 1 = 76

60. NUS.SOC.CS1102 OOI WEI TSANG 60 Formulation  To make a change of K cents, either make a change of (K-50) cents, or make a change of (K-20) cents, or make a change of (K-10) cents, or make a change of (K-5) cents, or make a change of (K-1) cents  Number of coins for K = 1 + minimum of all the above choices

61. NUS.SOC.CS1102 OOI WEI TSANG 61 Dynamic Programming …… min+1 K K-5K-10K-20

62. 62 NUS.SOC.CS5248 OOI WEI TSANG Theoretical Lower Bound

63. NUS.SOC.CS5248 OOI WEI TSANG 63 Goal Input: A packet trace A number of packet losses What is the minimum average playout delay T delay (.) ?

64. NUS.SOC.CS5248 OOI WEI TSANG 64 Trace k M talkspurts 1,k2,k3,kj,kn k,k

65. NUS.SOC.CS5248 OOI WEI TSANG 65 New Notations M: Number of Talkspurt N packet (k) or n k Number of packets in talkspurt k N total Total number of packets

66. NUS.SOC.CS5248 OOI WEI TSANG 66 Define.. d(k, i) minimum average playout delay for choosing i packets to be played out from k-th talkspurt k.. M 1,k2,k3,kj,kn k,k

67. NUS.SOC.CS5248 OOI WEI TSANG 67 Define.. D(k, i) minimum average playout delay for choosing i packets to be played out from k-th to M-th talkspurt k.. M

68. NUS.SOC.CS5248 OOI WEI TSANG 68 Dynamic Programming 1 2 3 M 012N

69. NUS.SOC.CS5248 OOI WEI TSANG 69 Formulation for Base Case if i = 0 then D(k,i) = 0 if k = M if i ≤ N packet (M) then D(k,i) = d(k,i) else D(k,i) = ∞

70. NUS.SOC.CS5248 OOI WEI TSANG 70 Formulation for Recursive Case k.. M

71. NUS.SOC.CS5248 OOI WEI TSANG 71 Loss Rate vs. Delay T delay (i) Loss Rate 1%

72. 72 NUS.SOC.CS5248 OOI WEI TSANG Video Playout

73. NUS.SOC.CS5248 OOI WEI TSANG 73 Two Methods I-Policy Display at fixed playout delay Drop late frames E-Policy Display late frame at next period Playout delay increase

74. NUS.SOC.CS5248 OOI WEI TSANG 74 I-Policy a b c d e f g h 1 2 3 4 5 6 7 8 9 10 ace fh

75. NUS.SOC.CS5248 OOI WEI TSANG 75 E-Policy a b c d e f g h 1 2 3 4 5 6 7 8 9 10 11 abdfgceh

76. NUS.SOC.CS5248 OOI WEI TSANG 76 I-Policy a b c d e f g h 1 2 3 4 5 6 7 8 9 10 abefhg

77. NUS.SOC.CS5248 OOI WEI TSANG 77 E-Policy a b c d e f g h 1 2 3 4 5 6 7 8 9 10 abcdfe

78. NUS.SOC.CS5248 OOI WEI TSANG 78 Pro and Cons I-Policy: Fixed latency E-Policy: No Loss Frame or Gap

79. NUS.SOC.CS5248 OOI WEI TSANG 79 Idea if queue length > L for T seconds drop incoming frames long queue -> small T short queue -> large T

80. NUS.SOC.CS5248 OOI WEI TSANG 80 Queue Monitoring 3 2 3 3 3 4 2 5 2 6 2 7 2 8 00 0 0 00 0 queue length threshold waiting time

81. NUS.SOC.CS5248 OOI WEI TSANG 81 Queue Monitoring 3 2 3 3 3 4 2 5 2 6 2 7 2 8 00 0 0 00 0 queue length threshold waiting time

82. NUS.SOC.CS5248 OOI WEI TSANG 82 Queue Monitoring 3 2 3 3 3 4 2 5 2 6 2 7 2 8 10 0 0 00 0 queue length threshold waiting time

83. NUS.SOC.CS5248 OOI WEI TSANG 83 Queue Monitoring 3 2 3 3 3 4 2 5 2 6 2 7 2 8 21 0 0 00 0 queue length threshold waiting time

84. NUS.SOC.CS5248 OOI WEI TSANG 84 Queue Monitoring 3 2 3 3 3 4 2 5 2 6 2 7 2 8 32 1 1 00 0 queue length threshold waiting time

85. NUS.SOC.CS5248 OOI WEI TSANG 85 Summary Preliminary work by Stone and Jeffay on Video Playout Ramjee’s and Moon’s adaptive audio playout algorithm

86. 86 NUS.SOC.CS5248 OOI WEI TSANG Possible Survey and Project Topic

87. NUS.SOC.CS5248 OOI WEI TSANG 87 Adapting Playback Besides adapting buffering time, we can adapt playback time Estimating and eliminating clock drift