1. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 1 Adaptive Playout

2. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 2 You are Here Network Encoder Sender Middlebox Receiver Decoder

3. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 3 How to recv and play? open socket while not done if socket is readable read packet from socket remove RTP header decode play back

4. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 4 What’s Wrong? packet ordering packet loss next packet arrive in-time? Especially bad for audio applications

5. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 5 Overview Network RTP Classifier Decode

6. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 6 Implementation Single Thread using select() Multi-Threads

7. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 7 Packet Buffer Sorted by sequence number When ADU is complete, send to decoder RTP Classifier

8. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 8 Playout Buffer Stored decoded data in playout order Post-processing/Mixing may happen Decode

9. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 9 Why Buffer?

10. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 10 Sending Packets Time Packet

11. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 11 Receiving Packets Time Packet

12. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 12 With Jitter Time Packet

13. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 13 With Jitter Time Packet

14. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 14 What causes Jitter? Network delay = Transmission Delay (fixed) + Propagation Delay (fixed) + Queuing Delay (variable) Delay jitter is caused by variable queuing delay

15. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 15 Delay Jitter Time Transit Time small jitter large jitter

16. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 16 Spike Time Transit Time

17. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 17 Today’s Question How big is the playout buffer? When to play back?

18. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 18 Types of Applications Non-interactive Buffer can be large Interactive As small as possible

19. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 19 Types of Applications Video Frames are discrete (easier problem) Audio Samples are “continuous”

20. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 20 Naive Answer How big is a buffer? Fixed at a small value, to reduce latency When to playback? Playback as soon as possible, to reduce latency

21. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 21 Client Buffer Management Bandwidth Smoothing for Non- interactive Applications

22. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 22 Multi-Threshold Flow Control R. Zimmermann, K. Fu, M. Jahangiri, C. Shahabi MTAP 2006

23. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 23 Video-on-Demand (VoD) High-quality playback required Buffer can be large Encoding may be VBR for high visual quality Playback time may be very long (2+ hours)

24. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 24 Two Approaches (1) Server-controlled Pre-compute transmission schedule Piece-wise linear approximation: compute a number of constant-rate segments Different optimization criteria Minimize: # of rate changes, # utilization of client buffer, peak rate, etc.

25. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 25 Two Approaches (2) Client-controlled Client adaptively informs server Advantages: No rate history necessary (VBR) Can adjust to network conditions Works easily with interactive commands such as FF, FR, Pause

26. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 26 Robust Stream Delivery Smoothing of VBR media traffic has the following quality benefits: Better resource utilization (less bursty) More streams with the same network capacity

27. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 27 Objectives Multi-Threshold Flow Control (MTFC) algorithm objectives: Online operation Content independence Minimizing feedback control signaling Rate smoothing

28. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 28 Example

29. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 29 MTFC Buffer Management Multiple Thresholds: goal is middle of buffer Send rate adjust command to server whenever threshold is crossed

30. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 30 How to Set Thresholds? Simple: equi-distant m: number of thresholds Overflow and underflow thresholds

31. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 31 How to Calculate Sending Rate?

32. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 32 Threshold Spacing Strategies Linear vs. arithmetic vs. geometric

33. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 33 Threshold Spacing Strategies Geometric and arithmetic spacing:

34. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 34 Sending Rate Computation Relevant factors: Target buffer level (i.e., 50%): TH N Current buffer level: b wobsv Predicted duration to reach target level Data consumed during predicted duration

35. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 35 MTFC Results

36. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 36 MTFC Results: # of Thresholds

37. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 37 MTFC Results: # of Thresholds Number Of Rate Changes 8MB 16MB 32MB 0 10 20 30 40 50 60 70 80 90 100 35917 Number of Thresholds

38. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 38 A Brief Introduction to Audio Conferencing

39. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 39 Audio Conferencing Live, interactive application Latency is important Each packet contains 20-30ms of audio

40. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 40 Silence Suppression Silence Detection if no sound, no need to send Talk spurt consecutive audio packets (between silence) hundreds of ms

41. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 41 Recall: RTP Header marker bit: depends on payload e.g. beginning of frame

42. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 42 RTP and Talkspurt First packet of a talkspurt will have marker bit set to 1

43. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 43 RTP and Talkspurt Deduce talkspurt from sequence number and timestamp 2 40 1 20 3 60 5 190 SeqNo TimeStamp

44. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 44 Consequences of Talkspurt Opportunity to adjust playout delay if jitter is large, increase delay if jitter is small, decrease delay

45. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 45 Fixed Playout Delay SEND RECV PLAY

46. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 46 Adaptive Playout Delay SEND RECV PLAY

47. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 47 Adaptive Playout Delay SEND RECV PLAY

48. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 48 Trade-Off Latency vs. Packet Loss

49. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 49 Latency vs Loss-Rate Loss Rate Latency

50. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 50 Adaptive Playout Mechanisms for Packetized Audio Applications in WAN R. Ramjee, J. Kurose, D. Towsley, H. Schulzrinne INFOCOM 1994

51. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 51 Variables and Notations

52. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 52 Variables and Notations T send (i) T play (i) T buf (i) T arrive (i) T delay (i) T net (i)

53. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 53 1 st Packet in Talkspurt We can estimate as

54. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 54 How to estimate V net (i)

55. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 55 How to estimate E net (i) Method 1: Jacobson’s Method

56. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 56 Spike Time T net

57. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 57 Problems Does not react to spike fast enough

58. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 58 How to estimate E net (i) Ramjee’s Method SPIKENORMAL

59. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 59 Ramjee’s Idea SPIKENORMAL if T net (i) suddenly increase if “slope” is small enough

60. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 60 In Spike Mode SPIKE i T net

61. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 61 In Normal Mode NORMAL i T net

62. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 62 Evaluations Delay Loss Rate

63. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 63 Problems with Ramjee’s Method Time Transit Time

64. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 64 Packet Audio Playout Delay Adjustment: Performance Bounds and Algorithms S. Moon, J. Kurose, D. Towsley Multimedia Systems 1998

65. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 65 Recall Previous Methods

66. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 66 How to Set T delay (i) Moon’s Method Collect statistics on packets that have arrived. Find t such that q% of last w packets have T net (i) < t.

67. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 67 Example (w =50, q = 90%) num of packets delay 1 2 3 4 5 6 7 8 9 10 11 12

68. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 68 Setting T delay (i) NORMALSPIKE

69. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 69 Setting T delay (i) Time Transit Time

70. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 70 Performance Bound Given a trace of packets, and a loss rate, find the minimum average playout delay. Use Dynamic Programming

71. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 71 A Packet Trace k M talkspurts 1,k2,k3,kj,kn k,k T net 131510…

72. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 72 More Notations M: Number of Talkspurt N packet (k) or n k Number of packets in talkspurt k N total Total number of packets

73. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 73 Definition minimum average playout delay for choosing i packets to be played out from k-th talkspurt k M talkspurts

74. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 74 How to find

75. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 75 Definition minimum average playout delay for choosing i packets to be played out from k-th to M-th talkspurt k M talkspurts M..

76. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 76 Base Case D(k, 0) = D(M, i) = minimum average playout delay for choosing i packets to be played out from k-th to M-th talkspurt

77. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 77 Recursive Case k.. M 1,k2,k3,kn k,k.. j

78. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 78 Performance Bound Given a trace of M talkspurts and n packets, and a loss rate e, find the minimum average playout delay. Answer: Minimum possible average playout delay is D(1, (1-e)n) minimum average playout delay for choosing i packets to be played out from k-th to M-th talkspurt

79. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 79 Evaluations Loss Rate Delay

80. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 80 Summary Playout Adjustment for Audio Conferencing Weighted Average Methods vs. Statistical Methods An Analysis of Minimum Playout Delay

81. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 81 Practical Complications Clock Drifts Route Change

82. NUS.SOC.CS5248-2014 Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 82 Advanced Techniques Speed-up Playback