1. NUS.SOC.CS5248 Ooi Wei Tsang 1 Adaptive Playout

2. NUS.SOC.CS5248 Ooi Wei Tsang 2 You are Here Network Encoder Sender Middlebox Receiver Decoder

3. NUS.SOC.CS5248 Ooi Wei Tsang 3 How to recv and play? open socket while not done if socket is readable read packet from socket remove RTP header decode play back

4. NUS.SOC.CS5248 Ooi Wei Tsang 4 What’s Wrong? packet ordering packet loss next packet arrive in-time? Especially bad for audio applications

5. NUS.SOC.CS5248 Ooi Wei Tsang 5 Overview Network RTP Classifier Decode

6. NUS.SOC.CS5248 Ooi Wei Tsang 6 Implementation Single Thread using select() Multi-Threads

7. NUS.SOC.CS5248 Ooi Wei Tsang 7 Packet Buffer Sorted by sequence number When ADU is complete, send to decoder RTP Classifier

8. NUS.SOC.CS5248 Ooi Wei Tsang 8 Playout Buffer Stored decoded data in playout order Post-processing/Mixing may happens Decode

9. NUS.SOC.CS5248 Ooi Wei Tsang 9 Why Buffer?

10. NUS.SOC.CS5248 Ooi Wei Tsang 10 Sending Packets Time Packet

11. NUS.SOC.CS5248 Ooi Wei Tsang 11 Receiving Packets Time Packet

12. NUS.SOC.CS5248 Ooi Wei Tsang 12 With Jitter Time Packet

13. NUS.SOC.CS5248 Ooi Wei Tsang 13 With Jitter Time Packet

14. NUS.SOC.CS5248 Ooi Wei Tsang 14 What causes Jitter? Network delay = Transmission Delay (fixed) + Propagation Delay (fixed) + Queuing Delay (variable) Delay jitter is caused by variable queuing delay

15. NUS.SOC.CS5248 Ooi Wei Tsang 15 Delay Jitter Time Transit Time small jitter large jitter

16. NUS.SOC.CS5248 Ooi Wei Tsang 16 Spike Time Transit Time

17. NUS.SOC.CS5248 Ooi Wei Tsang 17 Today’s Question How big is the playout buffer? When to playback?

18. NUS.SOC.CS5248 Ooi Wei Tsang 18 Types of Applications Non-interactive Buffer can be large Interactive As small as possible

19. NUS.SOC.CS5248 Ooi Wei Tsang 19 Types of Applications Video Frames are discrete (easier problem) Audio Samples are “continuous”

20. NUS.SOC.CS5248 Ooi Wei Tsang 20 Naive Answer How big is a buffer? Fixed at a small value, to reduce latency When to playback? Playback as soon as possible, to reduce latency

21. NUS.SOC.CS5248 Ooi Wei Tsang 21 A Brief Introduction to Audio Conferencing

22. NUS.SOC.CS5248 Ooi Wei Tsang 22 Audio Conferencing Live, interactive application Latency is important Each packet 20-30ms of audio

23. NUS.SOC.CS5248 Ooi Wei Tsang 23 Silence Suppression Silence Detection if no sound, no need to send Talk spurt consecutive audio packets (between silence) hundreds of ms

24. NUS.SOC.CS5248 Ooi Wei Tsang 24 Demo

25. NUS.SOC.CS5248 Ooi Wei Tsang 25 Recall: RTP Header marker bit: depends on payload e.g. beginning of frame

26. NUS.SOC.CS5248 Ooi Wei Tsang 26 RTP and Talkspurt First packet of a talkspurt will have marker bit set to 1

27. NUS.SOC.CS5248 Ooi Wei Tsang 27 RTP and Talkspurt Deduce talkspurt from sequence number and timestamp 2 40 1 20 3 60 5 190 SeqNo TimeStamp

28. NUS.SOC.CS5248 Ooi Wei Tsang 28 Consequences of Talkspurt Opportunity to adjust playout delay if jitter is large, increase delay if jitter is small, decrease delay

29. NUS.SOC.CS5248 Ooi Wei Tsang 29 Fixed Playout Delay SEND RECV PLAY

30. NUS.SOC.CS5248 Ooi Wei Tsang 30 Adaptive Playout Delay SEND RECV PLAY

31. NUS.SOC.CS5248 Ooi Wei Tsang 31 Adaptive Playout Delay SEND RECV PLAY

32. NUS.SOC.CS5248 Ooi Wei Tsang 32 Trade-Off Latency vs. Packet Loss

33. NUS.SOC.CS5248 Ooi Wei Tsang 33 Latency vs Loss-Rate Loss Rate Latency

34. NUS.SOC.CS5248 Ooi Wei Tsang 34 Adaptive Playout Mechanisms for Packetized Audio Applications in WAN R. Ramjee, J. Kurose, D. Towsley, H. Schulzrinne INFOCOM 1995

35. NUS.SOC.CS5248 Ooi Wei Tsang 35 Variables and Notations

36. NUS.SOC.CS5248 Ooi Wei Tsang 36 Variables and Notations T send (i) T play (i) T buf (i) T arrive (i) T delay (i) T net (i)

37. NUS.SOC.CS5248 Ooi Wei Tsang 37 1 st Packet in Talkspurt We can estimate as

38. NUS.SOC.CS5248 Ooi Wei Tsang 38 How to estimate V net (i)

39. NUS.SOC.CS5248 Ooi Wei Tsang 39 How to estimate E net (i) Method 1: Jacobson’s Method

40. NUS.SOC.CS5248 Ooi Wei Tsang 40 Spike Time T net

41. NUS.SOC.CS5248 Ooi Wei Tsang 41 Problems Does not react to spike fast enough

42. NUS.SOC.CS5248 Ooi Wei Tsang 42 How to estimate E net (i) Ramjee’s Method SPIKENORMAL

43. NUS.SOC.CS5248 Ooi Wei Tsang 43 Ramjee’s Idea SPIKENORMAL if T net (i) suddenly increase if “slope” is small enough

44. NUS.SOC.CS5248 Ooi Wei Tsang 44 In Spike Mode SPIKE i T net

45. NUS.SOC.CS5248 Ooi Wei Tsang 45 In Normal Mode NORMAL i T net

46. NUS.SOC.CS5248 Ooi Wei Tsang 46 Evaluations Delay Loss Rate

47. NUS.SOC.CS5248 Ooi Wei Tsang 47 Problems with Ramjee’s Method Time Transit Time

48. NUS.SOC.CS5248 Ooi Wei Tsang 48 Packet Audio Playout Delay Adjustment: Performance Bounds and Algorithms S. Moon, J Kurose, D. Towsley Multimedia Systems 1998

49. NUS.SOC.CS5248 Ooi Wei Tsang 49 Recall Previous Methods

50. NUS.SOC.CS5248 Ooi Wei Tsang 50 How to Set T delay (i) Moon’s Method Collect statistics on packets that have arrived. Find t such that q% of last w packets have T net (i) < t.

51. NUS.SOC.CS5248 Ooi Wei Tsang 51 Example (w =50, q = 90%) num of packets delay 1 2 3 4 5 6 7 8 9 10 11 12

52. NUS.SOC.CS5248 Ooi Wei Tsang 52 Setting T delay (i) NORMALSPIKE

53. NUS.SOC.CS5248 Ooi Wei Tsang 53 Setting T delay (i) Time Transit Time

54. NUS.SOC.CS5248 Ooi Wei Tsang 54 Performance Bound Given a trace of packets, and a loss rate, find the minimum average playout delay. Use Dynamic Programming

55. NUS.SOC.CS5248 Ooi Wei Tsang 55 A Packet Trace k M talkspurts 1,k2,k3,kj,kn k,k T net 131510…

56. NUS.SOC.CS5248 Ooi Wei Tsang 56 More Notations M: Number of Talkspurt N packet (k) or n k Number of packets in talkspurt k N total Total number of packets

57. NUS.SOC.CS5248 Ooi Wei Tsang 57 Definition minimum average playout delay for choosing i packets to be played out from k-th talkspurt k M talkspurts

58. NUS.SOC.CS5248 Ooi Wei Tsang 58 How to find

59. NUS.SOC.CS5248 Ooi Wei Tsang 59 Definition minimum average playout delay for choosing i packets to be played out from k-th to M-th talkspurt k M talkspurts M..

60. NUS.SOC.CS5248 Ooi Wei Tsang 60 Base Case D(k, 0) = D(M, i) = minimum average playout delay for choosing i packets to be played out from k-th to M-th talkspurt

61. NUS.SOC.CS5248 Ooi Wei Tsang 61 Recursive Case k.. M 1,k2,k3,kn k,k.. j

62. NUS.SOC.CS5248 Ooi Wei Tsang 62 Performance Bound Given a trace of M talkspurts and n packets, and a loss rate e, find the minimum average playout delay. Answer: Minimum possible average playout delay is D(1, (1-e)n) minimum average playout delay for choosing i packets to be played out from k-th to M-th talkspurt

63. NUS.SOC.CS5248 Ooi Wei Tsang 63 Evaluations Loss Rate Delay

64. NUS.SOC.CS5248 Ooi Wei Tsang 64 Summary Playout Adjustment for Audio Conferencing Weighted Average Methods vs. Statistical Methods An Analysis of Minimum Playout Delay

65. NUS.SOC.CS5248 Ooi Wei Tsang 65 Practical Complications Clock Drifts Route Change

66. NUS.SOC.CS5248 Ooi Wei Tsang 66 Advanced Techniques Speed-up Playback