NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 1 Adaptive Playout

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 2 You are Here Network Encoder Sender Middlebox Receiver Decoder

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 3 How to recv and play? open socket while not done if socket is readable read packet from socket remove RTP header decode play back

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 4 What’s Wrong? packet ordering packet loss next packet arrive in-time? Especially bad for audio applications

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 5 Overview Network RTP Classifier Decode

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 6 Implementation Single Thread using select() Multi-Threads

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 7 Packet Buffer Sorted by sequence number When ADU is complete, send to decoder RTP Classifier

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 8 Playout Buffer Stored decoded data in playout order Post-processing/Mixing may happen Decode

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 9 Why Buffer?

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 10 Sending Packets Time Packet

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 11 Receiving Packets Time Packet

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 12 With Jitter Time Packet

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 13 With Jitter Time Packet

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 14 What causes Jitter? Network delay = Transmission Delay (fixed) + Propagation Delay (fixed) + Queuing Delay (variable) Delay jitter is caused by variable queuing delay

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 15 Delay Jitter Time Transit Time small jitter large jitter

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 16 Spike Time Transit Time

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 17 Today’s Question How big is the playout buffer? When to play back?

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 18 Types of Applications Non-interactive Buffer can be large Interactive As small as possible

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 19 Types of Applications Video Frames are discrete (easier problem) Audio Samples are “continuous”

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 20 Naive Answer How big is a buffer? Fixed at a small value, to reduce latency When to playback? Playback as soon as possible, to reduce latency

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 21 Client Buffer Management Bandwidth Smoothing for Non- interactive Applications

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 22 Multi-Threshold Flow Control R. Zimmermann, K. Fu, M. Jahangiri, C. Shahabi MTAP 2006

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 23 Video-on-Demand (VoD) High-quality playback required Buffer can be large Encoding may be VBR for high visual quality Playback time may be very long (2+ hours)

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 24 Two Approaches (1) Server-controlled Pre-compute transmission schedule Piece-wise linear approximation: compute a number of constant-rate segments Different optimization criteria Minimize: # of rate changes, # utilization of client buffer, peak rate, etc.

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 25 Two Approaches (2) Client-controlled Client adaptively informs server Advantages: No rate history necessary (VBR) Can adjust to network conditions Works easily with interactive commands such as FF, FR, Pause

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 26 Robust Stream Delivery Smoothing of VBR media traffic has the following quality benefits: Better resource utilization (less bursty) More streams with the same network capacity

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 27 Objectives Multi-Threshold Flow Control (MTFC) algorithm objectives: Online operation Content independence Minimizing feedback control signaling Rate smoothing

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 28 Example

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 29 MTFC Buffer Management Multiple Thresholds: goal is middle of buffer Send rate adjust command to server whenever threshold is crossed

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 30 How to Set Thresholds? Simple: equi-distant m: number of thresholds Overflow and underflow thresholds

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 31 How to Calculate Sending Rate?

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 32 Threshold Spacing Strategies Linear vs. arithmetic vs. geometric

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 33 Threshold Spacing Strategies Geometric and arithmetic spacing:

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 34 Sending Rate Computation Relevant factors: Target buffer level (i.e., 50%): TH N Current buffer level: b wobsv Predicted duration to reach target level Data consumed during predicted duration

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 35 MTFC Results

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 36 MTFC Results: # of Thresholds

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 37 MTFC Results: # of Thresholds Number Of Rate Changes 8MB 16MB 32MB Number of Thresholds

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 38 A Brief Introduction to Audio Conferencing

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 39 Audio Conferencing Live, interactive application Latency is important Each packet contains 20-30ms of audio

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 40 Silence Suppression Silence Detection if no sound, no need to send Talk spurt consecutive audio packets (between silence) hundreds of ms

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 41 Recall: RTP Header marker bit: depends on payload e.g. beginning of frame

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 42 RTP and Talkspurt First packet of a talkspurt will have marker bit set to 1

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 43 RTP and Talkspurt Deduce talkspurt from sequence number and timestamp SeqNo TimeStamp

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 44 Consequences of Talkspurt Opportunity to adjust playout delay if jitter is large, increase delay if jitter is small, decrease delay

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 45 Fixed Playout Delay SEND RECV PLAY

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 46 Adaptive Playout Delay SEND RECV PLAY

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 47 Adaptive Playout Delay SEND RECV PLAY

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 48 Trade-Off Latency vs. Packet Loss

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 49 Latency vs Loss-Rate Loss Rate Latency

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 50 Adaptive Playout Mechanisms for Packetized Audio Applications in WAN R. Ramjee, J. Kurose, D. Towsley, H. Schulzrinne INFOCOM 1994

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 51 Variables and Notations

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 52 Variables and Notations T send (i) T play (i) T buf (i) T arrive (i) T delay (i) T net (i)

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 53 1 st Packet in Talkspurt We can estimate as

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 54 How to estimate V net (i)

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 55 How to estimate E net (i) Method 1: Jacobson’s Method

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 56 Spike Time T net

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 57 Problems Does not react to spike fast enough

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 58 How to estimate E net (i) Ramjee’s Method SPIKENORMAL

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 59 Ramjee’s Idea SPIKENORMAL if T net (i) suddenly increase if “slope” is small enough

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 60 In Spike Mode SPIKE i T net

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 61 In Normal Mode NORMAL i T net

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 62 Evaluations Delay Loss Rate

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 63 Problems with Ramjee’s Method Time Transit Time

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 64 Packet Audio Playout Delay Adjustment: Performance Bounds and Algorithms S. Moon, J. Kurose, D. Towsley Multimedia Systems 1998

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 65 Recall Previous Methods

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 66 How to Set T delay (i) Moon’s Method Collect statistics on packets that have arrived. Find t such that q% of last w packets have T net (i) < t.

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 67 Example (w =50, q = 90%) num of packets delay

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 68 Setting T delay (i) NORMALSPIKE

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 69 Setting T delay (i) Time Transit Time

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 70 Performance Bound Given a trace of packets, and a loss rate, find the minimum average playout delay. Use Dynamic Programming

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 71 A Packet Trace k M talkspurts 1,k2,k3,kj,kn k,k T net …

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 72 More Notations M: Number of Talkspurt N packet (k) or n k Number of packets in talkspurt k N total Total number of packets

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 73 Definition minimum average playout delay for choosing i packets to be played out from k-th talkspurt k M talkspurts

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 74 How to find

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 75 Definition minimum average playout delay for choosing i packets to be played out from k-th to M-th talkspurt k M talkspurts M..

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 76 Base Case D(k, 0) = D(M, i) = minimum average playout delay for choosing i packets to be played out from k-th to M-th talkspurt

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 77 Recursive Case k.. M 1,k2,k3,kn k,k.. j

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 78 Performance Bound Given a trace of M talkspurts and n packets, and a loss rate e, find the minimum average playout delay. Answer: Minimum possible average playout delay is D(1, (1-e)n) minimum average playout delay for choosing i packets to be played out from k-th to M-th talkspurt

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 79 Evaluations Loss Rate Delay

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 80 Summary Playout Adjustment for Audio Conferencing Weighted Average Methods vs. Statistical Methods An Analysis of Minimum Playout Delay

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 81 Practical Complications Clock Drifts Route Change

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 82 Advanced Techniques Speed-up Playback