1. Ali Saman Tosun Computer Science Department
Video Streaming
Ali Saman Tosun
Computer Science Department

2. Broadcast to True Media-on-Demand
Broadcast (No-VoD)
Traditional, no control
Pay-per-view (PPV)
Paid specialized service
Near Video On Demand (N-VoD)
Same media distributed in regular time intervals
Simulated forward / backward
True Video On Demand (T-VoD)
Full control for the presentation, VCR capabilities
Bi-directional connection

3. Streaming Stored Video
media stored at source
transmitted to client
streaming: client playout begins before all data has arrived
timing constraint for still-to-be transmitted data: in time for playout

4. Streaming Video
constant bit
rate video
transmission
variable
network
delay
client video
reception
constant bit
rate video
playout at client
client playout
delay
Cumulative data
buffered
video
time
Client-side buffering, playout delay compensate for network-added delay, delay jitter

5. Smoothing Stored Video
For prerecorded video streams:
All video frames stored in advance at server
Prior knowledge of all frame sizes (fi, i=1,2,..,n)
Prior knowledge of client buffer size (b)
workahead transmission into client buffer n 2 1
b bytes
Client
Server

6. Smoothing ConstraintsU
number of bytes
rate changes S L
time (in frames)
Given frame sizes {fi} and buffer size b
Buffer underflow constraint (Lk = f1 + f2 + … + fk)
Buffer overflow constraint (Uk = min(Lk + b, Ln))
Find a schedule Sk between the constraints
Algorithm minimizes peak and variability

7. Proxy-based Video Distribution
Server
Proxy adapts video
Proxy caches video
Proxy
Client
Client

8. Proxy Operations Drop frames Quality Adaptation
Drop B,P frames if not enough bandwidth
Quality Adaptation
Transcoding
Change quantization value
Most of current systems don’t support
Video staging, caching, patching
Staging: store partial frames in proxy
Prefix caching: store first few minutes of movie
Patching: multiple users use same video

9. Online Smoothing Source or proxy can delay the stream by w time units:
streaming
video
stream with
delay w
b bytes
Client
Source/Proxy
Larger window w reduces burstiness, but…
Larger buffer at the source/proxy
Larger processing load to compute schedule
Larger playback delay at the client

10. Online Smoothing ModelB b
Di-w Ai Si
proxy
client
Arrival of Ai bits to proxy by time i in frames
Smoothing buffer of B bits at proxy
Smoothing window (playout delay) of w frames
Playout of Di-w bits by client by time i
Playout buffer of b bits at client
Transmission of Si bits by proxy by time i

11. Online Smoothing max{Di-w, Ai - B} <= Si <= min{Di-w + b, Ai}
Must send enough to avoid underflow at client
Si must be at least Di-w
Cannot send more than the client can store
Si must be at most Di-w + b
Cannot send more than the data that has arrived
Si must be at most Ai
Must send enough to avoid overflow at proxy
Si must be at least Ai - B
max{Di-w, Ai - B} <= Si <= min{Di-w + b, Ai}

12. Online Smoothing Constraints
Source/proxy has w frames ahead of current time t:
number of bytes
don’t know
the future U L ?
time (in frames) t
t+w-1
Modified smoothing constraints as more frames arrive...

13. Smoothing Star Wars GOP averages 2-second window 30-second window
MPEG-1 Star Wars,12-frame group-of-pictures
Max frame bytes, mean frame 1950 bytes
Client buffer b=512 kbytes

14. Prefix Caching to Avoid Start-Up Delay
Avoid start-up delay for prerecorded streams
Proxy caches initial part of popular video streams
Proxy starts satisfying client request more quickly
Proxy requests remainder of the stream from server
smooth over large window without large delay
Use prefix caching to hide other Internet delays
TCP connection from browser to server
TCP connection from player to server
Dejitter buffer at the client to tolerate jitter
Retransmission of lost packets
apply to “point-and-click” Web video streams

15. Changes to Smoothing Model
Separate parameter s for client start-up delay
Prefix cache stores the first w-s frames
Arrival vector Ai includes cached frames
Prefix buffer does not empty after transmission
Send entire prefix before overflow of bs
Frame sizes may be known in advance (cached) Ai bs Si
Di-s bc bp

16. Best possible quality at possible sending rate
Scalable coding
Typically used as Layered coding
A base layer
Provides basic quality
Must always be transferred
One or more enhancement layers
Improve quality
Transferred if possible
Sending rate
Quality
Best possible quality at possible sending rate
Base layer
Enhancement layer

17. Temporal Scalability Frames can be dropped In a controlled manner
Frame dropping does not violate dependancies
Low gain example: B-frame dropping in MPEG-1

18. Better compression due to low values
Spatial Scalability
Base layer
Downsample the original image
Send like a lower resolution version
Enhancement layer
Subtract base layer pixels from all pixels
Send like a normal resolution version
If enhancement layer arrives at client
Decode both layers
Add layers 72 75 83 61 73 -1
-12 2 10
Base layer
Less data to code
Enhancement layer
Better compression due to low values

19. SNR Scalability SNR – signal-to-noise ratio Idea Base layer
Is regularly DCT encoded
A lot of data is removed using quantization
Enhancement layer is regularly DCT encoded
Run Inverse DCT on quantized base layer
Subtract from original
DCT encode the result
If enhancement layer arrives at client
Add base and enhancement layer before running Inverse DCT

20. Multiple Description Coding
Idea
Encode data in two streams
Each stream has acceptable quality
Both streams combined have good quality
The redundancy between both streams is low
Problem
The same relevant information must exist in both streams
Old problem: started for audio coding in telephony
Currently a hot topic

21. Delivery Systems Developments
Several Programs or Timelines
Network
Saving network resources:
Stream scheduling

22. Patching Join ! Central server 1st client 2nd client
Unicast patch stream
multicast
cyclic buffer
1st client
2nd client
Server resource optimization is possible

23. Proxy Prefix Caching Split movie Operation Goal Central server Prefix
Suffix
Operation
Store prefix in prefix cache
Coordination necessary!
On demand
Deliver prefix immediately
Prefetch suffix from central server
Goal
Reduce startup latency
Hide bandwidth limitations, delay and/or jitter in backbone
Reduce load in backbone
Unicast
Prefix cache
Unicast
Client

24. Interval Caching (IC) caches data between requests
following requests are thus served from the cache
sort intervals on length
I32
I33
I12
I31
I11
I21
S11
S12
S11
Video clip 1
S12
S11
S13
Video clip 2
S22
S21
Video clip 3
S33
S31
S32
S34
Video clip 1
Video clip 1
I11
I12
I21
I31
I32
I33

25. Receiver-driven Layered Multicast (RLM)
Requires
IP multicast
layered video codec (preferably exponential thickness)
Operation
Each video layer is one IP multicast group
Receivers join the base layer and extension layers
If they experience loss, they drop layers (leave IP multicast groups)
To add layers, they perform "join experiments“
Advantages
Receiver-only decision
Congestion affects only sub-tree quality
Multicast trees are pruned, sub-trees have only necessary traffic

26. Receiver-driven Layered Multicast (RLM)