Optimal Quality Adaptation for MPEG-4 Fine-Grained Scalable Video Taehyun Kim and Mostafa H. Ammar College of Computing, Georgia Institute of Technology

Problems Rate smoothing is not useful for a best effort network, since the Internet does not provide any information about the bandwidth evolution in advance. Rate smoothing is not useful for a best effort network, since the Internet does not provide any information about the bandwidth evolution in advance. A smooth data rate does not always guarantee a smooth quality for VBR video. A smooth data rate does not always guarantee a smooth quality for VBR video.

Problems Frequent adding and dropping of layers can incur significant quality variability. Frequent adding and dropping of layers can incur significant quality variability. Quality Adaptation for minimizing the perceptual video quality by using bidirectional optimum layer selection. Quality Adaptation for minimizing the perceptual video quality by using bidirectional optimum layer selection.

Problems Small time scale variability Small time scale variability Receiver buffer Receiver buffer Large time scale variability Large time scale variability Scalable (Layered) video encoding Scalable (Layered) video encoding

Goal Trying to accommodate the mismatch caused by both the available bandwidth variability and the encoded video variability. Trying to accommodate the mismatch caused by both the available bandwidth variability and the encoded video variability. To develop an optimal algorithm that minimizes the quality variability while at the same time maximizing the utilization of the variable network bandwidth. To develop an optimal algorithm that minimizes the quality variability while at the same time maximizing the utilization of the variable network bandwidth.

Rate variability in MPEG-4 FGS Base LayerFGS Layer (SNR).Fixed quantization step size A river runs through it, GOP=12.Max variation = 7.4 kBytes.Max variation = 33.6 kBytes

Rate variability in MPEG-4 FGS FGST Layer (SNR).Max variation = 29.9 kBytes

Two hybrid temporal-SNR scalability structures Two hybrid temporal-SNR scalability structures FGS-FGST

Two hybrid temporal-SNR scalability structures Two hybrid temporal-SNR scalability structures FGST-FGS

Video Quality – Base Layer 100 VOPs, 45 dB

Video Quality – Base+FGS Layer 100 VOPs, improved more than 20 dB

Video Quality – Base+FGST Layer 300 VOPs, inconsistent quality

Video Quality – All Layer 300 VOPs, 67.3 dB

Quality Adaptation Algorithm Quality adaptation is defined by a mechanism that adds and drops layers based on the available network bandwidth while maximizing the perceptual video quality. Quality adaptation is defined by a mechanism that adds and drops layers based on the available network bandwidth while maximizing the perceptual video quality. Consistent “ long runs ” of the same quality video. Consistent “ long runs ” of the same quality video.

Composed Algorithm The quality smoothing algorithm proposed in [13] accomplishes the maximum reduction of quality variability for layered CBR video using bidirectional layer selection. The quality smoothing algorithm proposed in [13] accomplishes the maximum reduction of quality variability for layered CBR video using bidirectional layer selection. Rate smoothing algorithm presented in [18] enables a sender to transmit a piecewise CBR sequence by using the work-ahead smoothing technique. Rate smoothing algorithm presented in [18] enables a sender to transmit a piecewise CBR sequence by using the work-ahead smoothing technique.

Composed Algorithm L: Number of layers N: Number of VOPs x i [k] : size of VOP k S i k : a feasible sequence of layer i

The cumulative selected data defined by Optimal Quality Adaptation The cumulative capacity The receiver buffer size for storing unplayed i - th layer video The VOP size of i -th layer, at time k The available bandwidth : the residual bandwidth after accommodating layers 1, 2, …, i-1. i -th layer, at time k.

Framework of quality adaptation Display buffered video + prefetch No display video

Framework of quality adaptation The constraint of rate adaptation is determined by the receiver buffer size and the source video rate, whereas the main constraint of quality adaptation is transmission resources. The constraint of rate adaptation is determined by the receiver buffer size and the source video rate, whereas the main constraint of quality adaptation is transmission resources.

State transition diagram specifying the quality adaptation mechanism SelectDiscard No capacity Available cumulative capacity ≥ threshold Available cumulative capacity <threshold Enough capacity

Optimal Adaptation Available network bandwidth is known Residual bandwidth for higher layer Stay as long as possible

Theorem 1 In the framework of the optimal quality adaptation, a threshold value equal to the receiver buffer size satisfies In the framework of the optimal quality adaptation, a threshold value equal to the receiver buffer size satisfies 1) minimum video quality variability 1) minimum video quality variability 2) the necessary condition of maximum network utilization 2) the necessary condition of maximum network utilization

Online Heuristic The optimal quality adaptation algorithm assumes the available bandwidth information is known in advance. An algorithm that minimizes quality variability without using future bandwidth information. The differences between the online heuristic and the optimal adaptation 1) the online heuristic makes a decision on which layer and which VOP to be transmitted in real time (lines 4-6) 2) a sender makes a receiver prefetch the next selected VOPs when there is a transition from the select state to the discard state (line 15).

Online Heuristic Algorithm Receiver prefetch the next selected VOPs Make decision on which layer and which VOP to be transmitted in real time

How to determine the next prefetch point at the transition time ? An MA (Moving Average) type estimator to determine the prefetch point. simple and widely known for the usage of TCP retransmission timeout estimation in [7].

Experimental model

TFRC throughput TFRC/UDP (TCP-Friendly Rate Control) Quality transition of the i th layer is defined by Rate smoothing Quality smoothing Receiver buffer

Performance over TFRC (1) QT=121QT=13 QT=87QT=9 Slow response time of TFRC Composed Optimal adaptation

Performance over TFRC (2) Online heuristic Threshold based QT=126 Target on minimize loss probability QT=16

TCP throughput Small time scale variability is significant as much as 3 Mbps

Performance over TCP (1) Composed Optimal adaptation

Performance over TCP (2) Threshold based Online heuristic

Two reasons contribute to superiority of TCP TCP achieves more throughput than TFRC in dynamic condition. TCP achieves more throughput than TFRC in dynamic condition. Although TCP exhibits significant small time scale variability, it can be successfully accommodated by the receiver buffer. Although TCP exhibits significant small time scale variability, it can be successfully accommodated by the receiver buffer.

Experiment results for 4 video streams Average Quality Transition Average Run Length [13]

Conclusion Considering a problem of providing perceptually good quality for layered VBR streaming video. Considering a problem of providing perceptually good quality for layered VBR streaming video. An optimal adaptation algorithm that minimizes quality variability while increasing the usage of the available bandwidth. An optimal adaptation algorithm that minimizes quality variability while increasing the usage of the available bandwidth. Companion web site Companion web site ple/Phd/tkim/qa.html ple/Phd/tkim/qa.html