Overview of Fine Granularity Scalability in MPEG-4 Video Standard Weiping Li, Fellow, IEEE

System configuration for Internet Streaming Video

FGS Scalability Structure n e(n) n+1 e(n+1) n+2 e(n+2) n+3 e(n+3)

Layered scalable coding Tech. SNR scalability Temporal scalability Spatial scalability

Illustration of video coding performance

SNR scalability decoder defined in MPEG-2

Layered scalable coding Tech. SNR scalability If the encoder uses the enhancement-layer information in the motion-prediction loop and the enhancement-layer information is (not) received by the decoder, the enhancement-layer coding efficiency is high (low). If the encoder does not use the enhancement-layer information in the motion-prediction loop and the enhancement-layer information is (not) received by the decoder, drift happens in the enhancement layer and coding efficiency is low. (the result is the same as using the base layer only.)

Layered scalable coding Tech. Temporal scalability

Layered scalable coding Tech. Spatial scalability

BIT-PLANE CODING OF THE DCT COEFFICIENTS

← CCIR-601 Sequences SIF and QCIF Sequences →

FGS USING BIT-PLANE CODING OF DCT COEFFICIENTS Overall Coding Structure of FGS Some Details of FGS Coding Profile Definitions in the Amendment of MPEG-4

Overall Coding Structure of FGS FGS encoder structure

Overall Coding Structure of FGS FGS decoder structure

Some Details of FGS Coding 1) Different Numbers of Bit-Planes for Individual Color Components 2) Variable-Length Codes 3) Decoding Truncated Bitstreams 4) Variations to the Standardized FGS Coding Structure

Different Numbers of Bit-Planes for Individual Color Components

Variable-Length Codes Statistics of the (RUN, EOP) symbols in the four VLC tables

Coding patterns for syntax element fgs_cbp

Decoding Truncated Bitstreams Decoding of the truncated bitstream is not standardized in MPEG-4. One possible method To look ahead 32 bits at every byte-aligned position in the bitstream. If the 32 bits are not fgs vop start code, the first 8 bits of the 32 bits are information bits of the FGS frame to be decoded. The decoder slides the bitstream pointer by one byte and looks ahead another 32 bits to check for fgs vop start code.

Variations to the Standardized FGS Coding Structure A possible variation of FGS encoder structure DCT Domain

Variations to the Standardized FGS Coding Structure A possible variation of FGS decoder structure

“Find Reference” To generate the reference signal in the DCT domain to be subtracted in the encoder and added in the decoder. The operation of taking residue is slightly different in the variation method.

The distribution of such a residual signal Taking Residue between Original and Reconstructed DCT Coefficients.

The distribution of such a residual signal Taking residue between original and lower boundary point of quantization bin.

Comparison of two different ways of generating the DCT residues

Encoder structure without the DCT unit in the enhancement layer

Experiment results on mismatch errors

Profile Definitions in the Amendment of MPEG-4 Advanced Simple Profile contains a subset of nonscalable video-coding tools to achieve high coding efficiency at any given bit rate within a wide range of bit rates. The base-layer coding tools defined in the Advanced Simple Profile include both P-VOP (forward pre-diction only) and B-VOP (bi- directional prediction) for coding motion- compensated residues. The FGS profile is defined to meet the requirement of optimizing video quality over a given bit rate range.

CODING EFFICIENCY PERFORMANCE OF FGS FGS versus Multilayer SNR Scalability FGS versus Non-Scalable Coding FGS versus Simulcast

Compare FGS with multilyer SNR scalability

Compare FGS with non- scalable coding

Compare FGS with simulcast

ADVANCED FEATURES IN FGS Frequency Weighting Selective Enhancement Error Resilience FGS Temporal Scalability

Frequency Weighting

Selective Enhancement

Error Resilience

FGS Temporal Scalability FGST organized into a separate layer from FGS FGST and FGS organized into a single enhancement layer.

Conclusion Advantages of using FGS for Internet streaming video applications It allows separation of encoding and transmission. The server can transmit enhancement layer at any bit rate without transcoding. It provides a solution to the video server overload problem.