Introduction to H.264 / AVC Video Coding Standard Multimedia Systems Sharif University of Technology November 2008

Scope of video coding standardization

Evolution of video coding standards of ITU-T VCEG and ISO/IEC MPEG.

H.264/AVC H.264 exploits the same video compression principles as its predecessors Simulation results have shown that it has substantial superiority of video quality over that achieved by H and MPEG-4. H.264 offer significantly higher quality levels for the same bit rates Rate-distortion curve of H.264 codec compared with its predecessors, Foreman QCIF at 10 fps

Selected new features in H.264 Advanced intra-coding Enhanced motion estimation with variable block size Multiple reference picture Integer block transform Improved deblocking filter Entropy coding H.264 Error Resilient Coding

H.264 advanced intra-coding (Cont.) Efficient video encoders mainly use inter-frame prediction, Use of intra-frame coding for parts of the picture is necessary to prevent error propagation But intra-frame coding generates a large bit rate, and hence in order for H.264 to be efficient, special attention is paid to intra-frame coding. H.264 takes advantage of correlations between neighboring blocks to achieve better compression in intra-coding.

H.264 advanced intra-coding (Cont.) Every intra 16 X 16 pixel MB in a picture is first predicted in an appropriate mode from the already coded and re-constructed samples of the same picture.

H.264 advanced intra-coding (Cont.) There are nine advanced intra-prediction modes for the samples when the MB is partitioned into 4 X 4 blocks

Advanced inter-coding Inter-frame predictive coding is where H.264 makes most of its gain in compression efficiency. Motion compensation (MC) on each 16 X 16 MB can be performed with various block sizes and shapes

Multiple reference picture The H.264 standard also offers the option of using several previous pictures for prediction. Every MB partition can have a different reference picture that is more appropriate for that particular block. –Increases the coding efficiency and produces a better subjective quality –Improve the robustness of the bitstream to channel errors

Deblocking filter In H.264 codec every reconstructed picture, is filtered by default using an adaptive deblocking filter. The filter removes visible block structures on the edges of the 4 X 4 blocks caused by block-based transform coding and motion estimation

Transformation, Quantization, and entropy coding H.264 employs a 4X4 integer transform The transform is an approximation of the DCT –It has a similar coding-gain to the DCT transform. –Since the integer transform has an exact inverse operation, there is no mismatch between the encoder and the decoder which was a problem in all DCT based codecs

Transformation, Quantization, and entropy coding (Cont.) A quantization parameter is used for determining the quantization of transform coefficients in H.264/AVC. The parameter can take 52 values. The quantized transform coefficients of a block generally are scanned in a zig-zag fashion and transmitted using entropy coding methods.

Transformation, Quantization, and entropy coding (Cont.) Before transmission, the generated data of all types are entropy coded. H.264 supports two different methods of entropy coding –context adaptive variable length coding (CAVLC) –context adaptive binary arithmetic coding (CABAC)

Transformation, Quantization, and entropy coding (Cont.) Their statistical distribution typically shows large values for the low frequency part decreasing to small values later in the scan for the high-frequency part. CAVLC –The number of nonzero quantized coefficients (N) and the actual size and position of the coefficients are coded separately. CABAC –The efficiency of entropy coding can be improved further if the Context-Adaptive Binary Arithmetic Coding (CABAC) is used –Allows the assignment of a non-integer number of bits to each symbol of an alphabet

H.264 Error Resilient Coding Spatial Error Propagation. –The use of entropy coding means that every coded bit within a slice requires the previous bits for its decoding. Hence, a single bit error in the transmitted stream may destroy the whole remaining coded bits of one slice Temporal error propagation –Dependencies of consecutive pictures, as a result of inter-coding. –Damage in one frame may propagate into many future frames, even if their information is received without error.

H.264 Error Resilient Coding (Cont.) First, the coded video data are grouped in network abstraction layer (NAL) units –Each NAL unit can be considered as a packet that contains an integer number of bytes including a header and a payload –The header specifies the NAL unit type and the payload contains the related data. Data Partitioning

H.264 Error Resilient Coding (Cont.) In a video sequence, each frame can be divided into several slices; each containing a flexible number of MBs. –In each slice, the arithmetic coder is aligned and the spatial predictions are reset. –Every slice in the frame is independently decodable, and therefore can be considered as a re-synchronization point that prevents spatial propagation of a probable error to the next slice.

H.264 Error Resilient Coding (Cont.) Intra-updates Multiple-reference selection

References [1] Thomas Wiegand, Gary J. Sullivan, Gisle Bjontegaard, and Ajay Luthra “Overview of the H.264 / AVC Video Coding Standard”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, JULY [2] Gary J. Sullivan, Pankaj Topiwala, and Ajay Luthra, “The H.264/AVC Advanced Video Coding Standard: Overview and Introduction to the Fidelity Range Extensions”, Presented at the SPIE Conference on Applications of Digital Image Processing XXVII Special Session on Advances in the New Emerging Standard: H.264/AVC, August, 2004.