Complexity varying intra prediction in H.264 Supervisors: Dr. Ofer Hadar, Mr. Evgeny Kaminsky Students: Amit David, Yoav Galon

Introduction to video coding  A video stream is composed of frames.

Introduction to video coding   Each frame is divided into macro-blocks.

General compression scheme  For each block, the following process is applied: DCTQuant’ Zig-Zag Entropy coding  The goal of this process is to obtain a minimal representation of the block in a lossy manner.

General compression scheme  Each block is considered as a 2 dimensional discrete function. The block is transformed a 2 dimensional discrete cosine transform – DCT. 8x8 DCT bases DCT Quant Zig-zag Entropy coding

General compression scheme  Each DCT block is quantized by a uniform quantization matrix (this is the lossy part of the process!). quantization Quantization matrix DCTQuant Zig-zag Entropy coding

General compression scheme  The DCT elements of the quantized matrix are collected in a zig-zag way…  The bit stream that was generated goes through Run Length coding and then through a VLC coder (Huffman/Arithmetic coding). DCTQuant Zig-zag Entropy coding

Important notice!!! A block with smaller values produces a shorter output bitstream…

H.264  H.264 is the newest standard among the video compression standards.  H.264 is capable of producing a smaller bitstream for the same level of quality (SNR) then MPEG-4 part 2 does.

Intra prediction in H.264  In order to obtain small representing values for each block, a new block is created: Using previously coded information, an estimation of the current block is created. Using previously coded information, an estimation of the current block is created. This estimation is called a prediction block. This estimation is called a prediction block. The differences between the original block and the prediction block are calculated and compose a new block (called difference block). The differences between the original block and the prediction block are calculated and compose a new block (called difference block). Instead of the original block, the difference block is sent to the next step of the compression process, the DCT transform. Instead of the original block, the difference block is sent to the next step of the compression process, the DCT transform.  The prediction utilizes temporal and spatial correlations based on previously encoded information.  Our project deals with spatial correlation known as INTRA PREDICTION. … prediction original DCT originaldifference

Intra prediction in H.264

Intra prediction - details  H.264 examines the prediction methods for different block sizes (16x16,8x8,4x4) and chooses the prediction that minimizes the rate-distortion cost: J = Distortion + *Rate J = Distortion + *Rate  For each prediction mode, The rate and distortion are calculated after the entire compression process is applied.

Intra prediction - details  Prediction modes for 4x4 blocks: A B C D E F G HQ I J K L DC  Cost function for 4x4: Where: VAL = A VAL = 0.5Q I A DC

Intra prediction - details  Prediction modes for 16x16 and 8x8 chroma blocks: min DiffT = H * Diff * H H V...… 1.Vertical 2.Horizontal 3.Plane 4.Mean (H+V)=DC DC  Cost function for 16x16:  Cost function for chroma 8x8:

Motivation  As we saw the computational complexity of examining all the possible prediction modes is very big.  The number of modes is 4x(9x16+4)=592.  Remember, for each mode the entire coding process is applied.  We need a method that allows us to reduce the processing time of the encoder without severely damaging the quality of the encoded video sequence. chroma4x4 luma16x16 luma

The algorithm  In order to produce the smallest differential block, we estimate the probabilities of the different modes using the information existing in the block.  The way we do so is by finding edges in the original block. The edges are translated into one of predefined directions that match the different prediction modes.

 What is an edge? The algorithm

 How can we find an edge?  One way is to use the first order derivative filters known as SOBEL MASKS The algorithm

 2D convolution between the original frame and the 3x3 masks: The algorithm

 For each pixel in the block, 2 values are calculated: Amplitude - Amplitude - Angle - Angle - The algorithm

 Each pixel is associated with one of the prediction modes according to it’s angle(=directions).  The amplitude of each pixel is added to the cumulative sum of it’s direction. The algorithm

 This process can be viewed as a creation of a histogram that represents the probabilities of each mode to produce the minimum cost function.  The modes are sorted according to the histogram.  According to a COMPLEXITY parameter, we can decide how many modes should be examined. The algorithm

 The COMPLEXITY parameter allows us to have control over the performances of the encoder. A larger complexity can be given when more computational resources, or less bandwidth, is available. A larger complexity can be given when more computational resources, or less bandwidth, is available. Whenever a large complexity is given, more prediction modes are taken into account. Whenever a large complexity is given, more prediction modes are taken into account. The algorithm

The implementation & setup  We have implemented the algorithm (for 4x4 block size) on the JM10.1 platform.  The code was written in C language and tested with MATLAB.  We have examined the performance of the algorithm on standard movie sequences.  All frames (150) where forced to be I frames only.  Each movie was tested under 72 configurations. 9 QP x 8 complexities=72.

The results  The results were processed and show the relations between 3 variables: Processing time Processing time Bit rate Bit rate Distortion Distortion  The results are compared to the performance of the standard algorithm.

Time (sec) SNR (dB) Bits (k) Num’ modes The results Qp = 28

Time (sec) SNR (dB) Bits (k) Num’ modes The results Qp = 33

The results Timebits distortion The relations between : processing time, bits per frame, distortion

The results Timebits distortion comparison between our algorithm and the original implementation