COMPLEXITY VARYING INTRA PREDICTION IN H.264 by Leonid Ostrovsky Supervisors: Dr. Ofer Hadar, Mr. Evgeny Kaminsky

H.264 overview (a.k.a. MPEG 4 Part 10) Advanced codec Superior to previous standards: Relies on multiple frames in GOP Uses both 4x4 and 16x16 macroblocks Up to quarter pixel precise Uses prediction on the basis of edges of neighboring macroblocks (instead of DC)

H.264 overview: Compression Figure 1:GOP (Group Of Pictures) GOP (Why do we need this?) I-frames B- and P-frames Spatial and temporal redundancy We’re able not to encode the whole picture

H.264 overview: Compression Figure 2: Encoding process We’re able to encode just the residue Inter- and Intra- prediction

H.264 overview: Intra-Prediction Reminder: What is it? How do we predict? Cost function J = Distortion + λmode *Rate Prediction modes Figure 3: 4x4 modes Figure 4: 16x16 modes

H.264 overview: Intra-Prediction Problem: Too many modes to check! (592 for each macroblock) Quite problematic for embedded sys. What if: We could cut this number down… …by finding the most probable modes to be chosen? This leaves some open questions: How many modes? Which modes ?

The method Sobel filter Figure 5: Sobel filter Now we can see which mode is the most probable one However, we cannot use static allocation (will greatly influence the buffering)

The method We must cut down the modes in such way that more complex macroblocks will be more compressed It is done in a relative fashion… …i.e. the most complex block will get 9 modes and the least complex one will get 1 By this we ensure lesser buffering

The method The final goal is: To achieve a situation in which by giving a certain time limit … …the CODEC will allocate modes in such way that computational complexity will not cross that limit By this we can assure that embedded systems with low energy resources could set the limit lower than usual ensuring by that low waste of energy

The outcome so far There is a fully working Sobel filter There is a dynamic mode allocating mechanism in 4x4 block mode Actually, even though that wasn’t my goal I already have CODEC that works more effective than the original standard. Let’s see some statistics:

Results:

Results:

Results: Also: average number of modes per macroblock is : 1.9868 Variance of rate per frame in my algorithm is 39594 when in 2-mode state , for instance, is 49325 i.e. lower buffering

Future goals To extend same algorithm to 16x16 mode (and if I get to it, to chroma component) To develop a new cost function from the following format: J = Distortion +λMODE *Rate As you can see it’s the same as the standard function. However this one will also include time limit To test and check the whole algorithm