1. Gaurav Hansda 1000721849 gaurav.hansda@mavs.uta.edu
Fast Decision of Block size, Prediction Mode and Intra Block for H.264 Intra Prediction EE 5359
Gaurav Hansda

2. H.264 Encoder
Fig. 1. Basic coding structure for H.264/AVC for a macroblock. [2]

3. Mode decision of H.264 encoder
Fig. 2. Mode decision hierarchy of an H.264 compliant encoder. [4]

4. Implication of Hierarchical Structure
To ensure the correctness of the decision at upper layer.
To ensure early termination is executed accurately and as early as possible.
Most fast mode decision algorithms developed so far, only deal with a single stage of the mode decision hierarchy [5]- [14] and fail to achieve the best possible complexity reduction.
We will concentrate on left branch of the mode decision hierarchy namely, intra/intra block decision of inter frames, block size decision of intra blocks, and the prediction mode decision of intra blocks.

5. Intra-Prediction
There are 3 macroblock (MB) modes for intra prediction of luma pixels: intra4x4 (I4MB), intra8x8 (I8MB), and intra16x16 (I16MB).
Intra4MB and Intra8MB have 9 prediction modes as shown in Fig. 3(a). Intra16MB has only 4 prediction modes as shown in Fig. 3(b).
Fig. 3. Prediction modes for (a) Intra4MB and (b) Intra16MB. [4]

6. Fig. 4. Prediction flow diagram [18]
Fig. 5. Intra-prediction [18]

7. Mode Decision
To achieve a better tradeoff between bit-rate and distortion, H.264 encoder adopts the rate-distortion (R-D) optimization framework and the Lagrangian technique for mode decision [2].
For intra frames, the best prediction mode of a block is defined as the mode that, among all prediction modes of the block, gives rise to the minimum R-D cost. The R-D cost of an MB mode is the sum of the minimum R- D cost of each individual block.

8. Related work on Fast Prediction Mode Decision
Can be classified into two categories: non-filter based and filter-based.
Non-filter based algorithms attempt to reduce the computation of the R-D cost and improve the R-D performance by modifying the matching criterion.
Most such algorithms, like that of developed by Huang et al. [5] and Tseng et al. [7], deal with the low complexity mode.

9. The best prediction mode of a block is highly correlated with its dominant edge direction.
Filter-based algorithms determine the dominant edge direction of each block and then select the associated prediction mode, the neighboring two prediction modes, and the DC mode as candidate modes.
Always include the DC mode as a candidate mode.
Algorithms developed by Pan et al. [6], Tsai et al. [9], Li et al. [10], and Bharanitharan et al. [12], all exploits various edge detection.
Due to limited prediction accuracy, filter-based algorithms introduce more the 2% increase in bit-rate.

10. Related work on Fast Inter/Intra Block decision
The goal of fast inter/intra block decision is to efficiently determine which coding type between inter coding and intra coding is more suitable for coding an MB of an inter frame.
Basic idea is to measure the correlation between spatially and temporally neighboring MBs.
If the spatial correlation is larger than the temporal correlation, the block under consideration is more likely to be an intra block.
Algorithms developed by Choi et al. [13] and Kim et al. [14] have high computational overhead.

11. Proposed Algorithm for Block Size Decision
Block size is highly correlated with texture complexity.
Variance of block corresponds to the total energy of the AC coefficients of the block, hence it is good measurement of the texture complexity.
Thus variance based classification of texture complexity is used [16].
If variance is above the threshold, Intra4MB and Intra8MB is selected; otherwise, Intra8MB and Intra16MB is chosen. This is simple way to skip the examination of Intra4MB mode.

12. Fig. 6. Variance-based MB mode decision [4]

13. Improved Prediction Mode Decision
Earlier algorithms only consider the edge information of the current block and do not take correlation between blocks into account.
Hence the Most Probable Mode(MPM) is used.
The MPM, which takes advantage of the spatial correlation of the prediction modes between the neighboring blocks and the current block for coding, is defined as the prediction mode of the left or the upper neighbor, whichever has the smaller prediction mode number.

14. Input 2x2 subsampled block
Pass through the filters separately
Determine the dominant edge
Choose the candidate modes
Fig. 7. Prediction mode decision [4]

15. Proposed algorithm for Intra Block Decision
Intra block decision, for inter frames, occupies a considerable percentage of the total computations of inter- frame coding.
Intra16MB takes much less computation time than the other modes.
Hence, scaled R-D cost is used [4].
An MB is less probable to be intra coded if the R-D cost difference between best inter mode and Intra16MB is small. Denoting the scaled R-D cost differences between Intra16MB and the inter MB mode by dˆJ, and based on the above observation, if dˆJ is small both I4MB and I8MB can be skipped.

16. References
[1] Draft ITU-T Recommendation and Final Draft International Standard of Joint Video Specification, document JVT-G050.doc, ITU-T Rec. H.264 and ISO/IEC AVC, [2] T. Wiegand et al, “Overview of H.264 video coding standard,” IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 560–576, Jul [3] G. J. Sullivan and T. Wiegand, “Rate-distortion optimization for video compression,” IEEE Signal Process. Mag., vol. 15, no. 6, pp. 74–90, Nov [4] Y.-W. Huang, T. Ou, and H.Chen,” Fast decision of block size, prediction mode and intra block for H.264 intra prediction,” I EEE Trans. Circuits Syst. Video Technol., vol. 20, no.8, pp , Aug [5] Y.-W. Huang et al, “Analysis, fast algorithm, and VLSI architecture design for H.264/AVC intra frame coder,” IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 3, pp. 378–401, Mar [6] F. Pan et al., “Fast mode decision algorithm for intra prediction in H.264/AVC video coding,” IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 7, pp. 813–822, Jul

17. [7] C. Tseng, H. Wang, and J. Yang, “Enhanced intra-4x4 mode decision for H.264/AVC coder,” IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 8, pp. 1027–1032, Aug [8] J.-C. Wang, J.-F. Wang, and J.-T. Chen, “A fast mode decision algorithm and its VLSI design for H.264/AVC intra-prediction,” IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 10, pp. 1414–1422, Oct [9] A.-C. Tsai et al., “Intensity gradient technique for efficient intra-prediction in H.264/AVC,” IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 5, pp. 694–698, May [10] H. Li, K. Ngan, and Z. Wei, “Fast and efficient method for block edge classification and its application in H.264/AVC video coding,” IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 6, pp. 756–768, Jun [11] A.-C. Tsai et al., “Effective subblock- based and pixel-based fast direction detections for H.264 intra prediction,” IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 7, pp. 975–982, Jul [12] K. Bharanitharan et al., “A low complexity detection of discrete cross differences for fast H.264/AVC intra prediction,” IEEE Trans. Multimedia, vol. 10, no. 7, pp. 1250–1260, Nov

18. [13] I. Choi, J. Lee, and B. Jeon, “Fast coding mode selection with rate-distortion optimization for MPEG-4. Part 10: AVC/H.264,” IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 12, pp –1561, Dec
[14] C. Kim and C. Jay Kuo, “Feature-based intra/inter coding mode selection for H.264/AVC,” IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 4, pp. 441–453, Apr
[15] B. Kim, “Fast selective intra-mode search algorithm based on adaptive thresholding scheme for H.264/AVC encoding,” IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 1, pp. 127–133, Jan
[16] A. Yu, G. Martin, and H. Park, “Fast inter-mode selection in the H.264/AVC standard using a hierarchical decision process,” IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 2, pp. 186– 195, Feb
[17] JM software –
This software is a product of Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG. The latest version of JM Software is 17.2
[18] Iain E. Richardson , “The H.264 Advanced Video Compression Standard”, Wiley publications,