1. H.264/AVC Baseline Profile Decoder Complexity Analysis Michael Horowitz, Anthony Joch, Faouzi Kossentini, and Antti Hallapuro IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, JULY 2003

2. Outline  Introduction  H.264/AVC decoder overview  Storage requirements  Time complexity  Comparative analysis  Experimental analysis  Conclusion

3. Introduction  To estimate the computational complexity of an H.264/AVC baseline decoder, it is important to understand its two major components ： Time complexity Space complexity

4. Introduction  Time complexity Time complexity is measured by the approximate number of operations required to execute a specific implementation of an algorithm.  Storage complexity Storage complexity is measured by the approximate amount of memory required to implement an algorithm.

5. Introduction  Develop and validate a methodology for estimating decoder time complexity.  Study the relationship between decoder time complexity and encoder characteristics, source content, resolution and bit rate

6. H.264/AVC decoder overview  H.264/AVC decoding process consists of two primary paths: the generation of the predicted video blocks the decoding of the coded residual blocks

7. H.264/AVC decoder overview

8.  The decoding first process includes the parsing and decoding of the entropy coded bitstream UVLC ( Complexity of CAVLC = 2 x UVLC )  Depending on the coding mode (I or P) of each macroblock, the predicted macroblock can be generated either temporally (intercoding) or spatially (intra-coding).

9. H.264/AVC decoder overview  Inter-coding MB: Block size 16x16 ~ 4x4 Quarter-sample accuracy Motion vectors are coded differentially using either median or directional prediction. Multiple reference frame

10. H.264/AVC decoder overview  Intra-coding MB 16x16 or 4x4 Intra-coding mode 9 possible mode for 4x4, 4 possible mode for 4x4 (ex : DC, vertical, horizontal …...)  Decoding of residual Inverse transform  Deblocking filter

11. Storage requirements  The storage required by an H.264/AVC baseline decoder is divided into ： Memory that is needed for the whole frame Memory that is needed for one line of macroblock Memory that is needed for a macroblock Memory that is needed for constant data

12. Storage requirements

13.  Frame buffers dominate the storage requirements, particularly for high- resolution video 95% for QCIF, 98% for CIF

14. Time complexity  Table Descriptions Decoder Subfunction Tables Operation Count Table Execution Subunit Table

15. Decoder Subfunction Tables

16. Operation Count Table

17. Execution Subunit Table

18. Time complexity  Analysis Methodology First ： compute the number of cycles required to execute a particular subfunction on a chosen hardware platform Second ： the cycle count estimate is derived by multiplying the result from the first step by the frequency with which the subfunctionwas used.

19. Time complexity  Example : 4x4 inverse transform and reconstruct on TRIMEDIA  Two case : inverse transform and reconstruct inverse transform only (no nonzero coefficient )

20. Time complexity  Case 1 : inverse transform and reconstruct

21. Time complexity  Case 2 : inverse transform

22. Time complexity  4 x 4 inverse transforms are 42165.7 ( 28266.6 luminance and 13899.1 chrominance, Mobile QP=21 )  42165.7 x 39 = 164462.3  242954.3 x 16 = 3887268.8  164462.3 + 3887268.8 = 5531731.1

23. Comparative analysis  Measure the cycles of 4 x 4 inverse transform and reconstruct on P3 Using the propose method 6.9 million cycles Using VTune Performance Analyzer 28.23 million cycles The ratio : 28.23 / 6.9 = 4.05

24. Comparative analysis

25.  The ratio is due to : Operation count table contains data for only fundamental operations. Overhead operations such as loop overhead, flow control, and boundary condition handling are not included. The software is designed so that the overhead due to instruction cache misses is negligible, hardware register counts are not exceeded and operation latency is hidden.

26. Comparative analysis  Through an analysis summary for the P3 that the theoretical estimates are approximately 2~6 times lower than the experimental results.  Specific factor depends mainly on the characteristics of the subfunction, such as the regularity of operations, amount of overhead.

27. Experimental analysis  Our experimental analysis shows that the time complexity of the decoder and its major subfunctions are strongly dependent upon the average bit rate of the coded bitstream  Optimality of the motion estimation and mode decision processes in the source encoder don’t have a significant impact on decoder complexity.

28. Experimental analysis  One of the most important pieces of information in the complexity analysis is the distribution of time complexity amongst subsystems Loop filtering 33% interpolation 25% Entropy decoding 13% inverse transforms and reconstruction 13%

29. Experimental analysis  The factors that can affect the complexity of each of subfunctions Inverse Quantization, Transforms, and Reconstruction  the number of blocks and macroblocks that contain nonzero coefficients. Bitstream Parsing and Entropy Decoding  bit rate: More time is spent with coefficients at higher bit rates

30. Experimental analysis  Interpolation :

31. Experimental analysis  Loop filter The most important factor in this variability is the percentage of edges that might be filtered due to the boundary strength determined for that edge.

32. Conclusions  Study the computational complexity of the H.264/AVC baseline decoder using both theoretical and experimental methods