Implementation, Performance Analysis & Comparison of H.264 and VP8 Submitted by: Keyur Shah ( ) Under guidance of Dr. K.R.Rao

Motivation: H.264 is the most widely-accepted video standard and has spawned a huge amount of software that implements it viz., JM, X264 and Ffmpeg. [3] H.264 is used in various applications like video broadcasting, video streaming, video conferencing, HDTV, Blu-ray Discs. VP8 is an open video compression format released by Google, originally developed by On2 Technologies.[25]GoogleOn2 Technologies VP8 offers significant gains in compression performance. VP8 is less compute intensive to decode. [8] VP8 is used as the codec for HTML5, video playback in Internet Explorer 9, Mozilla, Opera and Google Chrome. Adobe Flash Player also supports VP8 playback. [17]

Goal of the Project (Comparison of VP8 and H.264): Feature (parametric) comparison: 1)Prediction schemes 2)Transform 3)In-loop filter 4)Entropy coding Performance comparison: 1) Bit rate 2) PSNR 3) MSE 4) SSIM [26] 5) Compression ratio 6) Complexity (encoding time and decoding time)

Why H.264? One of the most widely accepted video coding standards. Block-oriented motion-estimation based codec. Higher coding efficiency than previous standards, MPEG-1,2,4, H.261 and H.263. [28] Simple syntax specifications. Seamless integration of video coding into all current protocols. More error robustness.

Profiles of H.264: Table 1: H.264 profiles [3]

Specific coding schemes of profiles Fig 1: H.264 coding profile schemes [3]

WHY VP8?: Inheriting many great innovations from its predecessors (VP7 and VP6) such as golden frames, processor-adaptive real-time encoding and a low-complexity loop filter, VP8 adds more than fifty new techniques to achieve its goal of outstanding quality at low bitrates, with very low complexity. [6] VP8 specifies exact values for reconstructed pixels. This greatly facilitates the verification of the correctness of a decoder implementation as well as avoiding difficult-to-predict visual incongruities between such implementations. [6] VP8 offers both VBR (variable bitrate) and CBR (constant bitrate) encoding options. CBR attempts to keep the bitrate more constant, i.e. the codec tries to remain within given buffering constraints. If the user sets CBR mode but gives very loose buffer restrictions, then the result will start to resemble VBR.

VP8 has been designed with a wide range of machines in mind, from 60 MHz ARM9 processors to today's highly parallel multi-core processors. It encodes in real-time on low-end machines and takes fewer cycles to decode than other leading algorithms. [8] Fig 2: Parallel processing in VP8. [8]

Fig. 3: H.264 encoder block diagram. [3]

Fig. 5: H.264 decoder block diagram. [3]

Parametric Comparison (VP8 and H.264): 1)Intra Prediction: Intra prediction is used to guess the content of a block without referring to other frames. H.264 has three Intra-Prediction modes [3] 1) 4x4 luma prediction modes – 9 prediction modes : 8 Directional predictions and 1 DC prediction ( vertical : 0, horizontal : 1, DC : 2, diagonal down left : 3, diagonal down right : 4, vertical right : 5, horizontal down : 6, vertical left : 7, horizontal up : 8) Fig 6: 4x4 luma prediction modes. [3] 2) 16x16 luma prediction modes : 4 prediction modes ( vertical : 0, horizontal : 1, DC : 2, plane : 3) 3) 8x8 chroma prediction mode: S imilar to 4x4 luma with low pass filtering of the predictor to improve prediction performance. a b c d e f g h i j k l m n o p ABCD I J K L MEFGH mode 1 mode 6 mode 0 mode 5 mode 4 a b c d e f g h i j k l m n o p ABCD I J K L MEFGH mode 8 mode 3 mode 7

VP8 has four Intra-Prediction modes [22] 1) H_PRED (horizontal prediction): Fills each column of the block with a copy of the left column, L. 2) V_PRED (vertical prediction): Fills each row of the block with a copy of the above row, A. 3) DC_PRED (DC prediction): Fills the block with a single value using the average of the pixels in the row above A and the column to the left of L. 4) TM_PRED (TrueMotion prediction): A mode that gets its name from a compression technique developed by On2 Technologies. In addition to the row A and column L, TM_PRED uses the pixel C above and to the left of the block. Horizontal differences between pixels in A (starting from C) are propagated using the pixels from L to start each row. Fig 7: VP8 intra prediction modes. [22]

2) Inter Prediction: Inter prediction is used to guess the content of a block by referring to past and/or future frames. There are two primary components to inter prediction: reference frames and motion vectors. The reference frame is a past frame from which to grab pixels from and the motion vectors index an offset into that frame. H.264 has three frames: I-frames, P-frames and B-frames [3] 1) The first image in a video sequence is always an I-frame. 2) A P-frame, which stands for predictive inter frame. 3) A B-frame, or bi-predictive inter frame. [6] VP8 has two frames common with H.264, I-frames and P-frame but B-frames are missing here, instead there are altref frames (alternative reference). VP8 also has alternate prediction frames, called golden frames.

3) Transform and Quantization: A) Transform [7] The general method for transformation in both H.264 and VP8 is the same, each 16×16 macroblock is divided into sixteen 4×4 DCT blocks, each of which is transformed by a bit-exact DCT approximation. The main difference is that in VP8, DC coefficients of these bit-exact DCT blocks are collected into another 4×4 group, which are then Walsh-Hadamard transformed. [27] B) Quantization [7] The general method here also remains the same but there is a small difference again, H.264 has a built-in macroblock-level quantizer (adaptive quantization) whereas VP8 does not have it.

4) Entropy Coding: Entropy coding is the process of taking all the information from all the other processes: DCT coefficients, prediction modes, motion vectors, and so forth — and compressing them lossless into the final output file. VP8 and H.264 use an arithmetic coder for this purpose. The difference is H.264 uses an adaptive arithmetic coder whereas VP8 uses non-adaptive arithmetic coder. [7] 5) Loop Filter: The loop filter is run after decoding or encoding a frame and serves to perform extra processing on a frame, usually to remove blockiness in DCT-based video formats. They have same kind of loop filter with two small differences, 1) VP8 has two modes in loop filtering: a simple mode and a normal mode. 2) VP8 filter has wider range when filtering between macroblocks as compared to the H.264 filter. [7]

Implementation of the Project (Comparison of Parameters for VP8 and H.264): Performance Comparison: 1) Bit Rate 2) PSNR 3) MSE 4) SSIM [21] 5) Compression Ratio 6) Complexity (encoding time and decoding time)

Test Sequences: The test sequences that can be used are in QCIF and CIF (formats) 1) CIF 2) QCIF Fig(a) : Common Intermediate Format(CIF) 4:2:0 chroma sampling. [3] Fig (b): Quadrature Common Intermediate Format (QCIF) 4:2:0 chroma sampling. [3]

Software that can be used: 1) Softwares for H.264: ffmpeg: Fast Forwarding Mpeg, version [18] ffmpeg is a free software / open source project that produces libraries and programs for handling multimedia data. [30] JM Software: Current version 17.2 [23] JM software is the most widely used software for testing H.264/10 AVC. 2) Softwares for VP8: libvpx: The VP8 codec SDK, version [6] libvpx is a free software/ open source project developed by On2 Technologies to test VP8 codec. ffvp8: ffmpeg team’s own VP8 decoder, version [18], [31] ffvp8 is a free software/ open source project. It is ffmpeg software itself but the latest version (mentioned above) has included libvpx libraries (encoder and decoder). [30], [31]

REFERENCES: [1] T. Wiegand and G. J. Sullivan, “The H.264 video coding standard”, IEEE Signal Processing Magazine, vol. 24, pp , Mar [2] D. Marpe, T. Wiegand and G. J. Sullivan, “The H.264/MPEG-4 AVC standard and its applications”, IEEE Communications Magazine, vol. 44, pp , Aug [3] S. K. Kwon, A. Tamhankar and K. R. Rao, “Overview of H.264/MPEG-4 Part 10” J. Visual Communication and Image Representation, vol. 17, pp , Apr [4] A. Puri, X. Chen and A. Luthra, “Video coding using the H.264/MPEG-4 AVC compression standard”, Signal Processing: Image Communication, vol. 19, pp , Oct [5] Thesis%20Project%20table%20docs/Padiathesis. pdfThesis%20Project%20table%20docs/Padiathesis. pdf – “Complexity reduction for VP6 to H.264 transcoder using motion vector reuse”, M.S. Thesis, EE Dept., UTA, May2010. [6] VP8 Data Format and Decoding Guide WebM Project Google On2,

[7] The first in-depth technical analysis of VP8. [8] VP8-video-codec-High-compression-low-complexity- EE Times Design: The VP8 video codec: High compression + low complexity. VP8-video-codec-High-compression-low-complexity- [9] G. J. Sullivan, “ The H.264/MPEG-4 AVC video coding standard and its deployment status”, SPIE/VCIP 2005, vol.5960, pp , Beijing, China, July [10] K. Onthriar, K. K. Loo and Z. Xue, “Performance comparison of emerging Dirac video codec with H.264/AVC”, IEEE International Conference on Digital Telecommunications, vol. 06, page: 22, issue: 29-31, Aug [11] - Video test sequenceshttp://trace.eas.asu.edu/yuv/index.html (YUV 4:2:0) [12] - JM software source codehttp://iphome.hhi.de/suehring/tml/ [13] I.E.Richardson, “The H.264 Advanced Video Compression Standard”, 2 nd Edition, Wiley, 2010.

[14] - VP8 Video Codec Library. [15] Explore the WebM Source Code for VP8. [16] - VP8: The Savior Codec. [17] - General VP8 information. [18] - FFmpeg software. [19] - General HTML5 information. [20] - The SSIM Index for Image Quality Assessment. [21] - VP8 encoder and decoder explanation. [22] - Inside WebM Technology: VP8 Intra and Inter Prediction. [23] - JM Software Download. [24] % 20Manual%20(JVT-AE010).pdf – JM Software Installation Guide. % 20Manual%20(JVT-AE010).pdf [25] – On2 Technologies (VP8 developers).

[26] Z. Wang, et al, “Image Quality Assessment: From Error Visibility to Structural Similarity”, IEEE Transactions on Image Processing, vol. 13, no. 4, Apr [27] - General information on Walsh Hadamard Transform. [28] StandardsH261H263.pdf – H.261 and H.263 overview StandardsH261H263.pdf [29] - Latest ffmeg software for win32 and win64. [30] - ffvp8 and libvpx comparison.