1. FINAL REPORT ON PERFORMANCE COMPARISON OF HEVC,H.264 and VP9
A PROJECT UNDER THE GUIDANCE OF
DR. K. R. RAO
COURSE: EE MULTIMEDIA PROCESSING,
SPRING 2015 By
DEEPIKA SREENIVASULU PAGALA

2. OBJECTIVE:
The objective of this project is to study video coding standards HEVC [1][34][35][36], H.264 [2] [34] and VP9 [3][4] and understand various techniques in video coding such as prediction, transform, quantization and coding.
A performance comparison of these video codecs based on various metrics such as computational time, PSNR [25], SSIM [5][20][31], BD-Bitrate [6] and BD-PSNR [7] will be carried out.
The HM 16.0 [26][33], JM 18.6 [27][32] and VPX encoder [28] from The WebM Project test models for HEVC, H.264 and VP9 respectively will be used for this purpose.

3. EVOLUTION OF VIDEO CODING STANDARDS [8]:

4. GROWING DEMAND FOR VIDEO [30]:
Video exceeds half of internet traffic and will grow to 86 percent by 2016 [30].
Increase in applications, content, fidelity, etc. -Need higher coding efficiency! [30].
Ultra-HD 4K broadcast expected for Japan in London Olympics Opening and Closing Ceremonies shot in Ultra-HD 8K. - Need higher throughput! [30].
25x increase in mobile data traffic over next five years. Video is a “must have” on portable devices. - Need lower power! [30].

5. Fundamental Concepts in Video Coding [10]:
Color Spaces
RGB color space – Each pixel is represented by three numbers indicating the relative proportions of red, green and blue colors.
YCrCb color space – Y is the luminance component and Cr ,Cb - two color difference or chrominance components.
As the human visual system is less sensitive to color than the luminance component, YCrCb has advantages over RGB space.

6. H.264/MPEG-4 [2][9]:
H.264/Advanced Video Coding (AVC) is video coding standard of the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group [2].
Most popular video standard- 80% of video is encoded on internet with H.264/AVC [9].
Developed for multimedia applications [9].
Adopted advanced coding techniques such as multiple-reference frame prediction, and context-based adaptive binary arithmetic coding (CABAC).

7. ENCODER IN H.264 [11]:

8. DECODER IN H.264 [11]:

9. FEATURES OF H.264: Prediction [12]:
Fig 5: Inter prediction in H.264 [12]
Fig 4: Intra prediction in H.264 [12]

10. FEATURES OF H.264(cotd..): Transform and Quantization [13]:
A block of residual samples is transformed using a 4x4 or 8x8 integer transform, an approximate form of the Discrete Cosine Transform (DCT) . The transform outputs a set of coefficients, each of which is a weighting value for a standard basis pattern. The output of the transform, a block of transform coefficients, is quantized, i.e. each coefficient is divided by an integer value .
Entropy Coding [15]:
H.264 uses CABAC(Context Adaptive Binary Arithmetic Coding )
or CAVLC(Context Adaptive Variable Length Coding) for entropy coding[14].

11. HEVC/H.265 [1]:
High Efficiency Video Coding (HEVC) is an international standard for video compression developed by a working group of ISO/IEC MPEG (Moving Picture Experts Group) and ITU-T VCEG (Visual Coding Experts Group).
High Efficiency Video Coding (HEVC) is the latest Video Coding format .
It challenges the state-of-the-art H.264/AVC Video Coding standard which is in current use in the industry by being able to reduce the bit rate by 50% and retaining the same video quality [1].

12. Fig 6 : Block Diagram of HEVC Encoder [17]
Encoder in H.265 [19][20]:
Fig 6 : Block Diagram of HEVC Encoder [17]

13. Fig 7 : Block Diagram of HEVC Decoder [21]
DECODER IN H.265 [19][21]:
Fig 7 : Block Diagram of HEVC Decoder [21]

14. FEATURES OF HEVC: Partitioning [19]:
Fig 8: Picture, Slice, Coding Tree Unit (CTU), Coding Unit (CU) [19]

15. FEATURES OF HEVC(cotd..):
Prediction [1] :
- Intra prediction: Each CU is predicted from neighboring image data in the same picture, using DC prediction, planar prediction or directional prediction.
Inter Prediction :Each PU is predicted from image data in one or two reference pictures, using motion compensated prediction.
Transform and Quantization [19]:
Any residual data remaining after prediction, is transformed using a block transform based on the Discrete Cosine Transform (DCT). The transformed data is quantized. One or more block transforms of size 32x32, 16x16, 8x8 and 4x4 are applied to residual data in each CU.
Entropy Coding:
- HEVC uses Context Adaptive Binary Arithmetic Coding (CABAC) [14] for Entropy coding.

16. Vp9 [3][4]:
VP9 is an open and royalty free video compression standard being developed by Google
VP9 is a successor to VP8.
One of the goals of VP9 is to reduce the bit rate by 50% compared to VP8 while having the same video quality [22].
VP9 has many design improvements compared to VP8.
VP9 supports the use of superblocks of 64x64 pixels.

17. ENCODER INVP9 [24]:
Fig 9: Encoder block diagram for VP9 [24]

18. DECODER IN VP9 [24]:
Fig 10: Decoder block diagram for VP9 [24]

19. Fig 11: Example partitioning of a 64x64 Super-block [4] [23]
FEATURES OF VP9:
Prediction block sizes [4] [23]:
Fig 11: Example partitioning of a 64x64 Super-block [4] [23]

20. FEATURES OF VP9(cotd..) Prediction [4]:
- Intra Prediction : VP9 supports a set of 10 Intra prediction modes for block sizes ranging from 4x4 up to 32x32
- Inter Prediction : VP9 supports a set of 4 inter prediction modes for block sizes ranging from 4x4 up to 64x64 pixels
Transform and Quantization [4]:
The residuals after subtraction of predicted pixel values are subjected to transformation and quantization . Transform blocks can be 32x32, 16x16, 8x8 or 4x4 pixels.
Entropy coding [4]:
- VP9 uses 8-bit arithmetic coding engine from VP8 known as bool-coder

21. COMPARISON METRICS: Peak Signal to Noise Ratio (PSNR) [25]
Structural Similarity Index (SSIM) [5][20][31]
BD-Bitrate and BD-PSNR [6]
Computational time as a measure of implementation complexity

22. PROFILES USED FOR COMPARISON:
The HM 16.2 [26][33] ,JM 18.6 [27] [32] and VPX encoder [28] from The WebM Project test models for HEVC , H.264 and VP9 respectively will be used for comparison in this project.

23. Fig.12 akiyo_qcif.yuv(176x144) [38]
TEST SEQUENCES :
Fig.12 akiyo_qcif.yuv(176x144) [38]

24. TEST SEQUENCES (Cotd..):
Fig.13 waterfall_cif.yuv(352x288) [38]

25. TEST SEQUENCES (Cotd..):
Fig.14 BasketballDrill_832x480.yuv [39]

26. TEST SEQUENCES (Cotd..):
Fig 15 :Jockey_1920x1080.yuv [29]

27. TEST SEQUENCES (Cotd..):
Fig 16: PeopleOnStreet_2560_1600_30_crop.yuv [29]

28. Configuration of HM 16.2 Main all-intra profile settings :
IntraPeriod : 1 # Period of I-Frame ( -1 = only first)
QP : 22 # Quantization parameter(0-51) (22, 27, 32 or 37 is used at a time)
Command line parameters for using HM 16.2 encoder:
TAppEncoder [-h] [-c config.cfg] [--parameter=value] Options:
-h Prints parameter usage
-c Defines configuration file to use. Multiple configuration files may be used with repeated –c options.
--parameter=value Assigns value to a given parameter.
Sample command line parameters for HM 16.2 encoder:
C:\HEVC\bin\vc10\Win32\Release>TAppEncoder.exe -c encoder_intra_main.cfg -wdt hgt fr 30 -f 20 -i C:\HEVC\bin\vc10\Win32\testsequences >> C:\HEVC\bin\vc10\Win32\final_results\ PeopleOnStreet_2560x1600_30_crop_qp37.txt

29. Configuration of JM 18.6 High all-intra profile settings :
FramesToBeEncoded : 20 # Number of frames to be coded
FrameRate : 50.0 # Frame Rate per second ( )
ProfileIDC : 100 # Profile IDC (66=baseline, 77=main, 88=extended; FREXT Profiles: 100=High, 110=High 10, 122=High 4:2:2, 244=High 4:4:4, 44=CAVLC 4:4:4 Intra, 118=Multiview High Profile, 128=Stereo High Profile)
IntraProfile : 1 # Activate Intra Profile for FRExt (0: false, 1: true)
LevelIDC : 40 # Level IDC (e.g. 20 = level 2.0)
IntraPeriod : 1 # Period of I-pictures (0=only first)
IDRPeriod : 1 # Period of IDR pictures (0=only first)
QPISlice : 22 # Quant. param for I Slices (0-51) (22, 27, 32 or 37 is used at a time)

30. Configuration of JM 18.6 (cotd..)
Command line parameters for using JM 18.6 encoder:
Options:
-h Prints parameter usage.
-d Use <defenc.cfg> as default file for parameter initializations. If not used then file defaults to “encoder.cfg” in local directory.
-f Read <curencM.cfg> for resetting selected encoder parameters. Multiple files could be used that set different parameters.
-p Set parameter <EncParamM> to <EncValueM>. The entry for <EncParamM> is case insensitive.
Sample command line parameters for JM 18.6 encoder:  
C:\jm 18.6\JM\bin>lencod.exe –f encoder.cfg -p InputFile=" C:\jm 18.6\JM\bin\ Jockey_1920x1080.yuv" –p SourceWidth=832 -p SourceHeight=480 >> C:\jm 18.6\JM\bin\final_results\ jockey_37.txt

31. Configuration of VP9 Sample command line parameters for VP9:
vpxenc akiyo_qcif.y4m ­o akiyo_22.webm \­­codec=vp9 ­­i420 ­­width=174 ­­height=144 ­­passes=2 ­t 0 \­­good ­­cpu-used=0 ­­end­usage=q \­­limit=25 ­­fps=15000/1001 ­­verbose ­­psnr \­­lag­in­frames=0 ­­kf-max­dist=1 \­­min­q=37 ­­max­q=37

32. Table 1 : Implementation results for akiyo_qcif.yuv sequence
Implementation results for akiyo_qcif.yuv , Number of Frames Encoded = 20
HEVC
H.264
VP9 QP
PSNR in dB
Bit Rate in Kbps
Encoding time in seconds 22
5.836
474.04
1.394
42.804
4.118 27
5.195
299.65
1.111
40.946
3.850 32
4.732
188.64
0.998
39.110
3.543 37
4.257
119.56
0.994
37.217
3.402
Table 1 : Implementation results for akiyo_qcif.yuv sequence

33. Results(cotd..)
Implementation results for waterfall_cif.yuv , Number of Frames Encoded = 20
HEVC
H.264
VP9 QP
PSNR in dB
Bit Rate in Kbps
Encoding time in seconds 22
31.461
6.508
39.108
20.283 27
25.583
5.424
37.204
18.439 32
21.392
4.467
35.262
16.740 37
19.569
3.911
33.515
15.791
Table 2 : Implementation results for waterfall_cif.yuv sequence

34. Results(cotd..)
Implementation results for BasketballDrill_832x480_50.yuv , Number of Frames Encoded = 20
HEVC
H.264
VP9 QP
PSNR in dB
Bit Rate in Kbps
Encoding time in seconds 22
22.822
40.374
79.937 27
94.157
17.026
38.795
69.721 32
75.877
14.691
37.229
60.568 37
67.456
14.375
35.812
56.062
Table 3 : Implementation results for BasketballDrill_832x480_50.yuv sequence

35. Results(cotd..)
Implementation results for Jockey_1920x1080.yuv , Number of Frames Encoded = 20
HEVC
H.264
VP9 QP
PSNR in dB
Bit Rate in Kbps
Encoding time in seconds 22
71.600
43.641 27
63.995
42.705 32
59.007
41.623 37
56.011
40.5
Table 4 : Implementation results for Jockey_1920x1080.yuv sequence

36. Results(cotd..)
Implementation results for PeopleOnStreet_2560_1600_30_crop, Number of Frames Encoded = 20
HEVC
H.264
VP9 QP
PSNR in dB
Bit Rate in Kbps
Encoding time in seconds 22
41.945 27
40.385 32
38.79 37
37.309
Table 5 : Implementation results for PeopleOnStreet_2560_1600_30_crop.yuv sequence

37. Results(cotd..)
Fig 17: R-D Plot for akiyo_qcif.yuv sequence

38. Results(cotd..)
Fig 18: R-D Plot for waterfall_cif.yuv sequence

39. Results(cotd..)
Fig 19: R-D Plot for BasketballDrill_832x480_50. yuv sequence

40. Results(cotd..)
Fig 20: R-D Plot for Jockey_1920x1080. yuv sequence

41. Results(cotd..)
Fig 21: R-D Plot for PeopleOnStreet_2560_1600_30_crop.yuv sequence

42. Results(cotd..)
Fig 22: Comparison of encoding time in all-intra configuration for akiyo_qcif.yuv sequence

43. Results(cotd..)
Fig 23 : Comparison of encoding time in all-intra configuration for waterfall-cif.yuv sequence

44. Results(cotd..)
Fig 24 : Comparison of encoding time in all-intra configuration for BasketballDrill_832x480_50.yuv sequence

45. Results(cotd..)
Fig 25 :Comparison of encoding time in all-intra configuration for Jockey_1920x1080.yuv sequence

46. Results(cotd..)
Fig 26 : Comparison of encoding time in all-intra configuration for PeopleOnStreet_2560_1600_30_crop.yuv sequence

47. Results(cotd..)
Fig 27: Comparison of HEVC and H.264 based on BD-Bitrate

48. Results(cotd..)
Fig 28: Comparison of HEVC and VP9 based on BD-Bitrate

49. Results(cotd..)
Fig 29: Comparison of HEVC and H.264 based on BD-PSNR

50. Results(cotd..)
Fig 30: Comparison of HEVC and VP9 based on BD-PSNR

51. CONCLUSIONS
The aim of this project is to experiment and present a thorough study, implementation and exhaustive comparison of intra-frame only of the contending video coding standards HEVC , H.264 and VP9 based on different performance metrics like PSNR, BD Metrics
etc. for different video coding sequences [29] [38] [39].
VP9 needs 33.57% more bit rate than HEVC and H.264 needs 47.05% more bit rate than HEVC for the same PSNR.
HEVC has 27% - 40% more encoding time than VP9 and VP9 has 100% more encoding time than H.264.

52. ACKNOWLEDGEMENT
I would sincerely like to thank Dr. K. R. Rao for his constant support and guidance throughout the duration of my project.
I would also like to thank Dr. Debargha Mukherjee , Member of Technical Staff, Google (Chrome), Mountain View , CA , for assisting me in my project.

53. List of Acronyms and Abbreviations:
ADST: Asymmetric Discrete Sine Transform.
AHG: Ad Hoc Group.
JCT-VC: Joint Collaborative Team on Video Coding.
AVC: Advanced Video Coding.
JM: H.264 Test Model.
BD-BR: Bjontegaard Delta Bitrate.
BD-PSNR: Bjontegaard Delta Peak Signal to Noise Ratio.
JPEG: Joint Photographic Experts Group.
CABAC: Context Adaptive Binary Arithmetic Coding.
KTA: Key Technical Areas (H.264 based exploration software of VCEG)
CAVLC: Context Adaptive Variable Length Coding.
MC: Motion Compensation.
CTB: Coding Tree Block.
ME: Motion Estimation.
CTU: Coding Tree Unit.
MPEG: Moving Picture Experts Group.
CU: Coding Unit.
MSE: Mean Square Error.
DBF: De-blocking Filter.
MVC : Multiview Video Coding..
DCT: Discrete Cosine Transform.
NGOV: Next Generation open Video
DST :Discrete Sine Transform.
PB: Prediction Block.
DPB :Decoded Picture Buffer.
PCS : Picture Coding Symposium
DVD: Digital Video Disk.
PSNR: Peak Signal to Noise Ratio.
HD: High Definition.
PU: Prediction Unit
HDR :High Dynamic Range.
QP: Quantization Parameter
HEVC: High Efficiency Video Coding.
RD: Rate Distortion
HM: HEVC Test Model.
SAO: Sample Adaptive Offset.
ICME: International Conference on Multimedia and Expo.
SCC: Screen Content Coding.
IEC: International Electro-technical Commission.
SSIM: Structural Similarity Index.
ISCAS : International Symposium on Circuits and Systems.
TB: Transform Block.
ISO: International Organization for Standardization.
TU: Transform Unit.
ITU-T: International Telecommunication Union- Telecommunication Standardization Sector.
VCEG: Visual Coding Experts Group.
WCG: Wide Color Gamut.
JCT: Joint Collaborative Team.

54. REFERENCES:
[1] G. J. Sullivan et al, “Overview of the high efficiency video coding (HEVC) standard”, IEEE Transactions on circuits and systems for video technology, vol. 22, no.12, pp – 1668, Dec 2012.
[2] JVT Draft ITU-T recommendation and final draft international standard of joint video specification (ITU-T Rec. H.264-ISO/IEC AVC), March 2003, JVT-G050 available on
[3] D. Grois et al, “Performance Comparison of H.265/ MPEG-HEVC, VP9, and H.264/MPEG-AVC Encoders”, IEEE PCS 2013, pp , San José, CA, USA, Dec 8-11, 2013
[4] D. Mukherjee et al, “The latest open-source video codec VP9–An overview and preliminary results”, Google Inc., United States
[5] Z. Wang et al, “Image quality assessment: From error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp , Apr. 2004
[6] G. Bjøntegaard, “Calculation of average PSNR differences between RD-curves”, ITU-T Q.6/SG16 VCEG 13th Meeting, Document VCEG-M33, Austin, USA, Apr. 2001

55. REFERENCES(Cotd..):
[7] X. Li et al, “Rate-complexity-distortion evaluation for hybrid video coding”, IEEE International Conference on Multimedia and Expo (ICME), pp , July. 2010
[8] N. Ling, “High efficiency video coding and its 3D extension: A research perspective,” Keynote Speech, ICIEA, pp , Singapore, July
[9] V. sze , M. Budagavi , " Design and Implementation of Next Generation Video Coding Systems (H.265/HEVC Tutorial) " , IEEE ISCAS Tutorial 2014 , Melbourne , Australia , June filehttp://
[10] I. E. G. Richardson, “Video Codec Design: Developing Image and Video Compression Systems”, Wiley, 2002
[11] A. Puri , X.Chen and A.Luthara , “Video coding using the H.264/MPEG-4 AVC compression standard”, Signal Processing: Image Communication, vol. 19, pp , Oct
[12] H.264 tutorial by I.E.G. Richardson:
[13] N. Ahmed , T. Natarajan and K. R. Rao, “Discrete Cosine Transform”, IEEE Transactions on Computers, Vol. C-23, pp , Jan

56. REFERENCES(Cotd..):
[14] D. Marpe, H. Schwarz, and T. Wiegand, “Context-based adaptive binary arithmetic coding in the H.264/AVC video compression standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, pp. 620–636, Jul
[15] J . Ostermann, et al, " Video coding with H.264/AVC tools, performance , and complexity", IEEE Circuits and Systems Magazine , Vol.4 , pp.7-28, Aug.2004.
[16] J. Ohm , et al, "Comparison of the Coding Efficiency of Video Coding Standards - including High Efficiency Video Coding (HEVC) ", IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, Issue: 12 , pp , Dec.2012.
[17] G. J. Sullivan et al, “Standardized Extensions of High Efficiency Video Coding (HEVC)”, IEEE Journal of selected topics in Signal Processing, Vol. 7, No. 6, pp , Dec
[18] HEVC white paper -
[19] HEVC tutorial by I.E.G. Richardson:
[20] W. Malpica and A. Bovik , "Range image quality assessment by structural similarity", IEEE ICASSP 2009, Apr

57. REFERENCES(Cotd..):
[21] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T / ISO-IEC Document: JCTVC J0292r1, July 2012.
[22] "VP-Next Overview and Progress Update" (PDF). WebM Project (Google). Retrieved Available on:
[23] M. P. Sharabayko et al, "Intra Compression Efficiency in VP9 and HEVC" Applied Mathematical Sciences, Vol. 7, no. 137, pp.6803 – 6824, Hikari Ltd, 2013
[24] J. Padia, “Complexity reduction for VP6 to H.264 transcoder using motion vector reuse,” M.S. Thesis, EE Dept., UTA, Arlington, TX, Available on:
[25] White paper on PSNR-NI -
[26] Access to HM Reference Software:
[27] Access to JM 18.6 Reference Software:

58. REFERENCES(Cotd..):
[28] Chromium® open-source browser project, VP9 source code, Online:
[29] - Video test sequences (4kx2k)
[30] Cisco Visual Networking Index -
[31] J. Wang et al, "Fractal image coding using SSIM", IEEE 18th International Conference on Image Processing, pp , Brussels, Belgium , sept
[32] H.264/AVC Software Reference Manual:
[33] HEVC Software Reference Manual :

59. REFERENCES(Cotd..): [34] K. R. Rao, D. N. Kim and J. J. Hwang,
“VideoCodingStandards:AVSChina,H.264/MPEG-4 Part10, HEVC, VP6, DIRAC and VC-1 ”, Springer, 2014.
[35] V. Sze , M. Budagavi ,and G. J.Sullivan "High Efficiency Video Coding (HEVC) : Algorithms and Architectures", Springer, 2014.
[36] M. Wien, "High Efficiency Video Coding : Coding Tools and Specification" , Springer , 2014.
[37] I. E. Richardson , "Coding Video : A practical guide to HEVC and beyond " , Wiley , 11 May 2015.
[38] - test sequences
[39] ftp://ftp.kw.bbc.co.uk/hevc/hm-11.0-anchors/bitstreams/ - test sequences
[40] G. Correa , et al , " Fast HEVC Encoding Decisions Using Data Mining " , IEEE Transactions on Circuits and Systems for Video Technology , Vol . 25 , No. 4 , pp , April 2015.
[41] D. K. Kwon and M. Budagavi , " Combined scalable and multiview extension of High Efficiency Video Coding (HEVC) " , IEEE Picture Coding Symposium , pp , Dec
[42] Encoding Time Evaluation Intel VTune Amplilfier  XE  Software profiler
Available