O BJECTIVE P ERFORMANCE E VALUATION OF THE HEVC M AIN S TILL P ICTURE P ROFILE EE5359 Multimedia Interim Report Under Guidance of Dr. K.R.Rao By Deepu Sleeba Philip

1. A CRONYMS A ND A BBREVIATIONS  AMVP: Advanced motion vector prediction  AVC: Advanced Video Coding  BD-PSNR: Bjontegaard metric calculation  CB: Coding Block  CIF: Common Intermediate Format  CU: Coding Unit  CTB: Coding Tree Block  CTU: Coding Tree Unit  DCT: Discrete Cosine Transforms  DST: Discrete Sine Transform  EBCOT: Embedded block coding with optimized truncation  GIF: Graphics interchange format  HD: High Definition  HEVC: High Efficiency Video Coding  JCT-VC: Joint Collaborative Team on Video Coding  MC: Motion Compensation  ME: Motion Estimation  MPEG: Moving Picture Experts Group

 MSP: Main Still Picture Profile  MV: Motion Vector  NGOV: Next Generation Open Video  PNG: Portable Network Graphic  PSNR: Peak Signal To Noise Ratio  PU: Prediction Unit  QP: Quantization Parameter  QCIF: Quarter Common Intermediate Format  RD: Rate Distortion  SAO: Sample Adaptive Offset  SAD: Sum of Absolute Differences  SATD: Sum of Absolute Transformed Differences (SATD)  SHVC: Scalable HEVC  SSIM: Structural Similarity  SVC: Scalable Video Coding  TU: Transform Unit  URQ: Uniform Reconstruction Quantization  VCEG: Video Coding Experts Group

O BJECTIVE  Rate-distortion performance analysis of the HEVC MSP profile in comparison to WebP  The peak-signal-to-noise ratio (PSNR) and the average bit rate savings in terms of Bjøntegaard delta rate(BD) is considered  The implementation complexity will be evaluated based on the encoding time.

O VERVIEW OF C OMPARISON S CHEMES  JPEG  Block based transform coding approach  Step1: Picture is partitioned into non- overlapping 8x8 blocks.  Step2:Each of the block is transformed into freq domain by 2-D DCT. -transmitted by using entropy coding based on Huffman Codes.

 JPEG XR  -The image is divided into 4X4 blocks for transform coding stage.  -An integer approximation of 4x4 DCT is used(Photo Core Transform)  -Adaptive Huffman Coding is used for entropy coding  JPEG 2000  -Multi resolution approach based on wavelet transform.  Context Adaptive arithmetic coding

HEVC  Initial goal 50% reduction in bit rate at the same quality compared to H.264  Better coding efficiency – more than 35% reduction in bit rate.  Support of stereo and multiview  Increased use of parallel processing  Resolution :  HEVC: up to 8k UHD(8192X4320)  H.264: up to 4K UHD (4096X2160)

E NCODER -D ECODER HEVC B LOCK D IAGRAM Figure.1 [3]

I NTRA PREDICTION MODES IN HEVC  HEVC has 35 luma intra prediction modes (Figure 2)  Intra prediction can be done at different block sizes, ranging from 4 X 4 to 64 X 64 (whatever size the PU has) (Figure 3)  HEVC also includes a planar and DC intra prediction modes Figure 3:Luma intra prediction modes for different PU sizes in HEVC [8] Figure 2: Modes and directional orientations for intra picture prediction for HEVC [1]

W EB P  It is based on the intra-frame coding of the VP8 video format [15]  It is a block-based transformation scheme with eight bits of color depth and a luminance- chrominance model with chroma sub sampling by a ratio of 1:2 (YCbCr4:2:0)

VP8 E NCODER BLOCK DIAGRAM Figure.4 [6]

I NTRA PREDICTION MODES IN WEBP  WebP has three types of blocks:  4x4 luma  16x16 luma  8x8 chroma Modes:  H_PRED (horizontal prediction): Fills each column of the block with a copy of the left column, L.  V_PRED (vertical prediction) : Fills each row of the block with a copy of the above row, A.

 DC_PRED (DC prediction): Fills the block with a single value using the average of the pixels in the row above A and the column to the left of L[16].  TM PRED (True Motion prediction): In addition to the row A and column L, TM_PRED uses the pixel C above and to the left of the block. Horizontal differences between pixels in A and vertical differences between pixels in L are propagated (starting from C) to form the prediction block.

X ij = L i + A j - C (i,j=0, 1, 2, 3). Figure 4a: Illustration of intra prediction mode TM_PRED [16]

A NIMATED IMAGE  This small gif image of 279x193, this short video weighs as much as 1.5MB.  The WebP animation above is generated on-the-fly by Cloudinary when the dynamic URL is accessed.  It looks the same as the original image, but weighs only 419KB. This means we saved 72% of file size, bandwidth and load time, compared to the original 1.5MB animated GIF. Figure 5. Bored_animation.gif [32]

BPG/JPEG COMPARISON ON THE L ENA PICTURE [33]

WEBP VS BPG[33]

I MAGE Q UALITY M EASUREMENT  The compression quality can be measured by  Objective quality measure- PSNR, MSE  Structural quality measure- SSIM  MSE and PSNR for a NxM pixel image are defined as (1) (2) dB0 where x is the original image and y is the reconstructed image. M and N are the width and height of an image and ‘L’ is the maximum pixel value in the NxM pixel image.

I MPLEMENTATION  For comparison purpose, open-source implementations of the reviewed codecs will be used. HEVC compression efficiency will be measured with the HM Test Model [12]. WebP is downloaded from [19].  The rate distortion is compared in HEVC and WebP for MSP is done by plotting graphs for PSNR and BD rate.  The implementation complexity will be evaluated based on the encoding time.

R ESULTS

Reference picture: barbara.jpg(512x512) HEVC-Main Still Picture Profile WebP QPQP PSNR in dB Bit rate in kbps Bits per Pixel Encodin g time in seconds Q PSNR in dB Bit rate in kbps Bits per Pixel Encodin g time in seconds Table.1

Figure.5 Encoding Time vs Bpp Plot for Barbara.jpg(512x512)

Figure.6 Encoding Time vs Quantization Parameter Plot for Barbara.jpg (512x512)

Figure.7 PSNR vs Bpp plot for Barbara.jpg(512x512)

Reference picture: lena.jpg(512x512) HEVC-Main Still Picture ProfileWebP QP PSNR in dB Bit rate in kbps Bits per Pixel Encoding time in seconds QPSNR in dB Bit rate in kbps Bits per Pixel Encoding time in seconds Table. 2

Figure.8 Encoding Time vs Bpp Plot for Lena.jpg (512x512)

Figure.9 Encoding Time vs Quantization Parameter Plot for Lena.jpg (512x512)

Figure.10 PSNR vs Bpp plot for Lena.jpg (512x512)

T EST IMAGES

Barbara.jpg(512x512) Lena.jpg(512x512)

T EST S OFTWARES /C ONFIGURATIONS  HM16.5  libwebp  ImageMagick  Intel Core i5 processor,1 Tb hard disk, 8GB RAM  Microsoft Visual Studio 2012

I NPUT COMMAND LINE PARAMETERS FOR HEVC

I NPUT COMMAND LINE PARAMETERS FOR W EB P

R EFERENCE  [1] Joint Collaborative Team On Video Coding Information website /16/Pages/video/jctvc.aspx /16/Pages/video/jctvc.aspx  [2] H.261: Video Codec for Audiovisual Services at px64 kbit/s,”  [3] G. J. Sullivan et al, “Overview of the High Efficiency Video Coding (HEVC) Standard”, IEEE Transactions Circuits and Systems for Video Technology, Vol. 22, No. 12, pp , Dec  [4] N. Ahmed, T. Natarajan and K.R. Rao, “Discrete Cosine Transform”, IEEE Transactions on Computers, Vol. C-23, pp , Jan  [5] P.K Ranjan, D. Pacharla, B. Ravindran and D. Mani, "Quality evaluation of HEVC Main Still Picture with limited coding tree depth and intra modes", Advances in Computing, Communications and Informatics, New Delhi.  [6] S. Bultje and M. Frost, Access website PPT on “WebM and the new Open Video Codec”.

 [7] M. Budagavi and V. Sze, content/uploads/2014/06/H.265-HEVC-Tutorial-2014-ISCAS.pdf, " Design and Implementation of Next Generation Video Coding Systems (H.265/HEVC Tutorial)".  [8] C Kodpadi,"Comparative study of Intra Frame Coding efficiency in HEVC and VP9" EE5359, UTA, spring 2014  [9] J. Bankoski et al, “Towards a Next Generation Open source Video Codec” SPIE, Vol. 8666, Page 2, Dec  [10] D. Grois et al, “Performance Comparison of H.265/ MPEG-HEVC, VP9, and H.264/MPEGAVC Encoders”, IEEE PCS 2013, pp , San José, CA, USA, Dec 8-11, 2013  [11] M.P. Sharabayko et al, "Intra Compression Efficiency in VP9 and HEVC" Applied Mathematical Sciences, Vol. 7, no. 137, pp.6803 – 6824, Hikari Ltd, 2013  [12] HM Reference Software-  [13]Software manual : e-manual.pdf

 [14]G. Bjøntegaard, “Calculation of average PSNR differences between RD-curves”, ITU-T Q.6/SG16 VCEG 13th Meeting, Document VCEG- M33, Austin, USA, Apr  [15]F. Liang, X. Peng and J. Xu, "A light weight HEVC Encoder for Image Coding"MSRA-MOE joint key lab, Univ. of Sci and Technology of China, Hefei China  [16] J. Bankoski, P. Wilkins and X. Yaowu, "Technical Overview of VP8,an open source video codec for the web", International conference on Multimedia and Expo,pages:1-6, 2011  [17] T. Nguyen and D. Marpe, "Objective Performance Evaluation of the HEVC Main Still Picture Profile" IEEE Transactions on circuits and systems for video technology, page:1,Sep2014  [18] “The WebM Project.” [Online]. Available:  [19] “WebP Google Developers.” [Online]. Available: com/speed/webp/  [20] “Kodak Lossless True Color Image Suite.” [Online]. Available:  [21]P.K. Bansal, M.N. Shukla and A.S. Motra, "VP8 Encoder-Cost effective implementation", SoftCOM, pages(1-6), 2012

 [22] Z. Xiong et.al, “A comparative study of DCT- and wavelet-based image coding,” IEEE Transactions on Circuits and Systems for Video Tech., vol.9, pp , Aug  [23] Visual studio download for students for free-  [24] Tortoise SVN download-  [25]MPL Website-  [26] K.R. Rao, D.N. Kim and J.J. Hwang, “Video Coding Standards: AVS China, H.264/MPEG-4 Part 10, HEVC, VP6, DIRAC and VC-1”, Springer,  [27] : program for converting to YUV format.  [28] V. Sze, M. Budagavi and G. J. Sullivan (Editors), "High Efficiency Video Coding (HEVC): Algorithms and Architectures," Springer,2014  [29] Iain Richardson/Vcodex.com “HEVC An Introduction to High Efficiency Video Coding” 2013  [30] Iain Richardson, “Video Codec Design: Developing Image and Video Compression Systems”, Wiley,  [32] 4/#image_sets, Lossy compressed image formats study, Mozilla corporation July 2014http://people.mozilla.org/~josh/lossy_compressed_image_study_july_201 4/#image_sets

T HANK YOU