Objective Video quality assessment of Dirac and H.265 SPRING 2016 INSTRUCTOR: Dr .K.R Rao. Satya sai krishna kumar Avasarala 1001090898

Acronyms AVC Advanced Video coding BBC-British Broadcast Corporation BD-Bjontegaard Delta CW-SSIM-Complex wavelet SSIM DVQ-Digital Video Quality. EPFL-PoLiMi - École Polytecnique Federale De Lausanne Politecnico milano FR-Full reference HDTV-High Definition television HEVC-High Efficiency Video Coding HM-HEVC Test Model HVS-Human Visual System IVC-Image and visual computing ISO-International Standards Organization IEC-International Electrotechnical Commission ITU-International Telecommunication Union JSVM-Joint Scalable Video Model LIVE-Laboratory for Image and Video Engineering. MAD-Mean Absolute Differences MOVIE-Motion Based Video Integrity Evaluation MMSP-Multimedia Signal Processing MPEG-Moving Picture Experts Group MSE –Mean squared error MOS –Mean Opinion of Scores.

NR-No Reference PSNR-Peak Signal to noise ratio PWSSIM- Perceptual Weighted Structural similarity QOE-Quality Of Experience SSIM-Structural similarity VCEG-Video Coding Experts Group. VQEG-Video quality experts group. QOS-Quality of Service. MOS –Mean Opinion of Scores. RR-Reduced Reference RAM-Random Access Memory STAQ-Spatio Temporal Assessment of quality STMAD-Spatio temporal MAD

MOTIVATION: An increasing demand for video. Increased used of applications, content, fidelity, etc. -Need higher coding efficiency. [1] 25 times increase in mobile data traffic over next five years. Video is a “must have” on portable devices. - Need lower power.[3] Network operators find very difficult to store huge amounts of data during transmission –therefore need for higher coding gain [6]

OBJECTIVE: The objective of this project is to study video coding standards DIRAC and H.265 and implement their performance in broadcasting environments especially in the areas of objective quality, delay, and complexity .[5] To evaluate the performance of these video coding standards various criteria are used such as PSNR, computational time, SSIM, BD-Bitrate and BD-PSNR.[6] The HM, Schroedinger softwares are used to evaluate various test sequences to determine the performance of HEVC,DIRAC respectively.

H.265/HEVC High Efficiency Video Coding (HEVC) is the latest video coding standard for video compression developed by ISO/IEC MPEG (Moving Picture Experts Group) in collaboration with ITU-T VCEG (Video Coding Experts Group). [3] It has higher compression efficiency than that of H.264/AVC Video Coding standard which is currently in use, by being able to reduce the bit rate by 50% and retaining the same video quality.[1]

H.265 ENCODER : Figure 2 : HEVC encoder block diagram [6]

HEVC DECODER: Figure 3 : HEVC decoder block diagram[7]

KEY FEATURES : Partitioning Prediction Transform and Quantization Entropy Coding [1]

DIRAC Dirac is open and royalty free video compression standard developed by BBC research group which was finalized in January 2008.[8] It is named after the two physicists Paul Dirac and   Erwin Schrödinger. It is used to provide high compression for the video resolutions of Ultra HDTV and beyond. This standard was first implemented by BBC to transmit HDTV pictures in Beijing Olympics in year 2008. Aimed to provide  significant savings in data rate and improvements in quality over video compression formats such as MPEG-2 Part 2, MPEG-4 Part 2.[10]

DIRAC ENCODER Figure 6 : Dirac encoder block diagram [1]

DIRAC DECODER Figure 7: Dirac decoder Block diagram [10]

Performance Metrics OBJECTIVE METRICS PSNR BD-PSNR BD-Rate SSIM PWSSIM SUBJECTIVE METRICS 1.MOS ( Mean Opinion of Scores)

where B= number of bits per sample MSE= Mean squared error PSNR – PSNR YUV is mostly used to evaluate the video quality for 4:2:0 format only.[21] -- (1) while the individual values for PSNR Y,PSNR U, PSNR V are calculated as follows [17] --(2) where B= number of bits per sample MSE= Mean squared error

BD-PSNR – This computes the average PSNR differences in dB for the same bit rate.[17] The average PSNR difference between two R-D curves is approximated by the difference between the integrals of the fitted R-D curves divided by the integration interval (delta D). BD –Bit rate – This computes the bit difference between the two R-D curves for a given bit rate.[17]

SSIM - It is calculated as the below formula [15] α, β, and γ are parameters that mediate the relative importance of the three components µx and µy = local sample means of x and y respectively σx and σy = local sample standard deviations of x and y respectively σxy = local sample correlation coefficient between x and y C1, C2, and C3 = constants that stabilize the computations when denominators become small

PWSSIM- It uses perceptual spatial information as a way of weighting the most important visual regions.[17] Spatial Information is calculated as follows. ---(10) PWSSIM is given by : ---(11) µs = mean of the gradient magnitude of a block N=Number of pixels in the block.

What Factors affect video quality? [27] Compression Transmission errors Display Reproduction systems Pre/post processing Many more..

Why go for Objective video quality assessment? Subjective video quality assessment methods are undoubtedly reliable methods than the objective methods.[28] Complexity is high in subjective methods. Need to follow strict evaluation conditions. Unable to provide instantaneous results. Due to all the above factors objective quality assessment algorithms have been developed.[27]

Objective quality methods [28] Media layer model: Uses speech or video signal to compute QoE. Parametric packet-layer model: These models predict QoE from the packet header information itself. Parametric planning model: These models make use of the quality planning parameters for networks and terminals to predict QoE. Bitstream layer model: These models use encoded bitstream information and packet layer information for predicting QoE. Hybrid model: These are combination of any two of the above models.

Objective quality methods Figure 8: Block diagram of objective quality assessment methods [27]

Media layer models [28] Figure 9: Block diagram of media layer models [28] (a)- Full reference model (b)-Reduced Reference model (c)-No reference model

Video Quality databases [21] VQEG FR-tc phase 1 IRCyN/IVC 1080i IRCCyN/IVC SD RoI EPFL-PoliMi LIVE video quality LIVE wireless video quality MMSP 3D video quality MMSP scalable video VQEG HDTV

VQA Metrics 1.MOVIE-Motion based video integrity evaluation Uses optical flow estimation to adaptively guide spatial-temporal filtering using 3D Gabor filter banks. [27] Computes the video quality as a combination of spatial MOVIE index and temporal MOVIE index . 2.DVQ-Digital video quality [28] Uses a pair of video sequences and computes a measure of the magnitude of the visible difference. Employs sampling, cropping and color transformations in the region of interest.

VQA Metrics contd. 3.V-factor- It is real time based VQM. Primarily used for video streaming over IP networks.[28] Uses transport stream headers, packetized elementary headers, video coding layer. 4.ST-MAD- Spatio temporal MAD. Spatio temporal slices of the image are constructed from time-based slices of the reference and distorted videos.[27] Performs better than other metrics in LIVE video quality database.

VQA Metrics contd. 5.STAQ-Spatial temporal assessment of quality. Uses both spatial and temporal parts of the video. Uses CW-SSIM to compute the motion vectors instead of MAD. Video quality is estimated from the values obtained from spatial and temporal domains. QoE is introduced as a function of motion activity.[28]

PROFILES USED FOR ASSESSMENT The HM 16.3 [16] and Schroedinger 1.0.11 [20] are the softwares that will be used for HEVC and Dirac respectively in this project. Various test sequences will be encoded in this project with the necessary profile settings to get the results.

Results Test sequence 1: QP=32 Bus_176x144_25.yuv

QP=32 but with random access _main10

Sequence QP Bit rate(Kbps) Time(Secs) No.of frames YUV-PSNR(dB) Bus.yuv 32 119.919 152.356 150 30.0015 Bus.yuv_main10 119.708 158.802 30.0129 10 1836.453 423.379 48.2606 Table 1:Bus.yuv metrics summary

Test sequence 2: QP=32 Foreman_352x288_30.yuv

QP=10 Sequence QP Bit rate(kbps) Time(secs) No.of frames Foreman.yuv 32 119.683 679.610 150 10 2796.023 1175.202 Table 2: Foreman.yuv metrics summary

Test platform Processor –i5 @ M450 @ 2.4Ghz RAM-4GB 64-bit operating system OS-windows 10 Home N

Test sequences Basketballdrive [18] BasketballDrill_832x480_50.yuv

Cactus [20] Racehorses [19] Cactus_1920x1080_30.yuv RaceHorses_416x240_30.yuv

Conclusions For a given objective quality assessment metric [28] to be reliable, it’s score should be equivalent to that of the subjective quality metric and also outlier ratio[27] should be low.

References [1] K. R. Rao, et al. “Video Coding Standards: AVS China, H.264/MPEG-4Part10, HEVC, VP6, DIRAC and VC-1”, Springer, 2014. [2] J-R Ohm , et al. "Comparison of the Coding Efficiency of Video Coding Standards - including High Efficiency Video Coding (HEVC) ", IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, Issue: 12 , pp. 1669 -1684 , Dec.2012 [3]https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding –Website on HEVC [4] I.E.G Richardson , The H.264 advanced video compression standard. Chichester, West Sussex: Wiley, 2010. [5] K .R .Rao and J .J .Hwang, “Techniques and standards for Image Video and Audio Coding”, Prentice Hall, 1996. [6]F. Bossen, et al. "HEVC Complexity and Implementation Analysis", IEEE Transactions on Circuits Systems Video Technology, vol. 22, no. 12, pp. 1685-1696, Dec.2012. [7] ] Z. Wang et al ,“Multiscale structural similarity for image quality assessment,” Conference Record of the Thirty-Seventh Asilomar Conference on Signals, Systems and Computers, 2003, vol.2, pp. 1398- 1402, 9-12 Nov. 2003. [8]https://en.wikipedia.org/wiki/Dirac_(video_compression_format) -Website on Dirac [9]J. Choi and Y. Ho, "Efficient residual data coding in CABAC for HEVC lossless video compression", Signal, Image and Video Processing, vol. 9, no. 5, pp. 1055-1066, Dec.2013. [10] A. Ravi and K.R Rao, "Performance Analysis and Comparison of the Dirac Video Codec with H.264/MPEG-4 Part 10 AVC", International Journal on Wavelets Multi resolution Information Processing, vol. 09, no. 04, pp. 635-654, July.2011 [11] G.J. Sullivan et al, “Standardized Extensions of High Efficiency Video Coding (HEVC)”, IEEE Journal of selected topics in Signal Processing, Vol. 7, No. 6, pp. 1001-1016, Dec. 2013.

[13] T. Wiegand, et al, “Overview of the H [13] T. Wiegand, et al, “Overview of the H.264/AVC Video Coding Standard”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 13, pp.560-576, July 2000. [14] Visual studio download for students for free- www.dreamspark.com. [15] Z. Wang et al, “Image Quality Assessment: From Error Visibility to Structural Similarity”, IEEE Transactions on Image Processing, Vol. 13, No. 4, pp. 600-612, Apr. 2004. [16] Access to HM 16.3 Reference Software: http://hevc.hhi.fraunhofer.de/ [17]P. Hanhart and T. Ebrahimi, "Calculation of average coding efficiency based on subjective quality scores", Journal of Visual Communication and Image Representation, Vol. 25, no. 3, pp. 555-564, Apr 2014. [18] https://hevc.aes.tu-berlin.de/web/testsuite -HEVC test sequences. [19] https://media.xiph.org/video/derf/ -test sequences [20] http://diracvideo.org/2012/01/schroedinger-1-0-11/ - Access to DIRAC reference software. [21] G. Bjøntegaard, Calculation of Average PSNR Differences Between RD Curves, document VCEG-M33, ITU-T SG 16/Q 6, Austin, TX, Apr.2001. [22] B. Li, G. J. Sullivan, and J. Xu, “Compression performance of high efficiency video coding (HEVC) working draft 4,” in Proc. IEEE International Conference on Circuits and Systems, pp. 886–889, May 2012. [23] K. Ramchandran and M. Vetterli, “Rate-distortion optimal fast thresholding with complete JPEG/MPEG decoder compatibility,” IEEE Transactions on Image Processing, Vol. 3, no. 5, pp. 700–704, Sep. 1994. [24] Tortoise SVN download- http://tortoisesvn.net/downloads.html

[25] D. Grois, et al, “HEVC/H.265 Video Coding Standard including the Range Extensions, Scalable Extensions, and Multiview Extensions,” (Tutorial), IEEE ICCE , Berlin, Germany, 6 – 9 Sept. 2015. [26] D. Grois, et al, “HEVC/H.265 Video Coding Standard (Version 2) including the Range Extensions, Scalable Extensions, and Multiview Extensions,” (Tutorial) Sunday 27 Sept 2015, 9:00 am to 12:30 pm), IEEE ICIP, Quebec City, Canada, 27 – 30 Sept. 2015. This tutorial is for personal use only. Password: a2FazmgNK https://datacloud.hhi.fraunhofer.de/owncloud/public.php?service=files&t=8edc97d26d46d4458a9c1a17964bf881. [27] S. Chikkerur, et al, "Objective Video Quality Assessment Methods: A Classification, Review, and Performance Comparison", IEEE Trans. on Broadcast., vol. 57, no. 2, pp. 165-182, Sept.2011. [28] T. Liu, et al, "Visual quality assessment: recent developments, coding applications and future trends", APSIPA Transactions on Signal and Information Processing, vol. 2, Dec.2013. [29] X. Ran and N. Farvardin, “A perceptually-motivated three-component image model - part I: description of the model,” IEEE Transactions on ImageProcessing,vol.4, no.4, pp.401-415, Apr.1995