1. Investigation of Image Quality of Dirac, H.264 and H.265
Biju Shrestha
UTA ID:
EE 5359: Multimedia Processing
9/21/2018

2. Overview Introduction Image quality assessment using SSIM and FSIM
Dirac
H.264
H.265
Image quality assessment using SSIM and FSIM
SSIM – structural similarity metric
FSIM – featured similarity index
Conclusion
References

3. Introduction Video codec compress/decompress digital video
Types of video codecs
Dirac
H.264
H.265
Source: [2]

4. Dirac
Developed by BBC (British Broadcasting Corporation) Research and is open source
Powerful and flexible despite using less number of core tools [1]
Features [1]
Multi-resolution transforms
Inter and intra frame coding
Frame and field coding
Dual syntax
CBR (constant bit rate) and VBR (variable bit rate) operations
Variable bit depths.
Multiple chroma sampling formats
Lossless and lossy coding
Choice of wavelet filters
Simple stream navigation
Source: [1]

5. Dirac Encoder Architecture
Source: [15]

6. Dirac Decoder Architecture
Source: [18]

7. H.264
Also referred as AVC (advance video coding) is a standard for video compression [2]
Joint development of video coding experts group (VCEG) of the ITU-T* and the moving picture experts group (MPEG) of ISO/IEC* [11]
Enhanced coding efficiency
Applications - video telephony, video conferencing, TV, storage, streaming video, digital video authoring, digital cinema, etc. [11]
*ITU-T : international telecommunication union – telecommunication
standardization sector
*ISO : international organization for standardization
*IEC : international engineering consortium
Source: [2, 11]

8. H.264 Decoder
Source: [2]

9. H.264 Encoder
Source: [2]

10. H.264 Profiles
Source: [12]

11. H.265 Also known as high efficiency video coding (HEVC) [3]
Can deliver significant improvement relative to AVC (ITU-T* H.264 | ISO/IEC ) [10]
Efficiency can be progressed by average of 37 % for hierarchical B structure and 36 % for IPPP structure [16]
Video codec is composed of many processes including intra prediction and inter prediction, transforms, quantization, entropy coding, and filtering [17]
*ITU-T: international telecommunication union – telecommunication
standardization sector
Source: [3, 10, 16, 17]

12. H.265 Block Diagram *IST : integer sine transform
Grey boxes – Proposed tools
White boxes – H.264/AVC tools
*IST : integer sine transform
Source: [17]

13. Image Quality Assessment
Digital images and videos are prone to several distortion [5]
Nonstructural distortion
Structural distortion
Phases when distortion occurs [5]
Acquisition
Processing
Compressing
Storage
Transmission
Reproduction
Results of distortion – poor visual quality [5]
Different metrics available to quantify visual quality [3, 8, 13, 14]
Source: [3, 5, 8, 13, 14]

14. Nonstructural and Structural Distortions
Source: [14]

15. Comparison Metrics FSIM – Featured Similarity index
SSIM – Structural Similarity metric
Bit rate
PSNR – Peak Signal to Noise Ratio
MSE – Mean Squared Error

16. FSIM/FSIMc Based on human visual system (HVS)
FSIM is designed for gray-scale images
FSIMc incorporates chrominance information
Mathematical model of FSIM
SL(x) = overall similarity between f1(x) and f2(x)
Mathematical model of FSIMc
λ > 0 is the parameter used to adjust the importance of the chrominance components.
Source: [3]

17. FSIM/FSIMc Index Computation
f1 is the reference image, and f2 is a distorted version of f1 [3].
Source: [3]

18. SSIM Based on degradation of structural information [5]
HVS is adapted to extract structural information from an images [14]
Figure: Block diagram of structural similarity (SSIM) measurement system [5]
Source: [5, 14]

19. Mathematical Representation of SSIM
General form of SSIM
α, β, and γ are parameters that mediate the relative importance of the three components
µx and µy = local sample means of x and y respectively
σx and σy = local sample standard deviations of x and y respectively
σxy = local sample correlation coefficient between x and y
C1, C2, and C3 = constants that stabilize the computations when denominators become small
Using α = β = γ = 1. We get,
Source: [7]

20. PSNR and MSE
Both are directly dependent on the intensity of an image [3]
Both do not correlate with subjective fidelity ratings [3]
Both cannot model human visual system very accurately [4]
Source: [3, 4]

21. Example: SSIM and MSE
*MSE is approximately same for all images but SSIM is different giving better comparison[13]
Source: [13, 22]

22. Ranking of Image Quality Assessment Metrics
Different metrics take different approach to quantify images
Ranking of metrics by different databases
TID2008
CSIQ
LIVE
IVC
MICT
A57
FSIM 1
SSIM 2
PSNR 3
Table : Ranking of image quality assessment metrics performance on six databases[5]
Source: [3]

23. Conclusion
Investigate qualitative performance of Dirac, H.264 and H.265
User comparative metrics like SSIM, FSIM and bitrate
Based on various test sequences, the performance of codec will be investigated
Software and tools that will be used:
MATLAB
Visual studio
JM software
KTA software
Source: [ ]

24. References
[1] Dirac Video (2008, September 23), “Dirac Specification” [Online]. Available: [2] I. Richardson (2011), “A Technical Introduction to H.264/AVC” [Online]. Available: [3] L. Zhang, L. Zhang, X. Mou, and D. Zhang, “FSIM: A feature similarity index for image quality assessment,” IEEE Transactions on Image Processing, vol.20, no.8, pp , Aug [4] Z.Li and A.M. Tourapis, “New video quality metrics in the H.264 reference software,” Input Document to JVT, Hannover, DE, Jul [5] Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli,“Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, issue 4, pp , Apr [6] Z. Wang, E.P. Simoncelli, and A.C. Bovik, “Multiscale structural similarity for image quality assessment,” Conference Record of the Thirty-Seventh Asilomar Conference on Signals, Systems and Computers, 2003, vol.2, pp , 9-12 Nov [7] C. Li, and A. C. Bovik, “Content-weighted video quality assessment using a three-component image model.” Journal of Electronic Imaging, vol.19, pp , Mar [8] X. Ran and N. Farvardin, “A perceptually-motivated three-component image model - part I: description of the model,” IEEE Transactions on Image Processing, vol.4, no.4, pp , Apr [9] J. L. Li, G. Chen, and Z. R. Chi, “Image coding quality assessment using fuzzy integrals with a three-component image model,” IEEE Transactions on Fuzzy Systems, vol.12, no.1, pp , Feb [10] G. J. Sullivan and J. Ohm, “Recent developments in standardization of high efficiency video coding (HEVC),” Proc. SPIE 7798, 77980V, [11] G. Sullivan, P. Topiwalla, and A. Luthra, “The H.264/AVC video coding standard: overview and introduction to the fidelity range extensions,” SPIE Conference on Applications of Digital Image Processing XXVII, vol. 5558, pp , Aug [12] A. Puri, X. Chen, and A. Luthra, “Video coding using the H.264/MPEG-4 AVC compression standard,” Signal Processing: Image Communication, vol. 19, pp , Oct

25. References
[13] Z. Wang et al (2003, February), “The SSIM index for image quality assessment” [Online]. Available: [14] C. Chukka, “A universal image quality index and SSIM comparison” [Online]. Available: [15] BBC Research, “The technology behind Dirac” [Online]. Available: [16] E. Alshina et al, “Technical considerations of new challenges in video coding standardization,” International Organization for Standardization Organization Internationale De Normalisation ISO/IEC JTC1/SC29/WG11 Coding of Moving Pictures and Audio, Oct [17] S. Jeong et al, “Highly efficient video codec for entertainment quality,” ETRI Journal, vol.33, no. 2, pp , Apr [18] K. R. Rao and D. N. Kim, “Current video coding standards: H.264/AVC, Dirac, AVS China and VC-1,” 42nd Southeastern Symposium on System Theory (SSST), pp.1-8, Mar [19] A. M. Tourapis (January 2009), “H.264/ AVC reference software manual” [Online]. Available: [20] F. Bossen, D. Flynn, and K. Sühring (July 2011), “HEVC reference software manual” [Online]. Available: [21] DiracPRO software: [22] D. T. Lee, “JPEG 2000: Retrospective and new developments,” Proc. IEEE, vol. 93, pp , Jan [23] KTA software: [24] H.264/AVC Reference Software: [25] A. Ravi, “Performance analysis and comparison of the Dirac video codec with H.264/MPEG-4 part 10 AVC,” M.S. thesis, Dept. Elect. Eng., Univ. of Texas at Arlington, 2009 [25] I.E.G. Richardson, “H.264 and MPEG-4 video compression: video coding for next generation multimedia,” Great Britain: Wiley, 2003, pp [26] MSU video quality measurement tool: