Objective Quality Assessment Metrics for Video Codecs - Sridhar Godavarthy

Contents Why are we doing this? Classification of metrics Aren’t we comparing apples and oranges? Results of experiments Conclusion 2

What is Video? Sequential combination of images Utilizes the persistence of vision High bandwidth due to large size 3

Compression To conserve Bandwidth Removes redundancy Spatial Within frame Temporal Between frames Transmit only changes from a base frame 4

Compression Continued… Lossless  Low compression  High quality Lossy  High compression  Obviously low quality 5

6/10/20166 Video Encoding Formats 1) MPEG-1 2) MPEG-2 3) MPEG-4 4) H.263 5) ASF 6) WMV and lots more… 6

6/10/20167 What is a Codec?  Coder Decoder.  Capable of encoding and decoding.  H/W or S/W.  Several codecs for each format.  Separate for audio/video. 7

Quality Measurement Subjective Objective 8

Subjective Mean opinion score - bunch of people watch and rate. Not stable Slow Expensive 9

Objective Measurable quantity Rate/Distortion Common Measurements: PSNR, MSE  Not accurate 10

Classification of Objective Measurements Full Reference No Reference Reduced Reference 11

Full Reference Measure distortion with full reference to original image/video (Eg. Pixel to Pixel) 12

No Reference Measure distortion with no reference to original image/video Less accurate Measure a specific (set of) distortion(s) Lower Complexity 13

Reduced Reference Extract information from the original image and use that information Compare with same information from distorted image. 14

Another Classification of Measurements Error Sensitivity Structural Similarity/Distortion Statistics of Natural Images Others 15

Error Sensitivity Based on Human Visual System FR method Decompose videos into spatio-temporal sub- bands followed by an error normalization and weighting process Metrics differ by the Sub-band decomposition method and the HVS model adapted 16

Structural Similarity/Distortion Extract structural information. Based on HVS. Top Down approach Quality is independent of intensity and contrast variations. Metrics for scaling, translation and rotation Similar to Reduced Reference methods. 17

Statistics of Natural Images Only for natural images Not applicable for text images, graphics, animations, Radar, X-rays etc. Just a measurement of information loss Uses statistical models. 18

Others Spatial information losses Edge shifting Luminance variations etc. Degradation caused by water markings 19

No Questions Slide All metrics converted to Predicted DMOS Allows for easier comparison Normalized PDMOS in some cases Mapping is obtained by a non-linear equation 20

Measurements All metrics trained on standard set to obtain objective and subjective indices These indices are used to calculate the unknowns in the previous equation Viola! Mapping for each metric to PDMOS 21

Training Set SSIM, VIF, RRIQA, DMOSpPSNR  Live2 database NRJPEGQS only with JPEG NRJPEG200 only with JPEG2K VQM-GM: 8 video sequences including foreman. Training only based on luminance(gray scale) 22

Analysis of Results – Why DMOSp PSNR shows differences even at high bit – rates. The objective measure DMOSp shows saturation at high bit- rates. Similar to Subjective indices. Hence the Subjective measure. 23

Analysis of Results Contd… High Bit rates results in saturation. Differences less than 5 are not perceivable. Lower DMOSp as Bit- rate increases(good!) But at low bit rates, wide variations. 24

Conclusions Metrics compared in the predicted DMOS space All Metrics were “trained” with the same dataset to attain the mapping. NRJPEG2000 gave wrong quality scores. NRJPEGQS does not accurately perceive the differences of quality VQM ranks certain formats wrongly. What does all this mean? 25

Conclusions Contd… Each metric can be used depending on  Application  Frame Size  Bit range VIF the most accurate of the lot RRIQA the best NR method DMOSp-PSNR the fastest 26

Sridhar Godavarthy Dept. of Computer Science and Engineering University of South Florida 27