Artefact-based methods for video quality prediction – Literature survey and state-of- the-art Towards hybrid video quality models

Savvas Argyropoulos, Deutsche Telekom Labs 2 Video quality prediction  Parameter-based vs signal-based models Parameter-basedSignal-based Hybrid Input network parametersreconstructed imageboth Advantages complexity, applicability in encrypted content results based on content characteristics and human visual system combination Disadvantages Accuracy computational cost, not applicable in encrypted content technology dependent

Savvas Argyropoulos, Deutsche Telekom Labs 3 Artifact-based video quality prediction No-Reference error concealment effectiveness  Estimate video quality caused by packet losses (but not error propagation!)  Error concealment effectiveness based on:  Motion level information  Luminance discontinuity  Video quality score is based on number of ineffectively concealed macroblocks [1] T. Yamada, Y. Miyamoto, M. Serizawa, “No Reference Video Quality Estimation based on Error Concealment Effectiveness,” Packet Video Workshop, 2007.

Savvas Argyropoulos, Deutsche Telekom Labs 4 Artifact-based video quality prediction No-Reference error concealment effectiveness  MVs for missing macroblocks are estimated from previous frames  EC is considered ineffective if:  where: mv x, mv y : (estimated) motions vectors of a missing macroblock a n : luminance value in the error region along the boundary A n : luminance value in the correctly decoded region along the boundary N : number of pixels along the boundary  Finally:  Where x is the number of ineffectively concealed macroblocks

Savvas Argyropoulos, Deutsche Telekom Labs 5 Artifact-based video quality prediction The ringing effect  Ringing: “Ripples or oscillations around high contrast edges”  Detect regions with perceived ringing  The annoyance caused by the ringing effect is determined by:  Luminance masking  Texture masking  Two-step approach:  Detect spatial location of regions with ringing  Estimate the visibility of the ringing effect  [2] H. Liu, N. Klomp, and I. Heynderickx, “A no-reference metric for perceived ringing,” in Proc. of VPQM 2009.

Savvas Argyropoulos, Deutsche Telekom Labs 6 Artifact-based video quality prediction The ringing effect Block diagram of the algorithm for the perceptual effect of ringing artifacts

Savvas Argyropoulos, Deutsche Telekom Labs 7 Artifact-based video quality prediction The ringing effect (a) Original image (b) Perceptual edge map (c) computational ringing region map (d) perceived ringing map

Savvas Argyropoulos, Deutsche Telekom Labs 8 Artifact-based video quality prediction Blurriness (1)  Blurriness is caused by the loss/attenuation of high spatial frequency values.  Blur is perceptually apparent along edges and in textured areas.  The algorithm measures the spread of the edges.  Subjective tests with ten subjects  Artificial blurriness noise:  Gaussian noise  JPEG 2000 compression  [3] P. Marziliano et al., “A no-reference perceptual blur metric,” ICIP, 2002.

Savvas Argyropoulos, Deutsche Telekom Labs 9 Artifact-based video quality prediction Blurriness (1)

Savvas Argyropoulos, Deutsche Telekom Labs 10 Artifact-based video quality prediction Blurriness (1)  Dashed lines: detected edges  Dotted lines: local minima/maxima around edges  Edge width at P1

Savvas Argyropoulos, Deutsche Telekom Labs 11 Artifact-based video quality prediction Blurriness (1) (a) Original image (b) Gaussian blurriness (c) JPEG 2000 compression blurriness

Savvas Argyropoulos, Deutsche Telekom Labs 12 Artifact-based video quality prediction Blurriness (2)  Global blur is estimated from a histogram of DCT coefficients.  8x8 DCT coefficients from all the blocks in the frame.  The distribution rather than the values of the DCT coefficients are considered.  Normalization with the number of non-zero DC coefficients.  [4] X. Marichal et al., “Blur determination in the compressed domain using DCT information,” in ICIP., Oct I-frameP-frameB-frame

Savvas Argyropoulos, Deutsche Telekom Labs 13 Artifact-based video quality prediction Blurriness (2)  The blur estimation algorithm examines the number of coefficients that are always zero in the image.  A weighted grid is applied to give more importance to the coefficients in the central diagonal since they better represent global blur.  The weighted sum of number of occurrence of each DCT coefficient provides a measure of the total blur

Savvas Argyropoulos, Deutsche Telekom Labs 14 Artifact-based video quality prediction Blurriness - Sharpness (3)  Blurriness is calculated as the inverse of sharpness.  Sharpness is estimated from kurtosis.  Kurtosis: measure of the non-gaussianity of a random variable  [5] J. Caviedes and S. Gurbuz, “No-Reference sharpness metric based on local edge kurtosis,” ICIP, 2002.

Savvas Argyropoulos, Deutsche Telekom Labs 15 Artifact-based video quality prediction Blurriness - Sharpness (3)  DCT is applied to 8x8 pixel blocks and the bivariate probability distribution p(x,y) is used to calculate kurtosis.  Edges are detected – Each edge pixel is assigned to the center of a 8x8 block.  The 2-D kurtosis is calculated for each block of the edge profile.  The average kurtosis over all 8x8 blocks is the frame sharpness measure.

Savvas Argyropoulos, Deutsche Telekom Labs 16 References  T. Yamada, Y. Miyamoto, M. Serizawa, “No Reference Video Quality Estimation based on Error Concealment Effectiveness,” Packet Video Workshop,  H. Liu, N. Klomp, and I. Heynderickx, “A no-reference metric for perceived ringing,” in Proc. of VPQM  P. Marziliano et al., “A no-reference perceptual blur metric,”ICIP, 2002, vol. 3, pp. 57–60.  X. Marichal et al., “Blur determination in the compressed domain using DCT information,” in ICIP., Oct  J. Caviedes and S. Gurbuz, “No-Reference sharpness metric based on local edge kurtosis,” ICIP, 2002.