Digital television systems (DTS) Video Sequence Processing Technical Univ. of Kosice Faculty of Electrical Engineering and Informatics Lˇ. Maceková, 2017

Videosequence processing and quality evaluation - basic principles j Videosequence processing and quality evaluation - basic principles Lˇ. Maceková, 2017 TU Kosice

j frame n+1 Fig. Digital (matrix) model of image sequence i pixel frame n frame n-1 M ... number of raws N ... number of columns time

Standard digital images and sequences - for relevant comparison of experimental results (of filtration methods, error correction, etc.)

Fig. For different purposes, there are more different color spaces for digital color representation wavelength [nm]   XYZ RGB

Models of artificial corruption of images and image sequences - computational softwares – for generation of more types noise (multiplicative or additive application principles, exchange: correct pixel x  corrupted one, random choise of noise value and pixel position, more types of value distribution, or salt /pepper noise application, etc. For example: Gauss Additive Noise (GAN) with probability distribution (appearance of error e ): e is the noise level in pixel x, μ is its mean value, σ2 is dispersion (variance) of noise values. Fig. Lena (Lg20) – illustration – GAN corruption, N(μ, σ2) = N(0,202) - then the overall brightness G of the pixel x with added noise is:

Fig.: Color Lena picture; impulsive noise – correlated/uncorrelated in R, G, B channels Fig. : Simulated random BW spots

Filtration methods dvojsmerové štruktúry – the mathematical (by computer) pixel intensity processing (correction / error concealment, noise suppresion, etc. (mathematic operations and operations of the choice) - For example: median filtration - that is the choice of the median from the set of values of the shifting filter window (It is suitable for impulsive noise suppresion, and partially for supressing „dirt and sparkle“ spots type) x* actual pixel Fig.: Examples of the 2D-shapes of filtering windows of the median filters. The use of each of them has another effect. The basic operation of median choice: dvojsmerové štruktúry - at the color pictures, the color vectors can be the elements of filter windows  vector filtering Fig.: Examples of 3D filter windows for image sequence filtering (there are the pixels of actual and 2 adjacent frames)

Detectors of noise, corruption spots, lines, etc. - The detection is needed before filtration process, to avoid the undesirable filtration effects. detector filter xi,j,n yi,j,n Fig. 3.1 Filtration with using of detector for noise or corruption detection. 1

Digital pictures and video quality evaluation 10 Digital pictures and video quality evaluation Mean absolute and mean squared errors respectively (of 1 picture and of a sequence of pictures : - pre obraz: - for sequence: Signal-to-Noise ratio: Peak SNR: color difference CD: successive transformations: lin.RGB XYZ  u’, v’, u’n, v’n  L*u*v* 

In the dynamic picture sequence, the change (error) of the motion is evaluated. The smoothness of the motion is corrupted  the correlation Rn between 2 frames is changed (Δ Rn): Notes: MxN – picture dimension o – original pixel y – new pixel value on the i,j position m – dimension of filter window ΔE*uv - color difference in CIE L*u*v* color space μ, σ – mean value and standard deviation of the brightness in the frames n, n+1

Older standardized quality criteria close to human visual system quality perception ... ITU-R 500 - very complex issue - there must be a lot of difficult constraints fulfilled (appearance of the room, other technician conditions, the steps of process of video watching and evaluating by the more subjects / people)  that is why the new objective criteria (SSIM) is developed and standardized - see next

Structural quality criteria SSIM - definition 13 Structural quality criteria SSIM - definition - The using of statistical parameters μ, σ, σx,y of the picture or frames - coefficients of similarity: C1=(K1L)2, C2=(K2L)2 , L = 255 (brightness range), K1, K2  10-2 ... user setting Description of the process: blocks or pixels i,j of the frame - SSIMij - average SSIM (i.e. MSSIM) for each image block or pixel  maps of the quality of the picture - in the case of color pictures – for each i,j component, calculation: - in the case of the image sequence (video) – for computing sparing, there is random choice of blocks realized – their wights assigned by brightnes -weights of the frames by the measure of the motion (using Block Matching methods or correlation method):

the subjective quality value The illustration of 3 types of the quality changing. The calculated MSE is very close of one to another. (b) Lena – increased contrast (a) Lena – original picture (c) Lena – corrupted image (blurring) (d) Lena – JPEG compression picture MSE the subjective quality value SSIM b 226.36 1,9 0.943 d 225.92 5,14 0.745 c 225.23 5,24 0.741 (a- picture – the best quality; the subjective value is 1)

Literature [1] Zeng, K. - Wang, Z.: 3D-SSIM for Video Qality Assesment, ICIP 2012, IEEE, pp. 621-624.