1. Digital television systems (DTS)
Video Sequence Processing
Technical Univ. of Kosice
Faculty of Electrical Engineering and Informatics
Lˇ. Maceková, 2017

2. Videosequence processing and quality evaluation - basic principlesj
Videosequence processing and quality evaluation - basic principles
Lˇ. Maceková, 2017
TU Kosice

3. j
frame n+1
Fig. Digital (matrix) model of image sequence i
pixel
frame n
frame n-1
M ... number of raws
N ... number of columns
time

4. Standard digital images and sequences - for relevant comparison of experimental results (of filtration methods, error correction, etc.)

5. Fig. For different purposes, there are more different color spaces for digital color representation
wavelength [nm]
XYZ
RGB

6. Models of artificial corruption of images and image sequences
- computational softwares – for generation of more types noise (multiplicative or additive application principles, exchange: correct pixel x  corrupted one, random choise of noise value and pixel position, more types of value distribution, or salt /pepper noise application, etc.
For example: Gauss Additive Noise (GAN) with probability distribution (appearance of error e ):
e is the noise level in pixel x, μ is its mean value, σ2 is dispersion (variance) of noise values.
Fig. Lena (Lg20) – illustration – GAN corruption, N(μ, σ2) = N(0,202)
- then the overall brightness G of the pixel x with added noise is:

7. Fig.: Color Lena picture; impulsive noise – correlated/uncorrelated in R, G, B channels
Fig. : Simulated random BW spots

8. Filtration methods dvojsmerové štruktúry
– the mathematical (by computer) pixel intensity processing (correction / error concealment, noise suppresion, etc. (mathematic operations and operations of the choice)
- For example: median filtration - that is the choice of the median from the set of values of the shifting filter window (It is suitable for impulsive noise suppresion, and partially for supressing „dirt and sparkle“ spots type) x*
actual pixel
Fig.: Examples of the 2D-shapes of filtering windows of the median filters. The use of each of them has another effect.
The basic operation of median choice:
dvojsmerové štruktúry
- at the color pictures, the color vectors can be the elements of filter windows  vector filtering
Fig.: Examples of 3D filter windows for image sequence filtering (there are the pixels of actual and 2 adjacent frames)

9. Detectors of noise, corruption spots, lines, etc.
- The detection is needed before filtration process, to avoid the undesirable filtration effects.
detector
filter
xi,j,n
yi,j,n
Fig. 3.1 Filtration with using of detector for noise or corruption detection. 1

10. Digital pictures and video quality evaluation10
Digital pictures and video quality evaluation
Mean absolute and mean squared errors respectively (of 1 picture and of a sequence of pictures :
- pre obraz:
- for sequence:
Signal-to-Noise ratio: Peak SNR:
color difference CD: successive transformations: lin.RGB XYZ  u’, v’, u’n, v’n  L*u*v* 

11. In the dynamic picture sequence, the change (error) of the motion is evaluated. The smoothness of the motion is corrupted  the correlation Rn between 2 frames is changed (Δ Rn):
Notes:
MxN – picture dimension
o – original pixel
y – new pixel value on the i,j position
m – dimension of filter window
ΔE*uv - color difference in CIE L*u*v* color space
μ, σ – mean value and standard deviation of the brightness in the frames n, n+1

12. Older standardized quality criteria close to human visual system quality perception ... ITU-R 500
- very complex issue - there must be a lot of difficult constraints fulfilled (appearance of the room, other technician conditions, the steps of process of video watching and evaluating by the more subjects / people)
 that is why the new objective criteria (SSIM) is developed and standardized - see next

13. Structural quality criteria SSIM - definition13
Structural quality criteria SSIM - definition
- The using of statistical parameters μ, σ, σx,y of the picture or frames
- coefficients of similarity:
C1=(K1L)2, C2=(K2L)2 , L = 255 (brightness range), K1, K2  user setting
Description of the process: blocks or pixels i,j of the frame - SSIMij - average SSIM (i.e. MSSIM) for each image block or pixel  maps of the quality of the picture
- in the case of color pictures – for each i,j component, calculation:
- in the case of the image sequence (video) – for computing sparing, there is random choice of blocks realized – their wights assigned by brightnes -weights of the frames by the measure of the motion (using Block Matching methods or correlation method):

14. the subjective quality value
The illustration of 3 types of the quality changing. The calculated MSE is very close of one to another.
(b) Lena – increased contrast
(a) Lena – original picture
(c) Lena – corrupted image (blurring)
(d) Lena – JPEG compression
picture
MSE
the subjective quality value
SSIM b
226.36
1,9
0.943 d
225.92
5,14
0.745 c
225.23
5,24
0.741
(a- picture – the best quality; the subjective value is 1)

15. Literature
[1] Zeng, K. - Wang, Z.: 3D-SSIM for Video Qality Assesment, ICIP 2012, IEEE, pp