1. Quantization Heejune AHN Embedded Communications Laboratory Seoul National Univ. of Technology Fall 2013 Last updated 2013. 9. 24

2. Heejune AHN: Image and Video Compressionp. 2 Outline Quantization Concept Quantizer Design Quantizer in Video Coding Vector Quantizer Rate-distortion optimization Concept Lagrangian Method Appendix Detail for Lloyd-max Quantization Algorithm

3. Heejune AHN: Image and Video Compressionp. 3 1. Quantization Reduce representation set N large points ([0 ~ N large ]) to N small points ([0, … N small ]) Inevitable when Analog to Digital Conversion Not Reversible ! Original (digital/analog) ADC/Quant Quanitzed Levels ReScale

4. Heejune AHN: Image and Video Compressionp. 4 Scalar Quantizer Quantize a single value (single input & single output) In contrast to VQ (Vector Quantizer, to be discussed later) Mapping function {[x i-1, x i ) => y i } Decision levels to reconstruction level Quantizer input Quantizer output x i-1 xixi yiyi Output : { y i } Input : x Q q Q q

5. Heejune AHN: Image and Video Compressionp. 5 Quantizer’s Effects + Bit Rate Reduction Bits required to represent the values: (E1bits=> E2 bits) N large (= 2 E1 )points ([0 ~ N]) to N small (= 2 E2 )points ([0, … N small ]) Bit reduction = E2 - E1 = log 2 (N Large /N small ) - Distortion e = y i - x (in [x i-1, x i )) MSE (mean squared error) Trade-off problem

6. Heejune AHN: Image and Video Compressionp. 6 2. Quantization Design Distortion Quantization error = difference original (x) and quantized (y i ) Distortion Measure Quantizer design parameters Step size (N) decision level (x i ) reconstruction level (y i ) Distortion depends on Input signal distribution (f(x)) Optimization Problem How to set the ranges and reconstruction values, i.e., { y i & [x i-1, x i )} for i =0, … N, so that the resulting MSE is minimized.

7. Heejune AHN: Image and Video Compressionp. 7 Optimal Quantizer Solutions Note performance depends on the input statistics. Simpler design Uniform (Fixed quant step) quantizer With/Without dead-zone near zero Lloyd-Max algorithm a optimal quantizer design algorithm More levels for dense region, Less levels for sparse region. (non-linear quantizer)

8. Heejune AHN: Image and Video Compressionp. 8 Uniform Quantizers Properties x i and y i are spaced evenly (so, uniform), with space Δ (step size) Mid-tread (odd # of recon. levels) vs Mid-rise (no zero recon. ) mid-tread is more popular than mid-rise. ∆ 2∆ 3∆ Input -3∆ -2∆ -∆ Reconstruction 3.5∆ 2.5∆ 1.5∆ 0.5 ∆ -0.5 ∆ -1.5∆ -2.5∆ -3.5∆ Uniform Midrise Quantizer -2.5 ∆ -1.5 ∆ -0.5 ∆ Reconstruction 3∆3∆ 2∆2∆ ∆ -∆-∆ -2∆ -3∆ Uniform Midtread Quantizer 0.5 ∆ 1.5 ∆ 2.5 ∆ Input What will be the mathematical representation for quant/inverse?

9. Heejune AHN: Image and Video Compressionp. 9

10. Heejune AHN: Image and Video Compressionp. 10 Uniform Quantizer Performance For Uniform dist. Input, Uniform quantizer is optimal Can you prove this? Uniform Input X ~ U(-A,A) Error signal e = y – x ~ U(-Δ/2, Δ/2)

11. Heejune AHN: Image and Video Compressionp. 11 Signal Power (variance) MSE SNR Note If one bit added, Δ is halved, noise variance reduces to 1/4, SNR increases by 6 dB. SNR does not depends on input range, MSE does.

12. Heejune AHN: Image and Video Compressionp. 12 Non-uniform Quantizer Issue Many data are non-uniformly distributed and even unbounded Concept General Intuition: Dense Levels for Dense Input region Boundness: Use saturation Post filter for unbounded input Generally no closed-form expression for MSE. Range and Reconstruction Level is cross-related

13. Heejune AHN: Image and Video Compressionp. 13 Lloyd-Max Algorithm Algorithm Iteration algorithm with decision level (x i ) and recon. level (y i ) Run until code book does not change any more Step 1: find Near Region R i ((x i-1, x i ]) R i = {x | d(x,y i ) <= d(x,y j ), for all j =\= I } Step 2: find level y i y i = E[X| x in R i ] centroid calculation range update

14. Heejune AHN: Image and Video Compressionp. 14 where T k is the k th interval [t k,t k+1 ) Lloyd-Max Algorithm Minimize mean squared error.

15. Heejune AHN: Image and Video Compressionp. 15 Example Quantization and Quality (Lloyd-Max applied) Original (8bpp) vs 6 bpp (spatial domain)

16. Heejune AHN: Image and Video Compressionp. 16 Original (8bpp) vs 4 bpp (spatial domain)

17. Heejune AHN: Image and Video Compressionp. 17 Original (8bpp) vs 2 bpp (spatial domain)

18. Heejune AHN: Image and Video Compressionp. 18 High Density Quantization conditional pdf of in the interval High Quality Condition Observation Slow change (Constant) over the quantization region Approximate with Uniform distribution

19. Heejune AHN: Image and Video Compressionp. 19 Quantization in Video Coding UTQ (uniform Threshold Quantization) often is used Difficult in adaptive design (a LM quantizer design) Video quantizers are high quality ones Quantization in VC Input signal: quantize the (DCT) coefficients Parameters q (Δ): quantization scale factor ( 2 ~ 62, usu. q = 2QUANT, 1 ~31 ) Index (Level) : index for the reconstruction values Note: index, not reconstruction values are transmitted E.g. Index: I (n,m) = floor(F(n,m)/q) = floor(F(n,m)/2QUANT) –i.e. [i-1, i) *q => i Reconstruction: F’(n.m) = (I(n,m) + ½) x q = (2*I(n,m) +1)xQUANT –i.e. i => (i +1/2) q (i.e, middle of range [i -1, i)*q) Note: with QUANT, no 1/2 in expression.

20. Heejune AHN: Image and Video Compressionp. 20 Uniform Threshold Quantizer Threshold (dead zone) for reducing dct coeff. Intra DC (zero threshold), Intra AC, inter DC, AC (non-zero) 2*Q(UANT) = q Quantizer input Quantizer output q/2 -q/2 q 2q -q -2q -q -2q -3q 2q 3q q -3q Quantizer input Quantizer output 2Q 3Q -2Q -3Q -Q -3Q -5Q 3Q 5Q q -5Q AC (With Threshold) DC (without Threshold, q = 8 ) dead zone

21. Heejune AHN: Image and Video Compressionp. 21 3. Quantization in Video Coding Applied on the Transform Coefficients Key Throttle for Bit-Rate and Quality Very Important Trade-off in Video coding Estimation (MC, IntraP) X orig Transform (e.g. DCT) e = X orig - X est Y = T(e) Quantizati on Entrophy Coding Y q = Q(Y)VLC(Y q)

22. Heejune AHN: Image and Video Compressionp. 22 Example #1 Example #2: Comparison with Coarse and Fine Quant See Textbook Table 7.4 Q(8) Zig-zag scan Transformed 8x8 block

23. Heejune AHN: Image and Video Compressionp. 23 DCT Coef. Quantization Human visual weight coarse quant to High frequency components NDZ-UTQ to Intra DC, DZ-UTQ to all others Intra block Case I (n,m) = round(F(n,m)/(q*W(n,n))/8 ) F’(n.m) = (I(n,m) + ½) x q x W(n,m)/8 816192226272934 16 222427293437 192226272934 38 22 262729343740 2226272932354048 2627293235404858 2627293438465669 2729353846566983 intra 16 inter

24. Heejune AHN: Image and Video Compressionp. 24 Quantizer Example MPEG-4 example Inverse Quantization only Defined if LEVEL = 0, REC = 0 Else If QUANT is ODD |REC| = QUANT *(2 |LEVEL| +1) Else |REC| = QUANT *(2|LEVEL| +1) -1 + 1 : Dead-zone (sorry, I cannot explain why odd/even) Quantization is not defined Why? Examples Q= 4 31 23 15 7 -7 -15 -23 -31 27 19 11 0 -11 -19 -27 3 Len = 14 > 8 34 26 18 10 -10 -18 -26 -34 27 19 11 0 -11 -19 -27 Len = 14 > 8 Un-biased biased

25. Heejune AHN: Image and Video Compressionp. 25 4. Vector Quantizater Vector Quantizer Use statistical correlation of N dimension Input

26. Heejune AHN: Image and Video Compressionp. 26 Vector Quantizer Design Generalized Loyd-Max Algorithm Why Not Vector Quantization in Video Standards Transform Coding utilizes the “Correlation”, the remaining signal does not has Correlation much.

27. Heejune AHN: Image and Video Compressionp. 27

28. Heejune AHN: Image and Video Compressionp. 28 Performance of VQ

29. Heejune AHN: Image and Video Compressionp. 29 Embedded quantizers Motivation: “scalability” – decoding of compressed bitstreams at different rates (with different qualities) Nested quantization intervals In general, only one quantizer can be optimum (exception: uniform quantizers) Q0 Q1 Q2 x

30. Heejune AHN: Image and Video Compressionp. 30 Rate Distortion Theory Rate vs Distortion Inverse relation between Information/Data Rate and Distortion Revisit to a uniform distributed r.v. with a uniform quantizer Plot it ! Exponential decreasing Proportional to Complexity of source in statistics

31. Heejune AHN: Image and Video Compressionp. 31 Rate Distortion Theory Rate-distortion Function and Theroy Inverse relation between Information/Data Rate and Distortion Example with Gaussian R.V.

32. Heejune AHN: Image and Video Compressionp. 32 Rate-Distortion Optimization Motivation Image coding Condition: we can use 10KBytes for a whole picture Each part has different complexity (i.e, simpler and complex, variance) Which parts I have to assign more bits? Analogy to school score Students are evaluated sum of Math and English grade. I gets 90pts in Math, 50pts in Eng. Which subject I have to study? (natural assumption: 90 to 100 is much harder than 50 to 60) simpler blocks More complex blocks

33. Heejune AHN: Image and Video Compressionp. 33 Lagrange multiplier Constrained Optimization Problem Minimize distortion with bits no larger than max bits Cannot use partial differential for minima and maxima Lagrange Multiplier technique Insert one more imaginary variable (called Lagrange) We have multivariable minimization problem

34. Heejune AHN: Image and Video Compressionp. 34 Side note: why not partial differential Simple example min subject to Solution By substitution minimum ½ at x = y = 1/sqrt(2) Wrong Solution With complex constraints, we cannot use substitution of variable method We cannot change partial different with ordinary difference If we check, x = y = 0 ? 1 1

35. Heejune AHN: Image and Video Compressionp. 35 An Example: Lagrange Multiplier Optimization Problem multiple independent Gaussian variables Lagrange multiplier optimization

36. Heejune AHN: Image and Video Compressionp. 36 Solution Multiplying M equations, and take M-square Lagrange value Bit-allocation Large variance, then more bit allocation But the increment is logarithmic (not linear)

37. Appendix Lloyd-Max Algorithm Details

38. Heejune AHN: Image and Video Compressionp. 38 Lloyd-Max scalar quantizer Problem : For a signal x with given PDF find a quantizer with M representative levels such that Solution : Lloyd-Max quantizer [Lloyd,1957] [Max,1960] M-1 decision thresholds exactly half-way between representative levels. M representative levels in the centroid of the PDF between two successive decision thresholds. Necessary (but not sufficient) conditions

39. Heejune AHN: Image and Video Compressionp. 39 Iterative Lloyd-Max quantizer design  Guess initial set of representative levels  Calculate decision thresholds  Calculate new representative levels  Repeat 2. and 3. until no further distortion reduction

40. Heejune AHN: Image and Video Compressionp. 40 Example of use of the Lloyd algorithm (I) X zero-mean, unit-variance Gaussian r.v. Design scalar quantizer with 4 quantization indices with minimum expected distortion D* Optimum quantizer, obtained with the Lloyd algorithm Decision thresholds -0.98, 0, 0.98 Representative levels –1.51, -0.45, 0.45, 1.51 D* =0.12=9.30 dB Boundary Reconstruction

41. Heejune AHN: Image and Video Compressionp. 41 Example of use of the Lloyd algorithm (II) Convergence Initial quantizer A: decision thresholds –3, 0 3 Initial quantizer B: decision thresholds –½, 0, ½ After 6 iterations, in both cases, (D-D*)/D*<1%

42. Heejune AHN: Image and Video Compressionp. 42 Example of use of the Lloyd algorithm (III) X zero-mean, unit-variance Laplacian r.v. Design scalar quantizer with 4 quantization indices with minimum expected distortion D* Optimum quantizer, obtained with the Lloyd algorithm Decision thresholds -1.13, 0, 1.13 Representative levels -1.83, -0.42, 0.42, 1.83 D* =0.18=7.54 dB Threshold Representative

43. Heejune AHN: Image and Video Compressionp. 43 Example of use of the Lloyd algorithm (IV) Convergence Initial quantizer A, decision thresholds –3, 0 3 Initial quantizer B, decision thresholds –½, 0, ½ After 6 iterations, in both cases, (D-D*)/D*<1%

44. Heejune AHN: Image and Video Compressionp. 44 Lloyd algorithm with training data  Guess initial set of representative levels  Assign each sample x i in training set T to closest representative  Calculate new representative levels  Repeat 2. and 3. until no further distortion reduction

45. Heejune AHN: Image and Video Compressionp. 45 Lloyd-Max quantizer properties Zero-mean quantization error Quantization error and reconstruction decorrelated Variance subtraction property