1. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 1Lecture 4 ECE-C490 Winter 2004 Image Processing Architecture Lecture 4, 1/20/2004 Principles of Lossy Compression Oleh Tretiak Drexel University

2. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 2Lecture 4 Compression:  Input is a sequence of discrete symbols  Main theoretical tool: Entropy Technology: Coding Methods  Huffman codes  Variable length code words, high probability symbols get short code words  Ideally, length of code word should be -log 2 p i  Arithmetic codes  Many symbols translate to many code bits  Each input symbol produces -log 2 p i  Other codes: Golomb-Rice Review: Lossless Compression

3. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 3Lecture 4 Successful preprocessing can reduce per/symbol entropy  Example: Run-length encoding  Example: Prediction on the basis of past values Review: Coder Structures High entropy Low entropy

4. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 4Lecture 4 Review: Coder Design Strategies Fixed encoder  Example: ITU-T, Rec. T.4 (Group 3), Rec. T.6 (Group 4)  Excellent compression on text documents of fixed resolution  Simple system Adaptive encoder  Example: JBIG  Adaptive predictor on basis of context  Arithmetic coding with p estimated from documents  Superior performance over a large range of documents

5. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 5Lecture 4 Review: Coder Designs  Example: Lossless JPEG  Prediction on basis of context  8 prediction rules, choose the best one Prediction context Available Prediction Rules

6. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 6Lecture 4 This Lecture Lossy Compression  Tradeoff between quality and number of bits  Bit reduction  Resolution reduction  Quantizers Rate distortion theory

7. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 7Lecture 4 Review: Lossless Compression of Images Can get quite good compression of graphics images Lossless compression of gray-scale images (JPEG) is ~ 2 What to do?  Keep trying  Give up  Take another approach Lossy Compression

8. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 8Lecture 4 Lossy Compression Ideas Tradeoff between bits and quality  Name of the game: Large reduction in bits at virtually no loss in quality Theoretical tool  Shannon Rate-Distortion Theory

9. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 9Lecture 4 Quality - Rate Tradeoff Given: 512x512 picture, 8 bits per pixel  Bit reduction  Fewer bits per pixel  Fewer pixels  Both Issues:  How do we measure compression?  Bits/pixel — does not work when we change number of pixel  Total bits — valid, but hard to interpret  How do we measure quality?  RMS noise  Peak signal to noise ratio (PSR) in dB  Subjective quality

10. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 10Lecture 4 Reduced bits/pixel Keep b most significant bits Compression = 8/b Distortion: o(i, j) original picture, r(i, j) reduced picture, we define the noise as RMS error: Peak Signal to Noise Ratio (PSNR) = 20log 10 (255/  )

11. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 11Lecture 4 Reducing bits per pixel. Upper row, left to right: 8, 4, 3 bits per pixel. Lower row, left to right: 2, 1 bits per pixel. With 5, 6, and 7 bits per pixel the image is almost the same as 8 bits.

12. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 12Lecture 4 Loss vs. Bit Rate

13. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 13Lecture 4 Resolution Reduction Start with 512x512 8 bit image Sample over a WxW point grid, keep 8 bits per pixel r(i, j) is interpolated to a 512 x512 grid Compression = (512/W) 2 n(i, j) = r(i, j) - o(i, j) Compute RMS Noise and PSNR

14. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 14Lecture 4 512382 256 181 128 Resolution reduction. Upper left is full resolution (512x512 pixels). Lower right image has 1/8 of the pixels, Compression = 8.

15. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 15Lecture 4 Difference Between Compressed and Original Difference between 128x128 (16 fold compression) and original image

16. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 16Lecture 4 Quantitative Performance

17. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 17Lecture 4 Comparison, Bit and Pixel Reduction

18. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 18Lecture 4 Conclusion and Questions For this image, resolution reduction is a much more effective compression method than bit reduction  8-fold reduction in bits at 30 dB PSNR with subsamplng  2.67-fold reduction bits at 29 dB PSNR with bit reduction What is the best combination of sampling and bits? Can entropy coding help? Is there a theory that can guide us?

19. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 19Lecture 4 Theory Theory tells us that the data we encode should be statistically independent  In a picture, adjacent pixels are correlated. Shannon’s rate-distortion theory provides a way of finding the tradeoff between rate (total bits per image) and distortion (rms. error)  Image should be transformed (preprocessed) so that statistically independent samples are encoded  Correlated images (with low detail) can be compressed more than uncorrelated images.

20. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 20Lecture 4 Typical Rate-Distortion Functions Rate is in bits per image, or bits per time Distortion is 1/(rms. error)

21. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 21Lecture 4 Information Theory of Continuous Sources Quantizers Decorrelation and Bit Allocation Rate-Distortion Theory DCT (Discrete Cosine Transform)

22. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 22Lecture 4 Quantizer: The Basics Given: 8 bit gray value samples, 0 ≤ data ≤ 255 Task: Keep 4 bits per sample Method: Shift 4 bits to right (divide by 2 4 = 64)  This is called quantization  Quantization step S = 64  All values betwee 0 and 63 are mapped to 0, values from 64 to 127 are mapped to 1, etc Restoring data: Multiply by 2 4 = 64 and add 32  This is called dequantization  What used to be between 0 and 63 is now at 32, data values between 64 and 1

23. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 23Lecture 4 Example of quantization Quantization with S = 20. Note that the dequantizer output is typically not equal to the input!

24. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 24Lecture 4 Quantizer Performance Questions:  How much error does the quantizer introduce (Distortion)?  How many bits are required for the quantized values (Rate)? Rate:  1. No compression. If there are N possible quantizer output values, then it takes ceiling(log 2 N) bits per sample.  2(a). Compression. Compute the histogram of the quantizer output. Design Huffman code for the histogram. Find the average lentgth.  2(b). Find the entropy of the quantizer distribution  2(c). Preprocess quantizer output,.... Distortion: Let x be the input to the quantizer, x* the de-quantized value. Quantization noise n = x* - x. Quantization noise power is equal to D = Average(n 2 ).

25. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 25Lecture 4 Quantizer: practical lossy encoding Quantizer  Symbols x — input to quantizer, q — output of quantizer, S — quantizer step  Quantizer: q = round(x/S)  Dequantizer characteristic x* = Sq  Typical noise power added by quantizer- dequantizer combination: D = S 2 /12 noise standard deviation  = sqrt(D) = 0.287S Example: S = 8, D = 8 2 /12 = 5.3, rms. quatization noise = sqrt(D) = 2.3 If input is 8 bits, max input is 255. There are 255/8 ~ 32 quantizer output values PSNR = 20 log10(255/2.3) = 40.8 dB Quantizer characteristic Dequantizer characteristic

26. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 26Lecture 4 Rate-Distortion Theory Probabilistic theory, developed by Claude Shannon in 1949 Input is a sequence of independent random variables, output are lossy (distorted) reproduction Joint probability density function p(x, y) Distortion: D =E[(X–Y) 2 ] (mean squared error, noise power), Rate-distortion function

27. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 27Lecture 4 Rate-Distortion Theorem When long sequences (blocks) are encoded, it is possible to construct a coder-decoder pair that achieves the specified distortion whenever bits per sample are R(D) +  Formula: X ~ Gaussian random variable, Q = E[X 2 ] ~ signal power D = E[(X–Y) 2 ] ~ noise power

28. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 28Lecture 4 Rate-Distortion Function for Gaussian Distribution ‘Theoretical’ plot ‘Practical’ plot

29. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 29Lecture 4 Rate - Distortion, in words Q - actual signal power (size) D - specified distortion power D > Q (specified distortion bigger than signal)  R = 0, send nothing, actual distortion = received - sent  distortion power = Q D < Q (specified distortion less than signal  Coded bits per sample = 0.5 log 2 (Q/D) This formula applies for coding statistically independent symbols!

30. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 30Lecture 4 Rate-Distortion Theory: Correlated Data Pixel values are correlated Above formula does not apply to correlated values Solution: pre-process data so that new values are de-correlated! On the left we show the contour plot of the pdf of two correlated random variables X 1 and X 2. By taking linear combinations (preprocessing) we get two new random variables Y 1 and Y 2. Feature: X 1 and X 2 have the same power (standard deviation). Y 1 and Y 2 have different standard deviation.

31. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 31Lecture 4 Rate-distortion theory — correlated samples Given: x = (x 1, x 2,... x n ), a sequence of Gaussian correlated samples Preprocess: convert to y = (y 1, y 2,... y n ), y = Ax, A ~ an orthogonal matrix (A -1 = A T ) that de-correlates the samples. This is called a Karhunen-Loeve transformation Perform lossy encoding of (y 1, y 2,... y n ) - get y* = (y 1 *, y 2 *,... y n *) after decoding Reconstruct: x* = A -1 y*

32. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 32Lecture 4 Bit allocation Before pre-processing, all samples x i have the same variance (size) After pre-processing, de-correlated samples y i have different variances, and are encoded differently. What is the best coding of non-uniform samples? Typical de-correlated sequence powers. Q(i) is the variance of y i

33. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 33Lecture 4 Rate-distortion theory: non- uniform random variables Sequence of independent Gaussian variables (y 1, y 2,... y n ), Var[y i ] = Q i. Distortion allocation: allocate D i distortion to Q i Rate (bits) for i-th variable is R i = max[0.5 log 2 (Q i /D i ), 0] Total distortion Total rate (bits) We specify R. What are the values of D i to get minimum total distortion D?

34. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 34Lecture 4 Basic Solution If the total distortion is small,  Make D i = D/n  R i = 0.5 log 2 (Q i /D i ) = 0.5 log 2 (nQ i /D) This holds as long as Q i ≥ D/n for all i

35. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 35Lecture 4 General solution Implicit solution (water-filling construction) Choose  (parameter) D i = min(Q i,  )  If (Qi >  ) then D i = , else D i = Q i R i = max[0.5 log 2 (Q i /D i ), 0]  If (Qi >  ) then R i = 0.5 log 2 (Q i /  ), else R i = 0. Find value of  to get specified R

36. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 36Lecture 4 Sample Calculation Table on left shows rate and distortion calculation for  = 0.1. The plot was produced by varying  SNR = 10 log 10 (Q tot /D), where Q tot = sum(Q i )=4.831.

37. Image Processing Architecture, © 2001-2004 Oleh TretiakPage 37Lecture 4 Rate Distortion: Summary Rate distortion theory gives a theoretical value of best bit-rate vs distortion tradeoff When source produces correlated data, it must be pre- processed to remove correlation Formula for independent source symbols: When symbols are correlated, the rate-distortion function can be computed by ‘water filling’ construction.