Digital Image Processing Chapter 8: Image Compression 11 August 2006

Data vs Information Information = Matter (สาระ) Data = The means by which information is conveyed Image Compression Reducing the amount of data required to represent a digital image while keeping information as much as possible

Relative Data Redundancy and Compression Ratio Types of data redundancy 1. Coding redundancy 2. Interpixel redundancy 3. Psychovisual redundancy

Example of Coding Redundancy : Different coding methods yield different amount of data needed to represent the same information. Example of Coding Redundancy : Variable Length Coding vs. Fixed Length Coding Lavg 3 bits/symbol Lavg 2.7 bits/symbol (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Variable Length Coding Concept: assign the longest code word to the symbol with the least probability of occurrence. (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Interpixel Redundancy (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Interpixel redundancy: Parts of an image are highly correlated. In other words,we can predict a given pixel from its neighbor.

Run Length Coding The gray scale image of size 343x1024 pixels Binary image = 343x1024x1 = 351232 bits Line No. 100 Run length coding Line 100: (1,63) (0,87) (1,37) (0,5) (1,4) (0,556) (1,62) (0,210) Total 12166 runs, each run use 11 bits  Total = 133826 Bits (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Psychovisual Redundancy 8-bit gray scale image 4-bit gray scale image 4-bit IGS image False contours The eye does not response with equal sensitivity to all visual information. (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Improved Gray Scale Quantization Pixel i-1 i i+1 i+2 i+3 Gray level N/A 0110 1100 1000 1011 1000 0111 1111 0100 Sum 0000 0000 0110 1100 1001 0111 1000 1110 1111 0100 IGS Code N/A 0110 1001 1000 1111 + Algorithm 1. Add the least significant 4 bits of the previous value of Sum to the 8-bit current pixel. If the most significant 4 bit of the pixel is 1111 then add 0000 instead. Keep the result in Sum 2. Keep only the most significant 4 bits of Sum for IGS code.

Fidelity Criteria: how good is the compression algorithm Objective Fidelity Criterion RMSE, PSNR Subjective Fidelity Criterion: Human Rating (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Image Compression Models Reduce data redundancy Increase noise immunity Source encoder Channel encoder Channel Noise Source decoder Channel decoder

Source Encoder and Decoder Models Mapper Quantizer Symbol encoder Reduce interpixel redundancy Reduce psychovisual redundancy Reduce coding redundancy Source decoder Inverse mapper Symbol decoder

Channel Encoder and Decoder - Hamming code, Turbo code, …

Information Theory Measuring information Entropy or Uncertainty: Average information per symbol

Simple Information System Binary Symmetric Channel Source Destination A = {a1, a2} ={0, 1} z = [P(a1), P(a2)] B = {b1,b2} ={0, 1} v = [P(b1), P(b2)] (1-Pe) P(a1) P(a1)(1-Pe)+(1-P(a1))Pe Pe Source Destination Pe 1-P(a1) 1 1 (1-P(a1))(1-Pe)+P(a1)Pe Pe= probability of error (1-Pe) (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Binary Symmetric Channel Source Destination A = {a1, a2} ={0, 1} z = [P(a1), P(a2)] B = {b1,b2} ={0, 1} v = [P(b1), P(b2)] H(z|b1) = - P(a1|b1)log2P(a1|b1) - P(a2|b1)log2P(a2|b1) H(z|b2) = - P(a1|b2)log2P(a1|b2) - P(a2|b2)log2P(a2|b2) H(z) = - P(a1)log2P(a1) - P(a2)log2P(a2) H(z|v) = H(z|b1) + H(z|b2) Mutual information I(z,v)=H(z) - H(z|v) Capacity

Binary Symmetric Channel Let pe = probability of error

Binary Symmetric Channel (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Communication System Model 2 Cases to be considered: Noiseless and noisy (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Noiseless Coding Theorem Problem: How to code data as compact as possible? Shannon’s first theorem: defines the minimum average code word length per source that can be achieved. Let source be {A, z} which is zero memory source with J symbols. (zero memory = each outcome is independent from other outcomes) then a set of source output of n element be Example: for n = 3,

Noiseless Coding Theorem (cont.) Probability of each aj is Entropy of source : Each code word length l(ai) can be Then average code word length can be

Noiseless Coding Theorem (cont.) We get from then or The minimum average code word length per source symbol cannot lower than the entropy. Coding efficiency

Extension Coding Example H = 0.918 Lavg = 1 H = 1.83 Lavg = 1.89 (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Noisy Coding Theorem Problem: How to code data as reliable as possible? Example: Repeat each code 3 times: Source data = {1,0,0,1,1} Data to be sent = {111,000,000,111,111} Shannon’s second theorem: the maximum rate of coded information is j = code size r = Block length

Rate Distortion Function for BSC (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Error-Free Compression: Huffman Coding Huffman coding: give the smallest possible number of code symbols per source symbols. Step 1: Source reduction (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Error-Free Compression: Huffman Coding Step 2: Code assignment procedure The code is instantaneous uniquely decodable without referencing succeeding symbols. (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Near Optimal Variable Length Codes (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Arithmetic Coding Nonblock code: one-to-one correspondence between source symbols And code words does not exist. Concept: The entire sequences of source symbols is assigned a single arithmetic code word in the form of a number in an interval of real number between 0 and 1. (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Arithmetic Coding Example The number between 0.0688 and 0.06752 can be used to represent the sequence a1 a2 a3 a3 a4 0.2x0.2 0.04+0.4x0.04 0.056+0.4x0.016 (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

LZW Coding Lempel-Ziv-Welch coding : assign fixed length code words to variable length sequences of source symbols. 24 Bits 9 Bits (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

LZW Coding Algorithm 0. Initialize a dictionary by all possible gray values (0-255) 1. Input current pixel 2. If the current pixel combined with previous pixels form one of existing dictionary entries Then 2.1 Move to the next pixel and repeat Step 1 Else 2.2 Output the dictionary location of the currently recognized sequence (which is not include the current pixel) 2.3 Create a new dictionary entry by appending the currently recognized sequence in 2.2 with the current pixel 2.4 Move to the next pixel and repeat Step 1

LZW Coding Example Dictionary Location Entry 0 0 1 1 … … 255 255 0 0 1 1 … … 255 255 256 39-39 257 39-126 258 126-126 259 126-39 260 39-39-126 261 126-126-39 262 39-39-126-126 Currently recognized Sequences 39 126 39-39 126-126 39-39-126 Input pixel 39 126 Encoded Output (9 bits) 39 126 256 258 260

Bit-Plane Coding Original image Bit 7 Binary image compression Bit 6 … Bit 0 Binary image compression Bit plane images Example of binary image compression: Run length coding (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Bit Planes Bit 3 Bit 7 Bit 2 Bit 6 Original gray scale image Bit 1 (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Gray-coded Bit Planes Original bit planes Gray code: a7 g7 and a6 g6 ai= Original bit planes a5 g5 = XOR a4 g4 (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Gray-coded Bit Planes (cont.) There are less 0-1 and 1-0 transitions in grayed code bit planes. Hence gray coded bit planes are more efficient for coding. a3 g3 a2 g2 a1 g1 a0 g0

Relative Address Coding (RAC) Concept: Tracking binary transitions that begin and end eack black and white run (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Contour tracing and Coding Represent each contour by a set of boundary points and directionals. (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Error-Free Bit-Plane Coding (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Lossless VS Lossy Coding Mapper Symbol encoder Source encoder Reduce interpixel redundancy coding Lossless coding Mapper Quantizer Symbol encoder Source encoder Reduce interpixel redundancy psychovisual coding Lossy coding (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Transform Coding (for fixed resolution transforms) Encoder Construct nxn subimages Input image (NxN) Forward transform Symbol encoder Quantizer Quantization process causes The transform coding “lossy” Compressed image Decoder Construct nxn subimages Decompressed image Inverse transform Symbol decoder Examples of transformations used for image compression: DFT and DCT (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Transform Coding (for fixed resolution transforms) 3 Parameters that effect transform coding performance: Type of transformation Size of subimage Quantization algorithm

2D Discrete Transformation Forward transform: where g(x,y,u,v) = forward transformation kernel or basis function T(u,v) is called the transform coefficient image. Inverse transform: where h(x,y,u,v) = inverse transformation kernel or inverse basis function

Transform Example: Walsh-Hadamard Basis Functions N = 2m bk(z) = the kth bit of z Advantage: simple, easy to implement Disadvantage: not good packing ability N = 4 (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Transform Example: Discrete Cosine Basis Functions DCT is one of the most frequently used transform for image compression. For example, DCT is used in JPG files. N = 4 Advantage: good packing ability, modulate computational complexity N = 4 (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Transform Coding Examples Error Fourier RMS Error = 1.28 Hadamard Original image 512x512 pixels Subimage size: 8x8 pixels = 64 pixels RMS Error = 0.86 DCT Quatization by truncating 50% of coefficients (only 32 max cofficients are kept.) (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. RMS Error = 0.68

DCT vs DFT Coding DFT coefficients have abrupt changes at boundaries of blocks 1 Block Advantage of DCT over DFT is that the DCT coefficients are more continuous at boundaries of blocks. (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Subimage Size and Transform Coding Performance This experiment: Quatization is made by truncating 75% of transform coefficients DCT is the best Size 8x8 is enough (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Subimage Size and Transform Coding Performance DCT Coefficients Reconstructed by using 25% of coefficients (CR = 4:1) with 8x8 sub- images Zoomed detail Subimage size: 2x2 pixels Zoomed detail Original Zoomed detail Subimage size: 4x4 pixels Zoomed detail Subimage size: 8x8 pixels (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Quantization Process: Bit Allocation To assign different numbers of bits to represent transform coefficients based on importance of each coefficient: - More importance coefficeitns  assign a large number of bits - Less importance coefficients  assign a small number of bits or not assign at all 2 Popular bit allocation methods 1. Zonal coding : allocate bits based on the basis of maximum variance, using fixed mask for all subimages 2. Threshold coding : allocate bits based on maximum magnitudes of coefficients

Example: Results with Different Bit Allocation Methods Reconstructed by using 12.5% of coefficients (8 coefficients with largest variance are used) Reconstructed by using 12.5% of coefficients (8 coefficients with largest magnitude are used) Threshold coding Error Zonal coding Error Zoom details (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Zonal Coding Example Zonal mask Zonal bit allocation (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Thresholded coefficient Threshold Coding Example Threshold mask Thresholded coefficient ordering (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Thresholding Coding Quantization 3 Popular Thresholding Methods Method 1: Global thresholding : Use a single global threshold value for all subimages Method 2: N-largest coding: Keep only N largest coefficients Method 3: Normalized thresholding: each subimage is normalized by a normalization matrix before rounding Bit allocation Restoration before decompressing Example of Normalization Matrix Z(u,v) (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

DCT Coding Example (CR = 38:1) (CR = 67:1) Method: - Normalized Thresholding, - Subimage size: 8x8 pixels Error image RMS Error = 3.42 Blocking Artifact at Subimage boundaries Zoom details (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Wavelet Transform Coding: Multiresolution approach Quantizer Symbol encoder Input image (NxN) Compressed image Inverse wavelet decoder Decompressed Decoder Encoder Unlike DFT and DCT, Wavelet transform is a multiresolution transform.

What is a Wavelet Transform One up on a time, human uses a line to represent a number. For example = 25 With this numerical system, we need a lot of space to represent a number 1,000,000. Then, after an Arabic number system is invented, life is much easier. We can represent a number by a “digit number”: X,XXX,XXX An Arabic number is one kind of multiresolution Representation. The 1st digit = 1x The 2nd digit = 10x The 3rd digit = 100x … Like a number, any signal can also be represented by a multiresolution data structure, the wavelet transform.

What is a Wavelet Transform Wavelet transform has its background from multiresolution analysis and subband coding. Other important background: - Nyquist theorem: The minimun sampling rate needed for sampling a signal without loss of information is twice the maximum frequency of the signal. We can perform frequency shift by multiplying a complex sinusiodal signal in time domain.

Wavelet History: Image Pyramid If we smooth and then down sample an image repeatedly, we will get a pyramidal image: Coarser, decrease resolution Finer, increase resolution Pyramidal structured image (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Image Pyramid and Multiscale Decomposition NxN Down Sampling By 2 Image N/2xN/2 Smooth Question: What Information is loss after down Sampling? Up Sampling By 2 Interpolate Answer: Loss Information is A prediction error image: Prediction Error (loss details) NxN Predicted Image NxN + S -

Image Pyramid and Multiscale Decomposition (cont.) Hence we can decompose an image using the following process Image NxN Smooth and down sampling by 2 Approxi- -mation Image N/2xN/2 Smooth and down sampling by 2 Approxi- -mation Image N/4xN/4 …. Up sampling by 2 and interpolate Up sampling by 2 and interpolate - - + S + S Prediction Error NxN Prediction Error N/2xN/2

Image Pyramid and Multiscale Decomposition (cont.) Original Image NxN Approximation image N/8xN/8 Prediction error N/4xN/4 = Prediction error N/2xN2 Prediction error (residue) NxN Multiresolution Representation

Multiresolution Decomposition Process Note that this process is not a wavelet decomposition process ! (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Example of Pyramid Images Approximation Images (using Gaussian Smoothing) Prediction residues (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Subband Coding Subband decomposition process h0(n) Down LPF Sampling Approximation Down Sampling by 2 LPF a(n) N/2 points x(n) N points h1(n) Detail Down Sampling by 2 HPF Freq. shift by N/2 d(n) N/2 points All information of x(n) is completely preserved in a(n) and d(n).

Subband Coding (cont.) Subband reconstruction process g0(n) Up Sampling by 2 a(n) N/2 points Interpolation x(n) N points S g1(n) Up Sampling by 2 d(n) N/2 points Interpolation Freq. shift by N/2

Subband Coding (cont.) (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

2D Subband Coding (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Example of 2D Subband Coding Vertical detail: filtering in x- direction using h0(n) and in y- h1(n) Approximation: filtering in both x and y directions using h0(n) Horizontal detail: filtering in x- direction using h1(n) and in y- h0(n) Diagonal detail: filtering in both x and y directions using h1(n) (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

1D Discrete Wavelet Transformation h y(n) 2 d1(n) N/2 points x(n) N points h j(n) 2 h y(n) 2 d2(n) N/4 points h j(n) 2 Note that the number of points of x(n) and wavelet coefficients are equal. h y(n) 2 d3(n) N/8 points h j(n) 2 a3(n) N/8 points y(n) = a wavelet function j(n) = a scaling function Wavelet coefficients (N points)

1D Discrete Wavelet Transformation

2D Discrete Wavelet Transformation Original image NxN d = diagonal detail h = horizontal detail v = vertical detail a = approximation d1 h1 v1 a1 d2 h2 Level 1 Level 3 v2 a2 d3 h3 Level 2 v3 a3

2D Discrete Wavelet Transformation (cont.) Original image NxN a3 h3 h2 h1 v3 d3 v2 d2 v1 d1 Wavelet coefficients NxN d = diagonal detail: filtering in both x and y directions using h y(n) h = horizontal detail: filtering in x-direction using h y(n) and in y direction using h j(n) v = vertical detail: filtering in x-direction using h j(n) and in y direction using h y(n) a = approximation: filtering in both x and y directions using h j(n)

Example of 2D Wavelet Transformation Original Image Original image

Example of 2D Wavelet Transformation (cont.) LH1 HL1 HH1 LL1 LL1 The first level wavelet decomposition

Example of 2D Wavelet Transformation (cont.) LL2 LL2 LH1 HL1 HH1 LH2 HH2 HL2 The second level wavelet decomposition

Example of 2D Wavelet Transformation (cont.) LL3 HL3 HL2 HL1 LH3 HH3 LH2 HH2 LH1 HH1 The third level wavelet decomposition

Example of 2D Wavelet Transformation Orig One Layer Two Layers The image is first decomposed in different frequency regions, by using filter banks. A filter bank separates high and low frequencies in a reversible way. The high frequencies (3/4) are wavelet coded, while the low-frequencies are again processed by a filter bank. This is done repeatedly a number of times. The image can be progressively decoded resolution by resolution. Level 1 Level 2

Example of 2D Wavelet Transformation (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Examples: Types of Wavelet Transform Daubechies wavelets Haar wavelets Biorthogonal wavelets Symlets (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Wavelet Transform Coding for Image Compression Quantizer Symbol encoder Input image (NxN) Compressed image Inverse wavelet decoder Decompressed Decoder Encoder Unlike DFT and DCT, Wavelet transform is a multiresolution transform.

Wavelet Transform Coding Example (CR = 38:1) (CR = 67:1) Error Image RMS Error = 2.29 Error Image RMS Error = 2.96 Zoom details No blocking Artifact (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Wavelet Transform Coding Example (cont.) (CR = 108:1) (CR = 167:1) Error image RMS Error = 3.72 Error image RMS Error = 4.73 Zoom details (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Wavelet Transform Coding vs. DCT Coding DCT 8x8 (CR = 67:1) (CR = 67:1) Error image RMS Error = 2.96 Error image RMS Error = 6.33 Zoom details (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Type of Wavelet Transform and Performance (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

No. of Wavelet Transform Level and Performance

Threshold Level and Performance (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Table 8.14 (Cont’) (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Table 8.19 (Con’t) (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Lossless Predictive Coding Model (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Lossless Predictive Coding Example (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Lossy Predictive Coding Model (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Delta Modulation (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Linear Prediction Techniques: Examples (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Quantization Function (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Lloyd-Max Quantizers (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Lossy DCPM (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

DCPM Result Images (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Error Images of DCPM (Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.