S.R.Subramanya1 Outline of Vector Quantization of Images

S.R.Subramanya2 VQ Coding Outline Divide data (signal) into non-overlapping vectors (Each vector contains ‘n’ elements (pixels/samples)) For each image vector : Find closest vector in codebook Get its index in codebook Encode indices

S.R.Subramanya3 VQ Decoding Outline Entropy decode the indices Lookup codebook for each index and retrieve vector Combine vectors to reconstruct image (data)

S.R.Subramanya4 VQ Terminology Vector : Group of ‘n’ elements (pixels/samples) Codebook : (Final) set of codevectors Training set : (Initial) set of codevectors Codevector : Vector derived from image/group of images

S.R.Subramanya5 Why VQ is better than scalar Quantization Scalar Quantizer :  Treats each pixel independently  Does not use correlation between neighboring pixels Vector Quantizer: Image (data) divided into vectors (blocks) Correlation among pixels in vectors is exploited Block size should be appropriate: Too large block : correlation is lost Too small block : More code vectors If no interpixel correlation, then no gain

S.R.Subramanya6 VQ Bitrate Vector size : n (say, n = p x p) pixels Codebook size : L vectors Codebook index size : bits Bit Rate of VQ : bits/pixel

S.R.Subramanya7 Distortion Measures xkxk Image closest matching code vector Image vectors : X j Codebook codevectors : V i, V1V1 V2V2 VkVk VLVL Mean Square Error (MSE) (Euclidean Distance) Weighted MSE

S.R.Subramanya8 Two Basic Kinds of Codebook Local Codebook  One codebook for each image  Codebook derived from vectors of one image  Good performance (Quality of reconstruction)  More overhead : (1) computation (2) Transmission of CB to decoder

S.R.Subramanya9 Two Basic Kinds of Codebook Global Codebook  One codebook for a class of images  Codebook derived from vectors of all images in the class  Less overhead (compared to local codebook)  Lower performance

S.R.Subramanya10 Major Issues in VQ Generation (construction) of codebook  concerns what needs to be included in the codebook Design of codebook  concerns structuring codebook entries to minimize search time

S.R.Subramanya11 Codebook Generation Generate codebook from a Training set Training Set: Set of vectors derived from image vectors Codevectors should minimize distortion Most commonly used algorithm : LBG algorithm LBG : Linde–Buzo–Gray algorithm

S.R.Subramanya12 LBG Algorithm Outline

S.R.Subramanya13 Codebook Initialization Three basic schemes: Random Perturb and Split (Bottoming) Pairwise Nearest Neighbor (PNN) clustering (PNN)

S.R.Subramanya14 Lloyd’s algorithm 1. Choice of an initial codebook. 2. All points x i are encoded; E VQ is evaluated. 3. If E VQ is small enough, then stop. 4. All centroids y j are replaced by the center- of-gravity of the data x i associated to y j in step Back to step 2.

S.R.Subramanya15 Lloyd: example 1. Initialization of the code boook. X i : data. y j :centroids

S.R.Subramanya16 Lloyd: example 2. Encoding (nearest-neighbor). x i : data. y j :centroids

S.R.Subramanya17 Lloyd: example 4. Decoding (center-of-gravity)

S.R.Subramanya18 Lloyd: example 2. Encoding (nearest-neighbor) → new borders

S.R.Subramanya19 Lloyd: example 4. Decoding (center-of-gravity) → new positions of centroids

S.R.Subramanya20 Lloyd: example 2. Encoding (nearest-neighbor) → new borders

S.R.Subramanya21 Lloyd: example 4. Decoding (center-of-gravity) → new positions of centroids

S.R.Subramanya22 Lloyd: example 2. Encoding (nearest-neighbor) → new borders

S.R.Subramanya23 Lloyd’s algorithm: the names Lloyd’s algorithm Generalized Lloyd’s algorithm Linde- Buzzo-Gray (LBG) algorithm K-means ISODATA … All based on the same principle.

S.R.Subramanya24 Lloyd’s algorithm: properties The codebook is modified only after the presentation of the whole dataset. The mean square error (E VQ ) decreases at each iteration. The risk of getting trapped in local minima is high. The final quantizer depends on the initial one

S.R.Subramanya25 How to initialize Lloyd’s algorithm ? 1. Randomly in the input space 2. The Q first data points x i Use the first 2 R vectors (code book vectors) in the training sequence as the initial codebook. 3. Q randomly chosen data points x i select several widely spaced words from the training sequence. This approach is sometimes called random code generation.

S.R.Subramanya26 4. ‘Product codes’: the product of scalar quantizers Collection of codebooks C i, each consisting of M i vectors. Then the product codebook C is defined as the collection of all M possible concatenations of m words drawn successively from the m codebooks C i. 5. Growing initial set: – a first centroid y 1 is randomly chosen (in the data set) – a second centroid y 2 is randomly chosen (in the data set); if d(y 1, y 2 ) > threshold, y 2 is kept – a third centroid y 3 is randomly chosen (in the data set); if d(y 1, y 3 ) > threshold and d(y 2, y 3 ) > threshold y 3 is kept – … How to initialize Lloyd’s algorithm ? (Cont.)

S.R.Subramanya27 How to initialize Lloyd’s algorithm ? (Cont.) 6. ‘pairwise nearest neighbor’: – a first codebook is built with all data points x i – the two centroids nearest one from another are merged (center-of gravity) – … Variant: – the increase of distortion ( E VQ ) is evaluated for the merge of each pair of centroids y j ; the pair giving the lowest increase is merged

S.R.Subramanya28 How to initialize Lloyd’s algorithm ? 7. Splitting – a first centroid y 1 is randomly chosen – a second centroid y 1 +ε is created; Lloyd algorithm is applied to the new codebook – two new centroids are created by perturbing the two existing ones; Lloyd algorithm is applied to this 4-centroids codebook – …

S.R.Subramanya29 Splitting

S.R.Subramanya30 Codebook Design Basic objective : Minimize search time for codevector Full (Exhaustive) Search : very expensive Design emphasis : Organization of codebook

S.R.Subramanya31 Tree-structured codebook Product codebook Mean/Residual VQ Interpolative/Residual VQ Gain/Shape VQ Classified VQ Finite State VQ Codebook Organizations

S.R.Subramanya32 Tree – Structured Codebook Codebook organized as M–ary tree Number of Levels : Code vectors stored at the leaves Intermediate node : ‘average’ of ‘codevectors’ of children Improved search time Increased Storage cost Performance inferior to full search

S.R.Subramanya33 Variations of Tree – Structured VQ Tapered trees:  Non – Uniform number of branches at nodes  Branches per node increases going down the tree ( Ex: 2 branches at level 1, 3 at level 2 etc.) Pruned Trees  Start with (full) large initial tree  Remove code vectors that do not reduce distortion

S.R.Subramanya34 Product Codebook Codebook : Cartesian product of many smaller codebooks Vector characterized by many independent features:f 1,f 2...,f N Separate codebook for each feature Smaller codebbok sizes : L 1,L 2...,L N Effective codebook size : L 1,L 2...,L N Actual storage + search complexity O(L 1 +L L N )

S.R.Subramanya35 Prediction/Residual Class VQ Predict Original Image Derive Residual Image Data used in prediction: Scalar Quantization + Encoding Residual Image : Vector Quantization Major Types:  Mean/Residual VQ  Interpolative/Residual VQ

S.R.Subramanya36 Mean/Residual VQ Image Vectors have similar variations about different mean/ends Remove mean from vectors fewer code vectors

S.R.Subramanya37 Mean/Residual VQ Scheme : Compute ‘mean’ of image vectors : M={m 1,m 2...,m N } Quantize M using scalar quantization : (Apply DPCM before Quantization further bitrate reduction) Subtract Quantized means from vector elements Residual vectors Quantize Residual vectors using VQ

S.R.Subramanya38 Interpolative/Residual VQ Subsample original image by l in each dimension (typically, l=8) Quantize subsampled value Upsample using Quantized subsampled values (typically, bilinear interpolation is used) Form Residual : (Original – Upsampled) Segment Residual to form 4 x 4 vectors Quantize Residual vectors using VQ

S.R.Subramanya39 Classified VQ Several codebooks Each codebook for a specific feature  ex. edges, smooth areas, etc. Codebooks could be of different sizes