Color Imaging Analysis of Spatio-chromatic Decorrelation for Colour Image Reconstruction Mark S. Drew and Steven Bergner School of Computing Science, Simon Fraser University, Canada

Color Imaging /27 - Use of PCA vs. ICA — what’s the difference? - How do you do ICA? - What does this have to do with images? - The objective: best characterize image blocks using ICA on color image block data == spatio (blocks are 16x16, say)- chromatic (x3); assign bits in bit allocation according to the importance of each ICA coefficient  data compression. I. Overview

Color Imaging /27 Best characterize image  colour and spatial information. Colour: we think of using PCA (Principal Component Anaysis): discover main colour axes. Is this best, given our objective? Spatial: use spatial Fourier filters? Gabor wavelets? Etc. Here, we’ll use ICA (Independent Component Anaysis) to derive best colour and spatial decomposition at once, for decorrelation, compression, and reconstruction.

Color Imaging /27 II. ICA  What is it? ICA is a form of “Blind Source Separation”  To explain, consider audio signals (in an Imaging conference!). Consider 2 speakers, and 2 microphones: s1s1 s2s2 -sources x1x1 x2x2 -data

Color Imaging /27 Can we disentangle s 1, s 2 from measured data x 1, x 2 ? == The “cocktail party problem”. An example:

Color Imaging /27 ICA: Order and sign not determined.

Color Imaging /27 What about PCA? Writing the signals in terms of reduced set of sources s 1, s 2, s 3,..., for higher-dimensional data, we can do a better job in compression. 

Color Imaging /27 III. ICA  How to do it? Model: ( x was 2xN in the audio example.) mixing matrix separating matrix

Color Imaging /27 Driving idea for finding sources: s 1, s 2 are statistically independent == information about one gives no knowledge re. the other. Not just uncorrelated: covariance = 0 ==PCA

Color Imaging /27 If independent as well, the pdf is separable: joint pdf marginal pdf’s which implies for any functions, !  useful for solving.

Color Imaging /27 So, to do ICA, start with uncorrelated signals (using PCA) == simplifies. Main tool: Non-Gaussian is independent. Central Limit Theorem: the sum of two independents is more like a Gaussian than is either one. So  we have sums. To get s, make a linear combination of x ’s that is as non-Gaussian as possible.

Color Imaging /27 One way: (…many others) A Gaussian has zero kurtosis. For zero mean y, Rescale y to variance=1:  just use We seek a signal that maximizes kurtosis.

Color Imaging /27 Algorithm  “whiten” the data: zero mean, + linear transform to make uncorrelated, variance=1. First, PCA: orthogonal U with In the new coordinate system, Why?  Now with orthogonal  simpler to search for.

Color Imaging /27 Algorithm -whiten x -we seek a column w of orthogonal W, with, that maximizes kurtosis: Euler eqn.: Code 1. Initialize w randomly, with stop when

Color Imaging /27 Matlab

Color Imaging /27 IV. ICA for Images Previous work: Greyscale and colour imagery using PCA and ICA. For colour images, x could be 3-vector pixels. But get spatial as well if use n  n tiles (nice illustration in Süsstrunk et al., CGIV’04 [using PCA on raw CFA data]) We show here that compression is better using ICA+colour+spatial info.

Color Imaging /27 16 x 16 greyscale tiles ICA finds “sparse” features: ICA (16 2 x1 greyscale data) localization in space

Color Imaging /27 PCA vs. ICA (3x1 data) (no spatial information) With colour:

Color Imaging /27 PCA (4x4 x3) DCT (4x4 x3) -less axis-aligned -ordering by variance-accounted- for is different: pure colour axes appear first -pure colour axes appear later, after luminance frequencies -separates colour from luminance PCA vs. DCT (4x4 x3 data) -Colour: luminance, blue-yellow, red-green

Color Imaging /27 ICA (4x4 x3) PCA (4x4 x3) again PCA vs. ICA -colour less separate from spatial information -combined localization in space and frequency -patterns not rectangular  more like Gabor functions (Gaussian-modulated sine functions) -localization in frequency

Color Imaging /27 ICA (4x4) ICA (5x5) ICA (8x8) ICA (16x16)

Color Imaging /27 SNRSNR Colour vs. Greyscale:  Compression  performance (Generic basis) Colour Greyscale - Higher reconstruction quality (SNR) for larger patches - Colour has better quality than grey, at equal compression Better quality 

Color Imaging /27 ICA vs. PCA (Specific basis: image = ) - ICA much better than PCA: higher compression for same SNR - ICA  increased quality with larger patches, for equal compression ICA PCA Better quality 

Color Imaging /27 ICA vs. PCA A. ICA does better separating axes such that they influence each other least  better entropy coding B.Colour aids in compression C.Large patch sizes and low rate encoding  At equal compression, SNR (quality) better for ICA

Color Imaging /27 ICA vs. PCA: Image reconstruction (compression ratio: 1:12) ICA PSNR= DCT: PSNR= 31.97

Color Imaging /27 Another image ICA PSNR= DCT: PSNR=  7:1 Orig ICA DCT --blocking

Color Imaging /27 The Future: Video Bases [submitted] ICA (6x6x6) PCA (6x6x6)