1. 2D Fourier Theory for Image Analysis Mani Thomas CISC 489/689

2. Roadmap 2D image basis Fourier basis Scale-space representation  Gaussian pyramid  Laplacian pyramid Image mosaicing Gabor filters

3. Different basis representation Recall our discussion of basis vectors for coordinate systems:  Describe point as linear combination of ortho- gonal basis vectors: x = a 1 v 1 +... + a n v n The standard basis for images is the set of unit vectors corresponding to each pixel. A toy example:

4. Hadamard basis The standard basis is not the only one we can use to describe an image E.g., the Hadamard basis (basis images shown here for 2 x 2 images, where black = +1, white = -1)  For the previous example, we can express the image with these new (normalized) basis vectors as:  Coefficients of sum = projection of I onto new basis (dot product)  These are the coordinates of the image in “Hadamard space”  We can also say that I has undergone a Hadamard transform H :

5. Sinusoidal Basis Binary-valued, rectangular wave pattern of Hadamard basis doesn’t capture real image gradients well  Idea: Use smoothly-varying sinusoidal patterns at different frequencies, angles for basis images

6. Fourier Basis The Fourier basis uses the family of complex sinusoidal functions

7. 2D DFT Forward 2D DFT Inverse 2D DFT  (u, v) are the frequency coordinates while (x, y) are the spatial coordinates  M, N are the number of spatial pixels along the x, y coordinates

8. Fourier Basis v Real (cos) part Imaginary (sin) part (u, v)(1, 0)(0, 5)(1, 1)

9. Fourier transform in Matlab Discrete, 2-D Fourier & inverse Fourier transforms are computed by fft2 and ifft2, respectively fftshift : Move origin (DC component) to image center for display Example: >> I = imread(‘test.png’); % Load grayscale image >> F = fftshift(fft2(I)); % Shifted transform >> imshow(log(abs(F)),[]); % Show log magnitude >> imshow(angle(F),[]); % Show phase angle

10. Phase and Magnitude Output of the Fourier transform is a complex number  Decompose the complex number as the magnitude and phase components In Matlab: u = real(z), v = imag(z), r = abs(z), and theta = angle(z)

11. Image pyramid representation Smoothing means removing high frequencies  Smoothing required to avoid aliasing Fourier transform of a Gaussian is a Gaussian  Convolution is a multiplication  Gaussian suppresses high frequencies

12. Gaussian Pyramid Downsampling: Cut width, height in half at each iteration: Upsampling S " (I) : Double size of image, interpolate missing pixels Let the base (the finest resolution) of an n -level Gaussian pyramid be defined as P 0 = I. Then the i th level is reduced from the level below it by: Gaussian pyramid from Forsyth & Ponce

13. Laplacian pyramid The tip (the coarsest resolution) of an n -level Laplacian pyramid is the same as the Gaussian pyramid at that level: L n (I) = P n (I) The i th level is expanded from the level above according to L i (I) = P i (I) ¡ S " (P i+1 (I)) Synthesizing the original image: Get I back by summing upsampled Laplacian pyramid levels

14. Gaussian and Laplacian Gaussian – Smoothing pyramid  Each level is a smoothed and decimated signal of the previous Laplacian – Band pass filter of the images  Each level is the difference of a more smoothed and less smoothed image courtesy of Wolfram

15. Summation Property If L 0, L 1  L N is the sequence of laplacians L i = G i – EXPAND[G i + 1 ], 0<i<N L N = G N The steps used to construct the Laplacian can be reversed to get the original  Expand L i and add it to L i-1 to G i-1 G 0 =  i=0 N L i

16. Applications – Image Mosaicing Seamless joining of images to get a larger view

17. Multi-resolution spline interpolation

18. Laplacian level 4 Laplacian level 2 Laplacian level 0 left pyramidright pyramidblended pyramid

20. Image mosaicing Automatic mosaicing  Cross correlation to compute translation between images Matlab demo – Burt and Adelson’s paper  http://www.cs.huji.ac.il/course/2003/impr/splin e83.pdf http://www.cs.huji.ac.il/course/2003/impr/splin e83.pdf

21. image I image J JwJw warp refine + Pyramid of image JPyramid of image I image I image J Application - Coarse-to-Fine Estimation u=10 pixels u=5 pixels u=2.5 pixels u=1.25 pixels Slide from CS 223-B L9 by Richard Szeliski

22. Gabor filters Gaussian windowed Fourier Transform  Make convolution kernels from product of Fourier basis images and Gaussians £= Odd (sin) Even (cos) Frequency

23. Texture Representation: Filter Responses Choose a group of filters  Edge/Bar filters: Something like Gabor filters at different orientations, scales  Spot filters: Center-surround filters like a Gaussian/difference of Gaussians at multiple scales Run filters over image to get a set of response images  Each contains specific texture information

24. Example: Filter Responses from Forsyth & Ponce Filter bank Input image

25. Texture Similarity based on Response Statistics Collect statistics of responses over an image or subimage  Mean of squared response  Mean and variance of squared response Euclidean distance between vectors of response statistics for two images is measure of texture similarity

26. Conclusions 2D Fourier Theory Image pyramid representation  Gaussian pyramid  Laplacian pyramid Applications of Image Pyramids  Image Mosaicing Gaussian + Laplacian pyramids (Burt and Adelson) Texture statistics  Gabor filters