1. MSU CSE 803 Stockman Fall 2009 Vectors [and more on masks] Vector space theory applies directly to several image processing/representation problems

2. MSU CSE 803 Stockman Fall 2009 Image as a sum of “basic images” What if every person’s portrait photo could be expressed as a sum of 20 special images?  We would only need 20 numbers to model any photo  sparse rep on our Smart card.

3. MSU CSE 803 Stockman Fall 2009 Efaces 100 x 100 images of faces are approximated by a subspace of only 4 100 x 100 “images”, the mean image plus a linear combination of the 3 most important “eigenimages”

4. MSU CSE 803 Stockman Fall 2009 The image as an expansion

5. MSU CSE 803 Stockman Fall 2009 Different bases, different properties revealed

6. MSU CSE 803 Stockman Fall 2009 Fundamental expansion

7. MSU CSE 803 Stockman Fall 2009 Basis gives structural parts

8. MSU CSE 803 Stockman Fall 2009 Vector space review, part 1

9. MSU CSE 803 Stockman Fall 2009 Vector space review, Part 2 2

10. MSU CSE 803 Stockman Fall 2009 A space of images in a vector space M x N image of real intensity values has dimension D = M x N Can concatenate all M rows to interpret an image as a D dimensional 1D vector The vector space properties apply The 2D structure of the image is NOT lost

11. MSU CSE 803 Stockman Fall 2009 Orthonormal basis vectors help

12. MSU CSE 803 Stockman Fall 2009 Represent S = [10, 15, 20]

13. MSU CSE 803 Stockman Fall 2009 Projection of vector U onto V

14. MSU CSE 803 Stockman Fall 2009 Normalized dot product Can now think about the angle between two signals, two faces, two text documents, …

15. MSU CSE 803 Stockman Fall 2009 Every 2x2 neighborhood has some constant, some edge, and some line component Confirm that basis vectors are orthonormal

16. MSU CSE 803 Stockman Fall 2009 Roberts basis cont. If a neighborhood N has large dot product with a basis vector (image), then N is similar to that basis image.

17. MSU CSE 803 Stockman Fall 2009 Standard 3x3 image basis Structureless and relatively useless!

18. MSU CSE 803 Stockman Fall 2009 Frie-Chen basis Confirm that bases vectors are orthonormal

19. MSU CSE 803 Stockman Fall 2009 Structure from Frie-Chen expansion Expand N using Frie- Chen basis

20. MSU CSE 803 Stockman Fall 2009 Sinusoids provide a good basis

21. MSU CSE 803 Stockman Fall 2009 Sinusoids also model well in images

22. MSU CSE 803 Stockman Fall 2009 Operations using the Fourier basis

23. MSU CSE 803 Stockman Fall 2009 A few properties of 1D sinusoids They are orthogonal Are they orthonormal?

24. MSU CSE 803 Stockman Fall 2009 F(x,y) as a sum of sinusoids

25. MSU CSE 803 Stockman Fall 2009 Spatial direction and frequency in 2D

26. MSU CSE 803 Stockman Fall 2009 Continuous 2D Fourier Transform To compute F(u,v) we do a dot product of our image f(x,y) with a specific sinusoid with frequencies u and v

27. MSU CSE 803 Stockman Fall 2009 Power spectrum from FT

28. MSU CSE 803 Stockman Fall 2009 Examples from images Done with HIPS in 1997

29. MSU CSE 803 Stockman Fall 2009 Descriptions of former spectra

30. MSU CSE 803 Stockman Fall 2009 Discrete Fourier Transform Do N x N dot products and determine where the energy is. High energy in parameters u and v means original image has similarity to those sinusoids.

31. MSU CSE 803 Stockman Fall 2009 Bandpass filtering

32. MSU CSE 803 Stockman Fall 2009 Convolution of two functions in the spatial domain is equivalent to pointwise multiplication in the frequency domain

33. MSU CSE 803 Stockman Fall 2009 LOG or DOG filter Laplacian of Gaussian Approx Difference of Gaussians

34. MSU CSE 803 Stockman Fall 2009 LOG filter properties

35. MSU CSE 803 Stockman Fall 2009 Mathematical model

36. MSU CSE 803 Stockman Fall 2009 1D model; rotate to create 2D model

37. MSU CSE 803 Stockman Fall 2009 1D Gaussian and 1 st derivative

38. MSU CSE 803 Stockman Fall 2009 2 nd derivative; then all 3 curves

39. MSU CSE 803 Stockman Fall 2009 Laplacian of Gaussian as 3x3

40. MSU CSE 803 Stockman Fall 2009 G(x,y): Mexican hat filter

41. MSU CSE 803 Stockman Fall 2009 Convolving LOG with region boundary creates a zero-crossing Mask h(x,y) Input f(x,y)Output f(x,y) * h(x,y)

42. MSU CSE 803 Stockman Fall 2009

43. LOG relates to animal vision

44. MSU CSE 803 Stockman Fall 2009 1D EX. Artificial Neural Network (ANN) for computing g(x) = f(x) * h(x) level 1 cells feed 3 level 2 cells level 2 cells integrate 3 level 1 input cells using weights [-1,2,-1]

45. MSU CSE 803 Stockman Fall 2009 Experience the Mach band effect

46. MSU CSE 803 Stockman Fall 2009 Simple model of a neuron

47. MSU CSE 803 Stockman Fall 2009 Output conditioning: threshold versus smoother output signal

48. MSU CSE 803 Stockman Fall 2009 3D situation in the eye Neuron c has + input to neuron A but - input to neuron B. Neuron d has + input to neuron B but – input to neuron A. Neuron b gives no input to neuron B: it is not in the receptive field of B.

49. MSU CSE 803 Stockman Fall 2009 Receptive fields

50. MSU CSE 803 Stockman Fall 2009 Experiments with cats/monkeys Stabilize/drug animal to stare Place delicate probe in visual network Move step edge across FOV Probe shows response function when the edge images to receptive field Slightly moving the probe produces similar signal when edge is nearby

51. MSU CSE 803 Stockman Fall 2009 Canny edge detector uses LOG filter

52. MSU CSE 803 Stockman Fall 2009 Summary of LOG filter Convenient filter shape Boundaries detected as 0-crossings Psychophysical evidence that animal visual systems might work this way (your testimony) Physiological evidence that real NNs work as the ANNs