1. Computer Vision Spring 2006 15-385,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm

2. Announcements Homework 1 is due today in class. Homework 2 will be out later this evening (due in 2 weeks). Start homeworks early. Post questions on bboard.

3. Image Processing and Filtering Lecture #5

4. Image as a Function We can think of an image as a function, f, f: R 2  R –f (x, y) gives the intensity at position (x, y) –Realistically, we expect the image only to be defined over a rectangle, with a finite range: f: [a,b] x [c,d]  [0,1] A color image is just three functions pasted together. We can write this as a “ vector- valued ” function:

5. Image as a Function

6. Image Processing Define a new image g in terms of an existing image f –We can transform either the domain or the range of f Range transformation: What kinds of operations can this perform?

7. Some operations preserve the range but change the domain of f : What kinds of operations can this perform? Still other operations operate on both the domain and the range of f. Image Processing

8. Linear Shift Invariant Systems (LSIS) Linearity: Shift invariance:

9. Example of LSIS Defocused image ( g ) is a processed version of the focused image ( f ) Ideal lens is a LSIS LSIS Linearity: Brightness variation Shift invariance: Scene movement (not valid for lenses with non-linear distortions)

10. Convolution LSIS is doing convolution; convolution is linear and shift invariant kernel h

11. Convolution - Example Eric Weinstein’s Math World

12. 12-2 1 1 1 1 1 Convolution - Example

13. Convolution Kernel – Impulse Response What h will give us g = f ? Dirac Delta Function (Unit Impulse) Sifting property:

14. Point Spread Function Optical System sceneimage Ideally, the optical system should be a Dirac delta function. Optical System point sourcepoint spread function However, optical systems are never ideal. Point spread function of Human Eyes

15. Point Spread Function normal visionmyopiahyperopia Images by Richmond Eye Associates astigmatism

16. Properties of Convolution Commutative Associative Cascade system

17. How to Represent Signals? Option 1: Taylor series represents any function using polynomials. Polynomials are not the best - unstable and not very physically meaningful. Easier to talk about “signals” in terms of its “frequencies” (how fast/often signals change, etc).

18. Jean Baptiste Joseph Fourier (1768-1830) Had crazy idea (1807): Any periodic function can be rewritten as a weighted sum of Sines and Cosines of different frequencies. Don’t believe it? –Neither did Lagrange, Laplace, Poisson and other big wigs –Not translated into English until 1878! But it’s true! –called Fourier Series –Possibly the greatest tool used in Engineering

19. A Sum of Sinusoids Our building block: Add enough of them to get any signal f(x) you want! How many degrees of freedom? What does each control? Which one encodes the coarse vs. fine structure of the signal?

20. Fourier Transform We want to understand the frequency  of our signal. So, let’s reparametrize the signal by  instead of x: f(x) F(  ) Fourier Transform F(  ) f(x) Inverse Fourier Transform For every  from 0 to inf, F(  ) holds the amplitude A and phase  of the corresponding sine –How can F hold both? Complex number trick!

21. Time and Frequency example : g(t) = sin(2pf t) + (1/3)sin(2p(3f) t)

22. Time and Frequency example : g(t) = sin(2pf t) + (1/3)sin(2p(3f) t) = +

23. Frequency Spectra example : g(t) = sin(2pf t) + (1/3)sin(2p(3f) t) = +

24. Frequency Spectra Usually, frequency is more interesting than the phase

25. = + = Frequency Spectra

26. = + =

27. = + =

28. = + =

29. = + =

30. =

32. FT: Just a change of basis...... * = M * f(x) = F(  )

33. IFT: Just a change of basis...... * = M -1 * F(  ) = f(x)

34. Fourier Transform – more formally Arbitrary functionSingle Analytic Expression Spatial Domain (x)Frequency Domain (u) Represent the signal as an infinite weighted sum of an infinite number of sinusoids (Frequency Spectrum F(u)) Note: Inverse Fourier Transform (IFT)

35. Also, defined as: Note: Inverse Fourier Transform (IFT) Fourier Transform

36. Fourier Transform Pairs (I) angular frequency ( ) Note that these are derived using

37. angular frequency ( ) Note that these are derived using Fourier Transform Pairs (I)

38. Fourier Transform and Convolution Let Then Convolution in spatial domain Multiplication in frequency domain

39. Fourier Transform and Convolution Spatial Domain (x)Frequency Domain (u) So, we can find g(x) by Fourier transform FT IFT

40. Properties of Fourier Transform Spatial Domain (x)Frequency Domain (u) Linearity Scaling Shifting Symmetry Conjugation Convolution Differentiation frequency ( ) Note that these are derived using

41. Properties of Fourier Transform

42. Example use: Smoothing/Blurring We want a smoothed function of f(x) H(u) attenuates high frequencies in F(u) (Low-pass Filter)! Then Let us use a Gaussian kernel

43. Image as a Discrete Function

44. Digital Images The scene is –projected on a 2D plane, –sampled on a regular grid, and each sample is –quantized (rounded to the nearest integer) Image as a matrix

45. Sampling Theorem Continuous signal: Shah function (Impulse train): Sampled function:

46. Sampling Theorem Continuous signal: Shah function (Impulse train): Sampled function:

47. Sampling Theorem Sampled function: Only if Sampling frequency

48. Nyquist Theorem If Aliasing When can we recover from ? Only if (Nyquist Frequency) We can use Thenand Sampling frequency must be greater than

49. Aliasing

50. Announcements Homework 1 is due today in class. Homework 2 will be out later this evening. Start homeworks early. Post questions on bboard.

51. Next Class Image Processing and Filtering (continued) Horn, Chapter 6