1. 3-D Computer Vision 83020 – Ioannis Stamos 3-D Computer Vision CSc 83020 Image Processing I/Filtering

2. 3-D Computer Vision 83020 – Ioannis Stamos Image Processing I/Filtering  Convolution (1-D)  Linear Shift Invariant Systems  Convolution (2-D)  Application: Noise:  Filtering: Averaging, Smoothing, Median..

3. 3-D Computer Vision 83020 – Ioannis Stamos Convolution (Important!) = g f h Used for: Derivatives, Edges, Matching, …

4. 3-D Computer Vision 83020 – Ioannis Stamos Convolution = g f h

5. Convolution f(ξ) ξ x h(ξ)

6. 3-D Computer Vision 83020 – Ioannis Stamos Convolution f(ξ) ξ x h(ξ) h(-ξ)

7. Convolution f(ξ) ξ x h(ξ) h(-ξ) ξ h(x-ξ) x

8. 3-D Computer Vision 83020 – Ioannis Stamos Convolution f(ξ) x ξ x ξ f(ξ) * h(x-ξ) g(x) : area under curve h(x-ξ)

9. Convolution f(ξ) x ξ ξ f(ξ) * h(x-ξ) g(x) : area under curve x Calculate g(x) for all x!! h(x-ξ)

10. 3-D Computer Vision 83020 – Ioannis Stamos Convolution g(x) x Calculate g(x) for all x!! => g(x): 1-D function h(x) f(x)

11. 3-D Computer Vision 83020 – Ioannis StamosExample ξ a(ξ) 1 1 ξ b(ξ) 1 1 c = a * b ?

12. 3-D Computer Vision 83020 – Ioannis StamosExample ξ a(ξ) 1 1 ξ b(-ξ) 1 1

13. 3-D Computer Vision 83020 – Ioannis StamosExample ξ a(ξ) 1 1 ξ b(-ξ) 1 1 ξ 1 1 a(ξ) b(x-ξ) x<-2 x

14. 3-D Computer Vision 83020 – Ioannis StamosExample ξ a(ξ) 1 1 ξ b(-ξ) 1 1 ξ 1 1 a(ξ) b(x-ξ) x=-2 x

15. 3-D Computer Vision 83020 – Ioannis StamosExample ξ a(ξ) 1 1 ξ b(-ξ) 1 1 ξ 1 1 a(ξ) b(x-ξ) -2<x<-1 x

16. 3-D Computer Vision 83020 – Ioannis StamosExample ξ a(ξ) 1 1 ξ b(-ξ) 1 1 ξ 1 1 a(ξ) b(x-ξ) -2<x<-1 x

17. 3-D Computer Vision 83020 – Ioannis StamosExample ξ a(ξ) 1 1 ξ b(-ξ) 1 1 ξ 1 1 a(ξ) b(x-ξ) x=-1 x

18. 3-D Computer Vision 83020 – Ioannis StamosExample ξ a(ξ) 1 1 ξ b(-ξ) 1 1 ξ 1 1 a(ξ) b(x-ξ) -1<x<0 x

19. 3-D Computer Vision 83020 – Ioannis StamosExample ξ a(ξ) 1 1 ξ b(-ξ) 1 1 ξ 1 1 a(ξ) b(x-ξ) x=0 x

20. 3-D Computer Vision 83020 – Ioannis StamosExample ξ a(ξ) 1 1 ξ b(-ξ) 1 1 ξ 1 1 a(ξ) b(x-ξ) 0<x<1 x

21. 3-D Computer Vision 83020 – Ioannis StamosExample ξ a(ξ) 1 1 ξ b(-ξ) 1 1 ξ 1 1 a(ξ) b(x-ξ) x=1 x

22. 3-D Computer Vision 83020 – Ioannis StamosExample ξ a(ξ) 1 1 ξ b(-ξ) 1 1 ξ 1 1 a(ξ) b(x-ξ) 1<x<2 x

23. 3-D Computer Vision 83020 – Ioannis StamosExample ξ a(ξ) 1 1 ξ b(-ξ) 1 1 ξ 1 1 a(ξ) b(x-ξ) x=2 x

24. 3-D Computer Vision 83020 – Ioannis StamosExample ξ a(ξ) 1 1 ξ b(-ξ) 1 1 ξ 1 1 a(ξ) b(x-ξ) x>2 x

25. 3-D Computer Vision 83020 – Ioannis StamosExample ξ a(ξ) 1 1 ξ b(ξ) 1 1 x 1 1 -22 c(x) c = a * b

26. 3-D Computer Vision 83020 – Ioannis Stamos Properties of Convolution  Commutativity: b * a = a * b.  Associativity: (a * b) * c = a * (b * c)  Cascaded system f g h1 h2 f g h1*h2 f h2*h1 g Equivalent Systems

27. 3-D Computer Vision 83020 – Ioannis Stamos Convolution Discrete f(ξ) ξ m h(ξ) m1m2 Discrete samples of continuous signal. Sampling at regular intervals. Example : Scanline n2n1

28. 3-D Computer Vision 83020 – Ioannis Stamos Convolution Discrete f(ξ) ξ m h(ξ) m1m2 n2n1 m-n1m-n2

29. 3-D Computer Vision 83020 – Ioannis Stamos One Scan Line: 1-D discrete signal 0 450 f(ξ), m1=0, m2=450.

30. 3-D Computer Vision 83020 – Ioannis Stamos One Scan Line: 1-D discrete signal 0 450 4-4 1/9 f(ξ), m1=0, m2=450. h(ξ), n1=-4, n2=4. f*h = ?

31. One Scan Line: 1-D discrete signal 0 450 f(ξ) h(m- ξ ) mm+4m-4 m 1/9 f*h = ?

32. One Scan Line: 1-D discrete signal 0 450 f(ξ) h(m- ξ ) mm+4m-4 m 1/9 f*h = ?

33. One Scan Line: 1-D discrete signal 0 450 f(ξ), m1=0, m2=450. h: filter or mask. f*h: filtered version of f. In this case h spatially averages f in a neighborhood of 9 samples.

34. 3-D Computer Vision 83020 – Ioannis Stamos Recap  1-D Convolution  Continuous vs. Discrete.  Finite vs. Infinite signals (spatial domain).  Filtering.

35. 3-D Computer Vision 83020 – Ioannis Stamos Linear Shift Invariant Systems f(x) g(x) Linearity: f1(x) g1(x) f2(x) g2(x) af1(x)+bf2(x)ag1(x)+bg2(x)

36. Linear Shift Invariant Systems f(x) g(x) Shift Invariance: f(x-a) g(x-a) x x f(x) g(x) x x f(x-a) g(x) a a

37. 3-D Computer Vision 83020 – Ioannis Stamos Properties of Convolution  Commutativity: b * a = a * b.  Associativity: (a * b) * c = a * (b * c)  Cascaded system f g h1 h2 f g h1*h2 f h2*h1 g Equivalent Systems

38. 3-D Computer Vision 83020 – Ioannis Stamos Convolution Used for: Derivatives, Edges, Matching, … f(x) g(x) h(x) Convolution: LINEAR & SHIFT INVARIANT Also, any LSIS is doing a CONVOLUTION!

39. 3-D Computer Vision 83020 – Ioannis Stamos Properties of Convolution  Commutativity: b * a = a * b.  Associativity: (a * b) * c = a * (b * c)  Cascaded system f g h1 h2 f g h1*h2 f h2*h1 g Equivalent Systems

40. Example of LSIS: g f Defocused image g: Processed version of Focused image f. Ideal Lens: f(x) g(x) LSIS Linearity: Brightness Variations. Shift Invariance: Scene Movement. Note: Not valid for lenses with non-linear distortions (aberrations). Study of LSIS leads to useful algorithms for processing images!

41. f g h Can we find h? What f will give us g=h? x δ(x) 1/(2ε) 2ε System as a black box x 1/(2ε) 2ε δ(x) 1/(2ε) 2ε x Decrease ε

42. 3-D Computer Vision 83020 – Ioannis Stamos Unit Impulse Function: x δ(x) 1 1/(2ε) 2ε System as a black box f(x)=δ(x) h(x) IMPULSE RESPONSE Impulse Response

43. 3-D Computer Vision 83020 – Ioannis Stamos Impulse Response f(x)=δ(x) h(x) IMPULSE RESPONSE

44. 3-D Computer Vision 83020 – Ioannis Stamos Image Formation Scene Image Optical System Point Source δ(x) Optical System Point Spread Function h(x) In an ideal system h(x)=δ(x) Optical Systems are never ideal! Human Eye: Point Spread Function….

45. 2-D Convolution = f(x,y) Input Image h(x,y) Filter g(x,y) Output Image

46. Discrete Convolution 11111 12221 12321 12221 11111 h g (larger than f) (0,0) f x y

47. Discrete Convolution 11111 12221 12321 12221 11111 1111112221 12321 12221 11111 fh

48. 3-D Computer Vision 83020 – Ioannis Stamos Discrete Convolution – 2D flip 11111 43221 45321 45531 444411444413554 13354 13334 11111

49. 3-D Computer Vision 83020 – Ioannis Stamos Discrete Convolution 1 2 3 4 5 4 3 2 1 2 6 10 14 16 14 10 6 2 3 10 20 28 32 28 20 10 3 4 14 28 42 48 42 28 14 4 5 16 32 48 57 48 32 16 5 4 14 28 42 48 42 28 14 4 3 10 20 28 32 28 20 10 3 2 6 10 14 16 14 10 6 2 1 2 3 4 5 4 3 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 = gfh

50. 3-D Computer Vision 83020 – Ioannis Stamos Discrete Convolution 1 2 3 4 5 4 3 2 1 2 6 10 14 16 14 10 6 2 3 10 20 28 32 28 20 10 3 4 14 28 42 48 42 28 14 4 5 16 32 48 57 48 32 16 5 4 14 28 42 48 42 28 14 4 3 10 20 28 32 28 20 10 3 2 6 10 14 16 14 10 6 2 1 2 3 4 5 4 3 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1

51. 3-D Computer Vision 83020 – Ioannis Stamos Discrete Convolution 1 2 3 4 5 4 3 2 1 2 6 10 14 16 14 10 6 2 3 10 20 28 32 28 20 10 3 4 14 28 42 48 42 28 14 4 5 16 32 48 57 48 32 16 5 4 14 28 42 48 42 28 14 4 3 10 20 28 32 28 20 10 3 2 6 10 14 16 14 10 6 2 1 2 3 4 5 4 3 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1

52. 3-D Computer Vision 83020 – Ioannis Stamos Discrete Convolution 1 2 3 4 5 4 3 2 1 2 6 10 14 16 14 10 6 2 3 10 20 28 32 28 20 10 3 4 14 28 42 48 42 28 14 4 5 16 32 48 57 48 32 16 5 4 14 28 42 48 42 28 14 4 3 10 20 28 32 28 20 10 3 2 6 10 14 16 14 10 6 2 1 2 3 4 5 4 3 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1

53. 3-D Computer Vision 83020 – Ioannis Stamos Discrete Convolution 1 2 3 4 5 4 3 2 1 2 6 10 14 16 14 10 6 2 3 10 20 28 32 28 20 10 3 4 14 28 42 48 42 28 14 4 5 16 32 48 57 48 32 16 5 4 14 28 42 48 42 28 14 4 3 10 20 28 32 28 20 10 3 2 6 10 14 16 14 10 6 2 1 2 3 4 5 4 3 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1

54. 3-D Computer Vision 83020 – Ioannis Stamos Discrete Convolution 1 2 3 4 5 4 3 2 1 2 6 10 14 16 14 10 6 2 3 10 20 28 32 28 20 10 3 4 14 28 42 48 42 28 14 4 5 16 32 48 57 48 32 16 5 4 14 28 42 48 42 28 14 4 3 10 20 28 32 28 20 10 3 2 6 10 14 16 14 10 6 2 1 2 3 4 5 4 3 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1

55. 3-D Computer Vision 83020 – Ioannis Stamos Discrete Convolution 1 2 3 4 5 4 3 2 1 2 6 10 14 16 14 10 6 2 3 10 20 28 32 28 20 10 3 4 14 28 42 48 42 28 14 4 5 16 32 48 57 48 32 16 5 4 14 28 42 48 42 28 14 4 3 10 20 28 32 28 20 10 3 2 6 10 14 16 14 10 6 2 1 2 3 4 5 4 3 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1

56. 3-D Computer Vision 83020 – Ioannis Stamos Discrete Convolution 1 2 3 4 5 4 3 2 1 2 6 10 14 16 14 10 6 2 3 10 20 28 32 28 20 10 3 4 14 28 42 48 42 28 14 4 5 16 32 48 57 48 32 16 5 4 14 28 42 48 42 28 14 4 3 10 20 28 32 28 20 10 3 2 6 10 14 16 14 10 6 2 1 2 3 4 5 4 3 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1

57. 3-D Computer Vision 83020 – Ioannis Stamos Discrete Convolution 1 2 3 4 5 4 3 2 1 2 6 10 14 16 14 10 6 2 3 10 20 28 32 28 20 10 3 4 14 28 42 48 42 28 14 4 5 16 32 48 57 48 32 16 5 4 14 28 42 48 42 28 14 4 3 10 20 28 32 28 20 10 3 2 6 10 14 16 14 10 6 2 1 2 3 4 5 4 3 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1

58. Discrete Convolution 1 2 3 4 5 4 3 2 1 2 6 10 14 16 14 10 6 2 3 10 20 28 32 28 20 10 3 4 14 28 42 48 42 28 14 4 5 16 32 48 57 48 32 16 5 4 14 28 42 48 42 28 14 4 3 10 20 28 32 28 20 10 3 2 6 10 14 16 14 10 6 2 1 2 3 4 5 4 3 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 = g f h g is larger than f. Finite borders!

59. 3-D Computer Vision 83020 – Ioannis Stamos Discrete Convolution = g h f

60. 3-D Computer Vision 83020 – Ioannis Stamos Discrete Convolution = g h f Commutativity

61. 3-D Computer Vision 83020 – Ioannis Stamos Gaussian Noise x y

62. 3-D Computer Vision 83020 – Ioannis Stamos Linear Filtering m: size of filter (odd number) m/2: integer (i.e. if m=5, m/2=2)

63. 3-D Computer Vision 83020 – Ioannis Stamos Linear Filtering m=3

64. 3-D Computer Vision 83020 – Ioannis Stamos Mean Filtering - Averaging x y m=3

65. Gaussian Filtering x y Separable Kernel

66. 3-D Computer Vision 83020 – Ioannis Stamos Separable convolution x x x x x x x x x 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 y y y y y = IrIr I grgr

67. 3-D Computer Vision 83020 – Ioannis Stamos Separable convolution x x x x x x x x x 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 y y y y y =

68. 3-D Computer Vision 83020 – Ioannis Stamos Separable convolution x x x x x x x x x 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 y y y y y =

69. 3-D Computer Vision 83020 – Ioannis Stamos Separable convolution x x x x x x x x x 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 y y y y y =

70. 3-D Computer Vision 83020 – Ioannis Stamos Separable convolution x x x x x x x x x 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 y y y y y =

71. 3-D Computer Vision 83020 – Ioannis Stamos Separable convolution x x x x x x x x x 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 y y y y y =

72. 3-D Computer Vision 83020 – Ioannis Stamos Separable convolution x x x x x x x x x 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 y y y y y =

73. 3-D Computer Vision 83020 – Ioannis Stamos Separable convolution x x x x x x x x x 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 y y y y y = IrIr I grgr

74. 3-D Computer Vision 83020 – Ioannis Stamos Separable convolution x x x x x x x x x = x x x x x x x x x y IGIG IrIr gcgc

75. 3-D Computer Vision 83020 – Ioannis Stamos Separable convolution x x x x x x x x x = x x x x x x x x x y

76. 3-D Computer Vision 83020 – Ioannis Stamos Separable convolution x x x x x x x x x = x x x x x x x x x y

77. 3-D Computer Vision 83020 – Ioannis Stamos Separable convolution x x x x x x x x x = x x x x x x x x x y IGIG IrIr gcgc

78. 3-D Computer Vision 83020 – Ioannis Stamos Separable convolution Two 1-D convolutions are more efficient than one 2-D convolution!

79. 3-D Computer Vision 83020 – Ioannis Stamos Gaussian Filtering

80. 3-D Computer Vision 83020 – Ioannis Stamos Constructing a Gaussian Filter 1-D Gaussian Mask g w: width of mask (in pixels) σ: continuous Gaussian kernel Relation between w and σ

81. 3-D Computer Vision 83020 – Ioannis Stamos Size of the mask

82. 3-D Computer Vision 83020 – Ioannis Stamos Noise – Median Filter Cannot implement with a convolution mask.

83. 3-D Computer Vision 83020 – Ioannis Stamos Noise – Median Filter Cannot implement with a convolution mask.