1. Image processing Honza Černocký, ÚPGM

2. 1D signal 2

3. Images 2D – grayscale photo Multiple 2D – color photo 3D – medical imaging, vector graphics, point-clouds (depth information, Kinect) Video – time is another dimension 3

4. Image 4 K L

5. Mathematically 5

6. Representation of pixels For storage – quantization 8 (normal) / 16 (HDR) bits 0…255 etc For computing [0…1] floats 6

7. Operations on pixels – more brightness 7

8. Operations on pixels – less brightness 8

9. More contrast 9

10. Less contrast 10

11. What about bad values ? x[k,l] 1 Let them be … Clip them: 11

12. Use statistics of values Histogram 12

13. Histogram equalization 13

14. Thresholding 14

15. Filtering 15 signal filter signal Task ? Obtain a new signal with desired properties.

16. Filtering – reminder of 1D 16

17. Filtering – reminder of 1D 17

18. 2D filter 18 I J

19. Filtering … Edge ? Insert zeros for example … 19

20. Coefficients h[k,l] Terminology Coefficients Mask Convolution kernel … We want them to have some sense … not to change the dynamics of the signal 20

21. Wire 21

22. Smoothing 22

23. Vertical edge detector 23

24. What to do with negative samples? Let them as they are for further computation… Or convert to [0…1] for visualization – for example take absolute value 24

25. Horizontal edge detector 25

26. Both together … 26

27. Denoising… Mask 9x9, values 1/81 … zoom noise_low_pass.png 27

28. Denoising II – median filter zoom noise_median.png 28

29. Spectral analysis Task: Determine, what is in the signal on different frequencies Why ? –Visualization, –Feature extraction (think of Facebook) –Filtering (conversion to spectral domain and multiplication therein can be more efficient) –Coding (think of JPEG) 29

30. Reminder – 1D What is in ? 30

31. Reminder – 1D cosine bases 31 Problem with the phase – the signal “wave” begins not necessarily at zero …

32. OK … 32

33. Also OK … 33

34. Bad bad bad  34 How come zero when ?

35. Complex exponentials 35

36. 36

37. 37

38. 1D ultimate result - DFT 38

39. Now finally 2D Correlation = determination of similarity = projection to bases We have seen this, it’s boring, it’s all the time the same … YES IT IS ! 39

40. Analyzing signal = d.c. c=4.8287e+03 c=6.8287e+03 c=8.8287e+03 40

41. Analyzzing signal – horizontal cosine 41 Changes only in one direction We’ll need to visualize negative values too.

42. Geeks: how to do this in Matlab ?? 42

43. Analysis for 43 c=4.5475e-13 c=2500 c=3.4106e-13

44. 2 x faster horizontal cos 44 c=2.9843e-13 c=1.5064e-12 c=2.5000e+03

45. Vertical cos 45 c= -7.7716e-14 c=2.5000e+03 c=6.9944e-13

46. 2 x faster vertical cos 46 c= 4.9960e-13 c=2.5000e+03 c=-65.4694

47. Mix of both directions … 47

48. Generalization m/K – horizontal frequency n/L – vertical frequency 48

49. Reasonable ranges of frequencies ? m/K and n/L 0 … ½ => OK > ½ => hm… 49 K L x[k,l] X[k,l] K/2 L/2 0 0

50. More on frequencies … 1D 2D 50

51. Phase – problem again  51 c=2.5000e+03 c=9.9476e-13 c=-5.6843e-14

52. Solution – complex exponentials (again) Reminder what is what : k,l indices of pixels (input) m,n indices of frequencies (result) m/K normalized vertical frequency n/L normalized horizontal frequency 52

53. Imagining it I. 53

54. Imagining it II. 54

55. Correlation with x[n] 55 Re(c) = 2500 Im(c) = 0 |c| = 2500

56. Correlation with x[n] 56 Re(c) = 1250 Im(c) = -2165 |c| = 2500

57. Correlation with x[n] 57 Re(c) = -772 Im(c) = 2378 |c| = 2500

58. Change in both directions…X[7,3] 58 Re(c) = 0 Im(c) = 0 |c| = 0 … not similar

59. Change in both directions…X[7,3] 59 Re(c) = 2165 Im(c) = 1250 |c| = 2500 … similar

60. 2D DFT using 2 x 1D DFT so that 60

61. 2D DFT for a real signal 61

62. 62

63. Something with higher frequencies 63

64. 64

65. DFT produces K x L points ! 65 K L x[k,l] K L X[k,l]

66. The whole result for k=0…K-1, l=0…L-1 66

67. Symmetry ? Source: http://www-personal.umich.edu/~hyunjinp/notes/n-dft.pdfhttp://www-personal.umich.edu/~hyunjinp/notes/n-dft.pdf 67 K/2 0 K-1 0 L/2 L-1

68. Re-shuffling… Low frequencies to the center 68

69. Inverse 2D DFT 69

70. Playing with frequencies 70

71. 71

72. DCT Why ? People don’t like complex exponentials People don’t like symmetries and “useless” values … For image K x L, we want K x L real values. 72

73. 1D DCT 73 + normalization of coefficients…

74. 2D DCT bases Source: https://en.wikipedia.org/wiki/Di screte_cosine_transform#Mult idimensional_DCTs https://en.wikipedia.org/wiki/Di screte_cosine_transform#Mult idimensional_DCTs 74

75. Filtering in DCT 75

76. 76

77. JPEG Source: http://www.eetimes.com/document.asp?doc_id=1225736http://www.eetimes.com/document.asp?doc_id=1225736 More in computer graphics courses and lab exercise. 77

78. SUMMARY Image is a 2D signal Filtering –Convolution, analogy with 1D FIR filters –No feedback (IIR) in images Sweeping mask over the image, multiplying everything that is underneath with its coefficients, adding. 78

79. SUMMARY II. Frequency analysis 2D –Similar to 1D – projection to bases –Image frequencies also have some sense. –Cos bases are a good exercise, but not enough. 2D DFT –Projection/similarity/correlation with complex exponentials –Coefficients are complex – magnitude and angle. –Lots of symmetries in the resulting DFT matrix –Can use spectrum for filtering 79

80. SUMMARY III. DCT –2x „slower“ bases than DFT –Slightly More complicated definition –Produces real coefficients, low frequencies (only!) at the beginning. –Use in JPEG 80

81. TO BE DONE Determining frequency response of a 2D filter How is it exactly with the symmetries of 2D DFT ? Why are 1D and 2D DCT exactly defined and why this half-sample shift ? Colors (color models, etc). How does the face matching on Facebook work? –Hint: https://research.facebook.com/researchers/68463 9631606527/yann-lecun/ https://research.facebook.com/researchers/68463 9631606527/yann-lecun/ 81

82. The END 82