1. Antal Nagy Department of Image Processing and Computer Graphics University of Szeged 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary1

2.  Human perception  Image degradation  Convolution, Furier Transform  Noise  Image operations ◦ Frequency filters ◦ Spatial filtering  Inverse filtering  Wiener filtering 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary2

3.  Aim ◦ to improve the perception of information images  for human viewers ◦ to provide ‘better’ input  for other automated image processing techniques 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary3

4.  No general theory for determining what is good image enhancement ◦ If it looks good, it is good!? 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary4

5. 5

6. 6

7.  http://www.youtube.com/watch?v=_d_l5nsnIvM 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary7

8.  Focus ◦ Noise reduction techniques  Quantitative measures can determine which techniques are most appropriate ◦ How does it improve e.g. the result of the next automated image processing step?  E.g. image segmentation 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary8

9.  The first stage of any vision system ◦ Can we do it in perfect way?  Sometimes yes  Industrial applications  Ideal background  Ideal lighting  Faultless camera  Sometimes not  Industrial applications  Despite of supreme conditions we got degraded image  Accumulation of the faults of the electrical components  Physical phenomena  E.t.c.  Medical image acqusition 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary9

10.  Non-linear mapping ◦ E.g., non-linear sensitivity, image of the straight line is not straight e.t.c.  Blurring ◦ Image of a point is blob  Moving during the image acquisition  Probabilistic noise 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary10

11. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary11 Frequency domain Spatial domain

12. ◦ Multiplication point by point 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary12

13. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary13 * = = · Multiplication Convolution Fourier transf. Inverse Fourier transf.

14. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary14 Definition

15. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary15

16. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary16 x x x

17.  Even functions that are not periodic can be expressed as the integrals of sines and/or cosines multiplied by a weighting function.  The formulation in this case is the Fourier transform. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary17 ∑ =∑ =∑ =∑ =

18. Taught mathematics in Paris Eventually traveled to Egypt with Napoleon to become the secretary of the Institute of Egypt After fall of Napoleon worked at Bureau of Statistics Elected to National Academy of Sciences in 1817 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary18

19. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary19 (invers transform) (continous) base-functions

20. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary20 base-functions (invers transform)

21. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary21 u=0, v=0 u=1, v=0u=2, v=0 u=-2, v=0u=-1, v=0 u=0, v=1u=1, v=1u=2, v=1 u=-2, v=1u=-1, v=1 u=0, v=2u=1, v=2u=2, v=2 u=-2, v=2u=-1, v=2 u=0, v=-1u=1, v=-1u=2, v=-1 u=-2, v=-1u=-1, v=-1 u=0, v=-2u=1, v=-2u=2, v=-2 u=-2, v=-2u=-1, v=-2 u v wavelength:

22.  F (0,0) - value is by far the largest component of the image,  Other frequency components are usually much smaller,  The magnitude of F ( X,Y ) decreases quickly ◦ Instead of displaying the |F(u,v)| we display log( 1 + |F(u,v)| ) real function usually 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary22

23. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary23 x y v u

24.  The 2D Fourier transform can be separated  The edges on the image appears as point series in perpendicular direction in Fourier transform of the image and vice versa. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary24

25. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary25 Image-space Frequency space original rotation linearity shift scale

26.  Noise unknown  subtraction not possible  Periodic noise ◦ N ( u,v ) can be estimated from G ( u,v ) 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary26

27. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary27  Gaussian ◦ In an image due to factors  Electronic circuit noise  Sensor noise due to  poor illumination  High temperature  Rayleigh ◦ Range imaging  Exponential and gamma ◦ Laser imaging  Impulse ◦ Faulty switching  Uniform density ◦ Practical situations

28. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary28

29. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary29

30. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary30

31. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary31

32. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary32

33.  Electrical and electromechanical interference  Spatial dependent noise  Can be reduced via frequency domain filtering ◦ Pair of impulses 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary33

34. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary34 T: A=[a(i,j)] → B=[b(i,j)] b(i,j)=T{a(i,j), S(i,j), i, j} intensity enviroment position

35.  Global: b(i,j)=T{A} (S(i,j)=A) (e.g. Fourier-transformation)  Local: T{a(i,j), S} given size of S and independent from the position (e.g. convolution with a mask)  Local, adaptive: T{a(i,j), S(i,j), i, j} the size of S(i,j) is independent from the size of image (e.g. adaptive thresholding)  Point operation: T{a(i,j)} (e.g. gamma-correction, histogram-equalization) 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary35

36. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary36 D 0 : cutoff frequency All frequencies less than D 0 will be passed, Other frequencies will be filtered out. Bluring and ringing properties Scope: noise filtering

37. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary37 F F -1. Input image Frequency- mask

38. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary38 Original5 1530 80230 Cutoff frequencies

39. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary39 n: order of the filter Properties:  Smooth transition in blurring  No ring effect (continouos filter)  Smoothed edges

40. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary40 Original5 1530 80230 Cutoff frequencies

41. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary41

42. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary42 Original5 1530 80230 Cutoff frequencies

43. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary43

44. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary44

45.  Bandreject and Bandpass Filters  Notch Filters ◦ Rejects or passes frequencies in a predefined neighborhood about the frequency rectangle ◦ Zero-phase-shift filters  Symmetric about the origin  ( u 0,v 0 )  ( -u 0,-v 0 ) ◦ Product of highpass filters whose centers have been translated to the centers of the notches. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary45

46. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary46 noisy image frequency mask frequency image filtered image 1 0

47. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary47

48.  Mean Filter 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary48 where g input image, S (x,y) neighborhood of (s,t) point, mn number of pixels in neighborhood. 3x3 neighborhood

49. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary49

50.  Averaging ◦ same weight for every pixels in neighborhood,  Weighted average ◦ weights for pixels in the neighborhood (generally decreasing with the distance).  The sum of the Noise Filtering/smoothing masks elements is 1! 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary50

51.  Smooth when the difference between the intensity value of the given pixel and the mean of the neighborhood is larger than threshold value defined in advance. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary51

52.  Arithmetic mean filter  Geometric mean filter ◦ Lose less image details  Harmonic mean filter ◦ Works well for  salt noise, Gaussian ◦ Fails for pepper noise  Contraharmonic mean filter ◦ Q >0 pepper, Q <0 salt 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary52

53. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary53

54. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary54

55. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary55

56.  Median filter  Max and min filters 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary56 50% 0% 100%

57. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary57

58. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary58

59.  Behavior changes based on statistical characteristic in the filter region ◦ Improved filtering power ◦ Increase in filter complexity ◦ Noise only! 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary59

60.  Adaptive, Local noise reduction filter ◦ Mean ◦ Variance  Local region S xy ◦ g(x,y) intensity value ◦ the variance of the noise corrupting f(x,y) to form g(x,y) ◦ m L local mean ◦ local variance 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary60

61.  Adaptive, Local noise reduction filter 1. If is zero, the filter should return the value of g(x,y)  Zero noise case 2. If the local variance is high relative to the filter should return a value close to g(x,y)  Edges should be preserved 3. If the variances are equal the filter should return the arithmetic mean value  Local noise averaging 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary61

62.  Simplest approach to restoration is direct inverse filtering 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary62 experience: H: quickly decreasing function, N: not (so quickly) decreasing let us cut the high frequencies No additive noise

63.  If we only know the degradation function ◦ Can not recover the undegraded image  H ( u,v ) is not known  Get around the zero or small value problem ◦ To limit the filter frequencies to values near the origin  H (0,0) is usually the highest value of H ( u,v ) in the frequency domain 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary63

64.  Inverse filtering makes no explicit provision for handling noise  Approach incorporates ◦ Degradation function ◦ Statistical characteristics of noise 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary64

65.  Method ◦ Considering images and noise as random variable ◦ Objective is to find an estimate of the uncorrupted image f such that the mean square error between them is minimized. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary65

66. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 66 spectrum Wiener filter input result original spectrum of the result

67.  Edge preserving filters ◦ E.g. anisotropic diffusion  Wavelet denoising  Point operations ◦ E.g. gamma correction, histogram operations  Iterative filtering  E.t.c. 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary67

68.  We always should consider some kind of noise model ◦ Even when working on phantom data  Should do in automatic way ◦ Have to chose carefully the method  Depends on the given task ◦ Determining the parameters  What we gain? ◦ Less problem afterwards ◦ Better final result  Even no other technique will be applied 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary68

69.  Digital Image Processing ◦ Gonzalez and Woods ◦ www.ImageProcessingPlace.com www.ImageProcessingPlace.com  Course on Image Processing at University of Szeged ◦ Attila Kuba, Kálmán Palágyi  Image Restoration presentation ◦ Attila Kuba ◦ SSIP 2006 17th SSIP 2009, 2 - 11 July, Debrecen, Hungary69