1 Chapter 3: Filtering & Enhancing Images This chapter is about image processing, since the methods take an input image and create another image as output. Other appropriate terms often used are filtering, enhancement, or conditioning. The major motion is that the image contains some signal or structure, which we want to extract, along with uninteresting or unwanted variation, which we want to suppress. If decisions are made about the image, they are made at the level of a single pixel or its local neighborhood.

2 (1) What Needs Enhancement? (2) Image Enhancement by Point Operation   Contrast Stretching   Digital Negative   Intensity Level Slicing   Histogram Modeling (3) Image Enhancement by Spatial (mask) Operation   Noise Smoothing   Median Filtering   Sharping Masking   Zooming ContentsContents

3 Before launching into the methods of this chapter, it is useful to review some of the problems that need them. Two general categories of problems follow. An image needs improvement Low ‑ level features must be detected What Needs Enhancement? Example 1: Scratches from original photo of San Juan are removed Scratches

4 Example 2: Intensity of photo of Alaskan Pipeline rescaled to show much better detail Example 3: Image of airplane part has edges enhanced to support automatic recognition and measurement

5 This chapter deals mostly with traditional methods of image enhancement. Before moving on, it is important to define and distinguish the terms. Image enhancement operators improve the detectability of important image details or objects by man or machine.   Example operations include noise reduction (smoothing), contrast stretching, and sharpening (edge enhancement).

6 Image Enhancement Sharpening of image features ( such as edges or boundaries) to make a graphic display more useful for analysis. Do not increase information content but increase dynamic range of chosen feature to be detected more easily.

7 Left - Original sensed fingerprint; Center - Image enhanced by detection and thinning of ridges; Right - Identification of special features called minutia, which can be used for matching to millions of fingerprint representations in a database. Example

8 Image Enhancement - Operations (1) Point operations –Contrast stretching –Noise clipping –Window slicing –Histogram modeling

9 Image Enhancement -Operations (2) Spatial (mask) operation –Noise smoothing –Median filtering –Sharping masking –Zooming

10 Two ways to enhance images: Changing the intensity values of pixels Transforming the pixels via a single function that maps an input gray value into a new output value Remapping the gray values is often called stretching 伸缩 Points Operation f(x)f(x) Mapping function Input image Output image g in g out =f(g in )

11 Definition - A point operator applied to an image is an operator in which the output pixel is determined only by the input pixel, Out[x, y] = f (In[x, y]); possibly function f depends upon some global parameters. A contrast stretching 对比度伸缩 operator is a point operator that uses a piecewise smooth function f (In[x, y]) of the input gray level to enhance important details of the image.

12 Contrast Stretching Improve contrast due to poor /nonuniform lighting conditions or nonlinearity or small dynamic range of image sensor. Typical contrast stretching transformation:

13 Contrast Stretching Where u is the gray-level values of input image 0-255, v is the transformed gray-level values of output image 0-255, L=255

14 Output image Mapping function f(x)f(x) g in g out Input image Output image

15 Contrast Stretching (a) Original(b) Enhanced

16 Contrast Stretching (b) Enhanced (a) Original

17 Contrast Stretching Example: (a) Original(b) Enhanced

18 Contrast Stretching Clipping: when –Useful for noise reduction when input signal is known to lie in the range [a,b]. Thresholding: when a=b=t (threshold) –Output becomes binary. –Useful for binary or other images that have bimodal distance of gray levels. Can define other similar operations.

19 Contrast Stretching Cliping and thresholding

20 Contrast Stretching Cliping and thresholding

21 Digital Negative Create digital negative of images Display of medical images

22 Output image Input image Mapping function (Reverse) f(x)=x -1 g in g out 0 255

23 Digital Negative Digital negatives

24 Digital Negative Digital negatives

25 Digital Negative Digital negatives

26 Intensity Level Slicing Without background: With background:

27 Intensity Level Slicing Fully illuminates pixels lying in the interval [a,b] and removes the background. Segmentation of certain gray level region e.g. Image from remote sensing.

28 Intensity Level Slicing Level slicing of intensity window[175,250] (a) Visual and infrared images(b) Segment images

29 Various Mapping Functions g in g out g in g out g in g out g in g out g in g out g in g out

30 Histogram Modification The histogram of the “bad” image is very narrow (Fig. a), while the histogram of the “good” image is more spread (Fig. b). To change a “bad” image to the “good” image, we need modify the histogram. Fig.a - bad image Gray level r Number of pixels Gray level s Number of pixels Fig.b - good image When is it necessary to modify the Histogram of an image

31 Histogram Modification The histograms of four basic types of images : Gray level z H(z) Dark image The gray levels are concentrated toward the dark end of the grayscale range. Thus this histogram corresponds to an image with overall dark characteristics. Dark image

32 Histogram Modification Just is the opposite of the above. Gray level z H(z) Bright image

33 Gray level z H(z) Low-contrast image The histogram has a narrow shape, which indicates little dynamic range and thus corresponds to an image having low contrast. The histogram has significant spread, corresponding to an image with high contrast. Gray level z H(z) High-contrast image Low-contrast image High-contrast image

34 Suppose that the grey levels in the original image are given by the values, which are obtained from a variable r. The grey levels in the new image are given by the values, which are obtained from a variable s. We would like to find a transformation s = T (r) such that the probability density function p r (r) in original image (Fig. a) is transformed into a probability density function p s (s) in the new image, which looks like that in Fig. b. How can we modify the Histogram Gray level r pr(r)pr(r) Gray level s ps(s)ps(s) s=T(r)

35 Since p r (r) is the probability density function of random variable r. The number of pixels with grey level values in the range r to r +dr is P r (r)dr. The transformation we seek will transform this range to [s, s+ds]: [r, r+dr]  [s, s+ds] Gray level r Number of pixels r+drr [r, r+dr] Gray level s Number of pixels s [s, s+ds] s+ds T

36 The total number of pixels in this range will remain the same but in the enhanced image this number will be p s (s)ds: p s (s)ds = p r (r)dr This equation can be used to define the transformation T that must be applied to variable r to obtain variable s, provided we define function p s (s).

37 Histogram equalization is an important method in histogram modification. Histogram equalization is often tried to enhance an image. The two requirements on the operator are that (a) the output image should use all available gray levels This requirement means that the target output image uses all gray values z = z 1, z = z 2, … z = z n (b) the output image has approximately the same number of pixels of each gray level. Histogram Equalization

38  Ps(s) is uniform, i.e. a flat histogram to obtain an image with a more balanced contrast  i.e. the transformation required is just the cumulative distribution function (c.d.f.) of u

39 Discrete case  i.e. use the histogram in place of the p.d.f. in the continuous case (the discrete equivalent to the p.d.f.)  similarly, the continuous c.d.f. is now replaced by ⇒ a monotonically increasing sequence

40  if the original digital image has L gray levels, it may be necessary to re-quantize s k to the same set of L discrete levels also Method1: Method 2:

41 method 1 is more efficient but it pre- supposes that uniform quantization is appropriate Example 1 (for 3 bit image)

42 Note: (1) re-quantization method is arbitrary (2) may not get a 1-1 mapping ⇒ inverse problem (3) may not get a flat histogram unless L is large

43 Example of Histogram Equalization Histogram after equalization Original image Original histogram Modified image

44 Original image Original histogramModified image Histogram after equalization

45 Histogram Equalization (a) Input image(b) Processed image

46 Histogram Equalization (c) Input image (d) Processed image

47 Histogram Equalization (e) Input image(f) Processed image

48 Spatial (mask) Operations Operations performed on local neighborhood of input pixels. convolved with maskUsually image is convolved with a finite impulse response (FIR) filter-spatial mask.

49 Mask operation: 1. Centre the mask at (x, y). 2. Multiply every pixel that is contained within the mask area by the corresponding mask coefficient. 3. Add the products together. 4. Either replace f (x, y) by the sum or compare the sum with a threshold and replace f (x, y) according to the comparison. 5. Repeat 1 to 4 for the whole image.

50

51

52 Image Smoothing Often, an image is composed of some underlying ideal structure, which we want to detect and describe, together with some random noise or artifact, which we would like to remove. Image contains both Gaussian noise and bright ring artifacts Image with random noise Scratches Image contains artifacts

53 Box Filter Definition : Smoothing an image by equally weighting a rectangular neighborhood of pixels is called using a box filter. Output-Image [r,c] = Average of some neighborhood of Input-Image [r,c] Example: 5  5 Neighborhood Filter - averages 25 pixel values in a 5  5 neighborhood of the input image pixel in order to create a smoothed output image.

54 Example =

55 Neighborhood Averaging Original Image Noisy Image NAF (3-by-3) NAF (5-by-5) NAF (7-by-7)

56 Neighborhood Averaging Use for noise smoothing LP filtering and subsampling of images. Assuming white noise ηwith zero mean and variance Then the spatial average:

57 assuming equal weight where is the spatial average of. Note that has zero mean and i.e. Noise power is reduced by a factor of Remark: Neighborhood averaging introduces a distortion in the form of blurring.

58 Neighborhood Averaging (a) Original(b) noisy

59 Neighborhood Averaging (c) 3×3 filter (d) 5×5 filter Spatial averaging filters for smoothing images containing Gaussian noise.

60 Definition : When a Gaussian filter is used, pixel [x,y] is weighted according to x Gaussian Filter d is the distance of the neighborhood pixel [x, y] from the center pixel [x c, y c ] of the output image where the filter is being applied. [xc][xc] g(x)g(x) [x][x] d [x][x] Rather than weight all input pixels equally, it is better to reduce the weight of the input pixels with increasing distance from the center pixel I[x c, y c ]. The Gaussian filter does this and is perhaps the most commonly used of all filters.

61 [xc][xc] x g(x)g(x) Gaussian Function One-Dimensional Gaussian Function Two-Dimensional Gaussian Function

62 Example                          1 25 =

63 Examples Noisy imageIdeal image Pixel values in row 100 of the noisy image Pixel values in row 100 of the smoothed image Noise averaged using a 5  5 neighborhood

64 Examples Noisy image M = 32M = 16M = 8M = 2M = 128

65 Often, it is useful to remove small regions from an image. A small region might just be the result of noise, or it might represent low level detail that should be suppressed from the image description being constructed. Small regions can be removed by changing single pixels, or by removing components after connected components are extracted. Removal of Small Image Regions

66 The presence of single dark pixels in bright regions, or single bright pixels in dark regions, is called salt ‑ and ‑ pepper noise. Often, salt ‑ and ‑ pepper noise is the natural result of creating a binary image via thresholding. Salt corresponds to pixels in a dark region that somehow passed the threshold for bright. Pepper corresponds to pixels in a bright region that were below threshold for dark. Removal of Salt-and-Pepper Noise (Binary Image) Gray level z H(z) Salt Pepper

Salt noise in 8 ‑ neighbors Pepper noise in 8 ‑ neighbors

68 Binary image of bacteria Salt ‑ and ‑ pepper noise removed using 8 ‑ neighbors Pepper noise removed using 8 ‑ neighbors Salt noise removed using 8 ‑ neighbors Pepper noise Salt noise Removal of Salt-and-Pepper Noise by Mask

69 Binary image of bacteria Salt ‑ and ‑ pepper noise removed using 4 ‑ neighbors Pepper noise removed using 4 ‑ neighbors Salt noise removed using 4 ‑ neighbors Pepper noise Salt noise Removal of Salt-and-Pepper Noise by Mask

70 Binary image of bacteria Salt ‑ and ‑ pepper noise removed using 8 ‑ neighbors Salt ‑ and ‑ pepper noise removed using 4 ‑ neighbors Better !

71 Median Filtering ( Gray-level Image) Definition --- Let A[i] i = 0, 1, …(n-1) be a sorted array of n real numbers. The median of the set of numbers in A is A[(n ‑ 1)/2] For odd n, the array has a unique middle as defined above. If n is even, then we can define that there are two medians, at A[n/2] and A[n/2 ‑ 1], or one median, which is the average of these two values, namely: A[n/2] = Median value g 1 A[n/2 ‑ 1] = Median value g 2 Median value (g 1 + g 2 )/2 In Median Filter, a pixel value is replaced with the median value of the neighborhood.

72 Example A[(n-1)/2] = A[(13-1)/2] = A[6] = 13 A[i] = 0, 3, 5, 8, 9, 12, 13, 18, 20, 30, 40, 50, 80 i = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12

73 Examples Noisy imageIdeal image Pixel values in row 100 of the noisy image Median filtering using a 5  5 neighborhood Pixel values in row 100 of the smoothed image

74 Examples Input image contains both Gaussian noise and bright ring artifacts added to four previously uniform regions Result of applying a 7  7 median filter

75 (b) 3×3 median filtered(a) Gray-level Image with salt & pepper noise

76 (c) Gray-level Image with Gaussian noise (d) 3×3 median filtered

77 (a) Original Gray-level Image(b) With salt & pepper noise

78 (c) Five nearest neighbors spatial average (d) 3×3 median filtered Spatial averaging versus median filtering

79 Properties of Median Filter However, performance poor when noise pixel in a window>1/2 no. of pixels in the windows.

80 Computing the median requires more computation time than computing a neighborhood average, since the neighborhood values must be partially sorted. Moreover, median filtering is not so easily implemented in special hardware that might be necessary for real ‑ time processing, such as in a video pipeline. However, in many image analysis tasks, its value in image enhancement is worth the time spent.

81 Image Sharpening Crisping the edges A signal proportional to the unsharp, or LPF (low pass frequency) version of the image is subtracted from the image. [or adding the gradient, or HPF (high pass frequency) signal to the image]

82 Idea is to boost the HF (high frequency) components (edges) while the LF remain intact. High-Frequency Filtering –To correct blurs –Emphasis depends on frequency, chosen such as to just compensate the blurring. –Frequency with strong noise should not be emphasized –May require transform operations.

83 Image Sharpening LPF: spatial averaging operation HPF: by subtracting the LPF image from the original image.

84 Image Sharpening Sharping mask operation

85 Image Sharpening (a) Original (b) enhanced Figure 7.24

86 (a) Original image (b) enhanced

87 (a) Original (b) enhanced

88 Magnification of Image (Zoom) (a) Replication Zero Order Hold: each pixel along a scan line id repeated once, and each scan line is repeated. For an image, interlace it by rows and columns of zero to get a matrix, then convolve with

89 Magnification of Image (Zoom) Zooming by replication from 128×128 to 256 ×256 and 512× 512 images

90 Zooming by replication from 128×128 to 256 ×256 and 512× 512 images

91 (b) Linear Interpolation –First Order Hold-straight line fitted between pixels along row, then column. –Can also be obtained by interlacing the image by rows and columns of zero, and then convolve with

92 Zooming by linear interpolation from 128 ×128 to 256 × 256 and 512 × 512 images

93 Zooming by linear interpolation from 128 ×128 to 256 × 256 and 512 × 512 images

94 Magnification of Image (Zoom) Higher-order interpolation: (order p): –Padding each row and column by p rows and p columns of zeros. –Convolve it p times with H. –e.g. p=3-cubic spline interpolation between pixels.

95 Magnification of Image (Zoom)