2. Misc. Non-linear Image Processing - Pixel to pixel operations - Histogram equalization - Grey level and colour enhancement - Neighbourhood operations

3. Image presentation image sourceobservermodulation Adaptation of the image to the observer is essential! dynamic range Understanding the “display” technology gray-level – luminance (monitor) optimal for whole Image or ROI Identical data-set, different observation

4. Monochrome/Bilevel Image  Each pixel is stored as a single bit (0 or 1)  A 640 x 480 monochrome image requires 37.5 KB of storage

5. Gray-Scale Image  Represents a continuous range of tones from black, through many intermediate shades of gray, to white  Each pixel is usually stored as a byte (value between 0 to 255, i.e. 8 bits)  A 640 x 480 grayscale image requires over 300 KB of storage

6. Truecolor Image  Each pixel is represented by 24 bits (e.g. RGB)  Supports 256 x 256 x 256 possible combined colors (16,777,216)  A 640 x 480 24-bit color image requires 900 KB of storage  For a 32-bit images, the extra byte of data (alpha value or alpha channel) for each pixel is used to store transparency information

7. Pseudocolor Image  Typically 8 bits per pixel  Supports 256 out of the millions possible colors  Requires Color Look-Up Table (CLUT)  Each pixel value of the image represents an 8-bit index value identifying one of the entries in the CLUT

8. Pseudocolor Image (cont.) 4 R G B RGB 0 4 3 2 1 5 253 254 255 000 64 32 0416 1282 1 2 0 8 255 128255 64 2 1632 4128 64 255 2 Color Look-up Table Image Bitmap

9. Dithering  A process of displaying images using a random dot pattern to give the viewer an impression of more detail and colors than the file format or computer display can actually support  Increase the apparent color range by mixing dots of colors to approximate shades of colors that is doesn't have available –display a 24-bit image with 256 colors –display gray level with only black and white

10. Dithering (cont.) Original image (8-bit)Image after dithering (1-bit)

11. Dithering (Digital Halftoning)  Mach bands can be removed by adding noise along the boundary lines  General perceptive principle: replaced structured errors with noisy ones and people complain less  Old industry dating to the late 1800’s –(Can’t say since the turn of the century!) –Methods for producing grayscale images in newspapers and books

12. Dithering to Black-and-White  Black-and-white is still the preferred way of displaying images in many areas –Black ink is cheaper than color –Printing with black ink is simpler and hence cheaper –Paper for black inks is not special  To get color to black and white, first turn into grayscale: I=0.299R+0.587G+0.114B –This formula reflects the fact that green is more representative of perceived brightness than blue is

13. Sample Images

14. Threshold Dithering  For every pixel: If the intensity < 0.5, replace with black, else replace with white –0.5 is the threshold –This is the naïve version of the algorithm  To keep the overall image brightness the same, you should: –Compute the average intensity over the image –Use a threshold that gives that average –For example, if the average intensity is 0.6, use a threshold that is higher than 40% of the pixels, and lower than the remaining 60%  For all dithering we will assume that the image is gray and that intensities are represented as 0-1.0 –If you have a 0-255 image, you can scale all the thresholds (multiply by 255)

15. Naïve Threshold Algorithm

16. Dithering (cont.)  The pattern at level k is formed by adding an “ on ” position to the grid pattern at level k – 1  Thus, if a pixel position is on for one grid level, it is on for all higher levels

17. Dithering (cont.)  We can minimize the introduction of other visual effects by avoiding symmetrical patterns  The symmetrical patterns would produce vertical, horizontal, or diagonal streaks in any large area shaded.  Thus, in the following figure, pattern (a) is to be preferred to the patterns in (b) (a) (b)

18. Dithering (cont.)  A 2 x 2 pixel area of a bi-level display can be used to produce five different intensity levels at the cost of halving the spatial resolution along each axis  A block of n x n pixels can simulate n 2 + 1 different gray levels 0  I < 0.2 0.2  I < 0.4 0.4  I < 0.6 0.6  I < 0.8 0.8  I  1.0

19. Dithering (cont.)  Although the use of n by n pixel patterns increases the number of intensities that can be displayed, they reduce the resolution of the displayed picture by a factor of 1/n along each of the x and y axes  A 512 by 512 screen area, for instance, is reduced to an area containing 256 by 256 intensity points with 2 by 2 grid patterns  And with 4 by 4 patterns, we would reduce the 512 by 512 area to 128 intensity positions along each side

20. Dithering (cont.) 00 00 01 00 01 10 11 10 11 11 12 11 12 21 22 21 22 22 23 22 23 32 33 32 33 33  Consider a display with 2 bits per pixel and hence four intensity levels. We can use 2 by 2 pixel grid to extend the available intensity levels from 4 to 13 (4 x 3 + 1 = 13)

21. Dithering (cont.)  Similarly, we can use pixel-grid patterns to increase the number of intensities that can be displayed on a color system  Suppose we have a 3-bit per pixel RGB system. This gives one bit per color gun in the monitor, providing eight colors (including black and white). Using 2 by 2 pixel-grid pattern area to obtain 125 different colors: each pattern can display five intensities for each of red, green, and blue. This results in 5 x 5 x 5 = 125 color combinations

22. Size matters in Dithering  With 3 by 3 pixel grids on a bi-level system, we can display 10 intensity levels 0  I < 0.1 0.1  I < 0.20.2  I < 0.30.3  I < 0.40.4  I < 0.5 0.5  I < 0.60.6  I < 0.70.7  I < 0.80.8  I < 0.9 0.9  I < 1.0

23. Ordered Dithering  Break the image into small blocks  Define a threshold matrix –Use a different threshold for each pixel of the block –Compare each pixel to its own threshold  The thresholds can be clustered, which looks like newsprint  The thresholds can be “random” which looks better Threshold matrix

24. Clustered Dithering 1 8517325 156211 2174912 1819132416 1420102322

25. Random Modulation  Add a random amount to each pixel before thresholding –Typically add uniformly random amount from [-a,a]  Pure addition of noise to the image –As stated above, we are adding uniformly random noise –For better results, add better quality noise –For instance, use Gaussian noise  Should use same procedure as before for choosing threshold  Not good for black and white, but OK for more colors –Add a small random color to each pixel before finding the closest color in the table

26. Random Modulation

27. Pattern Dithering  Compute the intensity of each sub-block and index a pattern  NOT the same as before –Here, each sub-block has one of a fixed number of patterns – pixel is determined only by average intensity of sub-block –In ordered dithering, each pixel is checked against the dithering matrix before being turned on

28. 1 117610 41415 59812 133162

29. Floyd-Steinberg Dithering (aka error diffusion)  Start at one corner and work through image pixel by pixel –Usually scan top to bottom in a zig-zag  Threshold each pixel  Compute the error at that pixel: The difference between what should be there and what you did put there  Propagate error to neighbors by adding some proportion of the error to each unprocessed neighbor e 3/165/161/16 7/16

30. Floyd-Steinberg Dithering

31. Dithering (cont.) An RGB 2 by 2 pixel-grid pattern

32. Dithering (cont.) Original image (24-bit)Image after dithering (8-bit)

33. Color Dithering  All the same techniques can be applied, with some modification –Below is Floyd-Steinberg: Error is difference from nearest color in the color table, error propagation is the same

34. Linear Point Operations scale offset 0 K-1 Watch for saturation/cutoff 0 K-1 0 Bad, information is lost (x,y) is irrelevant

35. Pixel to Pixel transformations For all pixels do pixel(i,j) := function( pixel(i,j)) dynamic range adaptation thresholding digital negative contrast stretching histogram equalization histogram shaping color processing Color Look-Up Table (CLUT)

36. Some Basic Gray Lavel Transformations

37. Image Enhancement in the Spatial Domain Image Enhancement in the Spatial Domain  =c=1: identity

38. Image Enhancement in the Spatial Domain Image Enhancement in the Spatial Domain

39. Image Enhancement in the Spatial Domain Image Enhancement in the Spatial Domain

40. Contrast/Gamma manipulation Interactive Demo With Gimp…

41. Adaptive Pixel to Pixel transformations global transformation global transformation based on ROI local/adaptive transformation

42. Dynamic range adaptation linear luminance transformation y = a x + b

43. Histogram of Imag Processing Dark Bright Low-contrast High-contrast

44. Thresholding

46. Contrast streching (CS) By stretching the histogram we attempt to use the available full grey level range. The appropriate CS transformation : s k = 255·(r k -min)/(max-min)

47. Contrast streching II

48. Contrast streching III CS does not help here HE ?

49. Contrast streching IV CS HE

50. Contrast streching V CS 1% - 99%

51. Contrast streching VI HE CS 79, 136 CS Cutoff fraction: 0.8

52. Contrast streching VIII a more general CS: 0, if r k < p low s k = 255·(r k - p low )/(p high - p low ), otherwise 255,if r k > p high

53. Histogram Processing  The shape of the histogram of an image does provide useful info about the possibility for contrast enhancement.  Types of processing: Histogram equalization Histogram matching (specification) Local enhancement

54. Histogram Equalization CDF versus PDF

55. Histogram Equalization More contrast for most pixels, less contrast for some pixels

56. Histogram Processing  The histogram of a digital image with gray levels from 0 to L-1 is a discrete function h(r k )=n k, where: –r k is the kth gray level –n k is the # pixels in the image with that gray level –n is the total number of pixels in the image –k = 0, 1, 2, …, L-1  Normalized histogram: p(r k )=n k /n –sum of all components = 1

57. Histogram Equalization  As mentioned above, for gray levels that take on discrete values, we deal with probabilities: p r (r k )=n k /n, k=0,1,.., L-1 –The plot of p r (r k ) versus r k is called a histogram and the technique used for obtaining a uniform histogram is known as histogram equalization (or histogram linearization).

58. Histogram Equalization  Histogram equalization(HE) results are similar to contrast stretching but offer the advantage of full automation, since HE automatically determines a transformation function to produce a new image with a uniform histogram.

59. Histogram Equalisation Histogram equalization

60. Histogram Equalization

61. Histogram equalization V cumulative histogram

62. Histogram equalization VII HE

63. histogram can be taken also on a part of the image Histogram equalization VIII

64. Local Equalization Enhancement Original Global Histogram Equalization 7x7 Local Histogram equalization

65. Local Enhancement continued Original Image Global histogram equalization 7x7 local histogram equalization 15x15 local histogram equalization

66. Histogram projection (HP) assigns equal display space to every occupied raw signal level, regardless of how many pixels are at that same level. In effect, the raw signal histogram is "projected" into a similar-looking display histogram.

67. Histogram projection II HE HP IR image

68. Histogram projection III occupied (used) grey level: there is at least one pixel with that grey level B(k): the fraction of occupied grey levels at or below grey level k B(k) rises from 0 to 1 in discrete uniform steps of 1/n, where n is the total number of occupied levels HP transformation: s k = 255 ·B(k).

69. Plateau equalization By clipping the histogram count at a saturation or plateau value, one can produce display allocations intermediate in character between those of HP and HE.

70. Plateau equalization II HE PE 50

71. Plateau equalization III The PE algorithm computes the distribution not for the full image histogram but for the histogram clipped at a plateau (or saturation) value in the count. When that plateau value is set at 1, we generate B(k) and so perform HP; When it is set above the histogram peak, we generate F(k) and so perform HE. At intermediate values, we generate an intermediate distribution which we denote by P(k). PE transformation: s k = 255· P(k)

72. Histogram Specification (or Histogram matching )  Histogram equalization does not allow interactive image enhancement and generates only one result: an approximation to a uniform histogram.  Sometimes though, we need to be able to specify particular histogram shapes capable of highlighting certain gray-level ranges.

73. Histogram specification (HS) an image's histogram is transformed according to a desired function Transforming the intensity values so that the histogram of the output image approximately matches a specified histogram.

74. Histogram specification II ST S -1 * T histogram 1 histogram 2 ?

75. Histogram Specification  The procedure for histogram-specification based enhancement is: –Equalize the levels of the original image using: n: total number of pixels, n j : number of pixels with gray level r j, L: number of discrete gray levels

76. Histogram Specification –Specify the desired density function and obtain the transformation function G(z): –Apply the inverse transformation function z=G -1 (s) to the levels obtained in step 1. p z : specified desirable PDF for output

77. Histogram Specification  The new, processed version of the original image consists of gray levels characterized by the specified density p z (z). In essence:

78. Histogram Specification  The principal difficulty in applying the histogram specification method to image enhancement lies in being able to construct a meaningful histogram. So…

79. Histogram Specification –Either a particular probability density function (such as a Gaussian density) is specified and then a histogram is formed by digitizing the given function, –Or a histogram shape is specified on a graphic device and then is fed into the processor executing the histogram specification algorithm.

80. Image Enhancement in the Spatial Domain Image Enhancement in the Spatial Domain

81. Histogram Histogram Equalization Histogram Matching

83. Comparison of normalizations Securics Dual LUT normalizationCSU Standard Normalization Raw Normalized EquinoxEquinox

84. Histogram of color images RGB color can be converted to a gray scale value by Y = 0.299R + 0.587G + 0.114B Y: the grayscale component in the YIQ color space used in NTSC television. The weights reflect the eye's brightness sensitivity to the color primaries.

85. R R R G B Histogram of color images III

86. Color Histogram Equalization Each channel by itself.. Is this RGB or HSV?

87. Histogram of color images IV or a 3-D histogram can be produced, with the three axes representing the red, blue and green channels, and brightness at each point representing the pixel count