1. M. Wu: ENEE631 Digital Image Processing (Spring'09) Point Operations Spring ’09 Instructor: Min Wu Electrical and Computer Engineering Department, University of Maryland, College Park   bb.eng.umd.edu (Select ENEE631 S’09)   minwu@eng.umd.edu ENEE631 Spring’09 Lecture 3 (2/2/2009)

2. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [2] Overview Last Time –Color representation and perception u not a precise spectrum analyzer => perceive color via 3 types of cones –Human visual properties for monochrome vision u not a precise intensity/energy meter => depend on surrounding contrast and spatial frequency Today –Wrap up monochrome vision: spatial frequency and quality metrics –Point processing (zero-memory operations) See reading list on the last slide of each lecture notes Assignment-1 will be posted on course webpage UMCP ENEE631 Slides (created by M.Wu © 2004; 2009)

3. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [3] dot Visual Angle and Spatial Frequency Visual angle matters more than absolute size and distance –Smaller but closer object vs. larger but farther object –Eyes can distinguish about 25-30 lines per degree in bright illumination u 25 lines per degree translate to 500 lines for distance=4 x screenheight Spatial Frequency –Measures the extent of spatial transition in unit of “cycles per visual degree” Visibility thresholds –Eyes are most sensitive to medium spatial freq. and least sensitive to high frequencies ~ similar to a band-pass filter –More sensitive to horizontal and vertical changes than other orientations UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

4. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [4] Visibility Thresholds at Various Frequencies –Measuring visibility threshold using sinusoidal grating of varying contrast and frequencies –Human visual system’s frequency response is similar to a band-pass filter From Jain’s Fig.3.7 (pp55) w.r.t. radial spatial freq.

5. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [5] Overall Monochrome Vision Model From Jain’s Fig.3.9 (pp57)

6. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [6] Image Fidelity Criteria Subjective measures –Examination by human viewers –Goodness scale: excellent, good, fair, poor, unsatisfactory –Impairment scale: unnoticeable, just noticeable, … –Comparative measures u with another image or among a group of images Objective (Quantitative) measures –Mean square error and variations –Pro: u Simple, less dependent on human subjects, and easy to handle mathematically –Con: u Not always reflect human perception UMCP ENEE631 Slides (created by M.Wu © 2001)

7. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [7] Mean-square Criterion Average (or sum) of squared difference of pixel luminance between two images Signal-to-noise ratio (SNR) – SNR = 10 log 10 (  s 2 /  e 2 ) in unit of decibel (dB) u  s 2 image variance u  e 2 variance of noise or error – PSNR = 10 log 10 ( A 2 /  e 2 ) u A is peak-to-peak value u PSNR is about 12-15 dB higher than SNR UMCP ENEE631 Slides (created by M.Wu © 2001)

8. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [8] Image Enhancement via Point Operations UMCP ENEE631 Slides (created by M.Wu © 2004)

9. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [9] Point Operations / Intensity Transform Basic idea –“Zero memory” operation u each output of a pixel only depend on the input intensity at the pixel –Map a given gray or color level u to a new level v, i.e. v = f ( u ) –Doesn’t bring in new info. –But can improve visual appearance or make features easier to detect Example-1: Color coordinate transformations – RGB of each pixel  luminance + chrominance components  etc. Example-2: Scalar quantization – quantize pixel luminance/color with fewer bits input gray level u output gray level v UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

10. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [10] UMCP ENEE631 Slides (created by M.Wu © 2004) Typical Types of Gray-level Transformation Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 3) LL

11. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [11] Example: Negative Transformation UMCP ENEE631 Slides (created by M.Wu © 2004) Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 3)

12. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [12] Example: Log Transformation UMCP ENEE631 Slides (created by M.Wu © 2004) Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 3) L U ~ logL

13. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [13] Example: Effects of Different Gammas ( “vectors” sample image from Matlab ) L 0 2.2 L 0 1/2.2 L0L0 UMCP ENEE631 Slides (created by M.Wu © 2001)

14. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [14] Gamma Characteristics & Gamma Correction Non-linearity in CRT display –Voltage U vs. Displayed luminance L’ u L’ ~ U  where  = 2.0 ~ 2.5 Use preprocessing to compensate  -distortion –U ~ L 1/  –log(L) gives similar compensation curve to  - correction => used for many practical applications –Camera may have L 1/  c capturing distortion with  c = 1.0-1.7 Power-law transformations are also useful for general purpose contrast manipulation U L’ L’ = a U  L U ~ logL ~ L 1/  UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

15. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [15] Example of Gamma Correction UMCP ENEE631 Slides (created by M.Wu © 2004) Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 3)

16. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [16] Luminance Histogram Represents the relative frequency of occurrence of the various gray levels in the image –For each gray level, count the # of pixels having that level –Can group nearby levels to form a big bin & count #pixels in it ( From Matlab Image Toolbox Guide Fig.10-4 ) UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

17. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [17] Luminance Histogram (cont’d) Interpretation –Treat pixel values as instantiations of a random variable –Histogram is an estimate of the probability distribution of the r.v. “Unbalanced” histogram doesn’t fully utilize the dynamic range –Low contrast image ~ histogram concentrating in a narrow luminance range –Under-exposed image ~ histogram concentrating on the dark side –Over-exposed image ~ histogram concentrating on the bright side Balanced histogram gives more pleasant look and reveals rich content UMCP ENEE631 Slides (created by M.Wu © 2001)

18. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [18] UMCP ENEE631 Slides (created by M.Wu © 2004) Example: Balanced and Unbalanced Histograms Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 3)

19. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [19] Image with Unbalanced Histogram Apply contrast stretching to enhance ( From Matlab Image Toolbox Guide Fig.10-10 & 10-11 ) UMCP ENEE631 Slides (created by M.Wu © 2001/2007) input gray level u output gray level v a b o   

20. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [20] Contrast Stretching for Low-Contrast Images Stretch the over-concentrated graylevels in histogram via a nonlinear mapping –Piece-wise linear stretching function –Assign slopes of the stretching region to be greater than 1 UMCP ENEE631 Slides (created by M.Wu © 2001/2007) expandcompress dynamic range ( Images from Matlab Image Toolbox Guide Fig.10-10 & 10-11; Sketch from B. Liu ’06 )

21. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [21] Contrast Stretching: Example original stretched UMCP ENEE631 Slides (created by M.Wu © 2001)

22. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [22] Clipping and Thresholding Clipping –Special case of contrast stretching with  =  = 0 –Useful for noise reduction when the signal of interest mostly lie in range [a,b] Thresholding –Special case of clipping with a = b = T –Useful for binarization of scanned binary images u documents, signatures, fingerprints input gray level u output gray level v a b o  input gray level u output gray level v T o UMCP ENEE631 Slides (created by M.Wu © 2001)

23. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [23] Examples of Histogram Equalization ( From Matlab Image Toolbox Guide Fig.10-10 & 10-11 ) UMCP ENEE631 Slides (created by M.Wu © 2001)

24. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [24] Equalization Example (cont’d) original equalized UMCP ENEE631 Slides (created by M.Wu © 2001)

25. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [25] How to Do Histogram Equalization? Approach: map input luminance u to the corresponding v –Use cumulative distribution as an intermediate step –v will be approximately uniform for u with discrete prob. distribution u b/c all pixels in one bin are mapped to a new bin (no splitting) gray level u c.d.f P(U<=u) u0 o 1 255 F u0   gray level v c.d.f P(V<=v) v0 o 1 255 F u0   UMCP ENEE408G Slides (created by M.Wu © 2002)

26. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [26] The Math Behind Histogram Equalization Goal: Map the luminance of each pixel to a new value such that the output image has approximately uniform distribution of gray levels To find what mapping to use: first model pixel as r.v. –How to generate r.v. with desired distribution?  Match c.d.f Want to transform one r.v. with certain p.d.f. to a new r.v. with uniform p.d.f. –For r.v. U with continuous p.d.f. over [0,1], find c.d.f. –Construct a new r.v. V by a monotonically increasing mapping v(u) s.t. –Can show V is uniformly distributed over [0,1]  F V (v 0 ) = v 0 for any v 0 u F V (v 0 ) = P(V  v 0 ) = P( F U (u)  v 0 ) = P( U  F -1 U (v 0 ) ) = F U ( F -1 U (v 0 ) ) = v 0 u Or: p V (v) dv = p U (u) du ; dv/du = p U (u) => p V (v) = 1 –For u in discrete prob. distribution, rounding may be needed => the output v will be approximately uniform UMCP ENEE631 Slides (created by M.Wu © 2004)

27. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [27] Histogram Equalization Goal: Map the luminance of each pixel to a new value such that the output image has approximately uniform distribution of gray levels To find what mapping to use: first model pixels as r.v. –How to generate r.v. with desired distribution?  Match c.d.f Want to transform one r.v. with certain p.d.f. to a new r.v. with uniform p.d.f. –For r.v. U with continuous p.d.f. over [0,1], find c.d.f. –Construct a new r.v. V by a monotonically increasing mapping v(u) s.t. –Can show V is uniformly distributed over [0,1]  F V (v 0 ) = v 0 for any v 0 u F V (v 0 ) = P(V  v 0 ) = P( F U (u)  v 0 ) = P( U  F -1 U (v 0 ) ) = F U ( F -1 U (v 0 ) ) = v 0 u Or: p V (v) dv = p U (u) du ; dv/du = p U (u) => p V (v) = 1 –For u in discrete prob. distribution, rounding may be needed => the output v will be approximately uniform UMCP ENEE631 Slides (created by M.Wu © 2004)

28. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [28] Explanation from Girod’s lecture –Prob. distribution of one r.v. F to another r.v. G: match the small prob. mass under a widge of p.d.f. curves of the two

29. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [29] Histogram Equalization Algorithm v  [0,1] Map discrete v  [0,1] to v’  { 0, …, L-1 } v min is the smallest positive value of v v min  0 1  L-1 Uniform quantization uvv’ p U (x i ) UMCP ENEE631 Slides (created by M.Wu © 2001)

30. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [30] Histogram Equalization: A Mini-Example –x i 0 1 2 3 4 5 6 7 (L=8) –p(x i ) 0.1 0.2 0.4 0.15 0.1 0.05 0 0 –v 0.1 0.3 0.7 0.85 0.95 1.0 1.0 1.0 –v’ 0 [1.5] [4.7] [5.8] [6.6] 7 7 7 (L-1)/(1-v min ) = 7.78 0 2 5 6 7 7 7 7 UMCP ENEE631 Slides (created by M.Wu © 2001)

31. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [31] Summary: Contrast Stretching vs. Histogram Eq. What are in common? What are different? input gray level u output gray level v a b o    gray level u c.d.f P(U<=u) u0 o 1 255 F u0   gray level v c.d.f P(V<=v) v0 o 1 255 F u0   UMCP ENEE631 Slides (created by M.Wu © 2004)

32. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [32] Generalization of Histogram Equalization Histogram specification –Want output v with specified p.d.f. p V (v) –Basic idea: match c.d.f. u Can think as using uniformly distributed r.v. W as an intermediate step u W = F U (u) = F V (v)  V = F -1 V (F U (u) ) –Approximation in the intermediate step needed for discrete r.v. u W1 = F U (u), W2 = F V (v)  take v s.t. its w2 is equal to or just above w1 UMCP ENEE631 Slides (created by M.Wu © 2001/2004) Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 3)

33. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [33] Generalization of Histogram Equalization Histogram modification –u  v = f(u)  v’ = uniform quantized v –f(u) = u 1/2 or f(u) = u 1/3 –f(u) = log(1+|u|) ~ range compression (good for spectrum visualization) Histogram specification –Want output v with specified p.d.f. p V (v) –Use uniformly distributed r.v. W as an intermediate step u W = F U (u) = F V (v)  V = F -1 V (F U (u) ) –Approximation in the intermediate step needed for discrete r.v. u W1 = F U (u), W2 = F V (v)  take v s.t. its w2 is equal to or just above w1 Uniform quant. uv v’ f(u) UMCP ENEE631 Slides (created by M.Wu © 2001)

34. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [34] Visual Quantization Contouring effect –Visible contours on smoothly changing regions for uniform quantized luminance values with less than 5-6 bits/pixel –Human eyes are sensitive to contours (high contrast between two areas) How to reduce contour effect at lower bits/pixel? –Use “dithering” to break contours by adding noise before quantization 8 bits / pixel 4 bits / pixel 2 bits / pixel UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

35. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [35] Use Dithering to Remove Contour Artifacts 3031 2829 31 30 272829 303132 28 27 29 31323331 33 31 30 32 33 original 24 40 24 40 24 40 24 40 quantized (step=16) +3 +2 +1 +2 -3+2 0+2 0 +2 +3-2 0 -3 +2 noise pattern 3330 28 32 243028 303331 27 26 31 33353130 32 31 27 34 32 original + noise 4024 40 24 4024 40 24 40 24 40 quantized version of (original + noise) artificial contour UMCP ENEE408G Slides (created by M.Wu © 2003)

36. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [36] Summary of Today’s Lecture Image fidelity criteria Point operations (mostly with zero-memory) –Gamma correction –Contrast stretching –Histogram equalization Next time –Dithering / Halftoning –2-D Systems and Transforms Readings –Gonzalez’s book 3.1—3.3 Question for Today –Finish the proof in histogram equalization UMCP ENEE631 Slides (created by M.Wu © 2004)

37. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [37] Readings and Assignment Readings –Jain’s book 7.1—7.3, 4.12 (or) Gonzalez’s book 3.1—3.3 Computer Assignment –On color and pixel based operations –Also “play” with image editor software (Paintshop Pro) –Will be posted on course webpage and announced by email Question for Today: –Finish the proof in histogram equalization UMCP ENEE631 Slides (created by M.Wu © 2001,2004)

38. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [38]

39. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [39] Backup UMCP ENEE631 Slides (created by M.Wu © 2004)

40. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [40] 4 bits / pixel Example 3 bits / pixel

41. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [41] Tool-1: Contrast Quantization [recall] Visual Sensitivity –Non-uniform for luminance –Just noticeable difference is almost uniform in relative changes (“contrast”):  L / L ~ 0.02 u About 50 levels of contrast ~ 6 bits with uniform quantizer; ~ 4-5 bits with MMSE quantizer (take account of prob. distributions) Uniformly quantize contrast instead of luminance –Can help reduce some number of bits required per pixel while maintaining good visual quality –Overall provide non-uniform quantization on luminance f(.) lum.-to- contrast g(.) contrast- to luminance Quantizer u c c’u’ UMCP ENEE631 Slides (created by M.Wu © 2004/2007) 0 (black) 255 (white)

42. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [42] Tool-2: “Pseudorandom Noise” Quantization Break contours (boundaries of const. gray area) –Add uniform zero-mean pseudo random noise before quantization u keep average values unchanged –Can achieve reasonable quality with 3-bit quantizer random noise U[-A, A] Quantizer u(m,n) u’(m,n) (display) v(m,n) v’(m,n) _ r(m,n) “dither” UMCP ENEE631 Slides (created by M.Wu © 2004/2007)

43. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [43] Halftone How about 1-bit quantizer? –Extreme case of low bits/pixel –Can we obtain a grayscale look with only black and white? u Photo printing in newspaper and books u Image display in the old black/white monitor Change density of black dots –Low/band-pass filtering effect in HVS u can’t resolve individual tiny dots u gives a feel of grayscale based on the average # of black dots per unit area UMCP ENEE631 Slides (created by M.Wu © 2004/2007)

44. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [44] Dithering for Halftone Images Halftone images –Extreme case of low bits/pixel: use two colors (b/w) to represent gray shades Extend the idea of pseudorandom noise quantizer –Upsample each pixel to form a resolution cell of many dots –Form dither signal by tiling halftone screen to be same size as the image –Apply quantizer on the sum of dither signal and image signal Perceived gray level –Equal to the percentage of black dots perceived in one resolution cell r(m,n)  [0,A] Pseudorandom halftone screen Luminance u(m,n)  [0,A] Bi-level display v(m,n)v’(m,n) Threshold A 0 1 UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

45. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [45] Examples of Halftone Images Trade spatial resolution with pixel depth –Let eyes serve as a low-pass-filter –The average intensity of a cell matches the intensity of original pixel (or neighborhood) –May also apply error diffusion Applications –Photo printing in newspaper and books –Image display in the old black/white monitor –Often use a periodic “dither screen” as noise pattern UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)