Signal Processing: From 1-D to 2-D (m-D) 11/15/2018 ENEE631 Spring’09 Lecture 4 (2/4/2009) Signal Processing: From 1-D to 2-D (m-D) Spring ’09 Instructor: Min Wu Electrical and Computer Engineering Department, University of Maryland, College Park bb.eng.umd.edu (select ENEE631 S’09) minwu@eng.umd.edu

UMCP ENEE631 Slides (created by M.Wu © 2004) 11/15/2018 Overview Last Time: Point processing (zero-memory operations) Gamma and gamma correction Contrast stretching Histogram and histogram equalization Today Dithering and halftoning A systematic examination of 2-D signals and systems Assignment-1 Posted on course webpage. Due 8pm Monday 2/16 About prerequisite: ENEE620 and 630 UMCP ENEE631 Slides (created by M.Wu © 2004) M. Wu: ENEE631 Digital Image Processing (Spring'09)

Review: Contrast Stretching vs. Histogram Eq. 11/15/2018 Review: Contrast Stretching vs. Histogram Eq. input gray level u output gray level v a b o    L U ~ logL UMCP ENEE631 Slides (created by M.Wu © 2004) Contrast stretching: commonly be piecewise linear/uniform stretching on selected range [a,b] Histogram eq: the stretching can be non-piecewise linear; the mapping is determined by the cdf of histogram gray level v c.d.f P(V<=v) v0 o 1 255 Fu0   gray level u c.d.f P(U<=u) u0 o 1 255 Fu0   M. Wu: ENEE631 Digital Image Processing (Spring'09)

Review: Histogram Equalization 11/15/2018 Review: Histogram Equalization Goal: Map the luminance of each pixel to a new value such that the output image has approximately uniform distribution of gray levels To find what mapping to use: model pixels as r.v.; match c.d.f. For r.v. U with continuous p.d.f. over [0,1], find c.d.f. Construct a new r.v. V by a monotonically increasing mapping v(u) s.t. Can show V is uniformly distributed over [0,1]  FV(v0) = v0 for any v0 FV(v0) = P(V v0) = P( FU(u) v0) = P( U  F-1U(v0) ) = FU( F-1U(v0) ) = v0 Or: pV(v) dv = pU(u) du ; dv/du = pU(u) => pV(v) = 1 For u in discrete prob. distribution, rounding may be needed => the output v will be approximately uniform UMCP ENEE631 Slides (created by M.Wu © 2004) An “answer” slide for the previous one with equation/proof uncovered. M. Wu: ENEE631 Digital Image Processing (Spring'09)

Histogram Equalization Algorithm 11/15/2018 Histogram Equalization Algorithm Uniform quantization u v v’ pU(xi) UMCP ENEE631 Slides (created by M.Wu © 2001) v [0,1] Map discrete v [0,1] to v’ { 0, …, L-1 } vmin is the smallest positive value of v vmin  0 1  L-1 Note we handle v_min separately because it may have non-zero prob. mass M. Wu: ENEE631 Digital Image Processing (Spring'09)

Generalization of Histogram Equalization 11/15/2018 Generalization of Histogram Equalization Histogram specification Want output v with specified p.d.f. pV(v) Basic idea: match c.d.f. Can think as using uniformly distributed r.v. W as an intermediate step W = FU(u) = FV(v)  V = F-1V (FU(u) ) Approximation in the intermediate step needed for discrete r.v. W1 = FU(u) , W2 = FV(v)  take v s.t. its w2 is equal to or just above w1 UMCP ENEE631 Slides (created by M.Wu © 2001/2004) Histogram modification: Similar to companding transformations Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 3) M. Wu: ENEE631 Digital Image Processing (Spring'09)

UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002) 11/15/2018 Visual Quantization 8 bits / pixel 4 bits / pixel 2 bits / pixel UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002) Contouring effect Visible contours on smoothly changing regions for uniform quantized luminance values with less than 5-6 bits/pixel Human eyes are sensitive to contours How to reduce contour effect at lower bits/pixel? Use “dithering” to break contours by adding noise before quantization M. Wu: ENEE631 Digital Image Processing (Spring'09)

Tool-1: Contrast Quantization 11/15/2018 0 (black) 255 (white) Tool-1: Contrast Quantization [recall] Visual Sensitivity Non-uniform for luminance Just noticeable difference is almost uniform in relative changes (“contrast”): L / L ~ 0.02 About 50 levels of contrast ~ 6 bits with uniform quantizer; ~ 4-5 bits with MMSE quantizer (take account of prob. distributions) Uniformly quantize contrast instead of luminance Can help reduce some number of bits required per pixel while maintaining good visual quality Overall provide non-uniform quantization on luminance UMCP ENEE631 Slides (created by M.Wu © 2004/2007) f(.) lum.-to-contrast g(.) contrast-to luminance Quantizer u c c’ u’ M. Wu: ENEE631 Digital Image Processing (Spring'09)

Tool-2: “Pseudorandom Noise” Quantization 11/15/2018 Tool-2: “Pseudorandom Noise” Quantization Break contours (boundaries of const. gray area) Add uniform zero-mean pseudo random noise before quantization keep average values unchanged Exploit eye’s limitation in resolving fine details Can achieve reasonable quality with 3-bit quantizer “dither” UMCP ENEE631 Slides (created by M.Wu © 2004/2007) random noise U[-A, A] Quantizer u(m,n) u’(m,n) (display) v(m,n) v’(m,n) _ r(m,n) M. Wu: ENEE631 Digital Image Processing (Spring'09)

Use Dithering to Remove Contour Artifacts 11/15/2018 Use Dithering to Remove Contour Artifacts 24 32 31 40 16 47 artificial contour 30 31 28 29 27 32 33 original 24 40 quantized (step=16) UMCP ENEE408G Slides (created by M.Wu © 2003) [16,31] => quantized to 24; [32, 47] => quantized to 40 +3 -1 +2 +1 -3 -2 noise pattern 33 30 28 32 24 31 27 26 35 34 original + noise 40 24 quantized version of (original + noise) M. Wu: ENEE631 Digital Image Processing (Spring'09)

UMCP ENEE631 Slides (created by M.Wu © 2004/2007) 11/15/2018 Example 4 bits / pixel 3 bits / pixel UMCP ENEE631 Slides (created by M.Wu © 2004/2007) M. Wu: ENEE631 Digital Image Processing (Spring'09)

UMCP ENEE631 Slides (created by M.Wu © 2004/2007) 11/15/2018 Halftone How about 1-bit quantizer? Extreme case of low bits/pixel Can we obtain a grayscale look with only black and white? Photo printing in newspaper and books Image display in the old black/white monitor Change density of black dots Low/band-pass filtering effect in HVS can’t resolve individual tiny dots gives a feel of grayscale based on the average # of black dots per unit area UMCP ENEE631 Slides (created by M.Wu © 2004/2007) M. Wu: ENEE631 Digital Image Processing (Spring'09)

Dithering for Halftone Images 11/15/2018 Dithering for Halftone Images Extend the idea of pseudorandom noise quantizer Upsample each pixel to form a resolution cell of many dots Form dither signal by tiling halftone screen to be same size as the image Apply quantizer on the sum of dither signal and image signal Perceived gray level equal to % of black dots in one cell/neighborhood Trade spatial resolution with pixel depth Often use a periodic well-designed “dither screen” as noise pattern May also apply error diffusion to compensate changes in local intensity UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002; EE631 S’07) r(m,n) [0,A] Pseudorandom halftone screen Luminance u(m,n) [0,A] Bi-level display v(m,n) v’(m,n) Threshold A 1 M. Wu: ENEE631 Digital Image Processing (Spring'09)

Signals and Systems: 1-D to 2-D 11/15/2018 Signals and Systems: 1-D to 2-D UMCP ENEE631 Slides (created by M.Wu © 2004) Ref: Jain’s boo 2.2-2.3; Ray Liu’s EE624 lecture notes on Multi-dim chapter M. Wu: ENEE631 Digital Image Processing (Spring'09)

1-D and 2-D Sig. Proc: Similarity and Differences 11/15/2018 1-D and 2-D Sig. Proc: Similarity and Differences Many signal processing concepts can be extended from 1-D to 2-D to multi-dimension Major differences The amount of data involved becomes several magnitude higher Audio: CD quality 44.1K samples/second Video: DVD quality 720*480 at 30 frames/sec => 10.4 M samples/sec Less complete mathematic foundations for multi-dimension SP E.g. A 1-D polynomial can be factored as a product of first-order polynomials (as we see and use in ZT, filter design, etc) A general 2-D polynomial cannot always be factored as a product of lower-order polynomials Notion of causality: Causal processing a 2-D signal: from top to bottom & left to right Causality often matters more for temporal signal than spatial signal UMCP ENEE631 Slides (created by M.Wu © 2004) 1-D: given product form of 1st order polynomials, we can have the zeros and poles; from these, we’ll be able to both qualitatively and quantitatively learn about the freq. response, stability, and other properties. M. Wu: ENEE631 Digital Image Processing (Spring'09)

UMCP ENEE631 Slides (created by M.Wu © 2004) 11/15/2018 2-D Signals Continuously indexed vs. discretely indexed (sampled) 2-D Impulse (unit sample function) Any 2-D discrete function can be represented as linear combination of impulses UMCP ENEE631 Slides (created by M.Wu © 2004) M. Wu: ENEE631 Digital Image Processing (Spring'09)

UMCP ENEE631 Slides (created by M.Wu © 2004) 11/15/2018 2-D Signals (cont’d) 2-D step function Extensions: line impulse and 1-sided step function UMCP ENEE631 Slides (created by M.Wu © 2004) M. Wu: ENEE631 Digital Image Processing (Spring'09)

Periodicity of 2-D Discrete Signal 11/15/2018 Periodicity of 2-D Discrete Signal x[n1,n2] is (rectangular) periodic with a (positive integer-valued) period T1 -by- T2 at rectangular repetition grid if x[n1,n2] = x[n1+T1, n2] = x[n1, n2+T2] for  integers (n1, n2) Interpreting the conditions: periodic tiling of column and row strips => tiling a basic rectangular shaped cell over a rectangular grid Example: cos[ n1/2 +  n2] is periodic with a period 4-by-2 Is cos[ n1 + n2 ] periodic? Periodicity of continuous vs discrete signal: sampled version of two-variable sinusoids are generally not periodic, unless periods are integers. How about tiling of a non-rectangular cell or at a non-rectangular grid? => More general periodicity UMCP ENEE631 Slides (created by M.Wu © 2004) Answer: the period is 4 by 2 The definition implies X[n1, n2] = X[n1+T1, n2+T2], but not vice versa (which relates to more general periodicity that is not necessarily with rectangular cell or rectangular tiling grid). M. Wu: ENEE631 Digital Image Processing (Spring'09)

UMCP ENEE631 Slides (created by M.Wu © 2004) 11/15/2018 Separability x[n1,n2] is called a separable signal if it can be expressed as x[n1,n2] = f[n2]  g[n1] In matrix notation of the image X, an separable image can be represented by the outer product of two column vectors: X = f  gT E.g. the impulse signal is separable: [n1,n2] = [n1]  [n2] Separable signals form a special class of multi-dimensional signals Consider indices range: 0 n1 N11, 0 n2 N21 A general 2-D signal x[n1,n2] has N1  N2 degrees of freedom A separable signal has only N1+ N2  1 degrees of freedom = * UMCP ENEE631 Slides (created by M.Wu © 2004) Degree of freedom: two signals’ outer product is N1 + N2, but given a 2-D N1xN2 signal to be decomposed, we can always normalize say the first entry of the N1 length vector to be 1, thus the degree of freedom is caped at N1+N2-1 M. Wu: ENEE631 Digital Image Processing (Spring'09)

2-D System: How to Characterize it? 11/15/2018 2-D System: How to Characterize it? A 2-D system often refers to a system that maps a 2-D input signals to a 2-D output signal Such a system may be represented by y[n1,n2] = H ( x[n1,n2] ) Linear system H( ): for all a, b, x1[ ], x2[ ] H (ax1[n1,n2] + bx2[n1,n2]) = a  H (x1[n1,n2]) + b  H (x2[n1,n2]) Impulse responses: h(m,n; m’,n’) = H ( [m-m’, n-n’] ) is the output at location (m,n) in response to a unit impulse at (m’,n’) => Point Spread Function (PSF): impulse response for system with positive inputs & outputs (such as intensity of light in imaging system) A linear sys can be characterized by its impulse responses UMCP ENEE631 Slides (created by M.Wu © 2004) M. Wu: ENEE631 Digital Image Processing (Spring'09)

UMCP ENEE631 Slides (created by M.Wu © 2004) 11/15/2018 Shift Invariance Shift invariant system: If H (x[m, n]) = y[m,n], then H ( x[m - m0, n - n0] ) = y[m - m0, n - n0 ] Impulse response for Linear Shift-Invariant (LSI) System A function of the two displacement index variable only: h(m,n; m’,n’) = h[ m-m’, n-n’] i.e. the shape of the impulse response does not change as the location of input impulse moves in the (m,n) plane I/O relation for a LSI system: Equal to the 2-D convolution of the input with an impulse response UMCP ENEE631 Slides (created by M.Wu © 2004) M. Wu: ENEE631 Digital Image Processing (Spring'09)

UMCP ENEE631 Slides (created by M.Wu © 2004) 11/15/2018 2-D Convolution Rotate the impulse response array h(  ,  ) around the original by 180 degree Shift by (m, n) and overlay on the input array x(m’,n’) Sum up the element-wise product of the above two arrays The result is the output value at location (m, n) UMCP ENEE631 Slides (created by M.Wu © 2004) From Jain’s book Example 2.1 M. Wu: ENEE631 Digital Image Processing (Spring'09)

Summary of Today’s Lecture 11/15/2018 Summary of Today’s Lecture Dithering / Halftoning 2-D Signals and Systems Next time 2-D Fourier Transform Image enhancement via spatial filtering Take-home exercises: review Fourier theories in 1-D FS (Fourier Series); FT; DTFT; DFT ~ time and freq. domain Readings Gonzalez’s book 2.6, 5.5 For further exploration: Woods’ book 1.1; Special Issue on “Digital Halftoning”, IEEE Signal Processing Magazine, July 2003 (five articles) UMCP ENEE631 Slides (created by M.Wu © 2004) Jain’s book 4.12, 2.1—2.6 Gonzalez’s book 4.1—4.2, 2.6, 5.5 M. Wu: ENEE631 Digital Image Processing (Spring'09)

Review of 1-D Fourier Transform 11/15/2018 Review of 1-D Fourier Transform Transform Time Domain (cont’s or discrete? periodic?) Frequency Domain (transform domain) Fourier Series (FS) Fourier Transform (FT) Discrete-Time Fourier Transform (DTFT) Discrete Fourier Transform (DFT) UMCP ENEE624 Slides (created by M.Wu © 2003) M. Wu: ENEE631 Digital Image Processing (Spring'09)

M. Wu: ENEE631 Digital Image Processing (Spring'09)