1. M. Wu: ENEE631 Digital Image Processing (Spring'09) Back Projection for Radon Construction and Multi-dimensional Sampling Spring ’09 Instructor: Min Wu Electrical and Computer Engineering Department, University of Maryland, College Park   bb.eng.umd.edu (select ENEE631 S’09)   minwu@eng.umd.edu ENEE631 Spring’09 Lecture 25 (5/4/2009)

2. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [2] Overview and Logistics Last Time: –Useful image features and feature extraction techniques u Projection based: convert 2-D data to 1-D profile u Parameter space: low-complexity detection of line, circle & other structure u Gradient based: corner detection; stable key point extraction & representation –Radon transform and applications in medical image processing u Relations between 1-D projection and 2-D FT => Project Theorem Today: –Back projection for inverting Radon transform –Sampling theory: extending 1-D to 2-D Course Logistics –Team work guide and questionnaire –Project progress report: Due Today 5pm May 4 by email –TA evaluation (hard copy) –Course evaluation: Due May 13. https://www.courseevalum.umd.edu/portal

3. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [3] Recap: ENEE631 Grading Update –Assignments25% –Mid-term Exam 20% –Project 1 (image compression) 20% –Project 2 (mini R&D project) 30% Oral presentation; Written report Overall performance and depth –Base points (more for active class participation) 5% –No final exam; Remaining tasks include project-2 and 5 th assignment (framework design as “project architect/lead”) M Tu W Th F Sa Su 7 8 9 10 11 12 13 14 15 16 17 18 19 20

4. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [4] Follow-up: Morphological Processing on (Binary) Patterns Mathematical Morphology –A useful tool for representing region shape and pre/post processing u E.g. filling broken edges/contours; thinning; enlarging opening –Largely based on set theory and set operations  Binary image: the set includes coordinates of black (or white) pixels ~ Z 2  Grayscale image: the set includes pixel coordinates and gray value ~ Z 3 Figure from Gonzalez’s book resource

5. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [5] Fill in Holes on Binary Lines or Patterns 2-D operator for nonlinear binary filtering: e.g., majority voting Generalize: Basic binary morphological operations –Object X, Structuring element B => B x (translated to x) –Dilation: X (+) B = { x: B x  X   } ~ expanding –Erosion: X (  ) B = {x: B x  X } ~ shrinking Extending basic operations of erosion and dilation –Thickening and Thinning –Opening and Closing –Pruning and Skeleton Morphological op. for grayscale –See Gonzalez’s & Bovik’s Handbook Figure from Jain’s book Chapter 9

6. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [6] Examples of Morphological Processing Figures from Gonzalez’s book resource

7. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [7] See more examples in Gonzalez’s book Chapter 9 Table from Jain’s book Chapter 9

8. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [8] Radon Transform A linear transform f(x,y)  g(s,  ) –Line integral or “ray-sum” –Along a line inclined at angle  from y-axis and s away from origin Fix  to get a 1-D signal p  (s)= g(s,  ) (From Jain’s Fig.10.2)

9. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [9] Rotational Invariance Property of Fourier Transform Consider a 2-D continuous signal & its 2-D Fourier transform 2-D FT is rotationally invariant FT of a rotated version of function f(t1, t2) is just to apply same rotation to F(  1,  2)

10. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [10] Connection Between Radon & Fourier Transform Observations –Look at 2-D FT coeff. along horizontal frequency axis u FT of 1-D signal u 1-D signal is vertical summation (projection) of original 2-D signal –Look at FT coeff. along  =  0 ray passing origin u FT of projection of the signal perpendicular to  =  0 Projection Theorem Proof using FT definition & coordinate transform (Figure from Bovik’s Handbook Fig.10.2.7)

11. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [11] Projection Theorem (a.k.a. Projection-Slice Theorem) Let F(  1,  2) be the FT of a cont’s function f(t1, t2) Let Radon transform of f be given as p  (s) with 1-D FT P  (  ) Projection Theorem: P  (  ) = F(  cos ,  sin  ) for all  i.e. FT of a slice is the corresponding slice of the 2-D FT –We can obtain the full 2-D FT in terms of 1-D freq. components of the Radon transform at all angles.

12. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [12] Inverting Radon by Projection Theorem (Step-1) Filling 2-D FT data matrix with 1-D FT of Radon along different angles (Step-2) 2-D IFT Need Polar-to-Cartesian grid conversion for discrete scenarios –May lead to artifacts esp. with less densely sampled slices (From Jain’s Fig.10.16)

13. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [13] Back Projection: Example

14. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [14] Back-Projection Sum up Radon projection along all angles passing the same pixel location (x, y) Back-projection = Inverse Radon? –Most contribution is from pixel (x,y), but still has some small contribution from others B ( R f ) = conv( f, h ) –Blurring func. h = (x 2 + y 2 ) -1/2, FT(h) ~ 1 / |  | (radial freq.  2 =  x 2 +  y 2 reflect how far from DC) Need inverse filtering to recover original –For “sharpening”: multiply |  | in FT domain (From Jain’s Fig.10.6)

15. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [15] Back-projection = Inverse Radon ? Not exactly ~ Back-projection gives a blurred recovery –Intuition: most contribution is from the pixel (x,y), but still has some small contribution from other pixels –B ( R f ) = conv( f, h1 ) –Bluring function h1 = (x 2 + y 2 ) -1/2, FT( h1 ) ~ 1 / |  | where radial frequency  2 =  x 2 +  y 2 (reflect how far from DC) Need to apply inverse filtering to fully recover the original –Inverse filtering for “sharpening”: multiplied by |  | in FT domain

16. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [16] Inverting Radon via Filtered Back Projection f(x,y) = B H g Change coordinate (Cartesian => polar) Projection Theorem ( F_polar => G) Back Projection(FT domain) filtering (From Jain’s Fig.10.8)

17. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [17] Properties/Applications of Filtered Back Projection Convolution-Projection Theorem –Radon[ f1  f2] = Radon[ f1 ]  Radon[ f2 ] u Radon and filtering operations are interchangeable u Can prove using Projection Theorem –Also useful for implementing 2-D filtering using 1-D filtering Another view of filtered back projection –Change the order of filtering and back-projection u Back Projection => Filtering u Filtering => Back Projection => See more discussions and applications in Jain’s

18. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [18] Other Scenarios of Computerized Tomography Parallel beams vs. Fan beams –Faster collection of projections via fan beams u involve rotations only Recover from projections contaminated with noise –MMSE criterion to minimize reconstruction errors  See Gonzalez 3/e book and Bovik’s Handbook for details (From Bovik’s Handbook Fig.10.2.1)

19. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [19] Explore Imaging Model and Sparsity Recall the talk on Model-based Imaging by Dr. Bouman Exploit imaging model to improve SNR –Exploit sparsity in image data –E.g. use L 1 norm instead of L 2 as in compressed sensing => be aware sparsity with artificial model like head phantom may not completely reflect the real data

20. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [20] Sampling: From 1-D to 2-D and 3-D UMCP ENEE631 Slides (created by M.Wu © 2004)

21. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [21] Review: 1-D Sampling Time domain –Multiply continuous-time signal with periodic impulse train Frequency domain –Duality: sampling in one domain  tiling in another domain u FT of an impulse train is an impulse train (proper scaling & stretching) Review Oppenheim “Signals & Systems” Chapt.7 (Sampling); Chapt.3,4,5 (FS,FT,DFT) x(t) p(t) =  k  ( t - kT) T x s (t) P(  ) =  k  (  - 2k  /T) *2  /T 2  /T X(  )  Xs()Xs() 2  /T UMCP ENEE631 Slides (created by M.Wu © 2001)

22. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [22] Review: 1-D Sampling Theorem 1-D Sampling Theorem –A 1-D signal x(t) bandlimited within [-  B,  B ] can be uniquely determined by its samples x(nT) if  s > 2  B (sample fast enough). –Using the samples x(nT), we can reconstruct x(t) by filtering the impulse version of x(nT) by an ideal low pass filter Sampling below Nyquist rate (2  B ) cause Aliasing X s (  ) with  s < 2  B  Aliasing  s =2  /T BB X s (  ) with  s > 2  B  Perfect Reconstructable  s =2  /T BB -s-s UMCP ENEE631 Slides (created by M.Wu © 2001)

23. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [23] Extend to 2-D Sampling with Rectangular Grid Bandlimited 2-D signal –Its FT is zero outside a bounded region ( |  x |>  x0, |  y |>  y0 ) in spatial frequency domain Real-word multi-dimensional signals often exhibit diamond or football shape of support u with spectrum normalization => approx. spherical shape of support [Jain’s Fig.4.6] UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

24. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [24] 2-D Sampling (cont’d) 2-D impulse function (a.k.a. 2-D comb) comb(x,y;  x,  y) =  m,n  ( x - m  x, y - n  y ) ~ separable function FT: COMB(  x,  y ) = comb(  x,  y ; 1/  x, 1/  y) /  x  y Sampling vs. Replication (tiling) –Nyquist rates (2  x0 and 2  y0 )  Aliasing Jain’s Fig.4.7 UMCP ENEE631 Slides (created by M.Wu © 2001)

25. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [25] Examples of Aliasing Zig-zag / staircase at straight edges and ringing artifacts Moiré effects E.g. scanned image of a uniform gray field exhibits stripes instead of flat http://en.wikipedia.org/wiki/Moir%C3%A9_pattern –Input image contains periodicities that are close to half of sampling freq. u e.g. when display/printing dot size is small compared to sampling period u Anti-aliasing filter’s response at cut-off band is insufficient to suppress large spectrum values around half of sampling freq. –Also appear when sampling a periodic signal at lower rate UMCP ENEE631 Slides (created by M.Wu © 2007) From Woods’ book Fig.2.11

26. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [26] Examples of Aliasing Zig-zag / staircase at straight edges and ringing artifacts Moiré effects E.g. scanned image of a uniform gray field exhibits stripes instead of flat UMCP ENEE631 Slides (created by M.Wu © 2007) Figures from Gonzalez’s 3/e book online resource

27. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [27] Moiré Effects E.g. scanned uniform gray field exhibits stripes See also http://en.wikipedia.org/wiki/Moir%C3%A9_pattern Input image contains periodicities that are close to half of sampling frequency –e.g. when display/printing dot size is small compared to sampling period –Anti-aliasing filter’s response at cut-off band is insufficient to suppress large spectrum values around half of sampling freq. Also appear when sampling a periodic signal at lower rate Can overcome moiré effects of sampled halftone image by notch filtering UMCP ENEE631 Slides (created by M.Wu © 2007) (Image examples from wikipedia; see also Gonzalez 3/e 4.5.4, 4.10.1)

28. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [28] u E.g. Recent IEEE Spectrum Magazine (S’09) has an interesting article, opening with downsampling sinusoid at proper rate to create a sinusoid of any period

29. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [29] 2-D Sampling: Beyond Rectangular Grid Sampling at nonrectangular grid –M ay give more efficient sampling density when spectrum region of support is not rectangular u Sampling density measured by #samples needed per unit area –E.g. interlaced grid for diamond- shaped region of support u equiv. to rotate 45-degree of rectangular grid u spectrum rotate by the same degree From Wang’s book preprint Fig.4.2 UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

30. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [30] General Sampling Lattice Lattice  in K-dimension space R K –A set of all possible vectors represented as integer weighted combinations of K linearly independent basis vectors Generating matrix V (sampling matrix) V = [v 1, v 2, …, v k ] => lattice points x = V n e.g., identity matrix V ~ square lattice Voronoi cell of a lattice –A “unit cell” of a lattice, whose translations cover the whole space –Enclose all points that are closer to the origin than to other lattice points u cell boundaries are equidistant lines between surrounding lattice points From Wang’s book preprint Fig.3.1 UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

31. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [31] Sampling Density: d1 = 1 d2 = 2 /  3 From Wang’s book preprint Fig.3.1 Examples of Lattices UMCP ENEE631 Slides (created by M.Wu © 2004)

32. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [32] Periodicity in Lattice Representations Lattice represents a multi-dimensional periodic function –Repeats a basic pattern of a unit cell with all integer steps of translations –Voronoi cell describes the fundamental period of this periodic function Finding Voronoi cell (1) Draw straight line between origin and each of the closest nonzero lattice points; (2) Draw a perpendicular line that is half way between the two points => The polygon formed by these equidistant lines surrounding the origin is the Voronoi cell. u For 3-D or multi-dim, replace equidistant line with equidistant (hyper-)planes Sampling density d(  ) = 1 / |det(V)| |det(V)| measures volume of a cell; d(  ) is # lattice points in unit volume. UMCP ENEE631 Slides (created by M.Wu © 2007)

33. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [33] Frequency Domain View & Reciprocal Lattice Reciprocal lattice  # for a lattice  (with generating matrix V) –Generating matrix of  # is U = (V T ) -1 –Basis vectors for  and  # are orthonormal to each other: V T U = I –Denser lattice  has sparser reciprocal lattice  # : det(U) = 1 / det(V) Frequency domain view of sampling over lattice –Sampling in spatial domain  Repetition in frequency domain –Repetition grid in freq. domain can be described by reciprocal lattice –Intuition for “reciprocal” [e.g.] rectangular grid that sample faster horizontally than vertically =>the repetition in frequency domain is slower horizontally than vertically Aliasing and prefiltering to avoid aliasing –Aliasing happens when signal spectrum extends outside the Voronoi cell of reciprocal lattice UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

34. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [34] Generalized Sampling Theorem –Wang’s book Sec.3.2.2, Theorem 3.6 –(see also Sec.2.1-2.2 on multi-dimensional FT and DFT) UMCP ENEE631 Slides (created by M.Wu © 2004)

35. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [35] Sampling Efficiency Consider spherical signal spectrum support –Many real-world signals have symmetric freq. contents in many directions –The multi-dim spectrum can be approximated well by a sphere (with proper scaling of spectrum support) Voronoi cell of reciprocal lattice need to cover the sphere to avoid aliasing –Tighter fit of the Voronoi cell to the sphere requires lower sampling density What lattice gives the best sphere-covering capability? Sampling Efficiency  = volume(unit sphere) / d(  )  prefer close to 1 From Wang’s book preprint Fig.4.2 & 3.5 UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

36. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [36] Recall: 1-D Upsample and Downsample From Crochiere-Rabiner “Multirate DSP” book Fig.2.15-16 UMCP ENEE631 Slides (created by M.Wu © 2001)

37. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [37] General Procedures for Sampling Rate Conversion From Wang’s book preprint Fig.4.1 UMCP ENEE631 Slides (created by M.Wu © 2001)

38. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [38] Sampling Lattice Conversion From Wang’s book preprint Fig.4.4 Intermediate Original Targeted UMCP ENEE631 Slides (created by M.Wu © 2001)

39. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [39] Case Studies on Sampling and Resampling in Video Processing Reference Readings: Wang’s book Chapter 4 e.g. Interlaced 50 fields/sec  60 fields/sec UMCP ENEE631 Slides (created by M.Wu © 2004)

40. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [40] Summary of Today’s Lecture Radon transform and Inverse Radon transform –by Filtered Back Projection Sampling issues for image: extending 1-D to 2-D –Sampling lattice and frequency-domain interpretation –Sampling rate conversion Next Lecture: –More on Sampling on non-rectangular grid via the Lattice Theory Readings –Radon transform: Gonzalez 3/e book 5.11 [More readings] Bovik’s Handbook Sec.10.2; Jain’s Sec.10.1-10.6, 10.9 –2-D Sampling: Gonzalez 3/e book 4.3, 4.5 –Lattice Sampling: Wang’s book Sec. 3.1-3.3, 3.5; Chapter 4 UMCP ENEE631 Slides (created by M.Wu © 2004)

41. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [41] Example: Frame Rate Conversion Video sampling: formulate as a 3-D sampling problem Note: different signal characteristics and visual sensitivities along spatial and temporal dimensions (see Wang’s Sec.3.3 on video sampling) General approach to frame rate conversion –Upsample => LPF => Downsample Interlaced 50 fields/sec  60 fields/sec –Analyze in terms of 2-D sampling lattice (y, t) –Convert odd field rate and even field rate separately u do 25  30 rate conversion twice => not fully utilize info. in other fields –Deinterlace first then convert frame rate u do 50  60 frame rate conversion: 50  300  60 –Simplify 50  60 by converting 5 frames  6 frames u each of output 6 frames is from two nearest frames of the 5 originals u weights are inversely proportional to the distance between I/O –May do motion-interpolation for hybrid-coded video UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

42. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [42] From Wang’s book preprint Fig.4.3 UMCP ENEE631 Slides (created by M.Wu © 2001)

43. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [43]

44. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [44] Video Format Conversion for NTSC  PAL Require both temporal and spatial rate conversion –NTSC 525 lines per picture, 60 fields per second –PAL 625 lines per picture, 50 fields per second Ideal approach (direct conversion) –525 lines 60 field/sec  13125 line 300 field/sec  625 lines 50 field/sec 4-step sequential conversion –Deinterlace => line rate conversion => frame rate conversion => interlace UMCP ENEE631 Slides (created by M.Wu © 2001)

45. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [45] Video Format Conversion for NTSC  PAL Require both temporal and spatial rate conversion –NTSC 525 lines per picture, 60 fields per second –PAL 625 lines per picture, 50 fields per second Ideal approach (direct conversion) –525 lines 60 field/sec  13125 line 300 field/sec  625 lines 50 field/sec 4-step sequential conversion –Deinterlace => line rate conversion => frame rate conversion => interlace –Food for thought: compare the computation of ideal and sequential u Hint: filtering done for X lines @ Y field/sec => examine X & Y UMCP ENEE631 Slides (created by M.Wu © 2001)

46. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [46] From Wang’s book preprint Fig.4.9 UMCP ENEE631 Slides (created by M.Wu © 2001)

47. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [47] Simplified Video Format Conversion 50 field/sec  60 field/sec –After deinterlacing, s implify to 5 frames  6 frames –Conversion involves two adjacent frames only 625 lines  525 lines –Simplify to 25 lines  21 lines –Conversion involves two adjacent lines only UMCP ENEE631 Slides (created by M.Wu © 2001)

48. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [48] UMCP ENEE631 Slides (created by M.Wu © 2001) From Wang’s book preprint

49. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [49] Interlaced Video and Deinterlacing Interlaced video Odd field at 0  Even field at  t  Odd field at 2  t  Even field at 3  t … Deinterlacing –Merge to get a complete frame with odd and even field Examples from http://www.geocities.com/lukesvideo/interlacing.html UMCP ENEE631 Slides (created by M.Wu © 2001)

50. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [50] De-interlacing: Practical Approaches Spatial interpolation –Vertical interpolation within the same field (1-D upsample by 2) –Line averaging ~ average the line above and below D=(C+E)/2 Temporal interpolation –2-frame field merging => artifacts –3-frame field averaging D=(K+R)/2 u fill in the missing odd field by averaging odd fields before and after Spatial-temporal interpolation –Line-and-field averaging D=(C+E+K+R)/4 UMCP ENEE631 Slides (created by M.Wu © 2001) From Wang’s book preprint

51. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [51] De-interlacing: Practical Approaches Spatial interpolation –Vertical interpolation within the same field (1-D upsample by 2) –Line averaging ~ average the line above and below D=(C+E)/2 u interpolation filter: [ ½, 1, ½ ] T u DTFT =  x(n) e -j2  n  y fy = 1 + ( e -j2  y fy + e j2  y fy ) / 2 = 1 + cos 2  y f y Temporal interpolation –2-frame field merging => artifacts –3-frame field averaging D=(K+R)/2 u fill in the missing odd field by averaging odd fields before and after Spatial-temporal interpolation –Line-and-field averaging D=(C+E+K+R)/4 UMCP ENEE631 Slides (created by M.Wu © 2001) From Wang’s book preprint

52. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [52] Motion-Compensated De-interlacing Stationary video scenes –Temporary deinterlacing approach yield good result Scenes with rapid temporal changes –Artifacts incurred from temporal interpolation –Spatial interpolation alone is better than involving temporal interpolation Switching between spatial & temporal interpolation modes –Based on motion detection result –Hard switching or weighted average –Motion-compensated interpolation UMCP ENEE631 Slides (created by M.Wu © 2001)

53. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [53] Example: De-interlacing From Woods’ book resource

54. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [54] Example: Temporal Upsampling (5 to 30 fps) From Woods’ book resource

55. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec25 – Back Projection and M-D Sampling [55] Summary of Today’s Lecture Sampling and resampling issues in 2-D and 3-D –Sampling lattice and frequency-domain interpretation –Sampling rate conversion –Applications in video format and frame rate conversions Readings –Woods’ book Chapter 2, (9.2), 10.3.3-10.3.4. –Explore further: Wang’s book Sec. 3.1-3.3, 3.5; Chapter 4 UMCP ENEE631 Slides (created by M.Wu © 2004)