1. M. Wu: ENEE631 Digital Image Processing (Spring'09) Medical Imaging Topic: Radon Transform & Inverse Radon Transf. Spring ’09 Instructor: Min Wu Electrical and Computer Engineering Department, University of Maryland, College Park   bb.eng.umd.edu (select ENEE631 S’09)   minwu@eng.umd.edu ENEE631 Spring’09 Lecture 24 (4/29/2009)

2. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [2] Overview and Logistics Last Time: –Non-intrusive image forensics –Useful image features and feature extraction techniques u Projection based: convert 2-D data to 1-D profile u Parameter space: detect line & other structure with low computation complexity u Gradient based: corner detection via eigen properties of derivative matrix Today: –More on gradient based image features –Radon transform and applications in medical image processing Course Logistics u Project progress report: Due next Monday 5pm May 4 by email u Project Topic 2: start with 1-2 small datasets and 2-3 types of features; make critical evaluation/comparison; do your own feature extractions in final submission; can use and acknowledge open-source classifier. UMCP ENEE631 Slides (created by M.Wu © 2004)

3. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [3] ENEE631 End of Semester Plan & Update –Assignments20%=>25% u 4 Homework, 5% each u No final => 5 th assignment (no programming)=> 5% –Projects45%=>50% u Project 1 on wavelet image coding ~20% u Project 2 ~25%=> 30% – Exams35%=>20% u In-class midterm exam ~20% u Final take-home exam ~15%=> no final –Base points (and extra for active class participations) => 5% Project presentation & demo: Thurs. 5/21 …

4. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [4] –For each point (x 0, y 0 ), lines of all angles passing it form a sinusoid curve in ( ,  ) space –( ,  ) curves corresponding to colinear points intersect at a point (  0,  0 ) => useful for line detection Illustration of Hough Transform Figures from http://en.wikipedia.org/wiki/Hough_transform ( ,  ) space: x cos  + y sin  = 

5. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [5] Example: Detecting Lines in Hough Transf. Domain Intensity peaks of the transformed results correspond to the ( ,  ) of the respective lines Can extend to other patterns such as circles Figures from http://en.wikipedia.org/wiki/Hough_transform

6. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [6] Pattern Matching Minimum distance classifier: compare with mean feature vector of each class (if known or can be learned) Matching by correlation –Reduce sensitivity via normalized correlation (correlation coeff.) –Exhaustive search for size & orientations ~ computationally expensive –Learn more on pattern classification u Overview: Chapter 12 of Gonzalez’s book u ENEE731 Statistical Pattern Recognition; CS Machine Learning course Figure from Gonzalez’s 2/e book resource

7. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [7] Corner Detector Corner: intersection of 2 edges –Large variation in local intensity in multiple directions –Alleviate “aperture problem” in motion analysis & image matching Harris-Stephens detector (1988) –Examine (weighted) Sum of Squared Difference u between an image patch & neighborhood –Build a 2x2 Harris matrix A u Describes 2 nd order derivatives of the above SSD averaged over neighborhood –Examine the two eigen values of A u If both close to 0, there are no features of interest at pixel (x,y). u If one close to 0 and the other has large positive value, then an edge is found. u If both have large, distinct positive values, then a corner is found. u May avoid computing eigen: examine M = det(A) – k tr(A) 2 with k = 0.04~0.15 Ref. and examples at http://en.wikipedia.org/wiki/Corner_detection

8. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [8]

9. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [9] More on Gradient-based Salient Feature Points Recall operations for robust edge detection ~ e.g. Canny –Laplacian of Gaussian (LoG) [Gonzalez 3/e 10.2.6] u effectively bandpass filtering to suppress noise when taking derivatives –Ridge of gradient magnitude and edge linking u Local extrema of gradient; seek stable edge info Recall multiresolution analysis Achieve invariance to scaling and rotation –Take account of multiple resolution scale levels –Represent orientation w.r.t. dominant or canonical direction => Scale Invariant Feature Transform (SIFT) by D. Lowe u Give about 2000 stable “keypoints” for a typical 500 x 500 image u Each keypoint is described by a vector of 4 x 4 x 8 = 128 elements (over 4x4 array of 8-bin gradient histograms in keypoint neighborhood)

10. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [10] Recall: Robust Edge Detector Apply LPF to suppress noise, then apply edge detector or derivative operations E.g. Laplacian of Gaussian (LoG): in shape of Mexican hat Figures from Gonzalez- Woods 2/e online slides.

11. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [11] Canny Edge Detector –J. Canny: A Computational Approach to Edge Detection, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol 8, No. 6, Nov 1986. –See 10.2.6 in 3 rd ed of Gonzalez. –http://www.cee.hw.ac.uk/hipr/html/canny.htmlhttp://www.cee.hw.ac.uk/hipr/html/canny.html The Canny operator works in a multi-stage process. First of all the image is smoothed by Gaussian convolution.Gaussian convolution Then a simple 2-D first derivative operator (somewhat like the Roberts Cross) is applied to the smoothed image to highlight regions of the image with high first spatial derivatives.Roberts Cross Edges give rise to ridges in the gradient magnitude image. The algorithm then tracks along the top of these ridges and sets to zero all pixels that are not actually on the ridge top so as to give a thin line in the output, a process known as non-maximal suppression. The tracking process exhibits hysteresis controlled by two thresholds: T1 and T2 with T1 > T2. Tracking can only begin at a point on a ridge higher than T1. Tracking then continues in both directions out from that point until the height of the ridge falls below T2. This hysteresis helps to ensure that noisy edges are not broken up into multiple edge fragments.

12. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [12] SIFT: Employ “Scale Space Extrema” Examine Differences of Gaussian filtered images at nearby scale  and k  Avoid low contrast & poorly defined DoG peaks Figures from Lowe’s IJCV 2004

13. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [13] Examples of Difference of Gaussian Examples from SIFT tutorial notes by Estrada/Jepson/Fleet (2004)

14. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [14] Examples from SIFT tutorial notes by Estrada/Jepson/Fleet (2004)

15. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [15] (weighted) Gradient orientation histogram near a stable DoG extrema –Peaks in histogram correspond to dominant orientation –Also include other directions close to the peak as keypoints for higher stability –Measure properties of a keypoint relative to its assigned orientation to gain rotational invariance Keypoint Descriptor records gradient pattern of each keypoint –As a set of 8-bin orientation histograms over 4x4 nearby blocks SIFT: Rotational Invariant Keypoint Descriptor Figure from Lowe’s IJCV 2004

16. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [16] Boundary Representation Chain code 4-way or 8-way connectivity Features/measurement obtained from object’s chain code –Perimeter; Area; Centroid Figure from Gonzalez’s book resource

17. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [17] Boundary Descriptors Simple descriptors –boundary length, diameter –curvature (rate of change of slope/tangent); B-splines Shape number –The 1st difference of smallest magnitude of chain code u to normalize chain code’s start point and orientation Fourier descriptor –Record point coordinates w/ a sequence of complex # –DFT coeff. of the complex sequence as Fourier descriptors –First several coeff. represent coarse info of the shape Statistical moments –mean, variance, and higher-order moments of boundary segments and/or 1-D representation of boundary e.g. r(  )

18. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [18] Radon Transform and its Applications in Medical Image Processing

19. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [19] Non-Intrusive Medical Diagnosis (From Jain’s Fig.10.1)

20. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [20] Non-Intrusive Medical Diagnosis (cont’d) Observe a set of projections (integrations) along different angles of a cross-section –Each projection itself loses the resolution of inner structure –Types of measurements u transmission (X-ray), emission, magnetic resonance (MRI) Want to recover inner structure from the projections –“Computerized Tomography” (CT) (From Bovik’s Handbook Fig.10.2.1) Emission tomography: measure emitted gamma rays by the decay of isotopes from radioactive nuclei of certain chemical compounds affixed to body parts. MRI: based on that protons possess a magnetic moment and spin. In magnetic field => align to parallel or antiparallel. Apply RF => align to antiparallel. Remove RF => absorbed energy is remitted and detected by RF detector.

21. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [21] Radon Transform A linear transform f(x,y)  g(s,  ) –Line integral or “ray-sum” –Along a line inclined at angle  from y-axis and s away from origin Fix  to get a 1-D signal p  (s)= g(s,  ) (From Jain’s Fig.10.2)

22. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [22] Example of Image Radon Transform (From Matlab Image Processing Toolbox Documentation) [Y-axis] distance, [X-axis] angle

23. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [23] Inverting A Radon Transform To recover inner structure from projections Need many projections to better recover the inner structure Reconstruction from 18, 36, and 90 projections (~ every 10,5,2 degrees) (From Matlab Image Processing Toolbox Documentation)

24. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [24] Recall: Rotation & Translation of Image Coordinates –Note the relations with rotation and translation of image objects x y  y’ x’ UMCP ENEE631 Slides (created by M.Wu © 2001)

25. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [25] Rotational Invariance Property of Fourier Transform Consider a 2-D continuous signal & its 2-D Fourier transform 2-D FT is rotationally invariant FT of a rotated version of function f(t1, t2) is just to apply same rotation to F(  1,  2)

26. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [26] Connection Between Radon & Fourier Transform Observations –Look at 2-D FT coeff. along horizontal frequency axis u FT of 1-D signal u 1-D signal is vertical summation (projection) of original 2-D signal –Look at FT coeff. along  =  0 ray passing origin u FT of projection of the signal perpendicular to  =  0 Projection Theorem Proof using FT definition & coordinate transform (Figure from Bovik’s Handbook Fig.10.2.7)

27. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [27] Projection Theorem (a.k.a. Projection-Slice Theorem) Let F(  1,  2) be the FT of a cont’s function f(t1, t2) Let Radon transform of f be given as p  (s) with 1-D FT P  (  ) Projection Theorem: P  (  ) = F(  cos ,  sin  ) for all  i.e. FT of a slice is the corresponding slice of the 2-D FT –We can obtain the full 2-D FT in terms of 1-D freq. components of the Radon transform at all angles.

28. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [28] Inverting Radon by Projection Theorem (Step-1) Filling 2-D FT data matrix with 1-D FT of Radon along different angles (Step-2) 2-D IFT Need Polar-to-Cartesian grid conversion for discrete scenarios –May lead to artifacts esp. with less densely sampled slices (From Jain’s Fig.10.16)

29. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [29] Back-Projection Sum up Radon projection along all angles passing the same pixel location (x, y) (From Jain’s Fig.10.6)

30. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [30] Back Projection: Example

31. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [31] Back-projection = Inverse Radon ? Not exactly ~ Back-projection gives a blurred recovery –Intuition: most contribution is from the pixel (x,y), but still has some tiny contribution from other pixels –B ( R f ) = conv( f, h1 ) –Bluring function h1 = (x 2 + y 2 ) -1/2, FT( h1 ) ~ 1 / |  | where radial frequency  2 =  x 2 +  y 2 (reflect how far from DC) Need to apply inverse filtering to fully recover the original –Inverse filtering for “sharpening”: multiplied by |  | in FT domain

32. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [32] Inverting Radon via Filtered Back Projection f(x,y) = B H g Change coordinate (Cartesian => polar) Projection Theorem ( F_polar => G) Back Projection(FT domain) filtering (From Jain’s Fig.10.8)

33. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [33] Properties/Applications of Filtered Back Projection Convolution-Projection Theorem –Radon[ f1  f2] = Radon[ f1 ]  Radon[ f2 ] u Radon and filtering operations are interchangeable u Can prove using Projection Theorem –Also useful for implementing 2-D filtering using 1-D filtering Another view of filtered back projection –Change the order of filtering and back-projection u Back Projection => Filtering u Filtering => Back Projection => See more discussions and applications in Jain’s

34. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [34] Other Scenarios of Computerized Tomography Parallel beams vs. Fan beams –Faster collection of projections via fan beams u involve rotations only Recover from projections contaminated with noise –MMSE criterion to minimize reconstruction errors  See Gonzalez 3/e book and Bovik’s Handbook for details (From Bovik’s Handbook Fig.10.2.1)

35. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [35] Explore Imaging Model and Sparsity Recall the talk on Model-based Imaging by Dr. Bouman Exploit imaging model to improve SNR –Exploit sparsity in image data –E.g. use L 1 norm instead of L 2 as in compressed sensing => be aware sparsity with artificial model like head phantom may not completely reflect the real data

36. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec 24 – Radon Transf / Medical Imaging [36] Summary of Today’s Lecture Gradient based image features Medical imaging topic: computerized tomography –Radon transform and Inverse Radon transform u by Projection Theorem u by Filtered Back Projection Next week: 2-D sampling theory Readings –Stable Gradient Features ~ SIFT David G. Lowe (2004). "Distinctive Image Features from Scale-Invariant Keypoints". International Journal of Computer Vision 60 (2): 91–110. –Boundary representation: Gonzalez’s 3/e book 11.1 – 11.2 –Radon transform: Gonzalez 3/e book 5.11 [More readings] Bovik’s Handbook Sec.10.2; Jain’s Sec.10.1-10.6, 10.9 UMCP ENEE631 Slides (created by M.Wu © 2004)