1. A Pipeline for Computer Aided Polyp Detection Wei Hong, Feng Qiu, and Arie Kuafman Center for Visual Computing (CVC) and Department of Computer Science Stony Brook University

2. Related Work Shape based polyp detection method Shape based polyp detection method Vining et al. 99: colon wall thickness Vining et al. 99: colon wall thickness Tomasi et al. 00: sphere fitting Tomasi et al. 00: sphere fitting Summers et al. 01: local curvature variations Summers et al. 01: local curvature variations Yoshida et al. 01: shape index and curvedness Yoshida et al. 01: shape index and curvedness Paik et al. 04: intersecting normal vectors Paik et al. 04: intersecting normal vectors Wang et al. 06: global curvature Wang et al. 06: global curvature Sensitive to the irregularity of the Sensitive to the irregularity of the colon wall colon wall Relatively high false positive rate Relatively high false positive rate 2

3. Electronic Biopsy 3 Opaque transfer function Transparent transfer function Polyps have a slightly higher density and different texture.

4. Overview of Our CAD Pipeline 4 Segmentation and Digital Cleansing Colon Surface Extraction Conformal Colon Flattening Electronic Biopsy Image Generation Polyp Detection by Clustering False Positive Reduction Integration with Virtual Colonoscopy Contrast-enhanced CT

5. Step 1 5 Segmentation and Digital Cleansing Colon Surface Extraction Conformal Colon Flattening Electronic Biopsy Image Generation Polyp Detection by Clustering False Positive Reduction Integration with Virtual Colonoscopy

6. Segmentation & Digital Cleansing Input: contrast-enhanced CT scan of patients abdomen Input: contrast-enhanced CT scan of patients abdomen Goal: segment and cleanse colon lumen Goal: segment and cleanse colon lumen Challenges for digital cleansing: Challenges for digital cleansing: 1.Remove the interface layer between air and tagged fluid 2.Restore the CT densities in the enhanced mucosa layer 6 interface air tagged fluid mucosa

7. Partial Volume Segmentation Assumptions: Assumptions: 1.Four material classes within each voxel i (air, soft tissue, muscle, and bone) 2.Each material follows a Gaussian distribution : the observed density value at voxel i : Gaussian noise with zero mean : fraction of classes k Using expectation-maximization algorithm to estimate Using expectation-maximization algorithm to estimate 7

8. Segmentation Results 8 air bone air & fluid partial volume effect tissue & fluid partial volume effect

9. Digital Cleansing Results Original CT slice Cleansed slice Zoomed view 9 Cleansing equation:

10. Step 2 10 Segmentation and Digital Cleansing Colon Surface Extraction Conformal Colon Flattening Electronic Biopsy Image Generation Polyp Detection by Clustering False Positive Reduction Integration with Virtual Colonoscopy

11. 11 Genus zero Colon Surface Extraction Topological noise (i.e., tiny handles) Topological noise (i.e., tiny handles) makes our flattening algorithm complex makes our flattening algorithm complex introduces distortion introduces distortion Simple point: A point is simple if its Simple point: A point is simple if its addition to and removal from objects addition to and removal from objects does not change object topology. does not change object topology. Our 3D region growing based method Our 3D region growing based method Computing a distance field Computing a distance field Computing colon centerline Computing colon centerline Region growing all simple points Region growing all simple points Simple point Critical point

12. Step 3 12 Segmentation and Digital Cleansing Colon Surface Extraction Conformal Colon Flattening Electronic Biopsy Image Generation Polyp Detection by Clustering False Positive Reduction Integration with Virtual Colonoscopy

13. Virtual Colon Flattening Bartroli et al. 01: area preserving Bartroli et al. 01: area preserving Haker et al. 00: angle preserving Haker et al. 00: angle preserving Genus 0 surfaces Genus 0 surfaces Mapping to a planar parallelogram Mapping to a planar parallelogram Our Method: angle preserving Our Method: angle preserving Surfaces with arbitrary topology Surfaces with arbitrary topology Mapping to a 2D rectangle Mapping to a 2D rectangle 13

14. Conformal Colon Flattening 14 1.Computing a gradient field of the conformal map 2.Computing the conformal map by integration 3.Tracing a horizontal line 4.Cutting the colon surface along the horizontal line

15. 15 Angle Preserving 3D 2D cutting line

16. Step 4 16 Segmentation and Digital Cleansing Colon Surface Extraction Conformal Colon Flattening Electronic Biopsy Image Generation Polyp Detection by Clustering False Positive Reduction Integration with Virtual Colonoscopy

17. Electronic Biopsy Image Generation 17 1.Pre-defined translucent transfer function 2.Surface normals used as ray directions 3.Rays are allowed to traverse up to 40 steps (0.5mm/step) 4.Rays cannot enter colon lumen 5.GPU acceleration (300ms for 4000X196 image)

18. Step 5 18 Segmentation and Digital Cleansing Colon Surface Extraction Conformal Colon Flattening Electronic Biopsy Image Generation Polyp Detection by Clustering False Positive Reduction Integration with Virtual Colonoscopy

19. Polyp Detection by Clustering For each pixel, we use the color information in its small neighborhood as the feature vector Method: 1.PCA: reduce the dimension of the feature vectors 2.Clustering algorithm: classify each pixel 3.A labeling algorithm: extract the connected components We only consider polyps with a diameter > 5mm, small components are removed. 19

20. Step 6 20 Segmentation and Digital Cleansing Colon Surface Extraction Conformal Colon Flattening Electronic Biopsy Image Generation Polyp Detection by Clustering False Positive Reduction Integration with Virtual Colonoscopy

21. Reduction of False Positives Shape features are exploited for polyp detection Volumetric shape index & curvedness Yoshida et al. The computation of these volumetric shape features is time consuming We only do it at suspicious areas for FP reduction 21

22. Results of Clustering and False Positive Reduction 22 Electronic biopsy image The result of our clustering algorithm The result of FP reduction

23. Step 7 23 Segmentation and Digital Cleansing Colon Surface Extraction Conformal Colon Flattening Electronic Biopsy Image Generation Polyp Detection by Clustering False Positive Reduction Integration with Virtual Colonoscopy

24. The extracted colon mesh is used to accelerate volumetric ray-casting The extracted colon mesh is used to accelerate volumetric ray-casting Colon mesh is projected onto the image plane Colon mesh is projected onto the image plane Empty space between image plane and colon wall is skipped Empty space between image plane and colon wall is skipped Frame rate: 17-20/sec for image size of 512X512 Frame rate: 17-20/sec for image size of 512X512 Suspicious polyp candidates are highlighted in the endoscopic view to attract the attention of the radiologists Suspicious polyp candidates are highlighted in the endoscopic view to attract the attention of the radiologists A flattened colon image is also provided A flattened colon image is also provided Suspicious polyp locations Suspicious polyp locations Bookmarks Bookmarks 24

25. Enhanced Endoscopic View 25

26. The User Interface of Our CAD System 26

27. 27 Datasets 52 CT datasets from National Institute of Health (NIH) 52 CT datasets from National Institute of Health (NIH) 400~500 Raw DICOM images (512X512) 400~500 Raw DICOM images (512X512) VC reports and videos VC reports and videos OC reports and videos OC reports and videos Pathology reports Pathology reports 46 CT datasets from Stony Brook University Hospital (SB) 46 CT datasets from Stony Brook University Hospital (SB) 400~500 Raw DICOM images (512X512) 400~500 Raw DICOM images (512X512) VC reports and videos VC reports and videos OC reports OC reports Pathology reports Pathology reports

28. Experimental Results 28 Source Total Polyps Sensitivity FP per Dataset FP Reduction NIH58100%3.196.3% SB65100%2.997.1% Stage of our CAD Pipeline Timing Segmentation and Digital Cleansing 3 mins Colon Surface Extraction < 1 min Conformal Colon Flattening 7 mins Biopsy Image Generation 300 ms Polyp Detection by Clustering < 1 min False Positive Reduction < 1 min 3.6GHz Pentium IV, 3G Ram, Quadro FX4500, 512^ 3

29. Conclusions A novel method for automatic polyp detection by integrating direct volume rendering with conformal colon flattening 100% sensitive to polyps with a low FP rate 100% sensitive to polyps with a low FP rate Highlighting the polyp locations Highlighting the polyp locations Enhancing the user interface of VC Enhancing the user interface of VC Improve the efficiency and accuracy of VC Improve the efficiency and accuracy of VC 29

30. Future Work Improving detection algorithm to further reduce FPs Improving detection algorithm to further reduce FPs Porting our CAD pipeline to a clinical VC system Porting our CAD pipeline to a clinical VC system Supine and prone registration Supine and prone registration Applying our methods to other human organs Applying our methods to other human organs Blood vessel Blood vessel Bladder Bladder 30

31. Questions? Thank You! 31