1. Perceptual Organization based method in vessel extraction from real retina images Revised on Sept 17,2004 Frank Tao

2. Motivation & Objectives: Retina vessel map segmentation is very important to medical applications, such as diabetic retinopathy, aging related retina analysis etc. Retina vessel map segmentation is very important to medical applications, such as diabetic retinopathy, aging related retina analysis etc. Available effective solutions will either cause high computational cost or need users intervention Available effective solutions will either cause high computational cost or need users intervention Our objectives: Our objectives: –Develop an efficient, accurate automatic solution based on perceptual organization principle: perceptual curve partition & grouping Edge trace partition Edge trace partition Generic edge token grouping Generic edge token grouping

3. Review Available researches can be grouped into following classes based on a review paper: Available researches can be grouped into following classes based on a review paper: –Pattern recognition –Matched filter related methods (MFR) –Regional growing –Vessel tracking –Artificial intelligent –Others

4. Review -continuing All the available systems can also be re-grouped into following classes based on the different features they are trying to search for: All the available systems can also be re-grouped into following classes based on the different features they are trying to search for: –Linear segment structure MFR related methods MFR related methods Morphology models: snake, water shade Morphology models: snake, water shade Regional growing Regional growing Some tracking methods Some tracking methods –Center line and/or edge Zhou matched filter edge tracking Zhou matched filter edge tracking Quebec parallel matching edge tracking Quebec parallel matching edge tracking Sobel edge detection and tracking Sobel edge detection and tracking Others Others –Others Artificial intelligence Artificial intelligence –Fuzzy c mean –others

5. Pro and cons of current systems Line segment structure based : MFR, Pattern recognition, Artificial intelligence etc. Line segment structure based : MFR, Pattern recognition, Artificial intelligence etc. –Advantages Automatic system Automatic system Good noise suppression and vessel segmentation Good noise suppression and vessel segmentation Continues vessel map including junction structures Continues vessel map including junction structures –Disadvantages High computational cost High computational cost Center line and/or edge based : Vessel tracking Center line and/or edge based : Vessel tracking –Advantages Computational efficiency Computational efficiency Good noise suppression and vessel segmentation Good noise suppression and vessel segmentation –Disadvantages Non automatic Non automatic Non continues map with poor junction detection and breaking of vessel segments Non continues map with poor junction detection and breaking of vessel segments

6. Our proposed system System design: System design: –Robust vessel feature extraction based on Perceptual Organization –Effective vessel junction and breaking fixing and extracting using limited numbers of guided matched filters Targets: Targets: –Fully automatic –Very low computational cost –Good noise suppression and vessel segmentation –Continues vessel map including junctions and low intensity vessel segments

7. Perceptual Organization based method for vessel segment extraction GAO’s Curve Partitioning Methods GAO’s Curve Partitioning Methods –Image processing with edge map obtained from an Edge Tracker software which detected and extracted all the edge traces based on the following rules: Intensity similarity Intensity similarity Shortest distance Shortest distance Direction similarity Direction similarity Noise removal principle Noise removal principle –Linearity –length –Curve Partitioning and Grouping

8. Perceptual Partition & Grouping Gibson’s Observation: The qualities of a simple line observed by Gibson: (a) “Left slant… Zero slant… Right Slant” (b) “Convex…straight…concave”

9. Perceptual Partition & Grouping Psychological experiments of 2-D curve partitioning: 1) best mark those locations at which distinctive curve segments are “ glued ” together; 2) best allow the reconstruction of the complete curves; 3) best allow a viewer to distinguish a given curve from the others.

10. GCS Partition Model Analytic descriptors of curves: f (x,a) = 0 Analytic descriptors of curves: f (x,a) = 0 where x denotes an image point, a is a vector of parameters. A generic curve segment : GCS = { x | p (x) } A generic curve segment : GCS = { x | p (x) } where x is an edge point, p (x) indicates the point satisfies the property p. This property p can be represented by the following function: p (x) = { f (x), j (y), f’ (x), j’ (y)} p (x) = { f (x), j (y), f’ (x), j’ (y)} Where y = f (x) is a curve, x = j (y) is its inverse function, f’ (x) and j’ (y) are their first derivatives

11. GCS Partition Model GCSf(x)  (y) f’(x)  ’(y) CS1M+M+M+M- CS2M-M-M+M- CS3M+M+M-M+ CS4M-M-M-M+ LS1M-M-cc LS2M+M+cc LS3cN/A0 LS4N/Ac0 A set of generic curve segments (GCS) Definition of GCS, M+ is monotonic increase and vice verse

12. GCS Grouping Rule # DefinitionsG1 (CPP1, CS1, CS2) G2 (CPP2, CS2, CS3) G3 (CPP3, CS3, CS4) G4 (CPP4, CS4, CS1) G5 (CPP5, CS1, CS3) G6 (CPP6, CS2, CS4) G7 (CPP7, CS, LS) G8 (CPP8, LSi, LSj) Definition of CPPs and Curve Grouping Rules: Extra CPPs (dark dots) introduced to increase the sensitivity of junction detection

13. Retinal Image Based Knowledge Vessel map definition Vessel map definition –Junctions and Endings Junctions: Branching junctions (including Y junction and T junction); Crossing junctions Junctions: Branching junctions (including Y junction and T junction); Crossing junctions Endings Endings –Vessel Segments Perceptual Partitioning and Grouping of the edge trace map Perceptual Partitioning and Grouping of the edge trace map –Original CPP detection: Aligned CPP, Junction CPP, Ending CPP –Virtual CPP creation through two-sides-parallel-scanning Two Sides Parallel Scanning: stretched out from both side of the detected CPP, using the gradient of the original pixel to do a parallel scanning, try to find matching pair pixels with reverse gradient within a pre-defined vessel width Two Sides Parallel Scanning: stretched out from both side of the detected CPP, using the gradient of the original pixel to do a parallel scanning, try to find matching pair pixels with reverse gradient within a pre-defined vessel width –Associated parallel GCS grouping based on original and virtual CPPs How to find out the Vessel segments in the edge trace map? How to find out the Vessel segments in the edge trace map? –Extracting vessel segments through connecting all the directly linked associated parallel GCS pairs

14. Vessel Map Definition Original Retina ImageVessel Map definition

15. CPP and related structure Junction CPP and related structureNon-Junction CPP and related structure

16. CPP detection and Virtual CPP creation Original CPPVirtual CPP creation via Two- Side-Parallel-Scanning

17. Associated parallel GCS grouping and Vessel segment extraction Associated parallel GCS grouping Vessel segment extraction Original edge trace map

18. Vessel junction, breaking detection and extraction using guided matched filters Assume vessel segment has: Assume vessel segment has: –Gaussian shaped gradient profile perpendicular to it’s length direction –Piecewise linear structure –Vessel width very close thus can be treated as same Assume the junction, vessel breaking structure: Assume the junction, vessel breaking structure: –Vessel breakings: Sit between any two detected vessel segments –Vessel junctions: intersection, crossing or overlapping of different vessel branches Using the direction information from detected vessel segments to build up matched filter and convolving it over the junction and vessel breaking areas to detect then extract junctions, breakings Using the direction information from detected vessel segments to build up matched filter and convolving it over the junction and vessel breaking areas to detect then extract junctions, breakings

19. System Architecture Pre-Processing Extract Edge Traces Vessel Map Extraction Original CPP detection and GCS partition Virtual CPP creation via two sides parallel scanning, GCS further partition Associated parallel GCS pair grouping and Vessel segment extraction Junction & breaking detection with guided matched filters Gaussian Blurring Noise removal

20. System Architecture Extract edge traces from retina image: Extract edge traces from retina image: –Smooth image by Gaussian blurring –Apply the edge tracker to extract edge traces –Remove short and non-linear noise traces Vessel map extraction: Vessel map extraction: –Original CPP detection and GCS partitioning –Virtual CPP creation through two-side-parallel-scanning and GCS further partitioning –Associated parallel GCS grouping and vessel segment extraction –Vessel junction and breaking fixing with limited, guided Matched Filters

21. System evaluation General performance: General performance: –Automation: No user provided start or ending point needed for our system No user provided start or ending point needed for our system –Fast: Very efficient system: It only takes 2 seconds (average time) for step 1 and 3 seconds for step 2 (average time) It only takes 2 seconds (average time) for step 1 and 3 seconds for step 2 (average time) –Accuracy: Avoid human created noise VS from global MF enhancement Avoid human created noise VS from global MF enhancement –Continues vessel map structure: Junctions and breakings were correctly detected or fixed Junctions and breakings were correctly detected or fixed

22. Result comparison with A.Hoover’s system Two standard sets of manual drawing retina vessel map from two experts Two standard sets of manual drawing retina vessel map from two experts –A.Hoover (normal one) – –V. Kouznetsova (rich vessel map) By compare with the rich manual drawing vessel map, our system obtained high positive rate while the negative rate remain lower than AH system By compare with the rich manual drawing vessel map, our system obtained high positive rate while the negative rate remain lower than AH system Our system proved to be good at detecting even low intensity vessel map Our system proved to be good at detecting even low intensity vessel map

23. Image 0163- negative rate (green) A.Hoover’s standard vessel map V.K’s standard vessel map Our System A.Hoover’s System Matched Filter Enhance Image Original Retina Image

24. Image 0163 Positive rate (brown) Original Retina Image Matched Filter Enhance Image A.Hoover’s standard vessel map V.K’s standard vessel map Our System A.Hoover’s System

25. Negative Rates

26. Positive Rate

27. Summary Perceptual Curve Partitioning method provides an robust way in handling vessel map extraction Perceptual Curve Partitioning method provides an robust way in handling vessel map extraction –The proposed system has achieved the following targets: Automation Automation Efficiency Efficiency High Accuracy even for low intensity retina vessel map High Accuracy even for low intensity retina vessel map Limitation: Limitation: –For some abnormal retina images, like some strong bright patches in the background, this system will receive some false detected vessel segments. Future works: Future works: –Further verification method could be applied to minimize the negative detection rate –Investigate how to combine more domain heuristics of retina images into the perceptual edge tracking mechanism for improving our implementation

28. Acknowledgement The authors gratefully acknowledge that this research received funding support from both NSERC and Deep Vision Inc. Deep Vision Inc. also provided the authors with their edge tracker software which was used for producing original edge trace data.

29. References: [1] Ferris FL, "How effective are treatments for diabetic retinopathy?", JAMA 269, 1993, pp.1290-1291. [2] L. Pedersen, M. Grunkin, B. Ersbøll, K. Madsen, M. Larsen, N. Christoffersen, U. Skands, “Quantitative measurement of changes in retinal vessel diameter in ocular fundus images”, Patt. Recog. Lett., 21, 1215-1223, 2000. [3] Khoobehi, B., Peyman, G.A., Vo, K.D., “Relationship between Blood Velocity and Retinal Vessel Diameter”, ARVO Abstract, Invest. Opthalmol. Vis. Sci., 33, 4, 804, 1992. [4] M. Lalonde, L. Gagnon, M.-C. Boucher, “Non-recursive paired tracking for vessel extraction from retinal images”, Proceedings of the Conference Vision Interface 2000, 61-68, 2000. [5] Luo Gang, Opas Chutatape*, and Shankar M. Krishnan,"Detection and Measurement of Retinal Vessels in Fundus Images Using Amplitude Modified Second-Order Gaussian Filter,"IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 49, NO. 2, FEBRUARY 2002. [6]QI-GANG GAO and A.K.C. WONG, Curve Detection Based On Perceptual Organization, Pattern Recognition, Vol. 26, No. 7, pp.1039-1046, 1993. [7] A. Hoover, V. Kouznetsova, and M. Goldbaum, “Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response,” IEEE Trans. Med. Imag., Vol. 19, No. 3, pp. 203–210, 2000. [8] F. Zana and J.-C. Klein. A multimodal registration algorithm of eye fundus images using vessels detection and Hough transform. IEEE Trans. Medical Imaging, 18(5):419-428, 1999. [9] S. Chaudhuri, S. Chatterjee, N. Katz, and M. Goldbaum, “Detection of blood vessels in retinal images using two-dimensional matched filters,” IEEE Trans. Med. Imag., Vol. 3, pp. 263–269, Sept. 1989. [10] O. Chutatape, L. Zheng, and S. M. Krishnan, “Retinal blood vessel detection and tracking by matched Gaussian and Kalman filters,” in Proc.20th Annu Conf. IEEE Engineering in Medicine and Biology Society, 1998, pp. 3144–3149. [11] Can, H. Shen, J. Turner, H. Tanenbaum, and B. Roysam, “Rapid automated tracing and feature extraction from retinal fundus images using direct exploratory algorithms,” IEEE Trans. Inform. Technol. Biomed.,Vol. 3, pp. 125–138, June 1999. [12] V. Rakotomalala, L. Macaire, J-G. Postaire, and M. Valette. Identification of retinal vessels by color image analysis. Machine Graphics and Vision, 7:725-742, 1998. [13] L. Gagnon, M. Lalonde, M. Beaulieu, M.-C. Boucher,Procedure to Detect Anatomical Structures in Optical Fundus Images,Proc SPIE Vol 4322 Med Imaging:Img Processing 2001 1218-25. [14]D. H. Ballard, “Generalizing the Hough Transform To Detect Arbitrary Shapes”, Pattern Recognition 13, p111-122, 1981.