Visual Computing Research @ CTI, DePaul University Daniela Raicu Assistant Professor draicu@cs.depaul.edu http://facweb.cs.depaul.edu/research/vc

Visual Computing Group CTI Faculty: Gian Mario Besana Lucia Dettori Jacob Furst Gerald Gordon Steve Jost Yakov Keselman Daniela Raicu Collaborators: Department of Radiology, Northwestern University & Northwestern Memorial Hospital, Chicago, IL Dr. David Channin, Chief of Informatics, Department of Radiology Medical Image Processing 11/29/2018

Visual Computing Group Graduate Students: John Campion, Ramzy Darwish William Horsthemke, Gabriel Sanchez, Winnie Tsang Undergraduate Students: Stelian Aioanei, Andrew Corboy Jong Lee, Mikhail Kalinin Lindsay Semler, Dong-Hui Xu Visual Computing (VC) area: CSC381/CSC481: Introduction to Image Processing CSC382/CSC482: Image Analysis and its Applications CSC384/CSC484: Introduction to Computer Vision VC research seminar: Fall Quarter, Friday, 5:00 - 6:00pm VC workshop: Spring Quarter, Friday, April 15th , 2005 Intelligent Multimedia Processing (IMP) lab: http://facweb.cs.depaul.edu/research/vc Medical Image Processing 11/29/2018

Medical Image Processing Research problems Content-based Image Retrieval: Image retrieval systems that permit image searching based on features automatically extracted from the images’ own visual content are called content-based image retrieval (CBIR) systems. Domain-specific features: - fingerprints, human faces visual features (primitive or low-level image features) General features: - color, texture, shape Drawback:-lack of expressive power Medical Image Processing 11/29/2018

Content-based Image Retrieval Image Database Feature Extraction Semantic Gap ? Mountains and water-falls It is a nice sunset. Meaning: Sunset Text Database Medical Image Processing 11/29/2018

Content-based Image Retrieval Feature Representation: Two examples of original images and their representations. Medical Image Processing 11/29/2018

Content-based Image Retrieval Two examples of original images and their representations: Medical Image Processing 11/29/2018

Content-based Image Retrieval Similarity Measure: S(q1,t1) Image T: Image Q: , bi = masking bit Medical Image Processing 11/29/2018

Content-based Image Retrieval Query Retrieval Results Medical Image Processing 11/29/2018

Content-based Image Retrieval Image Search Medical Image Processing 11/29/2018

Content-based Image Retrieval Medical Image Processing 11/29/2018

Medical Image Processing Medical Imaging Problem statement: Human body organs’ classifications using raw data (pixels) from abdominal and chest CT images. labels for the organs present in the image backbone heart Medical Image Processing 11/29/2018

Medical Image Processing Medical Imaging Segmentation Organ/Tissue segmentation in CT images - Data: 340 DICOM images Segmented organs: liver (56), kidneys (55), spleen (39), backbone (140), & heart (50) Segmentation algorithm: Active Contour Mappings (Snakes) A boundary-based segmentation algorithm Input for the algorithm: a number of initial points & five main parameters that influence the way the boundary is formed. Medical Image Processing 11/29/2018

Segmentation: Matlab Demo Advantage: it detects complex shapes Disadvantage: it needs manual selection of the initial points and of the parameters Our Solution: perform clustering of similar regions using a neural network Medical Image Processing 11/29/2018

Segmentation: Examples Medical Image Processing 11/29/2018

Segmentation: Examples Medical Image Processing 11/29/2018

Texture Analysis & Classification Organ/Tissue segmentation in CT images IF HGRE <= 0.38 AND ENTROPY > 0.43 AND SRHGE <= 0.20 AND CONTRAST > 0.029 THEN Prediction = Heart Probability = 0.99 Classification rules for tissue/organs in CT images Calculate numerical texture descriptors for each region [D1, D2,…D21] Medical Image Processing 11/29/2018

Inverse Difference Moment Medical Imaging Texture Analysis Entropy Energy Contrast Homogeneity SumMean Variance Correlation Maximum Probability Inverse Difference Moment Cluster Tendency 3.892828 .034692 2.764427 .6345745 11.662886 7.308909 .110921 .112929 .44697 26.471211 Medical Image Processing 11/29/2018

Inverse Difference Moment Medical Imaging Texture Analysis Entropy Energy Contrast Homogeneity SumMean Variance Correlation Maximum Probability Inverse Difference Moment Cluster Tendency 3.4151415 .108713 6.224426 .631435 13.628323 9.340897 .0723125 .3081855 .280289 31.139159 Medical Image Processing 11/29/2018

Inverse Difference Moment Medical Imaging Texture Analysis Entropy Energy Contrast Homogeneity SumMean Variance Correlation Maximum Probability Inverse Difference Moment Cluster Tendency 3.38482 .055998 3.49784 .5577785 14.278469 3.737737 .1436305 .1250245 .437988 11.453111 Medical Image Processing 11/29/2018

Inverse Difference Moment Medical Imaging Texture Analysis Entropy Energy Contrast Homogeneity SumMean Variance Correlation Maximum Probability Inverse Difference Moment Cluster Tendency 3.3099875 .049172 3.066407 .5369255 12.309719 1.634463 .0377875 .0897425 .460422 3.471442 Medical Image Processing 11/29/2018

Medical Imaging Texture Analysis Entropy Energy Contrast Homogeneity SumMean Variance Correlation Maximum Probability Inverse Difference Moment Cluster Tendency 2.72509 .091388 1.618982 .6208175 11.755226 0.912752 .123976 .1742075 .506894 2.032082 Medical Image Processing 11/29/2018

Texture Descriptors: Matlab Demo Medical Image Processing 11/29/2018

Organ/Tissue Classification IF HGRE <= 0.38 AND ENTROPY > 0.43 AND SRHGE <= 0.20 AND CONTRAST > 0.029 THEN Prediction = Heart Probability = 0.99 Classification rules for tissue/organs in CT images Calculate numerical texture descriptors for each region [D1, D2,…D21] Algorithm: - decision trees Output: Decision Rules Performance estimated using: - sensitivity - specificity Advantage: Set of decision rules that can be used for annotation Medical Image Processing 11/29/2018

Organ/Tissue Classification Examples of Decision Tree Rules for Combined Data: IF (HGRE <= 0.3788) & (CLUSTER <= 0.0383095) & (INVERSE <= 0.768085) & (SUMMEAN <= 0.556015) & (SRLGE <= 0.101655) & (ENEGRY > 0.106715) THEN Prediction = Spleen, Probability = 0.928571 IF (HGRE <= 0.3788) & (CLUSTER <= 0.0383095) & (INVERSE <= 0.768085) & (SUMMEAN <= 0.556015) & (SRLGE > 0.101655) THEN Prediction = Liver , Probability = 1.000000 IF (HGRE <= 0.3788) & (CLUSTER <= 0.0383095) & (INVERSE <= 0.768085) & (SUMMEAN > 0.556015) & (GLNU <= 0.087365) THEN Prediction = Kidney, Probability = 0.924658 Medical Image Processing 11/29/2018

Organ/Tissue Classification Examples of Decision Tree Rules for Combined Data: IF (HGRE <= 0.3788) & (CLUSTER > 0.0383095) & (GLNU > 0.03184) & (ENTROPY > 0.433185) & (SRHGE <= 0.19935) & (CONTRAST > 0.0295805) THEN Prediction = Heart, Probability = 0.988372 IF (HGRE <= 0.3788) & (CLUSTER > 0.0383095) & (GLNU <= 0.03184) & (LRE <= 0.123405) THEN Prediction = Backbone, Probability = 1.000000 Medical Image Processing 11/29/2018

Organ/Tissue Classification Decision Tree Accuracy on Testing Data (Co-occurrence, Run-length, and Combined): ORGAN Sensitivity Specificity Precision Accuracy Backbone 96% / 98% / 98% 99% / 100% / 99% 99% / 99% / 99% 98% / 99% / 99% Liver 64% / 57% / 78% 96% / 98% / 95% 75% / 84% / 71% 92% / 92% / 92% Heart 79% / 82% / 75% 96% / 95% / 98% 80% / 77% / 90% 94% / 93% / 95% Kidney 89% / 89% / 89% 96% / 93% / 96% 80% / 67% / 77% 94% / 92% / 95% Spleen 60% / 44% / 60% 93% / 93% / 95% 53% / 45% / 63% 89% / 87% / 91% Medical Image Processing 11/29/2018

Tissue Classification: Matlab Demo Medical Image Processing 11/29/2018

Medical Image Processing Publications (CBIR) [1] Daniela Stan and Ishwar K. Sethi, “Image Retrieval using a Hierarchy of Clusters” in Lecture Notes in Computer Science: Advances in Pattern Recognition – ICAPR 2001, Springer-Verlag Ltd. (Ed), pp. 377-388, 2001. [2] Daniela Stan and Ishwar K. Sethi, “Mapping Low-level Image Features to Semantic Concepts” in Proceedings of SPIE: Storage and Retrieval for Media databases, pp. 172-179, 2001. [3] Ishwar K. Sethi, Ioana Coman, Daniela Stan, “Mining Association Rules between Low-level Image Features and High-level Concepts” in Proceedings of SPIE: Data Mining and Knowledge Discovery III, pp.279-290, 2001. [4] Daniela Stan and Ishwar K. Sethi, “Color Patterns for Pictorial Content Description”, ACM Symposium on Applied Computing, 2002. [5] Daniela Stan and Ishwar K. Sethi, “eID: A System for Exploration of Image Databases”, Information Processing and Management Journal,2002. [6] Daniela Stan and Ishwar K. Sethi, “Synobins: An intermediate level towards Annotation and Semantic Retrieval”, IEEE Trans. Multimedia Journal. Medical Image Processing 11/29/2018

Medical Image Processing Publications (MI) [1] D. Xu, J. Lee, D.S. Raicu, J.D. Furst, D. Channin. "Texture Classification of Normal Tissues in Computed Tomography", The 2005 Annual Meeting of the Society for Computer Applications in Radiology, June 2-5, 2005. (Submitted) [2] D.S. Raicu, W. Tsang, M. Kalinin, D. Xu, J.D. Furst, D. Channin. "Automatic Tissue Context Determination in Computed Tomography", SPIE Medical Imaging, February 12–17, 2005. (Submitted) [3] D. H. Xu, A. Kurani, J. D. Furst, & D. S. Raicu, "Run-length encoding for volumetric texture", The 4th IASTED International Conference on Visualization, Imaging, and Image Processing - VIIP 2004,  Marbella, Spain, September 6-8, 2004. [4] D. Channin, D. S. Raicu, J. D. Furst, D. H. Xu, L. Lilly, C. Limpsangsri, "Classification of Tissues in Computed Tomography using Decision Trees", Poster and Demo, The 90th Scientific Assembly and Annual Meeting of Radiology Society of North America (RSNA04), November 28, 2004. [5] A. Kurani, D. H. Xu, J. D. Furst, & D. S. Raicu, "Co-occurrence matrices for volumetric data", The 7th IASTED International Conference on Computer Graphics and Imaging – CGIM, August 16-18, 2004 . [6] D. S. Raicu, J. D. Furst, D. Channin, D. H. Xu, & A. Kurani, "A Texture Dictionary for Human Organs Tissues' Classification", Proceedings of the 8th World Multiconference on Systemics, Cybernetics and Informatics (SCI 2004), July 18-21, 2004. Medical Image Processing 11/29/2018

Intelligent Multimedia Processing Laboratory Daniela Raicu Intelligent Multimedia Processing Laboratory School of CTI DePaul University THE END! Medical Image Processing 11/29/2018