1. MedIX – Summer 06 Lucia Dettori (room 745) ldettori@cti.depaul.edu

2. Projects Texture classification  What has been done  Things I would like to explore next  Connection to other projects Evaluations of segmentation algorithms

3. Done so far … Given a pre-segmented organ region, can you tell me what it is: kidney, heart etc? It depends … on its texture Identify image features that give texture information Find rules that distinguish the texture features of one organ from another

4. Texture Classification Process at a glance Apply filter To the image Organ/Tissue Segmented Image Texture Descriptors Classifier (Decision Tree) Classification rules for tissue/organs in CT images

5. Step1 – Segmentation and cropping OrgansBackboneHeartLiverKidneySpleen Segmented14050565539 Cropped363446506411364 The image might need to be cropped, when using filters that are sensitive to areas of high contrast (background) Active Contour Mapping (Snakes) – a boundary based segmentation algorithm

6. Step2 – Filtering the image Apply a filter to the image Organ/Tissue Segmented Image For example: * Co-occurrence matrices *Run-length matrices Wavelets Ridgelets Curvelets AveragesHorizontal Activity Vertical Activity Diagonal Activity Wavelet transform

7. Haar Wavelet 84 9735 6 1 21 Original image Wavelet coefficients 621 AveragesDetails AD AD A D A D AAAD DADD

8. Step3 – Texture features extraction Apply a filter to an image Organ/Tissue Segmented Image Texture Descriptors For example: Mean, standard deviation, energy, entropy etc.. Array of texture descriptors [T1, T2, T3, …, Tn]

9. Step4 - Classification Apply a filter to an image Organ/Tissue Segmented Image Texture Descriptors Classification rules for tissue/organs in CT images The process of identifying a region as part of a class (organ) based on its texture properties. Decision tree Predicts the organ from the values of the texture descriptors Training / Testing Classification performance measures

10. Step5 – Evaluating the classifier Actual Category BackboneHeartLiverKidneySpleenTotal Predicted Backbone 182 6 1 6 0 195 CategoryHeart 3 18 4 0 0 25 Liver 0 3 30 1 7 41 Kidney 10 4 0 49 1 64 Spleen 0 0 4 0 8 12 Total 195 31 39 56 16 337 Misclassification matrix Performance Measures

11. OrganDescriptorSensitivitySpecificityPrecisionAccuracy Backbone Wavelet82.696.182.693.7 Ridgelet91.599.396.898.0 Curvelet99.498.895.398.9 Heart Wavelet59.092.167.085.0 Ridgelet82.597.588.594.6 Curvelet89.799.095.597.1 Kidney Wavelet77.791.469.988.6 Ridgelet95.493.382.093.8 Curvelet96.098.193.597.6 Liver Wavelet87.394.482.692.8 Ridgelet86.995.984.494.0 Curvelet95.998.594.398.0 Spleen Wavelet65.594.369.789.5 Ridgelet76.997.688.093.8 Curvelet91.898.994.997.6 Average Wavelet74.493.774.489.9 Ridgelet86.696.788.094.8 Curvelet94.698.794.797.9

12. Apply a filter to an image Organ/Tissue Segmented Image Texture Descriptors Classification rules for tissue/organs in CT images Decision tree Things I would like to explore Gabor filters Fractal Dimensions Performance measures Different patients Different organs Abnormal texture Different modalities

13. Connections to other project Can we use wavelet, ridgelet, curvelet- based texture descriptors for content based image retrieval? Can we use these descriptors in the volumetric segmentation? Instead of many 2D images, can we use the same process for 3D stack of slices?

14. Projects Texture classification Evaluations of segmentation algorithms  What has been done  Things I would like to explore next  Connection to other projects

15. Texture segmentation Given an image, can you tell me how many organs you have? That was easy enough. Can you tell which organs they are?  Identifying regions with similar texture  Identifying which texture it is to label the organ

16. A couple of key questions Can you do it better by varying a parameter? How do you choose the values of your segmentation parameters? If it looks better is it really better?

17. A couple of key questions Parameter optimization Performance evaluation

18. 1 3 4 2 0.87 0.56 0.50 0.75

19. Increasing value of a segmentation parameter Ground Truth Regions key Machine Segmentations How do I decide what the optimal value of the parameter is? How good a segmentation is it?

20. The “goodness” metric A single value that assigns a rating to a particular segmentation based on how well the machine segmented regions “match” the regions in the ground truth images

21. Region Categories Ground Truth vs. Machine Segmented Correctly Detected Over Segmented Under Segmented Missed Noise GT MS

22. CORRECTLY DETECTED OVER SEGMENTED UNDER SEGMENTED A Missed region is a GT region that does not participate in any instance of CD, OS, or US A Noise region is an MS region that does not participate in any instance of CD, OS, or US Index for each region

23. The “Goodness” Metric good = Correct Detection Index bad = 1-Correct Detection Index goodness = good-bad*weight 1.0 Ceiling = CDind Floor = 2*CDind-1 Weight Range = CDind-1

24. How can we use the metric? Create a set of ground truth mosaic using radiologist-labels images of pure patches of organ tissues Apply segmentation algorithm Optimize the segmentation parameters using the metric Apply optimized algorithm to the “real” image

25. Ground Truth Region key T=1000; GM= -.94 T=4000; GM=.74 T=3000; GM=.73 T=2000; GM= -.02 T=5000; GM=.75T=6000; GM=.08

26. Done so far Used the metric on a block-wise walevet- based segmentation algorithm on some sample mosaic

27. To be done Fully test the metric on a wide range of segmentation algorithms Decouple the various components of the metric and test the individual performance measures instead of the overall score Extend the metric to measure one region vs background segmentation

28. To be done Improve the wavelet-based algorithms we have implement to include other texture features Explore and compare other texture-based segmentation algorithm Use regions and metric to calculate changes in time of an abnormal region

29. Connections to other projects Use one of these algorithms to create a rough segmentation that will generate the starting point for a more sophisticated segmentation algorithm.

30. Some references ”Wavelet-based Texture Classification of Tissues in Computed Tomography”, L. Semler, L, Dettori, and Jacob Furst. 18th IEEE International Symposium on Computer-based Medical Systems, Dublin, Ireland, June 2005. “Ridgelet-based Texture Classification in Computed Tomography”, L. Semler, L. Dettori. and W.Kerr. 8th IASTED International Conference on Signal and Image Processing, Honolulu, HW, August 2006. “Curvelet-based Texture Classification of Tissues in Computed Tomography”, L. Semler, & L. Dettori. International Conference on Image Processing, Atlanta, GA, October 2006. “A Comparison of Wavelet-based and Ridgelet-based texture classification of Tissues in Computed Tomography”, with Lindsay Semler, International Conference on Computer Vision Theory and Applications, Setubal, Portugal, February 2006 “A Methodology and Metric for Quantitative Analysis and Parameter Optimization of Unsupervised, Multi-Region Image Segmentation”, William Kerr, Lucia Dettori, and Lindsay Semler, 8th IASTED International Conference on Signal and Image Processing, Honolulu, HW, August 2006.