1. Quantitative analysis of radiologic images: Image segmentation and registration, statistical atlases Christos Davatzikos, Ph.D. Professor of Radiology Section of Biomedical Image Analysis http://www.rad.upenn.edu/sbia

2. You can control a quantity if you can measure or weigh it Lord Kelvin, 1824-1907 Need to develop tools that obtain accurate and precise measurement from image data

3. Expert 1: Total Lesion volume: 15,635 mm^3 Expert 2: Total Lesion volume: 7,560 mm^3 Human limitations in measuring: inter-rater differences

4. Major limitation for: 1)Diagnosis of disease stage 2)Monitoring the effect of treatments

5. Quantification/measurement: - ~3% longitudinal atrophy of the hippocampus in early AD patients - Contraction pattern of the cardiac muscle - a 5% change in radiologic signal could be indicative of evolving pathology More human limitations

6. Visually detecting morphological abnormalities Scan 1Scan 2

7. Visually detecting morphological abnormalities Scan 1Scan 2 30% atrophy!

8. Manual Drawing of anatomical structures Visual evaluation of a 3% atrophy is practically impossible  Laborious and not well-reproducible manual outlining is required

9. Evaluating complex spatio-temporal patterns of radiologic signal change, especially if the magnitude of the signal change is small and anatomical variability is large Kahneman and Tversky in their Nobel prize winninng careers studied human reasoning under uncertainty and demonstrated the limitations of human reasoning in evaluating conjunctions, i.e. A and B and C … Even more fundamental limitations of human evaluation

10. Detecting spatially complex very subtle anatomical abnormalities Normal Schizophrenia patient ?

11. HealthyMildly Cognitively impaired: Prodromal stage to Alzheimer’s Detecting spatially complex very subtle anatomical abnormalities ?

12. Functional activity during truth telling and lying Lies Truths

13. Brain and criminal behavior

14. Computers can complement and assist humans in many ways

15. Statistical anatomical atlases: from single- individual anatomical examples, to atlases capturing variability in a population Analogous to training of a human reader

16. Disease identification (learn variation of normal anatomy  identify abnormality as a deviation from normal variation) Integration of data from multiple individuals in order to discover systematic relationships among radiologic and clinical measurements -Does a lesion in a particular part of the brain correlate with a certain neurological deficit? -Does prostate cancer appear uniformly throughout the prostate or does it tend to appear in certain regions more frequently  what is the optimal way of biopsying/treating a patient in order to maximize probability of cancer detection/elimination?) -What is the normal variation of hippocampal size for a given age? - What is the normal variation of cardiac shape and deformation?

17. Image Registration: Integration and Comparative Analysis of Images from different individuals / modalities / times /conditions Before Spatial Normalization After Spatial Normalization --Image integration and co-registration helps generalize from the individual to the group, and to construct normative data  abnormalities can be distinguished from normal statistical variation Underlying biological process that results in abnormal signal, or simply normal tissue whose normal variability, in terms of image properties, needs to be measured Overlay/Comparison of such images?

18. Registration and Measurement of Biological Shape  D’Arcy Thompson, 1917:

19. The deformation function measures the local deformation of the template: Deformation 1 Deformation 2 Local structural measurements can be measured by analyzing the deformation functions with standard statistical methodologies Template Shape 1 Shape 2 Red: Contraction Green: Expansion ≈ High-Dimensional Shape Transformations Template MR imageWarped template

21. Significant 4-year GM changes in 107 older adults From the cover page of the Lancet, Neurology

22. RIGHT LEFT Voxel-based analysis of tissue density maps Effect Size Maps NC > FTD NC > AD FTD > AD

23. Tissue atrophy map of an AD patient, relative to cognitively normal controls Template Space Patient’s scan

24. Regions of differences between schizophrenics and normal controls Average of 148 brain images, after deformable registration to the atlas

25. Atlas with optimal needle positions Apex Base Left Right 6 7 4 3 1 2 5 Apex Base Left Right Targeted Prostate Biopsy Using Mathematical Optimization 100 Samples Template … Segmented 3D Prostate Warped Prostate Atlas US prostate image MRI prostate image Deformable Segmentation of Prostate Images

26. 20 subjects, average age 64.7020 subjects, average age 83.05 Quantitative analysis meets visual image interpretation 40174 mm 3 20564 mm 3 “Younger Old Adult” Average Model “Older Old Adult” Average Model Average age 64.7 Average age 83

27. Using a statistical atlas to guide WM lesion segmentation Spatial distribution of WM abnormalities in 50 older adults (BLSA)

28. HAMMER: Hierarchical Attribute Matching Mechanism for Elastic Registration

29. Pattern Matching: Finding Anatomical Correspondences Attribute vector based on wavelet analysis of the anatomical context around each voxel  morphological signature of each voxel

30. Template A brain MRI before warping and after warping

31. Model Measuring volumes of anatomical structures : An atlas with anatomical definitions is registered to the patient’s images Subject HAMMER

32. To summarize: Anatomical definitions are used to create an atlas  analogous to the knowledge of anatomy by humans Pattern matching performed hierachically at various scales is used to match the atlas to the individual

33. Can we use these quantitative image analysis tools as diagnostic tools? -Combine all morphological, physiological, and clinical measurements into a broader phenotypic profile -Use high-dimensional pattern classification and machine learning techniques Problem: Potentially high statistical overlap for any single anatomical structure, if disease is not focal

34. Where is the problem? Data from Baltimore Longitudinal Study of Aging, Davatzikos et.al. Neurobiology of aging, in press

35. Pattern Classification Abnormality score A pattern is sampled by measuring brain volumes and blood flow in a number of brain regions Local tissue volumes and PET O 15 are combined 15-20 brain regions (clusters) build a multi- parametric imaging profile

36. Abnormality Score Measurement and Integration of Structural and Functional Patterns

37. PET-post cingulate Individual Diagnosis High-dimensional Pattern Classification (Machine learning) Evaluate spatial patterns of GM, WM, CSF, PET signal distribution Use these pattern to construct an image-based classifier, using support vector machines L-ERC w Anterior L-hipp

38. Brain regions that collectively contributed to classification All GM Effect size WM Effect size PET Effect size Images in radiology convention

39. Classification Rate vs. Number of Regions

40. Change of abnormality scores over time * Clinically normal, has now gone through autopsy with Braak 4 and moderate plaques  meets AD pathology criteria * After removing this one participant

41. Normals: -0.3 MCI at latest scan: 0.26 MCI at year of conversion: 0.15 Already significant structural abnormality on year of conversion to MCI Abnormality scores when converting from normal to MCI

42. Data from ADNI AD vs CN classifier applied to MCI: most MCI’s have AD-like MRI profiles MMSE decline

43. fMRI for Lie Detection: A Card Concealment Experiment Experiments performed by the Brain and Behavior Laboratory (Psychiatry) Particiapnts were asked to lie about the possession of a card of their choice 22 participants, both true/lie responses Parameter images were created using the GLM with double gamma HRF

44. Most discriminative brain region: 63.1%

45. Region1 / Structure 1 Region 2/ Structure 2 Focal effects Non-focal effects H P H P The power of true multi-variate analysis vs. mass-univariate

46. Pattern classification results Individual images Average images

47. Lie Truth Set of regions with predictive power Statistical maps of group differences

48. Multi-variate analysis continued…….. …..combining different types of images Image1 Image2 No single image says it all!

49. Computer result by combining 4 different MR acquisition protocols

50. Conclusion Computers can complement humans in: Quantification Increased reproducibility Analysis of non-focal disease Evaluating complex spatio-temporal patterns -patterns of longitudinal change of structure and function - patterns of tissue motion and deformation In the heart of computational image analysis is the notion of statistical atlases, which represent normal variation and help identify disease as a deviation from this normal range

51. http://www.rad.upenn.edu/sbia