Quantitative analysis of radiologic images: Image segmentation and registration, statistical atlases Christos Davatzikos, Ph.D. Professor of Radiology Section of Biomedical Image Analysis

You can control a quantity if you can measure or weigh it Lord Kelvin, Need to develop tools that obtain accurate and precise measurement from image data

Expert 1: Total Lesion volume: 15,635 mm^3 Expert 2: Total Lesion volume: 7,560 mm^3 Human limitations in measuring: inter-rater differences

Major limitation for: 1)Diagnosis of disease stage 2)Monitoring the effect of treatments

Quantification/measurement: - ~3% longitudinal atrophy of the hippocampus in early AD patients - Contraction pattern of the cardiac muscle - a 5% change in radiologic signal could be indicative of evolving pathology More human limitations

Visually detecting morphological abnormalities Scan 1Scan 2

Visually detecting morphological abnormalities Scan 1Scan 2 30% atrophy!

Manual Drawing of anatomical structures Visual evaluation of a 3% atrophy is practically impossible  Laborious and not well-reproducible manual outlining is required

Evaluating complex spatio-temporal patterns of radiologic signal change, especially if the magnitude of the signal change is small and anatomical variability is large Kahneman and Tversky in their Nobel prize winninng careers studied human reasoning under uncertainty and demonstrated the limitations of human reasoning in evaluating conjunctions, i.e. A and B and C … Even more fundamental limitations of human evaluation

Detecting spatially complex very subtle anatomical abnormalities Normal Schizophrenia patient ?

HealthyMildly Cognitively impaired: Prodromal stage to Alzheimer’s Detecting spatially complex very subtle anatomical abnormalities ?

Functional activity during truth telling and lying Lies Truths

Brain and criminal behavior

Computers can complement and assist humans in many ways

Statistical anatomical atlases: from single- individual anatomical examples, to atlases capturing variability in a population Analogous to training of a human reader

Disease identification (learn variation of normal anatomy  identify abnormality as a deviation from normal variation) Integration of data from multiple individuals in order to discover systematic relationships among radiologic and clinical measurements -Does a lesion in a particular part of the brain correlate with a certain neurological deficit? -Does prostate cancer appear uniformly throughout the prostate or does it tend to appear in certain regions more frequently  what is the optimal way of biopsying/treating a patient in order to maximize probability of cancer detection/elimination?) -What is the normal variation of hippocampal size for a given age? - What is the normal variation of cardiac shape and deformation?

Image Registration: Integration and Comparative Analysis of Images from different individuals / modalities / times /conditions Before Spatial Normalization After Spatial Normalization --Image integration and co-registration helps generalize from the individual to the group, and to construct normative data  abnormalities can be distinguished from normal statistical variation Underlying biological process that results in abnormal signal, or simply normal tissue whose normal variability, in terms of image properties, needs to be measured Overlay/Comparison of such images?

Registration and Measurement of Biological Shape  D’Arcy Thompson, 1917:

The deformation function measures the local deformation of the template: Deformation 1 Deformation 2 Local structural measurements can be measured by analyzing the deformation functions with standard statistical methodologies Template Shape 1 Shape 2 Red: Contraction Green: Expansion ≈ High-Dimensional Shape Transformations Template MR imageWarped template

Significant 4-year GM changes in 107 older adults From the cover page of the Lancet, Neurology

RIGHT LEFT Voxel-based analysis of tissue density maps Effect Size Maps NC > FTD NC > AD FTD > AD

Tissue atrophy map of an AD patient, relative to cognitively normal controls Template Space Patient’s scan

Regions of differences between schizophrenics and normal controls Average of 148 brain images, after deformable registration to the atlas

Atlas with optimal needle positions Apex Base Left Right Apex Base Left Right Targeted Prostate Biopsy Using Mathematical Optimization 100 Samples Template … Segmented 3D Prostate Warped Prostate Atlas US prostate image MRI prostate image Deformable Segmentation of Prostate Images

20 subjects, average age subjects, average age Quantitative analysis meets visual image interpretation mm mm 3 “Younger Old Adult” Average Model “Older Old Adult” Average Model Average age 64.7 Average age 83

Using a statistical atlas to guide WM lesion segmentation Spatial distribution of WM abnormalities in 50 older adults (BLSA)

HAMMER: Hierarchical Attribute Matching Mechanism for Elastic Registration

Pattern Matching: Finding Anatomical Correspondences Attribute vector based on wavelet analysis of the anatomical context around each voxel  morphological signature of each voxel

Template A brain MRI before warping and after warping

Model Measuring volumes of anatomical structures : An atlas with anatomical definitions is registered to the patient’s images Subject HAMMER

To summarize: Anatomical definitions are used to create an atlas  analogous to the knowledge of anatomy by humans Pattern matching performed hierachically at various scales is used to match the atlas to the individual

Can we use these quantitative image analysis tools as diagnostic tools? -Combine all morphological, physiological, and clinical measurements into a broader phenotypic profile -Use high-dimensional pattern classification and machine learning techniques Problem: Potentially high statistical overlap for any single anatomical structure, if disease is not focal

Where is the problem? Data from Baltimore Longitudinal Study of Aging, Davatzikos et.al. Neurobiology of aging, in press

Pattern Classification Abnormality score A pattern is sampled by measuring brain volumes and blood flow in a number of brain regions Local tissue volumes and PET O 15 are combined brain regions (clusters) build a multi- parametric imaging profile

Abnormality Score Measurement and Integration of Structural and Functional Patterns

PET-post cingulate Individual Diagnosis High-dimensional Pattern Classification (Machine learning) Evaluate spatial patterns of GM, WM, CSF, PET signal distribution Use these pattern to construct an image-based classifier, using support vector machines L-ERC w Anterior L-hipp

Brain regions that collectively contributed to classification All GM Effect size WM Effect size PET Effect size Images in radiology convention

Classification Rate vs. Number of Regions

Change of abnormality scores over time * Clinically normal, has now gone through autopsy with Braak 4 and moderate plaques  meets AD pathology criteria * After removing this one participant

Normals: -0.3 MCI at latest scan: 0.26 MCI at year of conversion: 0.15 Already significant structural abnormality on year of conversion to MCI Abnormality scores when converting from normal to MCI

Data from ADNI AD vs CN classifier applied to MCI: most MCI’s have AD-like MRI profiles MMSE decline

fMRI for Lie Detection: A Card Concealment Experiment Experiments performed by the Brain and Behavior Laboratory (Psychiatry) Particiapnts were asked to lie about the possession of a card of their choice 22 participants, both true/lie responses Parameter images were created using the GLM with double gamma HRF

Most discriminative brain region: 63.1%

Region1 / Structure 1 Region 2/ Structure 2 Focal effects Non-focal effects H P H P The power of true multi-variate analysis vs. mass-univariate

Pattern classification results Individual images Average images

Lie Truth Set of regions with predictive power Statistical maps of group differences

Multi-variate analysis continued…….. …..combining different types of images Image1 Image2 No single image says it all!

Computer result by combining 4 different MR acquisition protocols

Conclusion Computers can complement humans in: Quantification Increased reproducibility Analysis of non-focal disease Evaluating complex spatio-temporal patterns -patterns of longitudinal change of structure and function - patterns of tissue motion and deformation In the heart of computational image analysis is the notion of statistical atlases, which represent normal variation and help identify disease as a deviation from this normal range