1. Integrative Analysis of Pathology, Radiology and High Throughput Molecular Data Joel Saltz MD, PhD Director Center for Comprehensive Informatics

2. Objectives Reproducible anatomic/functional characterization at gross level (Radiology) and fine level (Pathology) Integration of anatomic/functional characterization with multiple types of “omic” information Create categories of jointly classified data to describe pathophysiology, predict prognosis, response to treatment Data modeling standards, semantics Data research issues

3. In Silico Program Objectives (from NCI) In silico is an expression used to mean "performed on computer or via computer simulation.“ (Wikipedia)computer computer simulation In silico science centers: support investigator-initiated, hypothesis- driven research in the etiology, treatment, and prevention of cancer using in silico methods –Generating and publishing novel cancer research findings leveraging caBIG tools and infrastructure –Identifying novel bioinformatics processes and tools to exploit existing data resources Encouraging the development of additional data resources and caBIG analytic services Assessing the capabilities of current caBIG tools Emory, Columbia, Georgetown, Fred Hutchinson Cancer, Translational Genomics Research Institute

4. In Silico Center for Brain Tumor Research Specific Aims: 1.Influence of necrosis/ hypoxia on gene expression and genetic classification. 2. Molecular correlates of high resolution nuclear morphometry. 3.Gene expression profiles that predict glioma progression. 4. Molecular correlates of MRI enhancement patterns.

5. Integrative Analysis: Tumor Microenvironment Structural and functional differentiation within tumor Molecular pathways are time and space dependent “Field effects” – gradient of genetic, epigenetic changes Radiology, microscopy, high throughput genetic, genomic, epigenetic studies, flow cytometry, microCT, nanotechologies … Create biomarkers to understand disease progression, response to treatment Tumors are organs consisting of many interdependent cell types From John E. Niederhuber, M.D. Director National Cancer Institute, NIH presented at Integrating and Leveraging the Physical Sciences to Open a New Frontier in Oncology, Feb 2008

6. Informatics Requirements Parallel initiatives Pathology, Radiology, “omics” Exploit synergies between all initiatives to improve ability to forecast survival & response. Radiology Imaging Patient Outco me Pathologic Features “Omic” Data

7. In Silico Center for Brain Tumor Research Key Data Sets REMBRANDT: Gene expression and genomics data set of all glioma subtypes The Cancer Genome Atlas (TCGA): Rich “omics” set of GBM, digitized Pathology and Radiology Vasari Feature Set: Standardized annotation of gliomas of all subtypes

8. TCGA Research Network Digital Pathology Neuroimaging

9. Distinguishing Among the Gliomas “There are also many cells which appear to be transitions between gigantic oligodendroglia and astrocytes. It is impossible to classify them as belonging in either group” Bailey P, Bucy PC. Oligodendrogliomas of the brain. J Pathol Bacteriol 1929: 32:735

10. OligodendrogliomaAstrocytoma Nuclear Qualities

11. Progression to GBM Anaplastic Astrocytoma (WHO grade III) Glioblastoma (WHO grade IV)

12. TCGA Neuropathology Attributes 120 TCGA specimens; 3 Reviewers Presence and Degree of: Microvascular hyperplasia  Complex/glomeruloid  Endothelial hyperplasia Necrosis  Pseudopalisading pattern  Zonal necrosis Inflammation  Macrophages/histiocytes  Lymphocytes  Neutrophils Differentiation:  Small cell component  Gemistocytes  Oligodendroglial  Multi-nucleated/giant cells  Epithelial metaplasia  Mesenchymal metaplasia Other Features  Perineuronal/perivascular satellitosis  Entrapped gray or white matter  Micro-mineralization

13. Feature Extraction TCGA Whole Slide Images Jun Kong

14. Astrocytoma vs Oligodendroglima Overlap in genetics, gene expression, histology Astrocytoma vs Oligodendroglima Assess nuclear size (area and perimeter), shape (eccentricity, circularity major axis, minor axis, Fourier shape descriptor and extent ratio), intensity (average, maximum, minimum, standard error) and texture (entropy, energy, skewness and kurtosis).

15. Nuclear Qualities Which features carry most prognostic significance? Which features correlate with genetic alterations?

16. Whole slide scans from 14 TCGA GBMS (69 slides) 7 purely astrocytic in morphology; 7 with 2+ oligo component 399,233 nuclei analyzed for astro/oligo features Cases were categorized based on ratio of oligo/astro cells Machine-based Classification of TCGA GBMs (J Kong) TCGA Gene Expression Query: c-Met overexpression

17. Nuclear Feature Analysis: TCGA Using the parallel computation infrastructure of Sun Grid Engine, we analyzed image tiles of 4096x4096 of 213 whole- slide TCGA images of permanent tissue sections. Approximately 90 million nuclei segmented. 79 patients:57 are diagnosed as GBM (‘oligo 0’) 17 are classified as GBM with ‘oligo 1’, 5 as GBM with ‘oligo 2+’. With each data file including all nuclear features from one patient, all nuclei were classified with color blue, green, and red representing nuclei scored as 1~3, 4~6, and 7~10, respectively.

18. The ratio of nuclei classified ≤ 5 to >5 was computed for each 79 patients. Ratios associated with ‘oligo 0’ and ‘oligo 2+’ patient populations were compared with two-sample t-test (p=0.0145) Nuclear Feature Analysis: TCGA

19. Discriminating Features (Grade 1 vs. Grade 7-10)

22. Determine the influence of tumor microenvironment on gene expression profiling and genetic classification using TCGA data Necrosis = Severe Hypoxia Microvascular Hyperplasia GBM

23. 3 Gene Families are Altered in GBM: RTK, p53 and RB

24. GBM: necrosis, hypoxia, angiogenesis and gene expression Does the presence or degree of necrosis within digitized frozen section slides correlate with specific gene expression patterns or determine algorithm- based unsupervised clustering of GBMs gene expression categories? Does the presence or degree of necrosis influence the type of angiogenesis or pro-angiogenic gene expression patterns within human gliomas?

25. GBM: % Necrosis

26. TCGA: GBM Frozen Sections 179 cases were assessed for % necrosis on frozen section slides for TCGA quality assurance. Cox-based regression analysis for % necrosis vs. gene expression (795 probe sets; 647 distinct genes)

27. Network Analysis based on % Necrosis Carlos Moreno David Gutman

28. Aim 4. Identify correlates of MRI enhancement patterns in astrocytic neoplasms with underlying vascular changes and gene expression profiles. No enhancement Normal Vessels Stable lesion ? Rim-enhancement Vascular Changes Rapid progression

29. Angiogenesis Segmentation H&E Image Color Deconvolution Hematoxylin Image Eosin Image Eosin Image Spatial Norm. Density Image Density Calculation Boundary Smoothing Density Image Object ID Segmented Vessels Eosin intensity image Angiogenic Segmentation

30. States of Angiogenesis Endothelial Hypertrophy Complex Microvascular Hyperplasia Endothelial Hyperplasia Lee Cooper Sharath Cholleti

31. Vessel Characterization Bifurcation detection

32. Vasari Imaging Criteria (Adam Flanders, TJU; Dan Rubin, Stanford, Lori Dodd, NCI) Require standardized validated feature sets to describe de novo disease. Fundamental obstacle to new imaging criteria as treatment biomarkers is lack of standard terminology: –To define a comprehensive set of imaging features of cancer –For reporting imaging results –To provide a more quantitative, reproducible basis for assessing baseline disease and treatment response

33. Defining Rich Set of Qualitative and Quantitative Image Biomarkers Community-driven ontology development project; collaboration with ASNR Imaging features (5 categories) –Location of lesion –Morphology of lesion margin (definition, thickness, enhancement, diffusion) –Morphology of lesion substance (enhancement, PS characteristics, focality/multicentricity, necrosis, cysts, midline invasion, cortical involvement, T1/FLAIR ratio) –Alterations in vicinity of lesion (edema, edema crossing midline, hemorrhage, pial invasion, ependymal invasion, satellites, deep WM invasion, calvarial remodeling) –Resection features (extent of nCE tissue, CE tissue, resected components)

37. Results: Reader Agreement High inter-observer agreement among the three readers –(kappa = 0.68, p<0.001) Percentage agreement was also high for most features individually –22 of 30 features (73%) had agreement greater than 50% –Twelve features (40%) had >80% agreement –No feature had less than 20% agreement Feature agreement rose substantially when used with tolerance (+/- 1).

38. Preliminary Relationships of Features to Survival Cox proportional hazards models were fit to each of the thirty features related to overall survival. Features associated with lower survival included (p<.0001): –Proportion of enhancing tissue at baseline. –Thick or nodular enhancement characteristics. –Contralateral hemisphere invasion. Proportion of non contrast enhancing tumor (nCET) had positive correlation with survival. Tumor size at baseline had no relationship to survival.

39. Coupling silico methodology with a clinical trial: Will Treatment work and if not, why not? Example: Avastin and Glioblastoma as in RTOG-0825 (plus institutional accrual) Treatment: Radiation therapy and Avastin (anti angiogenesis) Predict and Explain: Genetic, gene expression, microRNA, Pathology, Imaging Imaging/RT reproducibility, Integration with EMR, PACS, RT systems

40. DATAINTEGRATIONDATAINTEGRATION DATAINTEGRATIONDATAINTEGRATION Information WarehouseUser AccessAcquisition Transfer CPOE OR system Patient Mgmt Dictated reports Pathology reports Daily ADT Lab Respiratory Blood Endoscopy Cardiology Siemens Img Real time Patient Billing Practice Plans Pt Satisfaction Weekly Monthly Cancer Genetics Wound Images Tissue Pulmonary Genomic Data Web Multi-Dimensional Analysis & Data Mining Ad-hoc Query Error Report Benchmarking De- Identification Honest Broker Wound Center Research Web Scorecards & Dashboards Image Analysis Text Mining, NLP Business Clinical ResearchExternal Meta Data Overview Crucial to Leverage Institutional Data Ohio State Information Warehouse Infrastructure

41. Annotations and Imaging Markup (AIM) Annotations and Imaging Markup Developer (AIM) provides a standard for medical image annotation and markup for images used in the research space, and in particular, the image based cancer clinical trial. It is notable that there is no existing standard for radiology annotation and markup. The caBIG ® program is working with almost every standards body such as DICOM to elicit consensus regarding use of AIM as the accepted standard for radiology annotation and markup, and is positioned to extend AIM to digital pathology. The pixel at the tip of the arrow [coordinates (x,y)] in this image [DICOM: 1.2.814.234543.23243] represents the Ascending Thoracic Aorta [SNOMED:A3310657] Aim Data Service Emory (AIME) is a caGrid data service that manages AIM documents in XML databases. AIME supports query, enumerationQuery, queryByTransfer, submit and submitByTransfer methods

42. MicroAIM Provide a semantically enabled data model for pathology and microscopy image markups and observations Goal: interoperable data exchange and knowledge sharing Ongoing collaborative efforts to standardize via caBIG and Association for Pathology Informatics microAIM data service provides comprehensive query support, and can be efficiently implemented either through an XML-based or a relational based approach.

43. Pathology AIM; Semantic Annotation and Spatial Reasoning Observation on a single or multiple objects Support multiple objects and associate different observations to each object Calculation on a single or multiple objects Class of type to represent mask or field value Represent provenance information for computed markup and annotation Identify all instances of a set of cell types Complex spatial queries: Quantify areas of glomeruloid microvascular proliferation within X microns of pseudopalisading necrosis

44. Use of caBIG Tools in Emory in Silico Brain Tumor Research Center Osirix XIP/AVT Clear Canvas NBIA AIME TCGA Portal Modified WorkstationExtended AIME Service Enterprise Architecture 2.0 caB2B caIntegrator 2 CBIIT Research Team Carl Schaefer Jinghui Zhang TBPT ICR VCDE/ ARCH Science CIP: Research & Clinical Trials TCGA Portal In vivo Imaging

45. Data Science Research Challenges Driven by In Silico Discovery Research Data integration that targets multiple data sources with conflicting metadata and conflicting data Efficient methods for semantic query that targets questions involving complex multi-scale features associated with petascale and exascale ensembles of highly annotated images Computer assisted annotation and markup for very large datasets Systems to support combinations of structured and irregular accesses to exascale datasets

46. Data Science Research Challenges Structural and semantic metadata management: how to manage tradeoff between flexibility and curation Data and semantic modeling infrastructures and policies able to scale to handle distributed systems with an aggregate of 10*9 or more data models/concepts Three dimensional (time dependent) reconstruction, feature detection and annotation of 3-D microscopy imagery Workflow infrastructure for large scale data intensive computations

47. Final Data Science Challenge: Large Dataset Size –Basic small mouse is 10 cm 3 –1 μ resolution – very roughly 10 13 bytes/mouse –Molecular data (spatial location) multiply by 10 2 –Vary genetic composition, environmental manipulation, systematic mechanisms for varying genetic expression; multiply by 10 3 Total: 10 18 bytes per big science animal experiment

48. Thanks to: In silico center team: Dan Brat (Science PI), Tahsin Kurc, Ashish Sharma, Tony Pan, David Gutman, Jun Kong, Sharath Cholleti, Carlos Moreno, Chad Holder, Erwin Van Meir, Daniel Rubin, Tom Mikkelsen, Adam Flanders, Joel Saltz (Director) caGrid Knowledge Center: Joel Saltz, Mike Caliguiri, Steve Langella co-Directors; Tahsin Kurc, Himanshu Rathod Emory leads caBIG In vivo imaging team: Eliot Siegel, Paul Mulhern, Adam Flanders, David Channon, Daniel Rubin, Fred Prior, Larry Tarbox and many others In vivo imaging Emory team: Tony Pan, Ashish Sharma, Joel Saltz Emory ATC Supplement team: Tim Fox, Ashish Sharma, Tony Pan, Edi Schreibmann, Paul Pantalone Digital Pathology R01: Foran and Saltz; Jun Kong, Sharath Cholleti, Fusheng Wang, Tony Pan, Tahsin Kurc, Ashish Sharma, David Gutman (Emory), Wenjin Chen, Vicky Chu, Jun Hu, Lin Yang, David J. Foran (Rutgers)

49. Thanks!