The cancer Biomedical Informatics Grid™ (caBIG™): In Vivo Imaging Workspace Projects Fred Prior, Ph.D. Mallinckrodt Institute of Radiology Washington University in St. Louis

caBIG will facilitate sharing of infrastructure, applications, and data Common, widely distributed informatics platform Shared vocabulary, data elements, data models Common standard for developing applications

In Vivo Imaging Workspace Project 1: Middleware Project 2: AIM: Annotation and Image Markup Project 3: Vocabulary Project 4: XIP: Extensible Imaging Platform Terminologies & CDEs Image Annotation Metadata storage formats Queries & Analysis Vocabularies & CDEs Data Capture Formats & Tools Data Re-Use Applications NLP Metadata for Images

Search for Images Retrieve Results

One of the goals of the In Vivo Imaging Workspace is to facilitate the increasing use of imaging based end points in clinical trials. To achieve this an easily extensible open source platform to support image analysis and visualization was defined as a key priority The eXtensible Imaging Platform (XIP) is an open source environment for rapidly developing medical imaging applications from an extensible set of modular elements. This platform will make it easier and less expensive to access specific post-processing applications at multiple sites, simplifying clinical trials, and most importantly, increasing the uniformity of imaging and analysis. Imaging applications developed by research groups will more easily be accessible within the clinical operating environment, simplifying workflows and speeding data processing and analysis. Once validated, the software should be readily transitioned into products through streamlined Federal Drug Administration, (FDA), approval processes due to the re-use of already approved libraries and open source development processes.

Deliverables XIP.rad XIP.ws Development and application build environment Extensive and extensible set of libraries for imaging and visualization Uses Open Inventor framework Includes code generating wizards to create new objects and wrap existing libraries XIP.ws A reference implementation of a medical imaging workstation developed using XIP.rad Integrated via middleware into caGRID Optimized to support basic cancer research use cases Includes two key components: XIP.app – a use case specific “plug-in” application integrated via the DICOM WG-23 Interface XIP.host – the hosting environment that provides XIP.apps access to services such as data stores, remote processing, etc.

XIP.ws - Reference Implementation XIP.app XIP.host Standard XIP Classes Custom XIP Classes API (Plug) API (Socket)

Open Inventor Open Inventor ® (http://oss.sgi.com/projects/inventor/ ) is an object-oriented toolkit offering a comprehensive solution to interactive graphics programming problems. It presents a programming model based on the Model/View/Controller design pattern and the concept of Pipelines. C++ modules represent Engines, Nodes and Manipulators Engines enable the creation of processing pipelines Nodes support the concept of scene graphs, which are hierarchical structures of objects describing what needs to be visualized in 2D/3D Manipulators handle input devices, measurements and coordinate transforms in response to user interaction

Integrating existing toolkits into Automatic Wrapper generation for 2D/3D libraries/toolkits such as ITK and VTK Example: ITK for image processing, segmentation, registration User can review parsed results and choose to support only the desired data types, hide some methods, exclude some classes, etc. Wrappers for ITK functions such as Region Growing, Neighborhood, Isolated, Confidence, Watershed, Thresholding, Edge Detection, Laplacian, Gaussian, … Support for ITK Data Meshes and Vector Fields

XIP Framework & Architecture XIP Application XIP Development Tools XIP Modules Host Independent XIP Host Adapter API (Enables rapid development of applications) ITK VTK XIP LIB . . . Data Access Remote Processing Annotation and Markup For those of you that were not able to attend Fred Prior’s talk, I’ll first desc . . . WG23 WG23 WG23 WG23 Host-Specific Plugin Libraries caGrid Data Svc. Analytical Svc. caGrid AIM MetaData . . . XIP Host (Can be replaced with any WG23-compatible Host)

caGrid Grid Infrastructure for caBIG caGrid Components Language (metadata, ontologies) Security Advertisement and Discovery Workflow Grid Service Graphical Development Toolkit (Introduce) Efficient Bulk Data Transport (IVI middleware) DICOM compatibility (IVI middleware)

gridIMAGE Architecture Expose algorithms, human markup and image data as caGrid Services Here you give an overview of the system and then describe things in detail in the following slides.

The caBIG AIM Project An ontology of image annotations An ontology defines concepts in a domain and the relationships between those concepts An ontology of image markups Use of controlled terminologies RadLex, SNOMED, LOINC, UCUM Define a set of translatable standards-based representations Implement this functionality on the caBIG eXtensible Imaging Platform (XIP)

An Image, and an Annotation an Image Markup The pixel at the tip of the arrow [coordinates (x,y)] in this image [DICOM: 1.2.814.234543.23243] represents the Ascending Thoracic Aorta [SNOMED:A3310657] and an Annotation

caDSR, EVS, RadLex, AIM ontology, etc Client access Service access HW XIP Application Inventor Application Modules WG 23 System Services PLUG WG 23 System Services SOCKET GRID CLIENT SERVICES DICOM (DCMTK) OTHER VTK ITK AIMTK other OS NCIA IDE RadLex AIM NCI Protégé EVS MIDDLEWARE DICOM Services IVI Middleware caGrid CaBIG caDSR, EVS, RadLex, AIM ontology, etc Client access Service access Grid Data Service Grid Analytical Service AIM Data Service XIP App Service Host WG23 DICOM Image Sources

The caBIG 2007 Annual Meeting February 5 - 8, 2007 Marriott Wardman Park Hotel, Washington, DC Plenary sessions; 60 break out sessions; exhibits, demonstrations, and posters; hackathon Tailored sessions for newcomers February 5 and throughout the conference