Medical Image Database Resource for Software Tools Evaluation and MIRC Laurence P Clarke, PhD Barbara Croft PhD Carl Jaffe MD Gary Becker MD Dan Sullivan MD Cancer Imaging Program

Resource for Benchmarking Application Specific Software Performance. Need to develop a consensus or standards for assessment of application specific software for cancer imaging. Need to develop a consensus or standards for assessment of application specific software for cancer imaging. Software methods and related informatics tools lag behind the front end imaging technology in terms of their performance (MRI,CT, Molecular Imaging..). Software methods and related informatics tools lag behind the front end imaging technology in terms of their performance (MRI,CT, Molecular Imaging..). Need to engage the international and internationally community in the development and assessment of advanced software tools. Need to engage the international and internationally community in the development and assessment of advanced software tools. Need to explore if this resource could be employed to reduce delay or uncertainty in FDA approval. Need to explore if this resource could be employed to reduce delay or uncertainty in FDA approval. Need to develop IT resource to establish this database and provide web access to this resource to encourage standards for software assessment. Need to develop IT resource to establish this database and provide web access to this resource to encourage standards for software assessment.

Example Projects: Evaluation of Software Tools as required Lung Cancer using X-ray CT. Lung Image Database Consortium (LIDC): Database to permit the benchmarking of CAD methods for cancer screening. Lung Image Database Consortium (LIDC): Database to permit the benchmarking of CAD methods for cancer screening. Lung Database for Drug Response. Database to permit the benchmarking of software metrics such as volumetric measures of tumor response to drug or radiation therapy. Lung Database for Drug Response. Database to permit the benchmarking of software metrics such as volumetric measures of tumor response to drug or radiation therapy.

U01: Lung Image Database Consortium (LIDC) (U01: ): NCI: ACRIN/NLST ( : $200M/9 years) is supporting a large lung cancer screening trial using helical CT, where subsets of images can be collected. NCI: ACRIN/NLST ( : $200M/9 years) is supporting a large lung cancer screening trial using helical CT, where subsets of images can be collected. LIDC ($7M/5 years: 5 Academic Sites). Goals: Develop an image database for the comparison of CAD methods, and encourage standards for assessment of application-specific software. LIDC ($7M/5 years: 5 Academic Sites). Goals: Develop an image database for the comparison of CAD methods, and encourage standards for assessment of application-specific software. LIDC Steering Committee: Includes academic members from each site, FDA CDRH scientists and consultants that are engaged in the developing broad understanding of all the issues involved in the development of a public database, as a transparent process. LIDC Steering Committee: Includes academic members from each site, FDA CDRH scientists and consultants that are engaged in the developing broad understanding of all the issues involved in the development of a public database, as a transparent process.

Foundation for NIH: Facilitate Public Private Partnerships Authorized by Congress in and established in to support the NIH in its mission through collaboration with academia, industry, individuals, and other non-profits. Authorized by Congress in and established in to support the NIH in its mission through collaboration with academia, industry, individuals, and other non-profits. An independent non-profit, not a government organization. An independent non-profit, not a government organization. Governed by independent Board of Directors, including the Director of NIH and the FDA Commissioner as ex- officio members. Governed by independent Board of Directors, including the Director of NIH and the FDA Commissioner as ex- officio members.

Partnership Model Based on the Osteoarthritis Initiative Steering Committee FDA FNIH NIH Grants or Contracts Private Partners $$ Flow Information Flow

FNIH-NCI Plans ( ): Expand the proposed databases as an industry resource Form an overarching steering committee with FNIH, NCI, Academic and Industry representation Form an overarching steering committee with FNIH, NCI, Academic and Industry representation Identify specific milestones for early delivery of databases acquired from NCI clinical trails. Identify specific milestones for early delivery of databases acquired from NCI clinical trails. Lung Cancer Screening: Expand the number of sites for X-ray CT include digital chest X ray (Jan 2005)Lung Cancer Screening: Expand the number of sites for X-ray CT include digital chest X ray (Jan 2005) Lung Cancer Drug Response: X-ray CT. Expand Pilot Project (Sept 2004): PPP timetable: Summer Fall 2005Lung Cancer Drug Response: X-ray CT. Expand Pilot Project (Sept 2004): PPP timetable: Summer Fall 2005

The Database Resource Research Community MIRC sites: NCINCI FDA CDRHFDA CDRH Cornell UniversityCornell University MD Anderson Cancer CenterMD Anderson Cancer Center Memorial Sloan Kettering Cancer CenterMemorial Sloan Kettering Cancer Center University of California, Los AngelesUniversity of California, Los Angeles University of ChicagoUniversity of Chicago University of IowaUniversity of Iowa University of Michigan…University of Michigan… Industry PartnersIndustry Partners

4/4 Markings 2/4 Markings Meta Data: Results of Unblinded Reads: Nodule Location We will capture one aspect of reader variability in this way

Metadata: Description of Nodule Focal Abnormalities ()Nodules A spectrum of abnormalities? Scar Spiculated Nodule Calcified Nodule

Meta Data: Nodule Characteristics Library to Standardize Radiologist Interpretation.

Meta Data: Nodule Boundaries Example Spiculated Nodule: Comparison of Contours

Meta Data: Probabilistic Description of Boundary

RSNA: 2003:LIDC: Fundamental Issues for the Creation of a International Research Resource for the Image Informatics Community.

NCI Current and Future Clinical Trials. Potential Sources of Data and Timeline Colonoscopy collection ACRINLiver mets - MSK Digital breast- ACRIN UltrasoundMRI - ACRIN Year 1Year 2Year 3 PET alliances Enhance hardware and software infrastructure with NCICB NTROI optical database Demonstration Project: LIDC Colonoscopy collection ACRINLiver mets - MSK Colonoscopy collection - Navy Digital breast- ACRIN UltrasoundMRI - ACRIN Enhance hardware and software infrastructure with NCICB Lung Drug Response: Pilot Project NTROI optical database

Molecular Signatures Clinical Trials NCI Core access portals participating group nodes Cancer Genomics TGI GAI CCAP Mouse Models Meta Data: NCI Informatics Infrastructure Planned Reference Database Resource to be Fully Integrated into caBIG Informatics GRID. Open source Open access Open development Federated GRID Principles Group Nodes: Cancer Centers SPORES CGAP MMHCC Others

Database Resource Internet NCI and FDA Users MIRC sites PACS MIRC Site Work- stations n Cornell MDA MSK UCLA Chicago Iowa Mich.

Modality MIRC Clinical Trial Service CTMS PACS MIRC Storage Principal Investigator SiteField Center Sites (images) DICOM (csv data) HTTP MIRC Clinical Trial Service MIRC Storage DICOM HTTP(S) 8080 Database Resource Several Demonstration Projects Required extensions: 1. image export in groups 2. CSV converter to metadata for NID Desirable extensions: 1. name spaces 2. cde viewing / query 3. extend author service 8443