Program Update April 11, 2013 Andrew J. Buckler, MS Principal Investigator, QI-Bench WITH FUNDING SUPPORT PROVIDED BY NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY

Agenda Summary and close-out of 1H FY13 reporting period: – Covering what’s been accomplished from multiple points of view Progress since F2F: – QIBA Challenge support – Architectural components – Publications 22

3 Autumn 2012 (n=42) FY 2012 (n=110) Winter 2013 (n=13) In Queue (n=23) 333 Established initial structure Initial Specify, Formulate, and Iterate Substantial work in Execute and Analyze First work on test-beds Established initial structure Initial Specify, Formulate, and Iterate Substantial work in Execute and Analyze First work on test-beds V&V issues in Execute Substantial work in Analyze library Deployment support Advanced test-beds V&V issues in Execute Substantial work in Analyze library Deployment support Advanced test-beds Architectural design for next generation Expansion to contour- based analysis First comprehensive analysis library runs on vCT test-bed PACS interface to RDSM Early work on triple store Architectural design for next generation Expansion to contour- based analysis First comprehensive analysis library runs on vCT test-bed PACS interface to RDSM Early work on triple store QI-Bench REST First workstation support Realization of QI-Bench cohesive architecture Migrate worked example test-bed into triples QI-Bench REST First workstation support Realization of QI-Bench cohesive architecture Migrate worked example test-bed into triples

5555

Secondary Analysis Register all seg_SUBJID_0_ACQREP_rdrID.nii with respect to one of them within same acqrep For each SUBJID and for each ACREP STAPLE  Weights.nii Binarize  Weights-bin.nii Pair-wise comparison between seg_SUBJID_0_ACREP_rdrID.nii-r.nii and Weights-bin.nii Create input files for analysis of metrics: Intersection, Union, Jaccard, DICE Tertiary Analysis Register all seg_SUBJID_0_ACQREP_rdrID.nii with respect to one of them across different acqrep Primary Analysis Prepare input files for statistical modules Bland Altman, CCC Linear Mixed Effect Probability Density Function of Error Bland Altman, CCC Linear Mixed Effect Probability Density Function of Error (steps only needed to accommodate 1B) Segmentation Objects: 1.2…UID.dcm Extract tags such as SUBJID, readerID, Acqrep Rename to Seg_SUBJID_0_ACQREP_rdrID.dcm Extract tags such as pixel spacing, matrix Calculate volumes Convert from dcm to nii format Label map files from second 1B study 77777

Primary Analysis Prepare input files for statistical modules Bland Altman, CCC Linear Mixed Effect Probability Density Function of Error Bland Altman, CCC Linear Mixed Effect Probability Density Function of Error Files rdg_SUBJID_participantID.csv and seg_SUBJID_TPINDEX_participantID from participants Secondary Analysis Register all seg_SUBJID_0_ACQREP_rdrID.nii with respect to one of them within same acqrep For each SUBJID and for each ACREP STAPLE  Weights.nii Binarize  Weights-bin.nii Pair-wise comparison between seg_SUBJID_0_ACREP_rdrID.nii-r.nii and Weights- bin.nii Create input files for analysis of metrics: Intersection, Union, Jaccard, DICE Tertiary Analysis Register all seg_SUBJID_0_ACQREP_rdrID.nii with respect to one of them across different acqrep (steps only needed for 3A) Disregard distractors Re-assign TPINDEX to ACQREP Roll-up to form s_files, including cov and dcm metadata Supported segmentation object formats: Analyze mhd mha nrrd nii QIBA Challenge Support

DEVELOPMENT PROGRESS REPORT wernsing 999

10

11

12

13

14

15

Development plan Continue with core development. 1) Controller to support… 2) QI-Bench API 3) Connections to Computing Services 4) Connections to Business Logic and Data Services Parallel work Creating the initial Slicer plug-in Refining and augmenting the QI-Bench API Workflows and custom actors in Kepler Connections to the API Web applications 16

Recent Publications 17

18

Value proposition of QI-Bench Efficiently collect and exploit evidence establishing standards for optimized quantitative imaging: – Users want confidence in the read-outs – Pharma wants to use them as endpoints – Device/SW companies want to market products that produce them without huge costs – Public wants to trust the decisions that they contribute to By providing a verification framework to develop precompetitive specifications and support test harnesses to curate and utilize reference data Doing so as an accessible and open resource facilitates collaboration among diverse stakeholders 19

Summary: QI-Bench Contributions We make it practical to increase the magnitude of data for increased statistical significance. We provide practical means to grapple with massive data sets. We address the problem of efficient use of resources to assess limits of generalizability. We make formal specification accessible to diverse groups of experts that are not skilled or interested in knowledge engineering. We map both medical as well as technical domain expertise into representations well suited to emerging capabilities of the semantic web. We enable a mechanism to assess compliance with standards or requirements within specific contexts for use. We take a “toolbox” approach to statistical analysis. We provide the capability in a manner which is accessible to varying levels of collaborative models, from individual companies or institutions to larger consortia or public-private partnerships to fully open public access. 20