1. QIBA CT Volumetrics - Cross-Platform Study (Group 1C) December 23, 2009 CT Cross-platform Sizing of Phantom Lesions Quantitative Imaging Biomarker Alliance Charles Fenimore, NIST

2. Charge 1. To agree on the scanner settings and other protocol elements under which imagery is to be collected. 2. To agree on requirements of phantoms to be imaged/measured. 3. To agree on the platforms and centers to be selected for imagery collection. 4. To agree on the site(s) and methods for the reader study. 5. To identify the measurements and the algorithms for use in image processing. 6. To specify the analysis of the measurements.

3. Goals 1. Measure the volume of nodules on CT imagery collected from several CT scanners and sites. Protocol is to be defined in relation to QIBA Vol CT 2 specifications. Protocol is to include a branch specifying performance. 2. Measure image noise and other image quality factors and determine their impact on the measurement of volume. 3. Analyze the accuracy and precision of volume measurements for all design factors including: site, device, scanner settings, & reader/algorithm/software. 4. Determine the minimum detectable level of change that can be achieved when measuring nodules in phantom datasets.

4. Goals 1 and 2 1. Measure the volume of nodules on CT imagery collected from several CT scanners and sites, etc. 2. Measure “image noise” and determine its impact on the measurement of volume, to facilitate scanner comparison. Realization of these goals is discussed in the next few slides.

5. QIBA CT 1-C Protocol Acquisition –The phantom –Image datasets –The imaging sites –System parameters –Markup sites

6. QIBA CT 1-C Acquisition Protocol Phantom: FDA lung & NIST pocket phantoms –The FDA phantom accomodates 8 attached nodules, some are 20 mm in size. A typical configuration might be:  Left: 10 & 20 mm spheres @ - 10 & +100 HU  Right: 10 & 20 mm spiculated @ - 10 & +100 HU, 10 mm ovoid, lobulated, spiculated Options: ovoid & lobulated nodules are also available. 9-mm spheres are coming, providing graded sizes (5, 8, 9, 10 mm). –The NIST pocket phantom is a free standing Lego structure holding teflon balls (6 mm in diameter, > 1000 HU). It is to be attached to the exterior of the lung phantom. Specific request to include high contrast (+100 HU) 20 mm sphere in interior. These two phantoms provide information about the system step response.

7. Potential imaging & reading sites Potential imaging sites –FDA - Philips 16 detector row –Nick Petrick / Marios Gavrelides, contact –Duke – GE Lightspeed 16 and 64 –Ehsan Samei, Division Chief, Clinical Imaging Physics –1 GE LightSpeed VCT (64 slice), 4 GE LightSpeed 16 (16 slice, 1 with fluoroscopic capability) –Johns Hopkins – Siemens, Toshiba –Mahadevappa Mahesh, Chief Physicist –UCLA –Mike McNitt Gray, medical physicist –UM-MC, Philips 64 –Eliot Siegel / Joseph Chen, radiologists –Siemens (Germany), offer of multiple scanners –Mathias Thorn Potential reading sites, CROs –Rad Pharm – Initiated discussion of markup –Additional sites: Perceptive Informatics, Biomedical Systems

8. QIBA CT 1-C Protocol 1 st branch = a standard protocol Scan 2 - 3 times. No more the one or two are to be read. Use the developing QIBA protocol (akin to ACRIN 6678) to specify kVp, slice thickness, mAs, rotation time, pitch, reconstruction kernel (affects MTF). Include water and ACR phantoms to characterize the resolution and noise levels under this branch of the protocol.

9. Sample Protocol Chart for ACRIN 6678

10. QIBA CT 1-C Profiles 2 nd branch = performance specified Specify easily implemented PERFORMANCE metrics of resolution and noise. Guidance provided on:  kVp (affects contrast difference between materials)  Slice thickness, recon interval (affects z-axis resolution & noise)  Rotation time and pitch (coverage, breath hold, etc.) Performance required on:  Recon kernel performance - EXAMPLE: –Choose kernel such that you can see 6 or 7 (but no more than 7) lp/cm on ACR phantom….or –10% MTF should be between 6 and 7 lp/cm  mA level performance –Choose effective mAs level so that std dev is between 20 and 30 HU in a 20 cm water phantom

11. QIBA CT 1-C Markup Procedure Adapt the Rad-Pharm process used in QIBA volume-CT 1-A and 1-B to acquire RECIST (1D), WHO (2D), & volume (3D) reads.  5 - 6 readers  2 reads for each nodules (one repeat). –Can we avoid repeat reads for the full set of series? The 1A study may provide prior estimates of intra- and inter-rater variability. –Is it possible to reduce the number of reads?

12. QIBA CT 1-C Profile Notional sizing of study 4 sites; 2 branches to protocol (QIBA-2 profile & performance-based); 8 nodules 2 – 3 acquisitions, of which 1 or 2 are read 2 repeat reads Full factorial design implies 256 nodule reads per radiologist - too large. Aim for less than full- factorial design.

13. Goal 3 Compare the accuracy and precision of radiologists’ measurements of RECIST and Volume for these phantom datasets. a) RECIST vs. volume. b) Investigate variance & bias. c) Inter-system variation. d) Intra-system variation.

14. Goal 4 4. Determine the minimum detectable level of change that can be achieved when measuring nodules in phantom datasets.

15. Required resources  Use of FDA phantom, water and ACR phantoms. Also NIST pocket phantom.  Select clinical image collection sites through QIBA-CT group: Offers from FDA, Duke, John’s Hopkins. Other possible sites: UMMC, MSKCC, Siemens.  Mark up similar to QIBA CT 1-A procedures at Rad-Pharm. Generates  RECIST (1D)  WHO (2D)  Segmented volume.

16. Next steps  Refine Questions and Experimental Design. –Do we need the first branch, “standard” protocol? –Do we want the lesion mask available for study?  Complete clinic recruitment. Develop protocol with associated medical physicists.  Confirm availability and arrange reading with Rad-Pharm. Other CROs are possible.  Schedule use of FDA phantom.