1. N Petrick RSNA QIBA Phantom Group November 13, 20081 QIBA Proposed Phase 1A Project V5.0 QIBA Phantom Subgroup

2. N Petrick RSNA QIBA Phantom Group November 13, 2008 2 Outline of proposal  Modified 1A reader study  Outstanding issues

3. N Petrick RSNA QIBA Phantom Group November 13, 2008 3 Draft 1A: Study objective  Measure intra- and inter reader bias and variability on phantom lesions for:  Uni-dimensional size measurement  Semi-automated 3D volumetric measure  Compare with fully automated algorithm? Not going to size for specific effect or significance but just report as pilot

4. N Petrick RSNA QIBA Phantom Group November 13, 2008 4 Draft 1A: Nodule/scan dataset  Nodules (10 attached nodules)  -10 & +100HU  10, 20 mm spheres  10 mm ovoid, lobulated, spiculated  Scanner  Philips 16 slice  Exposure (120 kVp):  100 mAs  Slice thickness (50% overlap)  0.75 & 5.0 mm slices  Recon kernel  Standard/medium (Still on table: Detail/Lung kernel)  Pitch  1.2  2 repeat scans  40 segmentations in set Okay 100 mAs okay 0.75 mm slices okay

5. N Petrick RSNA QIBA Phantom Group November 13, 2008 5 Draft 1A: Study Protocol  Reader:  6 RadPharm radiologists  Software:  In-house review software (Siemens?)  Semi-automated 3D volume software  Uni-dimensional measure (RESIST)  Which fully automated software? Need to determine this? Wendy will visit RadPharm and provide more info next week.

6. N Petrick RSNA QIBA Phantom Group November 13, 2008 6 Draft 1A: Study Protocol  Reading session:  Readers read all cases in 2 different reading session  Random ordering  One-dimensional measure  Semi-automated segmentation  Session separated by 3 week?  Include duplicate cases within each read session (1/3-1/2 of cases for intra-reader estimates)  Any time restrictions?  Probably not  Any specific instructions?  Probably not

7. N Petrick RSNA QIBA Phantom Group November 13, 2008 7 Draft 1A: Analysis  Estimate intra- and inter-reader variability in the different volume estimate  Estimate bias from known truth  Estimate variability  Compare the bias and variability with the different methods Okay

8. N Petrick RSNA QIBA Phantom Group November 13, 2008 8 Draft 1A: Outstanding issues  Objective-Okay  This is pilot, no sizing will be done  May want to at least look at this to see if we have a reasonable study design  Data collection: Okay  +100 HU (all 1A scanner settings)  Acquired: 10 mm & 20 mm sphere, ovoid, spic, lob  -10 HU (all 1A scanner settings)  Acquired: 10 mm & 20 mm sphere  Missing: 10 mm ovoid, spic, lob (Est: 12/01/08)  10 mm aspherics okay  100 mAs okay

9. N Petrick RSNA QIBA Phantom Group November 13, 2008 9 Draft 1A: Outstanding issues  Readers-Okay  6 okay (base on expected effect size?)  Software-Open source option is available to use  Need to finalize which software  Contact Bob Ford to discuss  Study Issues  GUI  AVT designed interface?  Standard PACs interface?  How will nodule locations be provided to readers?  Full scans with nodule centers  Extracted ROIs  Can cases be directly uploading into PACs?  How to randomize cases during reading sessions  Data format for segmentations/measurements  Truth/scoring format  ROIs or full scans with lesion marked

10. N Petrick RSNA QIBA Phantom Group November 13, 2008 10 Draft 1A: Pilot Study  Study protocol  9 nodules for segmentation  2 slice thickness  10 mm -10 HU, sphere, spiculated  10 mm, +100 HU, lob., ovoid  20 mm, +100 HU, sphere  3 nodules with each segm. method  2 readers  Timeframe  Needs to be determined

11. N Petrick RSNA QIBA Phantom Group November 13, 2008 11 Draft 1A: Outstanding issues  Statistical  Can we measure intra-reader variance with only 2-4 repeat reads?  What effect size can we distinguish?  Again, may want to explore but this study will only measure as in a pilot

12. N Petrick RSNA QIBA Phantom Group November 13, 2008 12 Draft 1A: Outstanding issues  Other issues  Need to define QIBA publication policy  Determine research team makeup  Lisa Kinnard & Marios Gavrielides from FDA lab  Other Issues

13. N Petrick RSNA QIBA Phantom Group November 13, 2008 13 Draft 1A: Submissions  Conferences  ASCO  5/29-6/2/09 Orlando  Abstract Deadline: 1/6/09  RSNA  End Nov, 2009, Chicago  Abstract Deadline: 4/15/09  Journal  NEJM!  Radiology  Oncology journals?

14. N Petrick RSNA QIBA Phantom Group November 13, 2008 14 Action Items  Rick Avila to follow-up with caBIG AVT group  Rick Avila to lead effort of Lesion Ground Truth subgroup  Reporting Mechanism to be discussed at later date

15. N Petrick RSNA QIBA Phantom Group November 13, 2008 15 Work Items and workgroup  ??