Financial Disclosure Robert Nishikawa: –paid consultant for Carestream and Siemens –shareholder in and receives royalties & research funding from Hologic, Inc. But I am here representing myself

Guidance Document on CADe Where the research is clear, then the document is reasonable However, whenever there is uncertainty, the approach is to error on the side of requiring more rather than less This is extremely burdensome This document if it goes into affect, it could kill new and innovative products in CADe

FDA is in a No-Win Situation If requirements are:  too lenient »non-effective products may be marketed –Money wasted –Patients may be harmed  too restrictive »effective products will be delayed –Patients’ healthcare is compromised

Guidance Document Statement “Test data, used once before, does not constitute independent data for testing a CADe device” Datasets are extremely costly and time consuming to collect This extremely overly burdensome This will stifle innovation

Training to the Test Dataset Difficult unless the performance on individual cases is known –Only release results of the performance on the whole set, not individual cases –Mitigates the example Dr. Gwise gave on overfitting

Proposed Observer Study Protocol 1.a conventional reading without the CADe device (i.e., reader alone) 2.CADe output is displayed immediately after conducting a conventional interpretation [sequential read] 3.a concurrent or simultaneous read

Proposed Observer Study Protocol Three reads, when only one is needed, is overly burdensome: –Costs are higher –Much, much harder to recruit radiologists –Time to complete study is much longer The increase is more than a factor of 3!

Proposed Observer Study Protocol 1.a conventional reading without the CADe device (i.e., reader alone) 2.CADe output is displayed immediately after conducting a conventional interpretation 3.a concurrent or simultaneous read “Reading scenarios should be consistent with the intended use of the device.”

Proposed Observer Study Protocol 1.a conventional reading without the CADe device (i.e., reader alone) 2.CADe output is displayed immediately after conducting a conventional interpretation 3.a concurrent or simultaneous read “Reading scenarios should be consistent with the intended use of the device.”

Concurrent Reading CADe is labeled as a second reader There is a possibility that radiologists will use it as a concurrent reading There is no clinical data to show whether this is good or bad It is overly burdensome to have manufactures answer a research question in their submission that goes beyond their labeling claims

Proposed Observer Study Protocol 1.a conventional reading without the CADe device (i.e., reader alone) 2.CADe output is displayed immediately after conducting a conventional interpretation Two studies have shown that a conventional read and the unaided reading in a sequential read are comparable

Proposed Observer Study Protocol 1.CADe output is displayed immediately after conducting a conventional interpretation

Radiology: June 2009 Post-approval study Post-approval studies are not needed Clinical studies show that CADe can increase sensitivity with a comparable increase in recall rate CADe can increase radiologists’ sensitivity (~9%) while keeping PPV 1 approximately constant (4.8% vs 4.7%)

Clinical Evaluation Methodology Using cancer detection rate in clinical studies that employ historical controls is not viable for measuring the effectiveness of CADe Radiology: June 2009

Power Calculation Dr. Gwise power calculation for se at a fixed sp is not the same as the power required for (se,sp) true operating point