1. 12/10/02Sacks - Clinical Assessment1 Clinical Assessment – Part II William Sacks, PhD, MD Clinical Assessment – Part II William Sacks, PhD, MD COMPUTERIZED THERMAL IMAGING, INC. Breast Cancer System BCS 2100 P010035 Radiological Devices Advisory Panel December 10, 2002

2. 12/10/02Sacks - Clinical Assessment2 Safety of the BCS has two aspects: Adverse events Accuracy of BCS output Safety of the BCS has two aspects: Adverse events Accuracy of BCS output

3. 12/10/02Sacks - Clinical Assessment3 There were only 4 minor adverse events, two of which were thought not to be device related, during a clinical trial involving 2407 subjects.

4. 12/10/02Sacks - Clinical Assessment4 From the point of view of the BCS output: Safety is more closely related to sensitivity (cancers), or FNR, but also FPR Effectiveness is more closely related to specificity (benign masses) From the point of view of the BCS output: Safety is more closely related to sensitivity (cancers), or FNR, but also FPR Effectiveness is more closely related to specificity (benign masses)

5. 12/10/02Sacks - Clinical Assessment5 POINTS TO BE COVERED What BCS 2100 (BCS) is and is not intended to do How BCS does it What the clinical trial demonstrated Labeling issues POINTS TO BE COVERED What BCS 2100 (BCS) is and is not intended to do How BCS does it What the clinical trial demonstrated Labeling issues

6. 12/10/02Sacks - Clinical Assessment6 CLINICAL SUBMISSIONS PMA Amendment 4 Amendment 5 Amendment 7 CLINICAL SUBMISSIONS PMA Amendment 4 Amendment 5 Amendment 7

7. 12/10/02Sacks - Clinical Assessment7 After reviewing the PMA, the FDA sent a letter to CTI listing a number of deficiencies. CTI’s response was Amendment 4.

8. 12/10/02Sacks - Clinical Assessment8 In Amendment 4, for their conclusions concerning BCS effectiveness (p. 11), CTI retrospectively selected from the PMA data one of two analytical indices (Se/Sp, not ROC) and two of three lesion types (masses, and arch. dist., not microcalcifications).

9. 12/10/02Sacks - Clinical Assessment9 In Amendment 4 the revised labeling further deleted architectural distortion, and referred to masses alone.

10. 12/10/02Sacks - Clinical Assessment10 The FDA sent another deficiency letter to CTI. The response was Amendment 5.

11. 12/10/02Sacks - Clinical Assessment11 Amendment 5 was offered as a test of BCS in additional subjects, because Amendment 4 had contained retrospective selections. CTI called the additional dataset PPMA (Post-PMA).

12. 12/10/02Sacks - Clinical Assessment12 Amendment 5 confined its analysis of the PPMA data to the newly chosen analytical index (Se/Sp) in the newly chosen subgroup (masses).

13. 12/10/02Sacks - Clinical Assessment13 In addition to presenting data on a new set of subjects (PPMA), Amendment 5 also contained analysis of the combined datasets from Amendment 4 and the PPMA.

14. 12/10/02Sacks - Clinical Assessment14 Because of the retrospective selections in Amendment 4, FDA asked CTI to justify combining that data with the PPMA. CTI’s response was Amendment 7.

15. 12/10/02Sacks - Clinical Assessment15 In Amendment 7 CTI applied a Bonferroni correction in an attempt to compensate for retrospective selection of data in Amendment 4 and the smallness of the additional (PPMA) sample in Amendment 5. In Amendment 7 CTI applied a Bonferroni correction in an attempt to compensate for retrospective selection of data in Amendment 4 and the smallness of the additional (PPMA) sample in Amendment 5.

16. 12/10/02Sacks - Clinical Assessment16 Two overriding issues: Adequacy of data Interpretation of data, i.e., do the data demonstrate S&E of BCS? Two overriding issues: Adequacy of data Interpretation of data, i.e., do the data demonstrate S&E of BCS?

17. 12/10/02Sacks - Clinical Assessment17 Adequacy of data Can data from Amendment 4 contribute to judgment of S&E, when it consists of retrospective selection of mammographic masses, because the Se was too low in microcalcifications and in the tested population as a whole? Adequacy of data Can data from Amendment 4 contribute to judgment of S&E, when it consists of retrospective selection of mammographic masses, because the Se was too low in microcalcifications and in the tested population as a whole?

18. 12/10/02Sacks - Clinical Assessment18 Adequacy of data ( cont.) Is a Bonferroni correction applicable in this context? Adequacy of data ( cont.) Is a Bonferroni correction applicable in this context?

19. 12/10/02Sacks - Clinical Assessment19 Adequacy of data (cont.) Are the data from the PPMA alone (in Amendment 5) adequate to judge S&E? Adequacy of data (cont.) Are the data from the PPMA alone (in Amendment 5) adequate to judge S&E?

20. 12/10/02Sacks - Clinical Assessment20 Interpretation of the data It is noteworthy for the following discussion that no formal hypotheses were explicitly put forward for testing, either in the PMA or in the subsequent amendments. Interpretation of the data It is noteworthy for the following discussion that no formal hypotheses were explicitly put forward for testing, either in the PMA or in the subsequent amendments.

21. 12/10/02Sacks - Clinical Assessment21 To qualify as a testable hypothesis there must be a quantitative criterion, whereby either a point estimate may imply rejection, or a confidence interval may entail exclusion. To qualify as a testable hypothesis there must be a quantitative criterion, whereby either a point estimate may imply rejection, or a confidence interval may entail exclusion.

22. 12/10/02Sacks - Clinical Assessment22 There were two implicit hypotheses: that the ROC area for BCS and mammography combined would exceed that of mammography alone (with statistical significance) [Protocol, Section 6.0 Statistical Analysis] that the point estimate for sensitivity would be at least 99.3% (in at least 75% of simulations with the data) [PMA, Section 5.8.7.2, p. 489]. There were two implicit hypotheses: that the ROC area for BCS and mammography combined would exceed that of mammography alone (with statistical significance) [Protocol, Section 6.0 Statistical Analysis] that the point estimate for sensitivity would be at least 99.3% (in at least 75% of simulations with the data) [PMA, Section 5.8.7.2, p. 489].

23. 12/10/02Sacks - Clinical Assessment23 The protocol otherwise contained only non-quantitative statements of what CTI hoped to achieve in the clinical trial. For example, The objective of the study is to determine if the (BCS 2100), when used in conjunction with clinical examination and/or diagnostic mammography, increases the ability of physicians to differentiate benign from malignant, or suspicious, breast abnormalities. [Protocol, Section 3.1 Study Objective] The protocol otherwise contained only non-quantitative statements of what CTI hoped to achieve in the clinical trial. For example, The objective of the study is to determine if the (BCS 2100), when used in conjunction with clinical examination and/or diagnostic mammography, increases the ability of physicians to differentiate benign from malignant, or suspicious, breast abnormalities. [Protocol, Section 3.1 Study Objective]

24. 12/10/02Sacks - Clinical Assessment24 In the original PMA submission, the comparison of ROC areas failed to achieve statistical significance, except as an artifact of too few points in the mammography-alone curve, so it was not pursued in any of the amendments.

25. 12/10/02Sacks - Clinical Assessment25 In addition, the sensitivities failed to achieve a level of 99.3% with 75% confidence, in any of the datasets.

26. 12/10/02Sacks - Clinical Assessment26 PPMA (n=15) (Se 93.3%, Sp 25.4%) Mammography alone PMA (n=187) (Se 97.1%, Sp 14.3%) Am. 4 (n=90) (Se 100%, Sp 18.0%) Combined (n=105) (Se 99.0%, Sp 19.2%) 72.1% 99.7% 94.1% 98.8% 95.6% 96.7% Reference Se 99.3% SPECIFICITY SENSITIVITY 100% 0% Chance Line

27. 12/10/02Sacks - Clinical Assessment27 Potential safety and effectiveness in U.S. population:

28. 12/10/02Sacks - Clinical Assessment28 Percent of U.S. biopsies potentially obviated by the BCS if used on all eligible women: 1.3 million U.S. women biopsied each year, of which ~45% (585,000) have mammographic masses, of which ~80% (468,000) are benign, of which 15% to 20% (70,000 to 94,000) would be BCS(-) and saved a biopsy. Thus ~5% to 7% of 1.3 million U.S. biopsies would be obviated, and that’s if BCS were used on all 585,000 women with mammographic masses on their way to biopsy. In addition to saving biopsies on these benign masses, approximately 1% to 6% of the malignant masses, and 0.5% to 3% of all breast cancers, might be delayed in diagnosis. ( Percent of U.S. biopsies potentially obviated by the BCS if used on all eligible women: 1.3 million U.S. women biopsied each year, of which ~45% (585,000) have mammographic masses, of which ~80% (468,000) are benign, of which 15% to 20% (70,000 to 94,000) would be BCS(-) and saved a biopsy. Thus ~5% to 7% of 1.3 million U.S. biopsies would be obviated, and that’s if BCS were used on all 585,000 women with mammographic masses on their way to biopsy. In addition to saving biopsies on these benign masses, approximately 1% to 6% of the malignant masses, and 0.5% to 3% of all breast cancers, might be delayed in diagnosis. (

29. 12/10/02Sacks - Clinical Assessment29 POINTS TO BE COVERED What BCS 2100 (BCS) is and is not intended to do How BCS does it What the clinical trial demonstrated Labeling issues POINTS TO BE COVERED What BCS 2100 (BCS) is and is not intended to do How BCS does it What the clinical trial demonstrated Labeling issues

30. 12/10/02Sacks - Clinical Assessment30 Labeling issues Size of mass Depth of mass Labeling issues Size of mass Depth of mass

31. 12/10/02Sacks - Clinical Assessment31 Size of mass Effect of small lesion size on device sensitivity difficult to evaluate, since only 2/105 cancers < 5mm. Size of mass Effect of small lesion size on device sensitivity difficult to evaluate, since only 2/105 cancers < 5mm.

32. 12/10/02Sacks - Clinical Assessment32 Size of mass (cont.) Number of masses MalignantBenign <0.5 cm 212 0.5 cm - 1.0 cm 50188 >1.0 cm 53185 (Amendment 4, p.8, Table A4.4, Amendment 5, p.23, Table A5.14) (Amendment 4, p.8, Table A4.4, Amendment 5, p.23, Table A5.14)

33. 12/10/02Sacks - Clinical Assessment33 Depth of mass With the chosen IOS threshold (20.59) there was no definite effect of lesion depth on BCS result, but... Depth of mass With the chosen IOS threshold (20.59) there was no definite effect of lesion depth on BCS result, but...

34. 12/10/02Sacks - Clinical Assessment34 Depth of mass (cont.)...the effect of lesion depth was difficult to evaluate, since depth is not easily gauged on mammography. Depth of mass (cont.)...the effect of lesion depth was difficult to evaluate, since depth is not easily gauged on mammography.

35. 12/10/02Sacks - Clinical Assessment35 Conclusions Conclusions

36. 12/10/02Sacks - Clinical Assessment36 In summary Only 4 out of 2407 subjects had an adverse event, all minor. There were no explicit quantitative hypotheses. There were two implicit quantitative hypotheses. Neither hypothesis was fulfilled. Most data were selected retrospectively. Bonferroni correction is not applicable in this context. Using the trial results, if the BCS were in general use in the U.S., it would obviate 5% to 7% of the 1.3 million biopsies per year. Approximately 1% to 6% of these obviated biopsies would turn out to be malignant, and their diagnoses would thus be delayed. In summary Only 4 out of 2407 subjects had an adverse event, all minor. There were no explicit quantitative hypotheses. There were two implicit quantitative hypotheses. Neither hypothesis was fulfilled. Most data were selected retrospectively. Bonferroni correction is not applicable in this context. Using the trial results, if the BCS were in general use in the U.S., it would obviate 5% to 7% of the 1.3 million biopsies per year. Approximately 1% to 6% of these obviated biopsies would turn out to be malignant, and their diagnoses would thus be delayed.