1. Probe Validation Anne Wiktor September 21, 2006

2. Abnormalities identified by FISH

3. Precision Evaluation Study Pilot Study Familiarization Validation Process

4. Enumeration probe strategy Normal metaphase Abbott Molecular p53 D17Z1

5. Familiarization Validation Process

6.   Assess probe performance   Assess equipment   Calculate analytic sensitivity and specificity Familiarization Experiment Objectives

7.   For each sample, score:   target loci in 20 metaphases   50 interphase nuclei   Record signal pattern of each cell   expected pattern 2R2G 5 normal male PHA-stimulated peripheral bloods Familiarization Experiment

8. Probe evaluation Probep53D71Z1 Locus17p13.117cen Size ~ 145 Kb ColorOrangeGreen Manufacturer: VysisStrategy:EnumerationLot #: 987654 Probe Performance: Large, bright, intact signals, no cross-hybridization Equipment: Standard FISH microscopes and laboratory equipment are suitable

9.   Analytical specificity:   metaphase (% cells with signals at expected target)   Analytical sensitivity:   metaphase & interphase (% cells with expected signal pattern) Familiarization Experiment # scored − # false positive # scored x 100

10. Familiarization Results ID Metaphase cells Orange p53Green D17Z1 17p13.1Other17cenOther 1200200 2200200 3200200 4200200 5200200 Analytical sensitivity: 20/20 = 100% Analytical specificity: 20/20 = 100%

11. Enumeration Scoring Criteria p53 Orange (R), 17cen Green (G), Overlap/Fusion Yellow (F) 1R2G Loss of p53 Normal 2R2G Do not score overlapping cells 1R1G signals Loss of p53 and 17cen Do not score cells with too many or too few signals 2R1G Loss of 17cen Normal 1R1G1F Normal Overlap

12. Familiarization Results ID Interphase nuclei 2R2G2R1G1R1G1R2G3R3G 1491000 2490100 3500000 4481010 5500000 Analytical sensitivity: 246/250 = 98.4%

13. Familiarization Experiment signal patterns observed 2R2G 2R1G 1R1G 1R2G

14. Pilot Study Familiarization Validation Process

15.   Using previous experience w/ similar FISH strategies:   Establish scoring criteria   Determine number of cells to analyze   From normal samples, determine:   Maximum false-positive nuclei for each signal pattern   Analytical sensitivity   Calculate initial cutoff for normal specimens   Percent cells meeting scoring criteria   Confirm and/or redefine scoring criteria Pilot Study Objectives

16. How do we…. establish scoring criteria? Normal patterns 2R2G Abnormal patterns 1R2G

17. Pilot Study   Code and randomize   2 technologists - each score 100 nuclei   Record all signal patterns 5 normal & 5 representative abnormal specimens test intended tissue type

18. IDTech Expected patterns OtherTotal 2R2G1R2G1R1G2R1G3R3G 1SF972100100 KS941320100 2SF922420100 KS942220100 6SF942121100 KS923310100 7SF915220100 KS924410100 8SF7028200100 KS6732010100 9SF913420100 KS933310100 10SF5142610100 KS5239720100 Pilot Study Results

19. Pilot Study Summary   Analytical sensitivity   (% cells with expected signal pattern): 94.9%   Initial normal cutoff:   Percent cells meeting scoring criteria: 97.8% False +Cutoff 1R2G97.5 2R1G44.5 1R1G76.5 3R3G12.5

20. Evaluation Study Pilot Study Familiarization Validation Process

21. Prepare for clinical evaluation study  Write provisional standard operating procedure  Final review of safety of reagents & procedures  Select controls for clinical practice  Verify number of cells to score is appropriate for FISH strategy and clinical application  100 – 3.0%  200 – 1.5%  500 – 0.6%  6000 – 0.05% 95% CI

22. Clinical Evaluation Objectives   Simulate clinical practice  Establish normal cutoffs  Define abnormal reference range  Identify new scoring patterns

23. Clinical Evaluation 25 normal and a set of abnormal specimens; include variants and mosaics   Code and randomize   2 technologists - score cells using pilot study criteria   Record number of cells not meeting criteria   Record signal patterns of atypical cells

24. Study design   25 “normal” controls   15 abnormal samples   14 - del(17)   1 – del(17)/-17

25. Analysis of results   Normal cutoff: upper boundary of 95% CI for false-positive cells in normal specimens   Abnormal reference range: lowest & highest % abnormal cells of patient specimens   % of patterns not meeting scoring criteria: identify potential new additional signal patterns   Experimental clinical sensitivity: % correctly identified as true positives   Experimental clinical specificity: % correctly identified as true negatives

26. Clinical Evaluation Results IDTech Scoreable patterns Total Other “Mental bucket” Normal 2R2G Abnormal 1R2G 2SF991100 KS955100 6SF964100 KS982100 9SF305310017 (1R1G) KS374910014 (1R1G) 14SF946100 KS955100 22SF5347100 KS6436100 AW5644100 30SF964100 KS982100

27. Normal cutoff based on binomial expansion formula # cells scored False positive cells50100200500 0631.50.6 11052.51.0 21273.51.4 31484.01.6 41894.52.0 520105.52.2 622116.02.4 722126.52.6 824137.03.0 926147.53.2 1028158.03.4 1130168.53.6 1230179.53.8 Normal cutoff is based on 1-sided 95% confidence interval for binomial (N,p) given observed number of abnormal cells Beta inversion function (Microsoft Excel)

28. Beta Inverse Function

29. =BETAINV(0.95,12,200) CI # false positive + 1 # cells scored =BETAINV(0.95,12,200)

30.   Normal cutoffs (based on 200 nuclei with 95% confidence interval)  1R2G = 8.5% (17/200 cells)  1R1G = 6.5% (13/200 cells)  2R1G = 4.5% (9/200 cells)  3R3G = 2.5% (5/200 cells)  All other patterns = 1.5% (3/200 cells) Clinical Evaluation Summary

31. Cutoffs based on # cells scored # false positive cells/# cells analyzed 3/506/10011/20028/500 % cutoff14118.57.2 Max. # false +7111736

32. Workload Recording 44 minutes 13 minutes 8 minutes 4 minutes Initial handling 3 minutes Digital Imaging / Printing 4 minutes 12 minutes Recording/ reporting CYTOGENETICS LABORATORY Specimen processing FISH process Analysis

33. Precision Evaluation Study Pilot Study Familiarization Validation Process

34. Precision Objectives   Test the reproducibility of the assay   Confirm that the same result can be obtained consistently and accurately

35.   Perform & score FISH assay on 10 consecutive days   Precision: mean, SD, & range Precision 1 abnormal with known % of abnormal cells

36. Mean = 40.0 Precision Testing SD = 2.02 Range = 36 - 44

37. Precision Evaluation Study Pilot Study Familiarization Validation Process

38. In clinical practice, validation continues with…. Working Protocol Continuous QC testing Proficiency testing Instrument calibration Employee competency & training Clinical correlation

39. Wiktor AE, Van Dyke DL, Stupca PJ, Ketterling RP, Dewald G, et al: Preclinical validation of fluorescence in situ hyrbidization assays in clinical practice, Genetics in Medicine 8:16-23, 2006. Validation Reference