Probe Validation Anne Wiktor September 21, 2006

Abnormalities identified by FISH

Precision Evaluation Study Pilot Study Familiarization Validation Process

Enumeration probe strategy Normal metaphase Abbott Molecular p53 D17Z1

Familiarization Validation Process

  Assess probe performance   Assess equipment   Calculate analytic sensitivity and specificity Familiarization Experiment Objectives

  For each sample, score:   target loci in 20 metaphases   50 interphase nuclei   Record signal pattern of each cell   expected pattern 2R2G 5 normal male PHA-stimulated peripheral bloods Familiarization Experiment

Probe evaluation Probep53D71Z1 Locus17p13.117cen Size ~ 145 Kb ColorOrangeGreen Manufacturer: VysisStrategy:EnumerationLot #: Probe Performance: Large, bright, intact signals, no cross-hybridization Equipment: Standard FISH microscopes and laboratory equipment are suitable

  Analytical specificity:   metaphase (% cells with signals at expected target)   Analytical sensitivity:   metaphase & interphase (% cells with expected signal pattern) Familiarization Experiment # scored − # false positive # scored x 100

Familiarization Results ID Metaphase cells Orange p53Green D17Z1 17p13.1Other17cenOther Analytical sensitivity: 20/20 = 100% Analytical specificity: 20/20 = 100%

Enumeration Scoring Criteria p53 Orange (R), 17cen Green (G), Overlap/Fusion Yellow (F) 1R2G Loss of p53 Normal 2R2G Do not score overlapping cells 1R1G signals Loss of p53 and 17cen Do not score cells with too many or too few signals 2R1G Loss of 17cen Normal 1R1G1F Normal Overlap

Familiarization Results ID Interphase nuclei 2R2G2R1G1R1G1R2G3R3G Analytical sensitivity: 246/250 = 98.4%

Familiarization Experiment signal patterns observed 2R2G 2R1G 1R1G 1R2G

Pilot Study Familiarization Validation Process

  Using previous experience w/ similar FISH strategies:   Establish scoring criteria   Determine number of cells to analyze   From normal samples, determine:   Maximum false-positive nuclei for each signal pattern   Analytical sensitivity   Calculate initial cutoff for normal specimens   Percent cells meeting scoring criteria   Confirm and/or redefine scoring criteria Pilot Study Objectives

How do we…. establish scoring criteria? Normal patterns 2R2G Abnormal patterns 1R2G

Pilot Study   Code and randomize   2 technologists - each score 100 nuclei   Record all signal patterns 5 normal & 5 representative abnormal specimens test intended tissue type

IDTech Expected patterns OtherTotal 2R2G1R2G1R1G2R1G3R3G 1SF KS SF KS SF KS SF KS SF KS SF KS SF KS Pilot Study Results

Pilot Study Summary   Analytical sensitivity   (% cells with expected signal pattern): 94.9%   Initial normal cutoff:   Percent cells meeting scoring criteria: 97.8% False +Cutoff 1R2G97.5 2R1G44.5 1R1G76.5 3R3G12.5

Evaluation Study Pilot Study Familiarization Validation Process

Prepare for clinical evaluation study  Write provisional standard operating procedure  Final review of safety of reagents & procedures  Select controls for clinical practice  Verify number of cells to score is appropriate for FISH strategy and clinical application  100 – 3.0%  200 – 1.5%  500 – 0.6%  6000 – 0.05% 95% CI

Clinical Evaluation Objectives   Simulate clinical practice  Establish normal cutoffs  Define abnormal reference range  Identify new scoring patterns

Clinical Evaluation 25 normal and a set of abnormal specimens; include variants and mosaics   Code and randomize   2 technologists - score cells using pilot study criteria   Record number of cells not meeting criteria   Record signal patterns of atypical cells

Study design   25 “normal” controls   15 abnormal samples   14 - del(17)   1 – del(17)/-17

Analysis of results   Normal cutoff: upper boundary of 95% CI for false-positive cells in normal specimens   Abnormal reference range: lowest & highest % abnormal cells of patient specimens   % of patterns not meeting scoring criteria: identify potential new additional signal patterns   Experimental clinical sensitivity: % correctly identified as true positives   Experimental clinical specificity: % correctly identified as true negatives

Clinical Evaluation Results IDTech Scoreable patterns Total Other “Mental bucket” Normal 2R2G Abnormal 1R2G 2SF KS SF KS SF (1R1G) KS (1R1G) 14SF KS SF KS AW SF KS982100

Normal cutoff based on binomial expansion formula # cells scored False positive cells Normal cutoff is based on 1-sided 95% confidence interval for binomial (N,p) given observed number of abnormal cells Beta inversion function (Microsoft Excel)

Beta Inverse Function

=BETAINV(0.95,12,200) CI # false positive + 1 # cells scored =BETAINV(0.95,12,200)

  Normal cutoffs (based on 200 nuclei with 95% confidence interval)  1R2G = 8.5% (17/200 cells)  1R1G = 6.5% (13/200 cells)  2R1G = 4.5% (9/200 cells)  3R3G = 2.5% (5/200 cells)  All other patterns = 1.5% (3/200 cells) Clinical Evaluation Summary

Cutoffs based on # cells scored # false positive cells/# cells analyzed 3/506/10011/20028/500 % cutoff Max. # false

Workload Recording 44 minutes 13 minutes 8 minutes 4 minutes Initial handling 3 minutes Digital Imaging / Printing 4 minutes 12 minutes Recording/ reporting CYTOGENETICS LABORATORY Specimen processing FISH process Analysis

Precision Evaluation Study Pilot Study Familiarization Validation Process

Precision Objectives   Test the reproducibility of the assay   Confirm that the same result can be obtained consistently and accurately

  Perform & score FISH assay on 10 consecutive days   Precision: mean, SD, & range Precision 1 abnormal with known % of abnormal cells

Mean = 40.0 Precision Testing SD = 2.02 Range =

Precision Evaluation Study Pilot Study Familiarization Validation Process

In clinical practice, validation continues with…. Working Protocol Continuous QC testing Proficiency testing Instrument calibration Employee competency & training Clinical correlation

Wiktor AE, Van Dyke DL, Stupca PJ, Ketterling RP, Dewald G, et al: Preclinical validation of fluorescence in situ hyrbidization assays in clinical practice, Genetics in Medicine 8:16-23, Validation Reference