1. James E. Haddow, M.D. Women & Infants Hospital Alpert Medical School of Brown University BROWN Women & Infants’ EDUCATIONAL CHALLENGES PRESENTED BY NEW CAPABILITIES IN THE LABORATORY NCHPEG Annual Meeting September 24, 2009

2. Even though this talk’s focus will be limited to DNA testing: The number of new tests is large The range of potential applications is also large Some of these tests are complex, with multiple SNPs measured Collecting data to document their performance requires funding and time There is economic and consumer-driven pressure to put them into practice

3. Within contemporary society, the anticipation is that these new DNA tests will contribute to a longer and healthier life, by somehow altering our destiny, as depicted originally in Greek mythology by the three fates.

4. Groups in the U.S. using an evidence-based approach to assess genomic tests United States Preventive Services Taskforce (USPSTF), sponsored by the Agency for Health Care Quality (AHRQ) Evaluation of Genetic Applications in Practice and Prevention (EGAPP) working group, sponsored by the Office of Public Health Genomics at the Centers for Disease Control Similar government-sponsored groups exist in Canada, Europe and elsewhere

5. Criteria for Evaluating a Screening or Diagnostic Test (Includes both laboratory tests and other approaches, such as questionnaires) Wald N, Cuckle H. Reporting the assessment of screening and diagnostic tests. Br J Obstet Gynaecol, 1989;96:389-396

6. Three Critical Elements for Test Evaluation The detection rate (sensitivity) The false positive rate (1-specificity) The odds of being affected, given a positive result (OAPR) – equivalent to positive predictive value

7. Applying these criteria to a Prader- Willi syndrome question A problem presented to the New England Regional Genetics Group (NERGG) in 1995

8. Stated Problem: Lack of assurance about manufacturer’s quality control of molecular probes used for detection Prader-Willi syndrome

9. Preliminary discussion leads to conclusion that reagent quality appears satisfactory, but quality control on a lab-by-lab basis is unacceptable.

10. The evaluation of Prader-Willi testing shifts to the checklist. The evaluation now begins by focusing on the disorder.

11. The Medical Disorder Being Sought Prader-Willi syndrome is well defined and sufficiently serious to warrant consideration of testing

12. Target Populations that Might be Tested for Prader-Willi Syndrome, Using the Molecular Probe Individuals clinically diagnosed with Prader-Willi syndrome. Infants suspected clinically to have Prader-Willi syndrome. Pregnant women having amniocentesis for other purposes. Obese children whose parents want testing for Prader-Willi syndrome.

13. Prevalence of Prader-Willi Syndrome in the Four Target Populations Target PopulationPrevalence Individuals diagnosed with Prader-Willi 100:1 Infants suspected to have Prader-Willi 1:10 Women having routine amniocentesis 1:15,000 Obese children with no other findings 1:1,000,000

14. The test being used is a molecular probe for detection of a microdeletion. It is considered a diagnostic test. It can detect 7 out of 10 cases of Prader-Willi syndrome (the other three are caused by uniparental disomy).

15. The false positive rate for the microdeletion test is not known. Arbitrarily assigned a rate of 1 per 1000 tests (0.1%)

16. The prevalence of Prader-Willi in the general population is 1:15,000. Knowing the detection and false positive rates of the molecular probe, and the prevalence of the disorder, we can now calculate the OAPR (or PPV) for the four target populations.

17. Reliability of a Positive Test Result (microdeletion) for Prader-Willi Syndrome in Four Target Populations Odds that a positive test result Target Populationis correct (PPV) Individuals diagnosed with Prader-Willi700:1 Infants suspected to have Prader-Willi 70:1 Women having routine amniocentesis 1:21 All obese children 1:1000

18. ACCE A CDC-Sponsored Project AIM: To develop and test a model system to assess the available quantity and usefulness of existing data on DNA-based tests and testing algorithms. PURPOSE: To provide an up-to-date, accurate and complete summary of available information in forms that are useful to policy-makers, professionals and the general public.

19. The Model System is Interpreted in Five Steps Defining the disorder and setting A nalytic validity C linical validity C linical utility E thical, legal and social implications The ACCE project was supported by a cooperative agreement with CDC, Office of Genomics and Public Health (CCU319352)

20. Effective Intervention (Benefit) Natural History Economic Evaluation Quality Assurance Education Facilities Pilot Trials Monitoring & Evaluation Ethical, Legal, & Social Implications (safeguards& impediments) Health Risks Clinical Specificity Clinical Sensitivity Prevalence PPV NPV Penetrance Assay Robustness Quality Control Analytic Specificity Analytic Sensitivity Disorder Setting The ACCE Model System Test

21. Setting the stage for applying the conceptual framework Is the disorder well defined? What is it? Is it sufficiently serious to warrant testing? Is the prevalence known in the population to be tested? What is the specific test being applied to the population?

22. Analytic validity (ACCE) How effectively does the test measure the genotype/mutation of interest?

23. Clinical validity (ACCE) How closely associated is the genotype/mutation with the target clinical disorder in the setting to which it is applied?

24. Clinical validity (ACCE) The detection rate for the disorder being sought (sensitivity)? The false positive rate (1-specificity)? The odds of being affected, given a positive result (positive predictive value)? For a screening test, what is:

25. Clinical validity (ACCE) When a diagnostic test follows a positive screening test, what is: The detection rate? The false positive rate? The odds of being affected, given a positive result for that test?

26. Clinical utility (ACCE) What harms are associated with testing? What therapeutic intervention will follow a positive screening or diagnostic test? How effective are these interventions in reducing risk, or treating existing conditions? What are the harms associated with treatments? What costs are incurred/avoided?

27. Ethical, legal and social issues (ACCE) What ethical, legal, and social issues are associated with the screening test or inquiry? What safeguards have been described and are these safeguards in place and effective?

28. Asking family history questions about breast/ovarian cancer: the best current example Aim - identify women at high risk for hereditary breast/ovarian cancer caused by mutations in BRCA1 and BRCA2 genes. Strategy - use a standardized history form (computer-based), and interpret the histories using a well designed and validated algorithm.

29. What is known? The prevalence of disease-causing BRCA1/2 gene mutations in the U.S. –among Caucasian women is about 1 in 350. –among Ashkenazi Jewish women is about 1 in 50. The false positive rate is about 3% for one computerized history form. This means that 1 in 30 women completing the form will be identified as candidates for genetic testing but won’t carry a BRCA1/2 mutation.

30. What is not known? The detection rate of family history screening for identifying women with a BRCA1/2 mutation –it is estimated to be 40 – 60%. –among 3,500 women screened, 10 will carry a BRCA1/2 mutation. Four to six of these might be identified. –among 3,500 women screened, about 100 (3%) will be false positives. The overall odds of carrying a mutation, given a positive family history would be 5:100, or 1:20 (a 5% PPV).

31. What is known about the diagnostic sequencing test for BRCA mutations? Detection rate is c. 95% Variants of unknown significance occur in 5-10% of cases tested.

32. Why is this a good example? A positive screening test leads to a well defined diagnostic test. The high risk nature of a mutation is well understood. Risk reducing strategies (chemotherapy, surgery) are available. Family studies can amplify the potential benefits.

33. What implications can be drawn from this example? Family history screening applications for other disorders with a genetic component ought to be assessed via the same/similar framework. Validated questions and interpretations should be developed for each application. Both health benefits and harms need to be measured. Consistent and transparent policy making should be applied to findings from research-based clinical applications.

34. Evaluation of Genomic Applications in Practice and Prevention (EGAPP) www.egappreviews.org Example of Evidence Reports: Outcomes of Genetic Testing in Adults with a History of Venous Thromboembolism Example of Recommendations: Genetic Testing Strategies in Newly Diagnosed Individuals with Colorectal Cancer Aimed at Reducing Morbidity and Mortality from Lynch Syndrome in Relatives

35. Analytic Framework From the EGAPP review entitled “Testing for cytochrome P450 polymorphisms in adults with non-psychotic depression treated with SSRIs”

36. Meeting the Educational Challenges from New Laboratory Capabilities: Keep basic evaluative principles in mind Understand the characteristics of the population being tested Ask “How much the test will add to existing care?” Look to recommendations from independent panels