Evidence Review Workgroup Advisory Committee on Heritable Disorders and Genetic Diseases in Newborns and Children Report August 2008 James M. Perrin, MD Professor of Pediatrics, Harvard Medical School Director, MGH Center for Child and Adolescent Health Policy

Workgroup Team Members Marsha Browning, MD, MPH, MGH/Harvard (genetics) Anne Comeau, PhD, New England Newborn Screening Program/UMass Medical School (public health screening perspective) Nancy Green, MD, Columbia University Alex Kemper, MD, MPH, MS, Duke University (methods/screening) Lisa Prosser, PhD, Harvard Ambulatory Care and Prevention (cost/benefit analyses) Denise Queally, Consumer (PKU Family Coalition) Alixandra Knapp, MS, MGH/Harvard (project coordinator) Ellen Lipstein, MD, MGH/Harvard (health services research fellow) James M. Perrin, MD, MGH/Harvard (policy, chronic conditions) Marie Mann, MD, MPH, ex officio

Evidence (External) Advisory Group Jannine Cody, PhD, University of Texas Harvey Cohen, MD, PhD, Stanford University Robert L. Davis, MD, MPH, Kaiser Atlanta Celia Kaye, MD, University of Colorado

Issues in Evidence Review (More Later in Pompe’s Review) Rare conditions –Lack of randomized trials in many cases –Limited information on costs and benefits across all potential outcomes (ie, true and false positives and negatives) Access to evidence –Published evidence –Investigator findings (unpublished) –Proprietary data

Evidence Review Workgroup: Year 1 Activities Assisted by members of ACHDGDNC Development of data abstraction form Clear conflict of interest policy –Information on direct intellectual conflicts of interest and self/family financial conflicts (based on IOM policies) Evidence review outline

Evidence Review Rationale and Objective Rationale (for review at this time) –Nomination form and consideration by the AC Prospective pilot data re population-based assessment Spectrum of disease well described Screening test capable of identifying the condition Treatment is well described –Recent changes in treatments and/or screening Objectives of Review –Provide timely information to the Advisory Committee to guide recommendation decisions for a specific screening program

Evidence Review Main Questions I Natural history, including variations in phenotype Incidence, relative to genotype, phenotype and phenotypic variations Impact and severity Methods of screening and diagnosis (in screen positive individuals) Screening test utilities (sensitivity, specificity, predictive values) Feasibility and acceptability of screening

Evidence Review Main Questions II Benefits of treatment (both efficacy and effectiveness) –in screen positive individuals –In otherwise diagnosed individuals Harms or risks of –Screening –Diagnosis –Treatment Costs (screening, diagnosis, treatment, late treatment; failure to diagnose in newborn period)

Evidence Review Model and Methods Decision model and development of evidence questions Search methods (literature review time frame [typically no more than 20 years] and search engines used)

Systematic Review and Additional Data Collection and Review Study selection, data abstraction, and review –Inclusion/exclusion criteria Peer-reviewed published literature English only Gray literature – limited to pharmaceutical companies, unpublished studies (and related data) Exclude case series with <4 cases (will provide bibliography of case reports to AC) Review consensus statements as guides, not for abstraction –Data abstraction and quality assessment Standard quality assessment methods Analyses of (any) additional raw data from unpublished sources –Special issues of data format and constraints on use (data sharing agreement template in process) –Focus groups of experts (investigators and families) re impact and severity estimates –Data synthesis

Evidence Review Results and Summary Results –Follow order and content of main questions –Decision analyses/decision model findings (outcomes tables) Summary –Key findings in summary and table form –Indicate where evidence is absent and what information would be most critical What do we not know and level of uncertainty What new information/studies would most help AC decisions All decisions by AC – evidence group makes no recommendations

Next Steps Began initial two reviews (SCID and Pompe’s) in June Pompe’s aided by earlier work by Alex Kemper and need mainly to update Timeline –Expect Pompe’s report to AC in October, after review by EWG External Advisory Committee –Plan to send SCID report to External Advisory Committee in late September for later submission to the AC

Challenges in Evidence Review: Lessons from Pompe Disease (Alex Kemper, MD, MPH Marsha Browning, MD, MPH)

Identifying and Evaluating Unpublished Data Identifying those with unpublished data –Our solution is not comprehensive but identifies most researchers Previously developed relationships Discussions at meetings Citations in reviews and from advocacy groups

Identifying and Evaluating Unpublished Data Obtaining conflict of interest declarations –Somewhat overwhelming for investigators –Seems to be an “invasion” –Difficult to evaluate for completeness –Difficult to understand the impact of conflicts on subsequently shared data

Identifying and Evaluating Unpublished Data Obtaining data –Attempts at obtaining data via written report led to “high level” summaries –Follow-up interview clarified the written reports. However, the data still lack granularity

Identifying and Evaluating Unpublished Data Systematic Evaluation –Level of detail is not sufficient to fully assess the evidence –However, it does allow us to understand the direction of research –Highlights findings that may differ from the “accepted wisdom”

Strategy for Using Unpublished Data in the Reports Not sufficient for “Results” section However, an important component of “Discussion,” especially if findings –Suggest new methods for screening, diagnosis, or treatment –Differ from published findings

Thank You! Questions and Comments