Formulation of a Strategy: A Framework for Action IOM Workshop on Standards for Systematic Reviews and Clinical Practice Guidelines May 10-11, 2011 University of California-Washington Conference Center Stephanie Chang AHRQ

Agenda Disclaimers and Disclosures Disclaimers and Disclosures Reflections Reflections A Framework for action A Framework for action

Disclaimers and Disclosures Represent self not EPCs Represent self not EPCs Financial COI Financial COI – AHRQ employee Intellectual COI Intellectual COI – Generalist – Academic research vs EHC program – EPC Program – Funder of systematic reviews – Government

Reflections on standards CPG have less standards, but more controversy CPG have less standards, but more controversy – SR: 84 instructions, 21 standards, 4 areas – CPG: 20 instructions, 8 standards SR considered a more mature science SR considered a more mature science – In early adulthood? – From wild west to assembly line Definition of standard Definition of standard – Provisional, needing beta testing – Sets an ideal or goal – Intended to provide uniformity, justification – “doesn’t mean you have to do all of them” – No minimum threshhold, no weighting

Reflections on SR standards General agreement General agreement Areas of disagreement due to limited resources - Good science vs practicalities Areas of disagreement due to limited resources - Good science vs practicalities – Is there a minimum? – When evidence and resources are limited how to weigh values Validity Validity Transparency Transparency Patient centeredness Patient centeredness Timliness, efficiency, utility Timliness, efficiency, utility u Research bias? u Balance between internal and external validity – is no review better than review with potential bias or error? Implications on need for better coordination, tools to help automate processes Implications on need for better coordination, tools to help automate processes – May result in fewer SR, GL

A framework for action IOM work on outlining standards is done IOM work on outlining standards is done Pilot testing Pilot testing – Can it work? Has this been done by any group? Is the tool too unwieldy? How much resources will it take? – Will it work? Is there face validity to the criteria? Are groups willing to implement the standards? – Is it worth it? Are SRs (or CPGs) better, more valid? Do the benefits outweigh the costs? Uptake depends on: Uptake depends on: – Face validity and willingness by implementers – Environment that supports standards – Regulations and requirements Funders Funders Publishers Publishers ?Regulators? ?Regulators?

Evaluating the standards Generally accepted vs variation Generally accepted vs variation – Limited evidence, disagreement on validity – Prioritization due to limited resources Adopt and implement those that are agreed upon Adopt and implement those that are agreed upon Consider ability to gather empiric evidence Consider ability to gather empiric evidence – Standards to improve relevance/usefulness – Standards to improve validity – Standards to improve transparency/trustworthiness Consider if values outweigh resource issues in areas where evidence cannot help. Consider if values outweigh resource issues in areas where evidence cannot help.

AHRQ work Training materials Training materials – Training modules – Community Forum Tools Tools – SRDR, Abstraktr, Meta-analyst Funded partners Funded partners – GRADE workshop – CUE, Community Forum Systematic reviews, translation guides and other dissemination materials Systematic reviews, translation guides and other dissemination materials – Post all topic nominations Guidelines Clearinghouse, USPSTF Guidelines Clearinghouse, USPSTF Methods research Methods research