AHRQ’S Perspective on Comparative Effectiveness Research Jean Slutsky Director, Center for Outcomes and Evidence Agency for Healthcare Research and Quality TEACH 2011 NYAM August 10, 2011

Overview Role of effectiveness research in decision making Role of effectiveness research in decision making Understanding effectiveness research Understanding effectiveness research Comparative effectiveness research at AHRQ Comparative effectiveness research at AHRQ Lessons learned throughout Lessons learned throughout

Why the Need for Effectiveness Research? Interventions that are efficacious under a highly specific set of circumstances often fail to replicate across a wide variety of settings, conditions, patients. Glasgow, 2003

What Healthcare Decision Makers Need To Know Can it work? Can it work? Will it work? Will it work? – For this patient? – In this setting? Is it worth it? Is it worth it? – Do benefits outweigh harms? – Do benefits justify costs? – Does it offer important advantages over existing alternatives? adapted from Brian Haynes ACP Journal Club

CER – Helping to Make Better Health Care Decisions? Improve research base Improve research base – Relevance – Quality Improve use of data in decision making Improve use of data in decision making Informed decision making Informed decision making

Uses of Evidence for Decisionmaking Product approval Product approval Product purchasing and formulary selection Product purchasing and formulary selection Benefit and coverage decisions Benefit and coverage decisions Practice guidelines Practice guidelines Quality review and improvement Quality review and improvement Clinician/patient decision-making Clinician/patient decision-making Choosing plans or providers Choosing plans or providers Organizational and management decisions Organizational and management decisions Program financing and priority setting Program financing and priority setting

Alternatives to Evidence- Based Medicine Eminence-based Eminence-based Vehemence-based Vehemence-based Eloquence-based Eloquence-based Providence-based Providence-based Diffidence-based Diffidence-based Nervousness-based Nervousness-based Confidence-based Confidence-based Reimbursement-based Reimbursement-based Celebrity-based Celebrity-based Gastronomically-based Gastronomically-based K-street-based K-street-based Isaacs and Fitzgerald. BMJ (7225): 1618.

Some Perspective on Comparative Effectiveness Research at AHRQ

Comparative Effectiveness: AHRQ Effective Health Care Program Created in 2005, authorized by Section 1013 of the Medicare Prescription Drug, Improvement, and Modernization Act (MMA) of 2003 Created in 2005, authorized by Section 1013 of the Medicare Prescription Drug, Improvement, and Modernization Act (MMA) of 2003 To improve the quality, effectiveness, and efficiency of health care delivered through Medicare, Medicaid, and Children’s programs by focusing on To improve the quality, effectiveness, and efficiency of health care delivered through Medicare, Medicaid, and Children’s programs by focusing on – What is known now – What research gaps are critical to fill – Clinical effectiveness

HorizonScanning Evidence Need Need Identification Identification EvidenceSynthesis Evidence Generation GenerationStrategiesInterventionsConditionsPopulations TranslationDisseminationImplementation Improvements in in Health Care Health Care Research Platform Infrastructure – Methods Development – Training A Framework for CER at AHRQ Stakeholder Engagement

Engaging the Public

Categorical Investments in Evidence Generation Distributed data networks and related infrastructure development Distributed data networks and related infrastructure development Methods development Methods development Non-experimental/observational studies, both retrospective and prospective Non-experimental/observational studies, both retrospective and prospective Sponsorship of pragmatic clinical trials addressing key foci of comparative effectiveness Sponsorship of pragmatic clinical trials addressing key foci of comparative effectiveness

Recovery Act: Grants by Priority Condition* * Grants by Priority Condition, AHRQ Effective Health Care Program, May 2011

AHRQ Recovery Act Evidence Generation Activities Request for Registries: the creation or enhancement of national patient registries, with a primary focus on the 14 priority conditions Request for Registries: the creation or enhancement of national patient registries, with a primary focus on the 14 priority conditions DEcIDE Consortium Support: Expansion of multi-center research system and funding for distributed data network models that use clinically rich data from electronic health records DEcIDE Consortium Support: Expansion of multi-center research system and funding for distributed data network models that use clinically rich data from electronic health records Clinical and Health Outcomes Initiative in Comparative Effectiveness (CHOICE): Coordinated national effort to establish a series of 10 pragmatic clinical comparative effectiveness studies Clinical and Health Outcomes Initiative in Comparative Effectiveness (CHOICE): Coordinated national effort to establish a series of 10 pragmatic clinical comparative effectiveness studies iADAPT: Innovative dissemination and implementation grants for CER iADAPT: Innovative dissemination and implementation grants for CER

A Word About Translation A Word About Translation

Why is Translation Needed? “Evidence may be necessary, but it is certainly not sufficient. The findings of research need to be translated into information that is useful for each health care decision maker.” Eisenberg JAMA 1999; 282:

If Researchers Ruled the World Food in grocery stores would be alphabetized Food in grocery stores would be alphabetized Holiday sales would be advertised in peer reviewed journals Holiday sales would be advertised in peer reviewed journals School room size would be directly related to sample size needed for NNL (numbers needed to learn) School room size would be directly related to sample size needed for NNL (numbers needed to learn)

Sagan Effect One of the most frequently cited reasons for scientists’ reluctance to talk to the press is the so-called Carl Sagan effect, that is, renowned scientist Carl Sagan was criticized by his fellow scientists who assumed that because Sagan was spending so much time communicating with the public, he must not have been devoting enough time to his research. J. Hartz and R. Chappel, Worlds Apart: How the Distance Between Science and Journalism Threatens America’s Future. Nashville, TN: Freedom Forum First Amendment Center, 1997.

Fact or Fiction Throughout his career, Sagan averaged a scientific peer-reviewed paper per month. Michael B. Shermer (August 2002). Social Studies of Science 32/4:

A Quantitative Analysis of Carl Sagan’s Scientific and Popular Articles by Year

It is Hard to Change Beliefs Popularization…is traditionally seen as a low status activity, unrelated to research work, which scientists are often unwilling to do and for which they are ill-equipped…Essentially, popularization is not viewed as part of the knowledge production and validation process but as something external to research which can be left to non-scientists, failed scientists or ex-scientists … Richard Whitley (1995), ‘Knowledge producers and knowledge acquirers: popularizations as a relation between scientific fields and their publics,’ in Terry Shinn and Richard Whitley (eds.), Expository Science: Forms and Functions of Popularization. Dordrecht/Boston, MA: D. Reidel Publishing

Educating Clinicians Concise Concise Actionable Actionable Paired with consumer guides Paired with consumer guides Convey level of uncertainty/certainty of findings Convey level of uncertainty/certainty of findings

Informing Patients and Consumers There is a suspicious area on my mammogram and my doctor said I need to have a breast biopsy to tell if it is cancer. What are my options?

Continuing Medical Education/Continuing Education Activities

What We’ve Learned (1) It is necessary to: It is necessary to: – Understand policy and practice context – Involve stakeholders early – Broaden approach to evidence – Link evidence gaps to future research – Translate findings for different audiences

What Have We Learned Often, there are no clear winners and losers Often, there are no clear winners and losers Need to think of questions years in advance of decision makers asking Need to think of questions years in advance of decision makers asking It is amazing what we don’t know about common and ubiquitous treatments It is amazing what we don’t know about common and ubiquitous treatments

Contact Jean Slutsky Director, Center for Outcomes and Evidence Agency for Healthcare Research and Quality Or