1. Donna A. Messner, PhD Research Director Center for Medical Technology Policy

2.  Simplest definition (AHRQ) ◦ Core question for CER is: which treatment works best, for whom, and under what circumstances?  CER “Hypothesis” ◦ Goal of CER is to inform decision-makers (patients, healthcare consumers, clinicians, payers, policy makers). Therefore, these stakeholders should have greater influence in guiding the activities of the clinical research enterprise.

3. Researchers Decision makers 3

4. Basis for medical and health policy decisions… SR Tunis, DB Stryer, Carolyn M Clancy, JAMA 2003; 290(12): 1624-1632 “…widespread gaps in evidence-based knowledge suggest that systematic flaws exist in the production of scientific evidence...” Decision makers “interested in using high-quality evidence to support clinical and health policy choices... [but] the quality of available scientific evidence is often found to be inadequate.

5. Although RCTs provide “essential, high-quality evidence about the benefits and harms of medical interventions, many such trials have limited relevance to clinical practice.”

6. Operative question Will intervention work under narrowly prescribed, experimental conditions? Design Purpose Maximize chance of seeing treatment effect in some group of patients Design Choice Strict limits on population studied, treatment setting, follow-up, etc. Uses of Output FDA Marketing Clearance, Phase IV, efficacy for additional indications Efficacy Operative question Will intervention work under real- world, clinical conditions? Design Purpose Inform specific clinical decision Design Choice Use representative population, typical treatment setting and follow-up, etc. Uses of Output Clinician, Patient, Payer, Policy decision-making Effectiveness Need larger study populations to offset more liberal design criteria… Very efficient design

7. Uses of Efficacy Output FDA Marketing Clearance, Phase IV, efficacy for additional indications Uses of Effectiveness Output Clinician, Patient, Payer, Policy decision-making Who is likely to pay for each type of research?

8. Increased data availability from Payers 2003 2009 2010 2013 Medicare Modernization Act (AHRQ Effective Health Care Program) ARRA (FCC-CER; IOM Prioritization) BCBS TEC (1985) CMS Coverage with Evidence Development (2006) Veterans Affairs, NIH conduct of CER PCORI ECRI Hayes CMTP ICER (2008) Increasing Academic Private Centers for CER Implementation DERP (2001) Others? Public Investment Private Investment Adapted from The Lewin Group

9. *ARRA= $1.1 Billion ($400M NIH; $400M HHS; $300 ARHQ) **PCORI Trust Fund = $1/Covered Life in 2013; $2/Covered Life 2014 with enhancement adjusted for inflation

10. Definitions of CER by… Congressional Budget Office IOM Roundtable on EBM American College of Physicians IOM Committee on Reviewing Evidence to Identify Highly Effective Clinical Services Medicare Payment Advisory Commission Agency for Healthcare Research and Quality (AHRQ) American Recovery and Reinvestment Act (ARRA) Patient Protection and Affordable Care Act (ACA)

11. 1. CER has the objective of informing a specific clinical decision from the patient perspective or a health policy decision from the population perspective 2. CER compares at least two alternative interventions, each with the potential to be “best practice” 3. CER describes results at the population and subgroup levels *IOM (Institute of Medicine). 2009. Initial National Priorities for Comparative Effectiveness Research. Washington, DC: The National Academies Press.

12. 4. CER measures outcomes – both benefits and harms – that are important to patients 5. CER employs methods and data sources appropriate for the decision of interest 6. CER is conducted in settings that are similar to those in which the intervention will be used in practice *IOM (Institute of Medicine). 2009. Initial National Priorities for Comparative Effectiveness Research. Washington, DC: The National Academies Press.

13. IOM National Priorities Committee Definition: The generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.

14. 2009 ARRA Establishes Federal Coordinating Council for Comparative Effectiveness Research Describes CER as “research that compares the clinical outcomes, effectiveness, and appropriateness of items, services, and procedures that are used to prevent, diagnose, or treat diseases, disorders, and other health conditions” 2010 ACA Abolishes Federal Coordinating Council Establishes non-profit Patient-Centered Outcomes Research Institute (PCORI) Shift in language, but CER still defined as research “comparing health outcomes and the clinical effectiveness, risks, and benefits of 2 or more medical treatments, services, and items…” 2011 PCORI defines “patient-centered outcomes research” (PCOR)

15. Research to answer patient questions:  “Given my personal characteristics, conditions and preferences, what should I expect will happen to me?”  “What are my options and what are the benefits and harms of those options?”  “What can I do to improve the outcomes that are most important to me?”  “How can the health care system improve my chances of achieving the outcomes I prefer?” http://www.pcori.org/patient-centered-outcomes-research/

16.  Assesses the benefits and harms of preventive, diagnostic, therapeutic, or health delivery system interventions to inform decision making, highlighting comparisons and outcomes that matter to people;  Is inclusive of an individual’s preferences, autonomy and needs, focusing on outcomes that people notice and care about such as survival, function, symptoms, and health-related quality of life; ◦ Incorporates a wide variety of settings and diversity of participants to address individual differences and barriers to implementation and dissemination; and  Investigates (or may investigate) optimizing outcomes while addressing burden to individuals, resources, and other stakeholder perspectives. http://www.pcori.org/patient-centered-outcomes-research/

17. CERPCOR Inform specific clinical decision from patient or policy perspective “to inform decision-making” Compare two interventions“…highlighting comparisons…” Describe outcomes for populations and subgroups “address individual differences” Measure benefits and harms important to patients “assess benefits and harms... that matter to people” Appropriate methods and data sources [Not in def…but PCORI addresses in other ways] Use real-world settingsIncorporate a “wide variety of settings” CER “hypothesis”Integrates stakeholder perspective Shift in emphasis…language more individualistic and patient-focused

18.  Should it be? ◦ Often stated fear: cost pressures will result in healthcare “rationing” and stifling of innovation  Some considerations: ◦ Innovative process is incremental, not revolutionary  Most devices cleared under 510(k) ◦ “New” is not necessarily better. ◦ Even technically revolutionary products may offer only modest benefits to patients. ◦ When cost is factored in CER, goal is not lower cost but better value – higher cost is justified with superior patient outcomes. ◦ Rationing: should we base on benefit to patient or ability to pay?

19.  Prioritization/selection of research questions  Picking the best comparator ◦ Typically want to use “standard of care” or “current best practice” ◦ What if many approaches used, no consensus on best practice?  Balancing benefits and harms ◦ As much methodological as a social judgment  Other methodological challenges Stakeholder preferences inform many of these issues

21.  Est. 2010 Patient Protection and Affordable Care Act  “Independent, non-profit organization created to conduct research to provide information about the best available evidence to help patients and their health care providers make more informed decisions”  Pilot grants: About 40 two-year projects, $20 million; funding to be announced in May 2012 ◦ Identifying evidence gaps; priority setting ◦ Stakeholder engagement methods ◦ New measurement tools for patient-centered outcomes ◦ Other basic methods and tools for PCOR  Internally, methodology committee developing “translation framework” for doing CER/PCOR  ◦ Outcome/NPC/CMTP parallel effort

22. Source: PCORI Draft National Priorities for Research and Research Agenda, v.1, p. 14

23.  Funded through Agency for Healthcare Research and Quality (AHRQ) grants since 2000  Networks of primary care clinicians and practices working to answer community-based health care questions and translate research findings into practice  Recruit typical patients from real-world practice settings receiving routine standards of care  “Learning health care system”  Presents challenges for informed consent, IRB review, data management ◦ CMTP and U of Alabama CERT informed consent project

24. http://pbrn.ahrq.gov Over 130 PBRNs funded through four major competitive AHRQ grant programs since 2000

25.  Recent Funding Opportunity Announcements (FOAs) for PCT demos and “collaboratory” projects  Collaboratory program: ◦ Aim: “enable the participation of many health care systems in clinical research” because this type of research is “essential to strengthen the relevance of research results to ‘real world’ health practice.”  Demonstration projects: ◦ pragmatic trials “primarily designed to determine the effects of an intervention under the usual conditions in which it will be applied” ◦ Projects should provide innovative approaches to overcoming barriers to doing pragmatic clinical trials in networks of health care systems ◦ List of example “high-impact” study topics include many of the IOM top priorities for CER

26.  Partnership in Applied Comparative Effectiveness Science  Advance PCOR by leveraging heretofore unused stores of FDA data  Buccaneer, Lewin Group, Johns Hopkins University, and CMTP collaborating with FDA ◦ set research priorities and gather stakeholder input ◦ create secure platforms for FDA data usage ◦ work to standardize FDA data sets for analysis ◦ develop statistical methods for subgroup analysis  Will allow FDA to understand which interventions are most effective for patients under specific circumstances

27.  Increasing recognition that CER will become a necessary part of development programs ◦ FDA approval no longer the end game  CMTP partnered with the Office of Health Economics (UK) on response to RFI to: ◦ assess the future CER landscape ◦ anticipate CER requirements pharma companies are likely to have to meet ◦ describe at what point in the product development process pharma should design for CER ◦ publish findings on behalf of consortium of participating companies

28.  Original focus was on improving information for patients, clinicians, payers and policy makers ◦ Better decisions in context of anticipated payment and delivery system reforms  Emphasis now shifted to primary emphasis on information needs of patients, especially “patient-centered outcomes”  But health policy forces behind original interest in CER and creation of PCORI have not vanished: ◦ Innovation with real impact; reliable evidence for informed, independent decision-making; enhance care and reduce harms; use resources more effectively

29. Thank you