1. The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no 115546, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu A novel framework for guiding evidence generation strategies to support evaluation of relative effectiveness of new drugs Páll Jónsson 1, Mike Chambers 2,Maciej Czachorowski 1, Chris Chinn 3, Rob Thwaites 4, and Sarah Garner 1 on behalf of Work Package 1 of GetReal 1 National Institute for Health and Care Excellence (NICE), United Kingdom, 2 MC Healthcare Evaluation, United Kingdom, 3 Sanofi, United Kingdom, 4 Takeda, United Kingdom

2. The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no 115546, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu The Efficacy-Effectiveness Gap Clinical trialHealth system Gap? Study type/ sources Population RCTs Observational studies Pragmatic trials Primary care and hospital records (EHRs) Healthcare databases, etc… Type of outcome Efficacy Effectiveness  Internal validity  External validity  Internal validity  External validity Design features ‘Real-World Data’ YesNo

3. The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no 115546, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu Increasing Focus on Comparative Effectiveness “Effectiveness challenges” Require evidence of benefit of treatment compared with standard of care in own jurisdiction. Earlier understanding of effectiveness of new products. Better targeting to patients who will gain the greatest benefits. Increasing interest. Potential for improving public health if efficacy- effectiveness gap identified. *Illustrative examples PopulationInterventionComparatorOutcome Study population not consistent with the population for which reimbursement is being sought Uncertainty of treatment effects in subgroups based on pivotal trial populations Study medication schedule (dose, dose titration/ escalation, frequency, route of admin, monitoring) inconsistent with routine practice Effectiveness likely impacted by adherence (of intervention and/or comparators) in routine practice Study comparator(s) do not include current standard of care (SOC) for the reimbursement population Indirect comparison vs SOC is uncertain due to a small number of trials from which to form a valid network Primary Study outcomes of limited interest from reimbursement perspective, Study underpowered to deliver meaningful results for outcomes of interest (e.g. HRQoL)

4. The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no 115546, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu IMI GetReal Project Aim To develop a common understanding of the acceptability and usefulness of innovative development programmes which use real- world evidence (RWE) to estimate the effectiveness of new medicines Focus Use of RWE in an early setting, prior to marketing authorisation or reimbursement Overall vision To provide pharma R&D with guidance on the most appropriate ways for developing evidence of relative effectiveness For healthcare decision makers to have relevant evidence to assess relative effectiveness of new drugs when used in standard practice

5. The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no 115546, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu Example GetReal Outputs Original research Drivers of effectiveness Analytical methods Prediction models Methodological guidance Tools : Software Checklists & templates Design options for pragmatic clinical trials Case studies Retrospective analyses of relative effectiveness issues Disease area specific issues Stakeholder views Methods Detection of bias Adjustment of bias Aggregate RWD in NMAs Individual patent RWD in NMAs Summaries Study types Sources of data Methods Literature reviews *Illustrative examples – not a complete list of GetReal outputs

6. The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no 115546, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu Real-World Evidence Framework Where possible we provide stakeholders’ views on RWE options, obtained through: Stakeholder workshops Case studies Consultation of GetReal output Outlining development strategies that provide evidence of relative effectiveness Using an agreed glossary of different types of study designs that considers study attributes and suitability for different applications  Technical and practical challenges and resource implications from WP3  Stakeholder policy and perspectives with respect to alternative study designs  The impact of the inclusion of alternative study decisions on the decision-making processes of industry, regulators and HTA agencies Draws from outputs and learnings, including: How to characterise the efficacy-effectiveness gap Guidance on methods for evidence synthesis Links to tools developed by the consortium Guidance on how to overcome technical and practical challenges of implementing pragmatic trials Key categories -Study designs -Observational -Experimental with an increased pragmatic element (PCTs, enriched RCTs,…) -Sources of RWD -Current policies and perspectives -Identifying and mitigating bias -Analytic methods -Using RWE to apply trial outcome to relevant populations -Using RWE to predict long-term outcomes (based on trial data) -Evidence synthesis, NMA with IPD and aggregate RWD -Informing trial designs

7. The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no 115546, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu Real-world Evidence Framework An educational resource to help find out more in general about the potential use of RWD to support the development of new medicines Two main functions: An expert resource to guide users to specific types of analyses or study designs relevant to RWE, many of which have been tested by the GetReal project Interactive material responding to specific user situations Background and educational material

8. The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no 115546, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

9. The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no 115546, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu P opulation I ntervention C omparator O utcome

10. The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no 115546, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

11. The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no 115546, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

12. The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no 115546, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu The GetReal Real-World Evidence Framework: A Resource for Everyone! A shared platform addressing the inclusion of alternative study designs in medicine development strategies Reflects stakeholder perspectives as far as possible A comprehensive resource regarding alternative evidence development pathways An index of study designs, data sources and policies relating to real-world evidence Educational resource, clarifying potential effectiveness challenges and identifying potential RWE options Signposts to authoritative resources, tools and policies relevant to real-world evidence

13. The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no 115546, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu Follow us for updates! www.imi-getreal.eu @IMIGetReal