1. Chrissie Fletcher, Amgen Ltd on behalf of IMI GetReal
IMI GetReal - using real world evidence in R&D and healthcare decision making
Chrissie Fletcher, Amgen Ltd
on behalf of IMI GetReal

2. Disclaimer
The views expressed herein represent those of the presenter and do not necessarily represent the views or practices of Amgen or the views of the general Pharmaceutical Industry.

5. What is effectiveness
Effectiveness may be defined as the impact of drug efficacy, when all “interactions” are at play
Interaction being used in the broad sense of “interaction” and “effect modification”
3 levels of interaction are considered
The real/actual use of drug
The patient/disease-related characteristics
The healthcare system-related characteristics

6. What is effectiveness research
The purpose of Effectiveness Research is to assess these “interactions”:
Which ones are “universal”
How do they distribute locally
What is the magnitude of their impact
What is the mechanism of Action
This covers the entire span of epidemiologic and public health research methods
‘Inteactions’
‘Interactions’

7. Increasing Focus on Comparative Effectiveness
Earlier understanding of effectiveness of new products. Better targeting to patients who will gain the greatest benefits.
Increasing interest. Potential for improving public health if efficacy-effectiveness gap identified.
Require evidence of benefit of treatment compared with standard of care in own jurisdiction.
“Effectiveness challenges”
Population
Intervention
Comparator
Outcome
Study population not consistent with the population for which reimbursement is being sought
Uncertainty of treatment effects in subgroups based on pivotal trial populations
Primary Study outcomes of limited interest from reimbursement perspective,
Study underpowered to deliver meaningful results for outcomes of interest (e.g. HRQoL)
Study comparator(s) do not include current standard of care (SOC) for the reimbursement population
Indirect comparison vs SOC is uncertain due to a small number of trials from which to form a valid network
Study medication schedule (dose, dose titration/ escalation, frequency, route of admin, monitoring) inconsistent with routine practice
Effectiveness likely impacted by adherence (of intervention and/or comparators) in routine practice
*Illustrative examples

8. X X Why should I generate real-world evidence?
Prior to launch
Focus on analyses for submissions & phase IV commitments
Prior to phase 2b
Develop overall evidence plan
Prior to phase 3
Detailed plan for RCTS, other studies and analyses
Why should I generate real-world evidence?
Is there a risk for an efficacy-effectiveness gap?
Is there a compelling need to generate evidence of effectiveness, over and above RCTs for registration?
How could I understand the issue?
How could I understand the issue?
Identification of drivers of effectiveness: literature review, experts insight, data analyses
Population
HTA CASE HISTORIES
Intervention
NMA
modelling techniques X X
SCIENTIFIC ADVICE
Comparator
Qualitative research, Patient insights
COMMERCIAL FORECAST
Outcome

9. Generation of evidence on effectiveness: options
What should I do?
Methodological options for an integrated “effectiveness evidence generation plan” (design parameters and analyses)
Prior to launch
Focus on analyses for submissions & phase IV commitments
Prior to phase 2b
Develop overall evidence plan
Prior to phase 3
Detailed plan for RCTS, other studies and analyses
How could I address the issue?
Generation of evidence on effectiveness: options
Phase 3 RCT
Can I use enriched RCT design, to improve the heterogeneity in population and gain knowledge on effectiveness?
Pragmatic trial
Shall we plan for PCT?
Which aspect of PICO should be more pragmatic?
Which are the statistical challenges
Post-launch observational study
Is there a risk of channelling bias ?
How to correct for this risk?
NMA
modelling techniques

10. of study design and analytical tools, with options and recommendations
TOOLBOX
of study design and analytical tools, with options and recommendations
Which pre-authorization RCTs actually explored effectiveness and how?
Which design parameters and analytical tools are available for pragmatic trials?
A methods to improve heterogeneity of population in RCTs (The “enriched RCT” design)
ANALYTICAL APPROACH
Identifying drivers of effectiveness
Best methods to adjust for indication bias
Development of methods to identify a risk for channelling bias using 3 case studies
How is the “efficacy-to-effectiveness gap” understood and conceptualized? The concept of Drivers of Effectiveness
Development of methods to identify drivers of effectiveness using 3 case studies
Mapping-out existing design parameters and analytical tools
Reviewing 60 authorization dossiers and HTA reviews (HAS), to understand the importance of providing evidence on effectiveness before launch and explore how to improve in pre-registration studies
Improving analytical tools for the assessment of effectiveness through Pragmatic Trials or post- authorization studies
Implementing knowledge on drivers of effectiveness
in RCTs
Dealing with imbalance in selection bias in TwiCS trial designs
Dealing with heterogeneity in the comparator arm
STATISTICAL APPROACH
Mapping-out the rationale for effectiveness evidence generation
Pragmatic trials
Post-authorization observational studies

11. Why pragmatic trials?
Real World Evidence (RWE) on relative effectiveness needed to guide physicians/policy makers in their decisions
Traditional RCTs and observational studies have limitations in providing RWE on relative effectiveness
Pragmatic trials combine the strength of RCTs ( addresses confounding through randomization), while generating results that are generalizable to the real world
BUT: they have to be well designed and executed
Calvert et al. J Clin Epidemiol 2011, Hemkens LG, BMJ 2016;

12. PRECIS-2 Pragmatic Trials on a continuum
Design trials fit for purpose
Focus on applicability of a trial
No focus on validity of the trial
To be used by trial design team
Makes judgements explicit
Little guidance on impact of specific design choices & challenges of pragmatic trial conduct
*PRagmatic-Explanatory Continuum Indicator Summary 2 (PRECIS-2) wheel. (Loudon K et al. BMJ 2015;350:h2147)

13. Minimize bias/variability:
The goal of PCTs
Minimize bias/variability:
Use preference design?
Blinding outcome assessment?
Use ‘objective outcomes’/training/standardizing?
Use realistic/flexible treatment strategies?
Maximize Generalizability/Feasibility:
Randomization at cluster level?
Select real life sites/settings?,
Use realistic/flexible treatment strategies?,
Outcomes as in practice?,
Integrate data collection with care systems?,
Minimize ‘Hawthorne effect’?
Discuss design in early phase with all stakeholders?

14. http://www. imi-getreal

17. GetReal Outputs Methods Original research Tools Summaries Case studies
Detection of bias
Adjustment of bias
Aggregate RWD in NMAs
Individual patent RWD in NMAs
Original research
Drivers of effectiveness
Analytical methods
Prediction models
Methodological guidance
Tools
Software
Checklists & templates
Design options for pragmatic clinical trials
Summaries
Study types
Sources of data
Methods
Literature reviews
Case studies
Retrospective analyses of relative effectiveness issues
Disease area specific issues
Stakeholder views
*Illustrative examples – not a complete list of GetReal outputs

19. What the navigator provides

24. ADDIS – GetReal software platform24

26. ADDIS – web based user interface26

27. The GetReal Real-World Evidence Framework: A Resource for Everyone!
A shared platform addressing the inclusion of alternative study designs in medicine development strategies Reflects stakeholder perspectives as far as possible A comprehensive resource regarding alternative evidence development pathways
An index of study designs, data sources and policies relating to real-world evidence
Educational resource, clarifying potential effectiveness challenges and identifying potential RWE options
Signposts to authoritative resources, tools and policies relevant to real-world evidence