1. Key Elements in the Design of Pragmatic Randomized Clinical Trials
Valentina Bayer, Ph.D.
Boehringer Ingelheim Pharmaceuticals, Inc.
July 29, 2019
JSM 2019 invited papers HPSS-sponsored session
Pragmatic randomized clinical trials: challenges and impact on clinical practice and health policies

2. Disclaimer
This material represents the author’s perspective and does not necessarily reflect that of Boehringer Ingelheim Pharmaceuticals, Inc.
The case studies of pragmatic clinical trials presented here are from published literature.

3. Selected Examples of Pragmatic Randomized Clinical Trials
ClinicalTrials.gov search: 424 “pragmatic randomized”(as of July 7, 2019)

4. Randomized Clinical Trials
Gold standard to determine existence of a causal relationship between intervention and effect
Broad categorization based on purpose:
Explanatory
Efficacy: “Can the intervention work under ideal conditions?”
Hypothesis testing
Pragmatic
Effectiveness: “Does the intervention work under usual conditions?”
Help clinical practice choose between healthcare options

5. Relationship between Explanatory and Pragmatic Trials
The wide base of the pyramid depicts the relatively higher proportion of explanatory trials.
Patsopoulos N., “A pragmatic view on pragmatic trials”, Dialogues Clin Neurosci. 2011

6. Pragmatic Randomized Clinical Trial (PrCT)
Pragmatic trials are designed to measure effectiveness. i.e., whether an intervention works when used in regular clinical practice.
“A pragmatic trial can be broadly defined as a randomized controlled trial whose purpose is to inform decisions about practice.”
M. Zwarenstein, CONSORT statement extension for pragmatic trials, 2008
“A study comparing several health interventions among a randomized, diverse population representing clinical practice, and measuring a broad range of health outcomes.”
IMI GetReal Glossary, adapted from Schwartz, 1967, Tunis, 2003 & Roland, 1998

7. Key Design Elements of PrCT
Randomization
Population
Setting
Comparators
Endpoints

8. PrCT Example: Airwise Study
Dual
COPD: chronic obstructive pulmonary disease
McBryan D. et al., “Assessment In a Real-World Setting of the Effect of Inhaled Steroid-Based Triple Therapy Versus the Combination of Tiotropium and Olodaterol on Reducing Chronic Obstructive Pulmonary Disease Exacerbations: AIRWISE Study Design”, AMCP Nexus 2017
ClinicalTrials.gov Identifier: NCT

9. Randomization Generates comparable interventions groups
Avoids systematic differences between groups with respect to known or unknown baseline variables that could affect outcome (ICH E10)
Ensures a solid statistical foundation for comparing interventions
Cluster randomization is often employed
Groups, rather than individuals, are randomly allocated to treatment arms
Operationally simpler, but leads to larger sample size
Airwise:
Patient-level randomization
2 treatment arms, parallel group
Open label
ICHE9: Preservation of the initial randomisation in analysis is important in preventing bias and in providing a secure foundation for statistical tests

10. Pro Con Population Patients who receive the treatment in usual care
Minimal Inclusion/Exclusion criteria
Airwise:
COPD diagnosis as defined by the study physician
Physician determination that patient is not controlled on current pharmacotherapy
Age >= 40 years
No contraindication to any of the study drugs
No asthma, pregnancy, nursing
Heterogenous, real-world population
Generalizability (external validity)
May include vulnerable/special population
Pro
Smaller treatment effects
Higher variability
Larger sample size
Con
All patients who may receive drug in clinical practice should be considered eligible
Simpler enrollment process, for example simplified informed consent
Relate to objective of PrCT – pre-approval vs post

11. Setting Usual care setting (primary care instead of research sites)
Recruitment challenge: less experience, fewer resources
Fewer scheduled visits
Mimic the real-world interaction of patients and physicians
Impact on adherence, interim analysis
Medication supply
Post-launch: pharmacy prescription instead of sponsor supplying it
Difference in co-pay: full reimbursement or co-pay equalization
Identify patients
Point of care (recruit during routine care visit)
Select patients from data bases, EHR (mail-out, inform during visit)
Ads or social media
Site feasibility to gauge interest of physicians
Airwise:
Community-based physician sites
Patients recruited for enrollment as part of their routine care interaction with their physician
Only two required study visits
Study drugs obtained from pharmacies
Equalized out-of-pocket expenses (i.e., co-pays, co-insurance, deductibles) between the treatment arms
Full reimbursement or co-pay equalization to prevent switching and to mitigate selection bias where patients won’t participate if one arm is more expensive

12. Comparators and Endpoints
Airwise:
Comparator(s)
Real world treatment(s) instead of placebo
Open label
Endpoints: clinically-relevant to decision makers
Stakeholders: patients, physicians, regulatory, payers
Avoid surrogate endpoints
Health care resource utilization (HCRU) and costs
Dual vs. Triple therapy
Open label
Primary endpoint: time to first moderate/severe COPD exacerbation
Secondary and other endpoints: annual rate of exacerbations, HCRU and costs
Pre- vs post-approval

13. Data Sources Data may include:
Airwise:
Data may include:
Study-specific data: collected directly from the patients into case report forms
interferes with routine clinical practice
Routinely collected data extracted from existing data sources
electronic health records (EHR), registries, insurance claims
Hybrid data
Consistency of data from existing data sources
Achievable if there is a unified EHR (e.g., national health system in UK, VA in USA)
Different standards and requirements in databases
Case report forms
Claims data

14. Open Label Most PrCT are not blinded, in line with clinical practice
Bias: reporting non-serious AEs, treatment discontinuation and PROs
May help enrollment
Lower internal validity
Loss to follow-up: differential drop-out may not be avoided
To reduce open label bias:
Objective endpoints instead of surrogates
Blinded assessment of endpoints
Keep the analysis team blinded

15. Impact of PrCTs on Clinical Practice
Simpler trial design with increased generalizability
PrCT results are relevant to real-world populations
Provide real-world evidence (RWE) through comparative effectiveness
FDA draft guidance on RWD and RWE for submission
Guide decision makers on choosing treatments for clinical practice
“Do pragmatic trials that clearly answer meaningful questions to patients and providers”
RWE is the evidence derived form the analysis of RWD
Relevant submissions may include RWE used to support study objectives, such as :
IND submissions for randomized clinical trials that use RWD to capture clinical outcomes or safety data, including pragmatic and large simple trials
Califf R., “FDA and Pragmatic Trials” presentation , Collaboratory Ground Rounds 2017

16. PRECIS-2 Tool (https://www.precis-2.org)
There is a continuum between explanatory and pragmatic trials
PRECIS-2 tool helps trial team make design decisions consistent with the intended purpose
9 domains used to score how pragmatic or explanatory a study is
1. Very explanatory
2. Rather explanatory
3. Equally pragmatic/explanatory
4. Rather pragmatic
5. Very pragmatic
Recommend to score independently then discuss within team (report median)
Pragmatic-explanatory continuum indicator summary
Airwise scored 4 out of 5 on Recruitment and Organisation, and 5 everywhere else – but it was post-study design
(The AIRWISE study design was retrospectively analyzed using the PRECIS-2 tool to assess the degree to which the study meets this definition of a pragmatic study.)

17. Challenges in PrCT Smaller treatment effect Higher variability
Slower recruitment
Less adherence
Open label
Variety of data sources
Economic endpoints
Variety of stakeholders

18. Value of PrCT
Randomization provides a secure foundation for statistical tests
Minimal inclusion/exclusion criteria lead to a real-world population
Fewer scheduled visits mimic the real-world interaction of patients and physicians
Simpler trial design with increased generalizability
A variety of data sources such as case report forms, electronic health records and insurance claims
Endpoints that extend beyond efficacy and safety to include HCRU and costs
Provide real-world evidence through comparative effectiveness
Results are relevant to patients, physicians, payers and health policy decision makers

19. Role of Statisticians in PrCT
“Statisticians are well placed to critically examine research portfolio to identify gaps of knowledge”
“They serve as critical mediators in transforming data into (real world) evidence” to inform decision making
“Remind people about what a powerful tool randomization is”
Statistical expertise
Communication: collaborate well and speak up
Califf R., “Pragmatic clinical trials: Emerging challenges and new roles for statisticians”, Clinical Trials 2016

20. Selected References
Bakerly et al., “The Salford Lung Study protocol: a pragmatic, randomised phase III real-world effectiveness trial in chronic obstructive pulmonary disease”, Respiratory Research 2015, 16:101, 1-5
EMA 2013, “Highlights from the Workshop on methods for efficacy studies in the everyday practice”
Califf R., “Pragmatic clinical trials: Emerging challenges and new roles for statisticians”, Clinical Trials 2016, 13(5):471‐477
Califf R. and Sugarman J., “Exploring the ethical and regulatory issues in pragmatic clinical trials”, Clinical Trials 2016, 12(5):
FDA draft guidance, “Submitting Documents Using Real-World Data and Real-World Evidence to FDA for Drugs and Biologics”, 2019
Ford I. and Norrie J., “Pragmatic Trials”, NEJM 2016, 375:
Hernan, M. and Robins J., “Per-protocol analyses of pragmatic trials”, NEJM 2017, 377:
Loudon K. et al., “The PRECIS-2 tool: designing trials that are fit for purpose”, BMJ 2015, 350: 1-11
McBryan D. et al., “Assessment In a Real-World Setting of the Effect of Inhaled Steroid-Based Triple Therapy Versus the Combination of Tiotropium and Olodaterol on Reducing Chronic Obstructive Pulmonary Disease Exacerbations: AIRWISE Study Design”, AMCP Nexus 2017, Dallas, Texas
Patsopoulos N., “A pragmatic view on pragmatic trials”, Dialogues Clin Neurosci. 2011, 13(2): 217–224
Schwartz D. and Lellouch J., “Explanatory and pragmatic attitudes in therapeutical trials”, J. Chronic Dis. 1967, 3:
Sherman R. et al., “Real‐World Evidence - What Is It and What Can It Tell Us?”, NEJM 2016, 375:2293‐2297
Tunis S. et al., “Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy”, JAMA 2003, 290:
Woertman W. et al., “Stepped wedge designs could reduce the required sample size in cluster randomized trials”, J. Clin. Epi. 2013, 66:
Yusuf S. et al.” Why do we need some large, simple randomized trials?”, Stat Med. 1984, 3:
Zuidgest M. et al., “Series: Pragmatic trials and real world evidence: Paper 1. Introduction ”, J. Clin. Epi. 2017, 88: 7-13
Zwarenstein M. et al, “Improving the reporting of pragmatic trials: an extension of the CONSORT statement”, BMJ 2008, 337:1-8

21. Thank you