Key Elements in the Design of Pragmatic Randomized Clinical Trials Valentina Bayer, Ph.D. Boehringer Ingelheim Pharmaceuticals, Inc. July 29, 2019 JSM 2019 invited papers HPSS-sponsored session Pragmatic randomized clinical trials: challenges and impact on clinical practice and health policies

Disclaimer This material represents the author’s perspective and does not necessarily reflect that of Boehringer Ingelheim Pharmaceuticals, Inc. The case studies of pragmatic clinical trials presented here are from published literature.

Selected Examples of Pragmatic Randomized Clinical Trials ClinicalTrials.gov search: 424 “pragmatic randomized”(as of July 7, 2019)

Randomized Clinical Trials Gold standard to determine existence of a causal relationship between intervention and effect Broad categorization based on purpose: Explanatory Efficacy: “Can the intervention work under ideal conditions?” Hypothesis testing Pragmatic Effectiveness: “Does the intervention work under usual conditions?” Help clinical practice choose between healthcare options

Relationship between Explanatory and Pragmatic Trials The wide base of the pyramid depicts the relatively higher proportion of explanatory trials. Patsopoulos N., “A pragmatic view on pragmatic trials”, Dialogues Clin Neurosci. 2011

Pragmatic Randomized Clinical Trial (PrCT) Pragmatic trials are designed to measure effectiveness. i.e., whether an intervention works when used in regular clinical practice. “A pragmatic trial can be broadly defined as a randomized controlled trial whose purpose is to inform decisions about practice.” M. Zwarenstein, CONSORT statement extension for pragmatic trials, 2008 “A study comparing several health interventions among a randomized, diverse population representing clinical practice, and measuring a broad range of health outcomes.” IMI GetReal Glossary, adapted from Schwartz, 1967, Tunis, 2003 & Roland, 1998

Key Design Elements of PrCT Randomization Population Setting Comparators Endpoints

PrCT Example: Airwise Study Dual COPD: chronic obstructive pulmonary disease McBryan D. et al., “Assessment In a Real-World Setting of the Effect of Inhaled Steroid-Based Triple Therapy Versus the Combination of Tiotropium and Olodaterol on Reducing Chronic Obstructive Pulmonary Disease Exacerbations: AIRWISE Study Design”, AMCP Nexus 2017 ClinicalTrials.gov Identifier: NCT03265145

Randomization Generates comparable interventions groups Avoids systematic differences between groups with respect to known or unknown baseline variables that could affect outcome (ICH E10) Ensures a solid statistical foundation for comparing interventions Cluster randomization is often employed Groups, rather than individuals, are randomly allocated to treatment arms Operationally simpler, but leads to larger sample size Airwise: Patient-level randomization 2 treatment arms, parallel group Open label ICHE9: Preservation of the initial randomisation in analysis is important in preventing bias and in providing a secure foundation for statistical tests

Pro Con Population Patients who receive the treatment in usual care Minimal Inclusion/Exclusion criteria Airwise: COPD diagnosis as defined by the study physician Physician determination that patient is not controlled on current pharmacotherapy Age >= 40 years No contraindication to any of the study drugs No asthma, pregnancy, nursing Heterogenous, real-world population Generalizability (external validity) May include vulnerable/special population Pro Smaller treatment effects Higher variability Larger sample size Con All patients who may receive drug in clinical practice should be considered eligible Simpler enrollment process, for example simplified informed consent Relate to objective of PrCT – pre-approval vs post

Setting Usual care setting (primary care instead of research sites) Recruitment challenge: less experience, fewer resources Fewer scheduled visits Mimic the real-world interaction of patients and physicians Impact on adherence, interim analysis Medication supply Post-launch: pharmacy prescription instead of sponsor supplying it Difference in co-pay: full reimbursement or co-pay equalization Identify patients Point of care (recruit during routine care visit) Select patients from data bases, EHR (mail-out, inform during visit) Ads or social media Site feasibility to gauge interest of physicians Airwise: Community-based physician sites Patients recruited for enrollment as part of their routine care interaction with their physician Only two required study visits Study drugs obtained from pharmacies Equalized out-of-pocket expenses (i.e., co-pays, co-insurance, deductibles) between the treatment arms Full reimbursement or co-pay equalization to prevent switching and to mitigate selection bias where patients won’t participate if one arm is more expensive

Comparators and Endpoints Airwise: Comparator(s) Real world treatment(s) instead of placebo Open label Endpoints: clinically-relevant to decision makers Stakeholders: patients, physicians, regulatory, payers Avoid surrogate endpoints Health care resource utilization (HCRU) and costs Dual vs. Triple therapy Open label Primary endpoint: time to first moderate/severe COPD exacerbation Secondary and other endpoints: annual rate of exacerbations, HCRU and costs Pre- vs post-approval

Data Sources Data may include: Airwise: Data may include: Study-specific data: collected directly from the patients into case report forms interferes with routine clinical practice Routinely collected data extracted from existing data sources electronic health records (EHR), registries, insurance claims Hybrid data Consistency of data from existing data sources Achievable if there is a unified EHR (e.g., national health system in UK, VA in USA) Different standards and requirements in databases Case report forms Claims data

Open Label Most PrCT are not blinded, in line with clinical practice Bias: reporting non-serious AEs, treatment discontinuation and PROs May help enrollment Lower internal validity Loss to follow-up: differential drop-out may not be avoided To reduce open label bias: Objective endpoints instead of surrogates Blinded assessment of endpoints Keep the analysis team blinded

Impact of PrCTs on Clinical Practice Simpler trial design with increased generalizability PrCT results are relevant to real-world populations Provide real-world evidence (RWE) through comparative effectiveness FDA draft guidance on RWD and RWE for submission Guide decision makers on choosing treatments for clinical practice “Do pragmatic trials that clearly answer meaningful questions to patients and providers” RWE is the evidence derived form the analysis of RWD Relevant submissions may include RWE used to support study objectives, such as : IND submissions for randomized clinical trials that use RWD to capture clinical outcomes or safety data, including pragmatic and large simple trials Califf R., “FDA and Pragmatic Trials” presentation , Collaboratory Ground Rounds 2017

PRECIS-2 Tool (https://www.precis-2.org) There is a continuum between explanatory and pragmatic trials PRECIS-2 tool helps trial team make design decisions consistent with the intended purpose 9 domains used to score how pragmatic or explanatory a study is 1. Very explanatory 2. Rather explanatory 3. Equally pragmatic/explanatory 4. Rather pragmatic 5. Very pragmatic Recommend to score independently then discuss within team (report median) Pragmatic-explanatory continuum indicator summary Airwise scored 4 out of 5 on Recruitment and Organisation, and 5 everywhere else – but it was post-study design (The AIRWISE study design was retrospectively analyzed using the PRECIS-2 tool to assess the degree to which the study meets this definition of a pragmatic study.)

Challenges in PrCT Smaller treatment effect Higher variability Slower recruitment Less adherence Open label Variety of data sources Economic endpoints Variety of stakeholders

Value of PrCT Randomization provides a secure foundation for statistical tests Minimal inclusion/exclusion criteria lead to a real-world population Fewer scheduled visits mimic the real-world interaction of patients and physicians Simpler trial design with increased generalizability A variety of data sources such as case report forms, electronic health records and insurance claims Endpoints that extend beyond efficacy and safety to include HCRU and costs Provide real-world evidence through comparative effectiveness Results are relevant to patients, physicians, payers and health policy decision makers

Role of Statisticians in PrCT “Statisticians are well placed to critically examine research portfolio to identify gaps of knowledge” “They serve as critical mediators in transforming data into (real world) evidence” to inform decision making “Remind people about what a powerful tool randomization is” Statistical expertise Communication: collaborate well and speak up Califf R., “Pragmatic clinical trials: Emerging challenges and new roles for statisticians”, Clinical Trials 2016

Selected References Bakerly et al., “The Salford Lung Study protocol: a pragmatic, randomised phase III real-world effectiveness trial in chronic obstructive pulmonary disease”, Respiratory Research 2015, 16:101, 1-5 EMA 2013, “Highlights from the Workshop on methods for efficacy studies in the everyday practice” Califf R., “Pragmatic clinical trials: Emerging challenges and new roles for statisticians”, Clinical Trials 2016, 13(5):471‐477 Califf R. and Sugarman J., “Exploring the ethical and regulatory issues in pragmatic clinical trials”, Clinical Trials 2016, 12(5): 436-441 FDA draft guidance, “Submitting Documents Using Real-World Data and Real-World Evidence to FDA for Drugs and Biologics”, 2019 Ford I. and Norrie J., “Pragmatic Trials”, NEJM 2016, 375: 454-463 Hernan, M. and Robins J., “Per-protocol analyses of pragmatic trials”, NEJM 2017, 377:1391-1398 Loudon K. et al., “The PRECIS-2 tool: designing trials that are fit for purpose”, BMJ 2015, 350: 1-11 McBryan D. et al., “Assessment In a Real-World Setting of the Effect of Inhaled Steroid-Based Triple Therapy Versus the Combination of Tiotropium and Olodaterol on Reducing Chronic Obstructive Pulmonary Disease Exacerbations: AIRWISE Study Design”, AMCP Nexus 2017, Dallas, Texas Patsopoulos N., “A pragmatic view on pragmatic trials”, Dialogues Clin Neurosci. 2011, 13(2): 217–224 Schwartz D. and Lellouch J., “Explanatory and pragmatic attitudes in therapeutical trials”, J. Chronic Dis. 1967, 3:409-422 Sherman R. et al., “Real‐World Evidence - What Is It and What Can It Tell Us?”, NEJM 2016, 375:2293‐2297 Tunis S. et al., “Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy”, JAMA 2003, 290: 1624-1632 Woertman W. et al., “Stepped wedge designs could reduce the required sample size in cluster randomized trials”, J. Clin. Epi. 2013, 66: 752-758 Yusuf S. et al.” Why do we need some large, simple randomized trials?”, Stat Med. 1984, 3:409-422 Zuidgest M. et al., “Series: Pragmatic trials and real world evidence: Paper 1. Introduction ”, J. Clin. Epi. 2017, 88: 7-13 Zwarenstein M. et al, “Improving the reporting of pragmatic trials: an extension of the CONSORT statement”, BMJ 2008, 337:1-8

Thank you