DIA Virtual Journal Club Statistical Considerations on the Evaluation of Imbalances of Adverse Events in Randomized Clinical Trials Discussant Susan Duke, Safety Statistics AbbVie 12 December 2016

Points I will cover from the paper Risk difference, confidence intervals, graphics Time to first event Volcano plot Dropouts and their predictive capacity post approval Baseline risk factors and confounding Integrating trials for safety, considering multifactorial aspects eg, concomitant meds, subgroups, etc

Setup A bit of history Safety analysis – what matters to patients? Safety Statistics is emerging as a new sub-discipline of statistics

Some interesting history 1962 Thalidomide, a new sleeping pill, caused birth defects in thousands of babies born in western Europe. Dr. Frances Kelsey, FDA medical officer, keeps the drug off the U.S. market, arousing public support for stronger drug regulation. Kefauver-Harris Drug Amendments passed to ensure drug efficacy and greater drug safety. Drug manufacturers are required to prove to FDA the effectiveness of their products before marketing.

Some interesting history, continued 2005 Formation of the Drug Safety Board, consisting of FDA, NIH and VA staff. The Board will advise CDER on drug safety issues and work with the agency in communicating safety information to health professionals and patients. 2007 report, Institute of Medicine

The beginnings of Safety Statistics George Rochester, FDA statistician Served as FDA’s Expert Lead Statistician for Quantitative Safety in the Office of Biostatistics, developed the Quantitative Safety Program, renamed the Division of Biometrics VII. SPERT and Program Safety Analysis Plan Amy Xia, Amgen Brenda Crowe, Lilly

SPERT Recommendations A proactive approach: Create a Program Safety Analysis Plan early in development Planning of repeated, cumulative meta-analyses of the safety data 3-tier system for signal detection and analysis of adverse events Pre-specified hypothesis testing – AESIs Signal detection of common events Descriptive analysis (rare AESIs & others) Sponsors review aggregated safety data on a regular and ongoing basis throughout the development program, rather than waiting until the time of submission

Regulatory Motivation: CIOMS Working Groups on Safety CIOMS WG Descriptions Resulting Regulatory Guidance VI Management of Safety Information from Clinical Trials (2005) IND Safety Reporting X Considerations for applying good meta-analysis practices to clinical safety data within the biopharmaceutical regulatory process (2016)

Setup A bit of history Safety analysis – what matters to patients? Safety Statistics is emerging as a new sub-discipline of statistics

What matters to patients? Both efficacy and safety matter It may be argued that safety matters more Do no harm

How is analysis of safety different from that of efficacy? Scientific engagement Tools to analyze Process and data architecture needs The 4 Pillars of Safety Statistics From: ASA Safety Monitoring Working Group Thought Leader Interviews

1. Scientific engagement Safety events/questions often change as drug exposure increases. Efficacy questions are much less dynamic – the science of efficacy can be planned for in a way that the science of drug safety cannot The interchange between clinician and statistician to understand the science is vital to understanding how best to analyze Drug Safety and Statistics groups need each other to do this well This requires mutual respect, trust, frequent interaction Tools of hypothesis testing & power that are the backbone of efficacy are less applicable for safety

2. Tools to analyze safety Generally not in a hypothesis testing situation with safety data Study is powered for efficacy. Safety comes along for the ride What does a non-significant p-value mean in this setting? Underpowered or no association? What does a p-value for descriptive purposes mean? How does it relate to clinical relevance? False positives & negatives, sensitivity, specificity may be more relevant Risk difference, confidence intervals, graphics

2. Tools to analyze safety Risk difference, confidence intervals, graphics Graphics, well-designed to answer the question, are rigorous statistical evaluations too From Jiang and Xia, 2015. Quantitative Evaluation of Safety in Drug Development

2. Tools to analyze safety From the paper being discussed, Use of distributions of time to first event to better understand risks Weibull, generalized gamma shape parameters help to better understand whether these may be constant, increasing or decreasing risks

2. Tools to analyze safety In one picture, the entire story is clearly and transparently described Volcano plot From Jiang and Xia, 2015. Quantitative Evaluation of Safety in Drug Development

2. Tools to analyze safety Graphs can elucidate important statistical properties about safety Subgroup effects aid in identifying We need to clearly understand risk mitigation outliers. When are point estimates relevant? From Jiang and Xia, 2015. Quantitative Evaluation of Safety in Drug Development From Duke, et al, 2015. Seeing is Believing

2. Tools to analyze safety Apply statistical rigor to graphs just as we do to methods From Duke, et al, 2015. Seeing is Believing

3a. Well-considered process – dynamic and static evaluation Static evaluations happen a few times with a clearly defined end time. Dynamic evaluations occur repeatedly as information accrues. Control of type I error can be an important consideration in both. Static Evaluation Safety evaluation during pre-marketing at pre-determined timepoint, e.g., end of clinical trial, end of phase 2, aggregated data for regulatory filing Evaluations at pre-specified time in observational studies, claims data, pharmacovigilance studies Dynamic Evaluation Sequential safety monitoring of cumulative data over time for known or pre-specified adverse events of interest Evaluation after each event, or at successive duration of time intervals Dynamic evaluation includes a “process” of detecting an unknown issue and triggering closer monitoring Blinded review– companies are actively working on this methodology. From Gordon, et al, Quantitative Sciences for Safety Monitoring in Clinical Development. Deming conference, Dec 6, 2016

3b. Intelligent data architecture Nimble process/infrastructure at end of study is good business practice Nimble infrastructure for on-going safety monitoring is needed for appropriate safety patient management

Setup A bit of history Safety analysis – what matters to patients? Safety Statistics is emerging as a new sub-discipline of statistics A few thoughts to consider… Dropouts and their predictive capacity post approval Baseline risk factors and confounding Integrating trials for safety, considering multifactorial aspects eg, concomitant meds, subgroups, etc

How is analysis of safety different from that of efficacy? Scientific engagement Tools to analyze Process and data architecture needs The 4 Pillars of Safety Statistics From: ASA Safety Monitoring Working Group Thought Leader Interviews

Two pillars statisticians don’t necessarily consider If safety statisticians don’t play a role in guiding smart process and data architecture for nimble safety monitoring, what functional role will provide the steer?

Multifactorial problems Concomitant medications, subgroups will benefit from better methods, possibly network visualizations such as this one for rotavirus vaccines Integrating trials for safety, considering multifactorial aspects eg, concomitant meds, subgroups, etc From Duke, et al, 2015. Seeing is Believing

Role of meta-analysis in assessment of clinical safety data - Dropouts and their predictive capacity post approval - Baseline risk factors and confounding - Integrating trials for safety, considering multifactorial aspects eg, concomitant meds, subgroups, etc Time of Approval Information about benefit Premarket Phases Post Market Real World use B-R Balance B-R Balance Information about risk Meta Analysis Meta Analysis From CIOMS X, 2016. Described in Gordon, et al, Quantitative Sciences for Safety Monitoring in Clinical Development. Deming conference, Dec 6, 2016

References H. Ma, C Ke, Q Jiang, S. Snapinn, 2015. Statistical considerations on the evaluation of imbalances of adverse events in randomized clinical trials. Therapeutic Innovation & Regulatory Science 49(6): 957-965. M. Colopy, S. Duke, G. Ball, R. Gordon, F. Ahmad, Q. Jiang, W. Wang, and W. Wang. 2016. ASA Biopharm’s Safety Monitoring Work Group: Survey of statisticians, thought leaders, and regulatory guidance. In JSM Proceedings, Biopharmaceutical Section. Alexandria, VA: American Statistical Association. pp 3030-3042. Qi Jiang, Amy Xia (eds). 2015. Quantitative evaluation of safety in drug development: design, analysis and reporting. Chapman & Hall/biostatistics series. Chapter 11, Safety Graphics. (SP Duke, Q Jiang, L Huang, M Banach, M Cherny). SP Duke, F Bancken, B Crowe, M Soukup, T Botsis, R Forshee. 2015. Seeing is believing: Good graphic design principles for medical research. Statistics in Medicine 34(22):3040-3059. R. Gordon, J. Li, G. Ball, W. Wang. Dec 6, 2016. Short course: Quantitative Sciences for Safety Monitoring in Clinical Development. 72nd Annual Deming Conference on Applied Statistics. CIOMS X. 2016. Evidence Synthesis and Meta-Analysis for Drug Safety: Report of CIOMS Working Group X. Council for International Organizations of Medical Sciences (CIOMS).