1. Statistical Considerations on the Evaluation of Imbalances of Adverse Events in Randomized Clinical Trials
Haijun Ma, Chunlei Ke, Qi Jiang,
and Steven Snapinn
DIA Virtual Journal Club, December 12, 2016

2. Disclaimer
The views expressed herein represent those of the individual presenter and do not necessarily represent the views or practices of the presenter’s employer or any other party

3. Background
Safety monitoring and assessment are critical to establish the benefit:risk profile of an investigational drug and to protect patient safety.
Key elements for the establishment of safety profile in pre-market: the mechanism of action (MOA), class effects, animal model, toxicity studies, early safety and tolerability clinical studies in healthy volunteers and patients, safety data from moderate or large randomized controlled trials, etc.
Safety profile of a drug is further established and refined in post-market
Adverse events (AEs) data compose the main body of safety data in clinical trials.
An AE does not have to be drug related.
Medically important imbalances of AEs are signals of potential ADRs and are subject to further evaluation for treatment causality.
Statistical aspects for screening and evaluation of signals generated from imbalances of AEs in moderate or large RCTs

4. Randomized Clinical Trials for Safety Assessment
RCTs are often considered as of top quality in the pyramid of evidence-based medical research.
Safety data from clinical trials are of high quality
Randomization removing confounding
Concurrent control arm
Limitations of RCT for safety assessment
Low power for ADR detection
Large amount of endpoints to be screened/multiplicity
Study population not representative of patient population
Duration not long enough to assess long-term safety concerns

5. Statistical Methods for Identification of Medically Important Imbalances
Case review of severe, low frequency AEs
E.g. DILI, Stevens-Johnson syndrome
Event of Interest based on previous knowledge and/or evidence
Comparing aggregated data between treatment and control groups to assess causality
Event Search Algorithm
Statistical metrics
Graphical tools
Multiplicity adjustment

6. Event Search Algorithm
Standardized coding system: MedDRA
MedDRA hierarchy and level of granuality
SMQ and other search algorithms
Increasing specificity by incorporating other characteristics in the search
Severity, seriousness
Timing of onset
Co-occurrence of events
System Organ Class (SOC)
High Level Group Term (HLGT)
High Level Term (HLT)
Preferred Term (PT)
Lowest Level Term (LLT)
Standardised MedDRA Queries (SMQs)

7. Statistical Metrics Relative measures and absolute measures
RR=3 for an AE of 10% reference rate => AE in additional 20% treated patients;
RR=3 for an AE of 1% reference rate => AE in additional 2% patients
RD better reflects the safety impact to patients
A commonly seen practice for identification of ‘‘common’’ADR is based on RD or RR with a qualifier for risk size
at least a% in the treatment group and b% greater (at least x times greater) than that in the placebo group
these criteria could be sensitive to sample size
higher chance to observe more extreme estimates from smaller studies
Uncertainty and strength of evidence, e.g. CI, nominal p-values

8. Statistical Graphics
Useful for signal screening and data display, e.g. volcano plot, dot plot with CI, forest plot
Interactive graphics allow drilling down to individual subjects

9. Multiplicity Adjustment
Large amount of AEs to be screened
A phase 3 RCT with 1000 subjects per arm could have AEs coded to >1000 unique PTs, where 5% to 15% of them may have a frequency 1%.
High false discovery rate
Conventional correction for multiplicity would make a finding untenable
Different multiplicity adjustment methods have been proposed
Bayesian shrinkage
false discovery rate control techniques
Incorporating MedDRA hierarchy

10. Statistical Methods for Safety Signal Evaluation
Further investigation of potential signals to evaluate evidence for a causal relationship with the drug:
biological plausibility
nonclinical and clinical evidence
consistency across trials of the same or similar drugs
Evidence from RCTs is an important component of the signal evaluation
event characteristics
dose-response relationship
risk factors
other study design and conduction factors.
Medical knowledge of the AEs and drug’s MOA can guide the analysis and interpretation.

11. Adverse Event Characteristics
AE’s characteristics distinguish ADRs from AEs or further describe AEs and provide insight in risk management
time of onset, duration, recurrence, severity, seriousness, resolution, dose- response patterns, and cause of the event, etc.
Time of Event Onset
AEs related to underlying disease tend to occur randomly across the observation period, while those caused by drug exposure would display a temporal relationship
Acute AE vs AE with latency
A comparison of the onset time and risk change over time between treatment and control arms is informative for causality assessment
KM, parametric distributions
At-risk period differ by type of AE

12. Kaplan-Meier (KM), Weibull, and Generalized Gamma fits of Cardiac Failure AE in a long-term cohort
constant risk with a corresponding Weibull shape parameter estimate of (95% CI: 0.708, 1.463)

13. Recurrent Events
Multiple occurrences of an AE while subjects are on drug are more likely drug related
Analysis methods for multiple/repeated events
tabulation of percentage of subjects with 0, 1, 2, AEs during the follow-up
mean cumulative function
Person-time adjusted event rate as a single-number summary of event rate
Recurrent events models, e.g. Anderson-Gill model, PWP (Prentice-Williams- Peterson) model

14. Information from Other Safety Data
AEs directly reflect patients’ adverse experiences
Other safety data (eg, laboratory measurements, vital signs, use of concomitant medications) could provide important insight into the safety profile of a drug
Mean trend changes of biomarkers and/or outlying observations
box plot of changes in blood pressure can be examined for increasing trend as an indication of potential cardiovascular risk
Line plots of calcium can be compared across treatment groups to aid assessment of risk of hypocalcemia
Simultaneous elevation of aminotransferase and bilirubin helps identification of potential Hy’s law cases for assessment of drug induced liver injury.
Patient profile plots displays a patient’s safety data on one plot to facilitate review of individual cases

15. Dropout Rate and Pattern
Dropout due to AE is considered an important safety end point, which could be a clue to unexpected but important ADRs.
Tabulate reasons for dropout and compare between treatment groups
Dropout from study terminates AE data collection and creates a missing-data problem
exposure-adjusted analysis to adjust for shortened exposure time, assuming independence between dropouts and endpoints
dependent censoring where the dropout is related to the risk of an AE requires more complicated statistical analyses, e.g. weighted inverse probability of censoring

16. Baseline Risk Factors
Randomization achieves approximate balance among treatment groups in terms of known and unknown risk factors.
Confounding can be a bigger issue when randomization is not maintained—for example, in an open-label extension phase of a RCT or a subset of RCT
Identification of clinically important risk factors or subgroups primarily responsible for the imbalance
Subgroup or stratified analysis
Interaction with treatment
Graphic approaches, e.g. forest plot

17. External Evidence A safety signal will likely be real
if a similar issue exists for products in the same class (class effect)
Signal present in other clinical studies
Evidence from other data sources, e.g. non-clinical findings, observational data
Descriptive and graphical summaries, eg, side-by-side bar plot, forest plot
Meta-analysis methods to synthesize information from different studies
particularly useful for safety evaluation

18. Studies With Open-Label Extension Phase
Open-label extension of an RCT may be needed for long-term safety follow-up and ethical reasons
allows switching treatments or puts all subjects on the new treatment
Drop-in is a similar issue where control subjects are allowed to roll over to receive treatment after disease progression
Both randomization and blinding are lost in this phase
stratified analysis by period to address confounding
Potential ascertainment bias could be assessed by comparison to other studies or the original treatment phase
Less an issue with objective endpoint
Comparison may suffer from heterogeneity or confounding of time-varying patient characteristics (eg, age)

19. Discussion
We review signal screening approaches and provide systematic statistical methods to evaluate factors that may contribute to the imbalance of an AE.
Medical judgment (eg, case confirmation, biologic plausibility, MOA) was crucial in safety assessment, and analytic tools helped to quantify the evidences.
Preplanned analyses are strongly recommended
Building a process and tools proactively could improve transparency, increase effciency, and sustain the objectivity of safety analysis
A definitive conclusion cannot always be made using clinical trial data alone, and post-marketing data are helpful to gather more evidence to refute or confirm the signal and further characterize the safety profile of a drug.