Monitoring rare events during an ongoing clinical trial Haley Hedlin, PhD; Victoria Ding, MS Quantitative Sciences Unit, Department of Medicine, Stanford University

Motivation During oral presentation, start here, go to introductions, skip setup, touch on the simulations setup table, skip to examples, show figures quickly, then summarize findings in discussion We consider the setting where a data and safety monitoring board (DSMB) or another entity is monitoring a rare event during the course of a two-arm study and desires guidelines around whether the event is occurring more often in one arm. We do not consider stopping rules, but instead present guidelines to guide the monitoring in combination with other accumulating information during the trial. Our work is motivated by a real trial where a rare safety event was of interest and the monitoring approach was similar to the CI approach. We aim to provide useful guidelines to be used in monitoring rare events while information is accumulating during a trial. We present a simulation study to evaluate two approaches: the sequential probability ratio test (SPRT) and confidence intervals (CI). 1/8

Examples Figure 1a: Example from a single simulated trial of the SPRT score updated with each patient along with the point the loose and strict thresholds were crossed. Figure 1b: Example from a single simulated trial of the 95% CI updated after every 50 participants. 2/8

Simulation Setup - Table 1 Baseline Risk Treatment Risk Relative Risk 0.02 0.1 5 0.04 2 1 0.005 0.025 0.010 0.001 0.002 Table 1: Assumed proportion of participants in the control and treatment arms experiencing the event of interest and the corresponding relative risk. 3/8

Results – SPRT strict Figure 2: The panels at left display the proportion of trials exceeding their threshold. The panels at right display the median (25th, 75th percentile represented by the error bar) number of patients accrued when the threshold was crossed among trials where the threshold was crossed. 4/8 All analyses were performed in R version 3.5.0.

Results – SPRT loose Figure 2: The panels at left display the proportion of trials exceeding their threshold. The panels at right display the median (25th, 75th percentile represented by the error bar) number of patients accrued when the threshold was crossed among trials where the threshold was crossed. 5/8 All analyses were performed in R version 3.5.0.

Results – 95% Confidence Interval Figure 2: The panels at left display the proportion of trials exceeding their threshold. The panels at right display the median (25th, 75th percentile represented by the error bar) number of patients accrued when the threshold was crossed among trials where the threshold was crossed. 6/8 All analyses were performed in R version 3.5.0.

Discussion The SPRT approach crossed the preset threshold more often than the CI approach in settings where the RR > 1. When RR = 1, the SPRT approach never crossed the threshold while the CI approach crossed it in 0.6% of trials when the baseline risk = 0.005 and in 4.7% of trials when the baseline risk = 0.02. When the threshold was crossed, it was crossed earlier using both SPRT approaches than with the CI approach in nearly all settings. The operating characteristics of the SPRT approach are better than the CI approach in all settings considered. 7/8

Contact For more information please contact Haley Hedlin at hedlin@stanford.edu. For more on the Quantitative Sciences Unit, visit med.stanford.edu/qsu. 8/8

Monitoring rare events during an ongoing clinical trial Introduction Results Haley Hedlin, PhD; Victoria Ding, MS Quantitative Sciences Unit, Department of Medicine, Stanford University Figure 2: The panels at left display the proportion of trials exceeding their threshold. The panels at right display the median (25th, 75th percentile represented by the error bar) number of patients accrued when the threshold was crossed among trials where the threshold was crossed. We conduct a simulation study to evaluate approaches to monitoring rare events during an ongoing a clinical trial. This work builds on a history of interim monitoring in clinical trials and considers the performance of previously developed methods when applied to a rare event , the sequential probability ratio test (SPRT) and confidence intervals (CI).. We aim to provide useful guidelines to be used in monitoring rare events while information is accumulating during a trial. Importantly, we focus on settings where the initial sample sizes are relatively small and the proportion experiencing events are very small. Methods We consider the relative risk of the rare event in the two arms. The SPRT repeatedly evaluates whether a test statistic based on a likelihood ratio exceeds a preset threshold based on ⍺ and β. We assume ⍺ = β = 0.05 (loose) and 0.001 (strict)1,2. Characteristics considered include proportion of simulated trials where the loose and strict SPRT thresholds are crossed and the median and interquartile range of the number of patients when the threshold is crossed. These thresholds from SPRT are considered in combination with the thresholds obtained from central confidence intervals obtained from a Fisher’s exact test3. Characteristics here are: proportion of simulated trials where the lower bound of the 95% CI exceeds 1 and the median and interquartile range of the number of patients when the CI is entirely > 1. Still need 5 slides for oral presentation Motivation We consider the setting where a data and safety monitoring board (DSMB) or another entity is monitoring a rare event during the course of a two-arm study and desires guidelines around whether the event is occurring more often in one arm. We do not consider stopping rules, but instead present guidelines to guide the monitoring in combination with other accumulating information during the trial. Our work is motivated by a real trial where a rare safety event was of interest and the monitoring approach was similar to the CI approach. Discussion The SPRT approach crossed the preset threshold more often than the CI approach in settings where the RR > 1. When the threshold was crossed, it was crossed earlier using both SPRT approaches than with the CI approach in nearly all settings. The operating characteristics of the SPRT approach are better than the CI approach in all settings considered. Figure 1a: Example from a single simulated trial of the SPRT score updated with each patient along with the point the loose and strict thresholds were crossed. Figure 1b: Example from a single simulated trial of the 95% CI updated after every 50 participants.