1. 2019 Joint Statistical meetings
An evaluation of the trimmed mean approach in clinical trials with dropout
Ming-Dauh Wang, Ph.D.
Regeneron Pharmaceuticals INC.

2. Abstract
In some clinical trial scenarios for some estimands dropout does not necessarily result in missing data because the dropout is considered to be the endpoint.  For example, an NRI analysis of a categorical outcome can be seen as not actually imputing missing values, but rather defining dropouts as treatment failures.   For continuous endpoints BOCF can be seen in a similar light as those who dropout will not in the long run have any improvement from the drug.  Although taking into account intercurrent events in their definitions, these composite estimands entail the rigid and often invalid assumption that there is no benefit from non-pharmacologic factors – that is, that the response rate / mean change in the placebo group is zero.  The trimmed mean was developed and proposed by a prominent FDA statistician as another composite estimand and an alternative analytic approach for these scenarios.  However, the research supporting the approach was not extensive.  Therefore, we conducted a simulation study and an analysis of actual clinical trial data to better understand the operational characteristics of the trimmed mean method.  In the presentation we will review relevant background and present the results of the simulation study and real data analysis.

3. Acknowledgements
This is a joint work with Jiajun Liu and Craig Mallinckrodt of Biogen Inc.

4. Clinical trial example Conclusions
Outline
Background
Simulation study
Clinical trial example
Conclusions

5. Starting Point
Definition and meaning of missing value / data is situation dependent
NRI analysis of a categorical outcome can be seen as not actually imputing missing values, but rather defining dropouts as treatment failures
For continuous endpoints BOCF can be seen in a similar light as those who dropout will not in the long run have any improvement from the drug
Although taking into account intercurrent events in their definitions, these composite estimands entail the rigid and often invalid assumption that there is no benefit from non-pharmacologic factors – that is, that the response rate / mean change in the placebo group is zero 
The trimmed mean is another composite estimand and an alternative analytic approach for these scenarios
As research supporting the approach, we conducted a simulation study and an analysis of actual clinical trial data to better understand the operational characteristics of the trimmed mean method 

6. Percentage of bad scores trimmed away
Trimmed Mean
Consider dropout a bad outcome and each patient who discontinues is given an equally bad score
Percentage of bad scores trimmed away
Adaptive trimming: if dropout rates = 20% on exp and 15% on control, trim 20%
Statistical theory behind trimmed means is sound
Key assumptions are that discontinuation = a similarly bad outcome in all and adherence decisions in trial similar to practice

7. Clinical Settings Where Trimmed Mean is Applicable
Short trial in chronic condition, with repeated observations but end is important
Dropouts mostly toxicity or lack of efficacy, some unrelated or unknown
Dropout is bad outcome
Examples where applicable include chronic pain, diabetes, allergy
In this talk, single observation at endpoint, no repeated measures

8. Simulation Study

9. Technical Details Trimmed mean is asymptotically normal: Mean:
Variance: , where
However, there is no need to rely on large-sample approximation. Instead, a permutation test uses the exact distribution, conditional on the observed data

10. Technical Details (Cont’d)
Approximation by permutation test:
Calculate observed difference of the trimmed means ( 𝑑 )
Randomly re-assign subjects to treatments, and re-calculate the difference of the trimmed means for the permutation
Repeat 2. “many” times and obtain a permutation dist’n (𝐷)
p-value is the percentage of differences for the permutations more extreme than the observed 𝑑
Exact Confidence interval can be constructed by inverting the permutation test; however, the procedure would be computationally intense
Instead, ( 𝑑 + 𝑑 𝛼/2 , 𝑑 + 𝑑 1−𝛼/2 ) is approximately a 1−𝛼 CI, where 𝑑 𝛼/2 and 𝑑 1−𝛼/2 are the lower and upper 𝛼/2 quantiles of 𝐷

11. Simulation Set Up Treatments: Experimental (1), placebo or control (2)
Distributions of endpoint: 𝑁 𝜇 𝑖 , 𝜎 2 ,𝑖=1,2
Hypotheses: 𝐻 0 : 𝜇 1 − 𝜇 2 =0 vs. 𝐻 𝑎 : 𝜇 1 − 𝜇 2 ≠0
Assumptions for power calculation:
𝜇 1 =−40, 𝜇 2 =−20, 𝑑= 𝜇 1 − 𝜇 2
2-sided alpha of 0.05, 90% power
Sample size
𝜎=30:group sample size = 50
𝜎=43: group sample size =100

12. Reasons for Missing Data
R1: MCAR
R2: Intolerability
R3: Lack of efficacy independent of intolerability: missing probability for a response 𝑥 = 𝐹 𝑁 𝑚, 𝑠 2 (𝑥)
*Note: R1, R2 are non-overlapping, R3 could occur along with M1 or M2

13. Simulation Scenarios for Missing Due to R1 Alone
Input parameters for simulations scenarios A1-A8, with equal rates of missing data arising from a completely random mechanism:
Scenario
Treatment arm
Missing Rate (%)
Endpoint means A1
Exp 5
-40
Pbo
-20 A2 10 A3 15 A4 20 A5 A6 A7 A8
Exp = experimental arm, Con = control arm, Pbo = placebo arm

14. Simulation Results (A1-4)
Trimmed means and power from simulation scenarios A1-A4, with untrimmed treatment means of 40 and 20, and equal rates of missing data arising from a completely random mechanism:
No change in means from assumptions
Loss of power increases with higher missing rate
No impact of variance on power
Scenario
Treatment
Missing Rate (%)
Monte Carlo Estimation
n=50 (𝝈=𝟑𝟎)
n=100 (𝝈=𝟒𝟑)
Exp Mean
Pbo Mean
Mean Diff
Power A1
Exp 5
-40.00
-20.00
0.87
-40.12
-19.98
-20.14
0.84
Pbo A2 10
-39.89
-20.13
-19.75
0.78
-39.99
-20.07
-19.92
0.77 A3 15
-39.88
-20.12
-19.76
0.73
-39.90
-19.99
0.70 A4 20
-40.05
-19.87
-20.18
0.67
-40.04
-19.95
-20.09
0.65
Note: Analysis by adaptive trimming: 5,000 simulated trials; 1,000 permutations for each trial

15. Simulation Results (A5-8)
Trimmed means and type I error from simulation scenarios A5-A8, with untrimmed treatment means of 40 and 40, and equal rates of missing data arising from a completely random mechanism:
No change in means from assumptions
Lower 𝛼 with missing data
No impact of variance on 𝛼
Scenario
Treatment
Missing Rate
(%)
Monte Carlo Estimation
n=50 (𝝈=𝟑𝟎)
n=100 (𝝈=𝟒𝟑)
Exp Mean
Pbo Mean
Mean Diff 𝜶 A5
Exp 5
-20.06
-19.97
-0.09
0.04
-20.15
-19.93
-0.02
0.03
Pbo A6 10
-20.10
-20.03
-0.07
0.02
-20.05
-0.01 A7 15
-19.91
-20.13
-20.01
-0.12 A8 20
-20.02
0.002
0.01
-20.07
-0.10
Note: Analysis by adaptive trimming: 5,000 simulated trials; 1,000 permutations for each trial

16. Simulation Scenarios for Missing Due to R2 Alone
Input parameters for simulations scenarios B1-B8, with unequal rates of missing data arising from a completely random mechanism:
Scenario
Treatment arm
Missing Rate (%)
Endpoint means B1
Exp 5
-40
Pbo
-20 B2 10 B3 15 B4 20 B5 B6 B7 B8
Exp = experimental arm, Con = control arm, Pbo = placebo arm

17. Simulation Results (B1-4)
Trimmed means and power from simulation scenarios B1-B4, with untrimmed treatment means of 40 and 20, and unequal rates of missing data arising from a completely random mechanism:
Unchanged mean with Exp; higher mean with Pbo
Power loss increases with higher missing rate
More power loss with larger variance
Scenario
Treatment
Missing Rate
(%)
Monte Carlo Estimation
n=50 (𝝈=𝟑𝟎)
n=100 (𝝈=𝟒𝟑)
Exp Mean
Pbo Mean
Mean Diff
Power B1
Exp 5
-40.04
-22.55
-17.49
0.78
-39.99
-24.64
-15.35
0.64
Pbo
B2* 10
-40.01
-25.66
-14.37
0.53*
-40.08
-28.29
-11.79
0.36 B3 15
-40.00
-27.61
-12.39
0.38
-40.07
-31.66
-8.41
0.17 B4 20
-30.31
-9.69
0.20
-34.87
-5.19
0.06
Note: Analysis by adaptive trimming: 5,000 simulated trials; 1,000 permutations for each trial
*Simulation for this scenario was repeated 10 times, whereof the 95% confidence interval for the power is (0.527, 0.536)

18. Simulation Results (B5-8)
Trimmed means and type I error from simulation scenarios B5-B8, with untrimmed treatment means of 40 and 40, and unequal rates of missing data arising from a completely random mechanism:
Unchanged mean with Exp; higher mean with Pbo
Inflated type I error rate, increasing with higher missing rate
Higher inflated 𝛼 with higher missing rate
Scenario
Treatment
Missing Rate
(%)
Monte Carlo Estimation
n=50 (𝝈=𝟑𝟎)
n=100 (𝝈=𝟒𝟑)
Exp Mean
Pbo Mean
Mean Diff 𝜶 B5
Exp 5
-19.92
-22.70
2.77
0.06
-20.06
-24.61
4.55
0.09
Pbo
B6* 10
-25.68
5.62
0.11
-19.96
-28.30
8.34
0.21 B7 15
-19.94
-27.56
7.62
0.17
-19.99
-31.57
11.58
0.35 B8 20
-19.95
-30.17
10.22
0.27
-20.09
-34.92
14.83
0.50
Note: Analysis by adaptive trimming: 5,000 simulated trials; 1,000 permutations for each trial
*Simulation for this scenario was repeated 10 times, whereof the 95% confidence interval for the power is (0.527, 0.536)

19. Simulation Scenarios for Missing Due to R3 Alone
Input parameters for simulations scenarios C1-C8, with varying rates of missing data arising from lack of efficacy:
Scenario
Treatment arm1
Missing Rate (%)
Endpoint means C1
m=50,s=10
Exp 2
-40
Pbo 6
-20 C2
m=50,s=35 5 10 C3
m=35,s=35 9 16 C4
m=15,s=8 21 C5 C6 C7 C8
Exp = experimental arm, Con = control arm, Pbo = placebo arm

20. Simulation Results (C1-4)
Trimmed means and power from simulation scenarios C1-C4, with untrimmed treatment means of 40 and 20, and unequal rates of missing data arising from lack of efficacy:
Means of both arms are higher than assumptions
Treatment difference remains unchanged
Almost no loss of power (more bad performers from Exp than Pbo are trimmed off counter-balances loss of patients)
Scenario
Treatment
Missing Rate (%)
Monte Carlo Estimation
Exp Mean
Pbo Mean
Mean Diff
Power C1
m=50,s=10
Exp 2
-45.11
-25.18
-19.96
0.899
Pbo 6 C2
m=50,s=35 5
-48.07
-26.86
-21.20
0.890 10 C3
m=35,s=35 9
-51.70
-29.72
-21.98
0.884 16 C4
m=15,s=8
-55.61
-35.59
-20.02
0.878 21
Note: Analysis by adaptive trimming: 5,000 simulated trials; 1,000 permutations for each trial

21. Simulation Results (C5-8)
Trimmed means and type I error from simulation scenarios C5-C8, with untrimmed treatment means of 40 and 40, and equal rates of missing data arising from lack of efficacy:
Means of both arms are higher than assumptions
Treatment difference remains unchanged
Almost no change in 𝛼
Scenario
Treatment
Missing Rate (%)
Monte Carlo Estimation
Exp Mean
Pbo Mean
Mean Diff 𝜶 C5
m=50,s=10
Exp 6
-25.93
-25.95
0.03
0.049
Pbo C6
m=50,s=35 10
-28.25
-28.19
-0.06
0.056 C7
m=35,s=35 16
-31.30
-31.57
0.27
0.051 C8
m=15,s=8 21
-36.94
-36.97
0.048
Note: Analysis by adaptive trimming: 5,000 simulated trials; 1,000 permutations for each trial

22. Simulation Scenarios for Missing Due to R1/2/3
Input parameters for simulations scenarios D1-D8, with varying rates of missing data arising from a combination reasons:
Scenario
Treatment
Missing Rates (%)
 Endpoint Means R1 R2 R3
Overall D1
m=50,s=10
Exp 5 24 2 30 40
Pbo 6 10 20 D2
m=50,s=35 16 25 15 D3
m=35,s=35 8 9 D4
m=15,s=8 21 D5
Con 17 D6 D7 11 D8 D9
D10
Exp = experimental arm, Con = control arm, Pbo = placebo arm

23. Simulation Results (D1-4)
Trimmed means and power from simulation scenarios D1-D4, with untrimmed treatment means of 40 and 20, and varying rates of missing data arising from multiple reasons:
Higher means with both treatments
Treatment difference ranges from below to above assumed value of 20, with much decreased to almost perfect power, trending with higher Exp dropout rate than Pbo/Con to lower
Scenario
Treatment
Missing Rates (%)
Monte Carlo Estimation R1 R2 R3
Overall
Exp Mean
Pbo Mean
Mean Diff
Power D1
m=50,s=10
Exp 5 24 2 30
-42.15
-38.59
-3.56
0.04
Pbo 6 10 D2
m=50,s=35 16 25
-43.77
-35.10
-8.67
0.14 15 D3
m=35,s=35 8 9 20
-47.39
-31.37
-16.02
0.51 D4
m=15,s=8
-55.61
-35.49
-20.12
0.85 21 D5
-58.85
-26.82
-32.03
0.99
Con 17
Note: Analysis by adaptive trimming: 5,000 simulated trials; 1,000 permutations for each trial

24. Simulation Results (D5-8)
Trimmed means and type I error from simulation scenarios D5-D8, with untrimmed treatment means of 40 and 40, and varying rates of missing data arising from multiple reasons:
Higher means with both treatments
Higher or lower difference than assumed value of 0, as well as inflated 𝛼, when the overall dropout rates are unequal
Scenario
Treatment
Missing Rates (%)
Monte Carlo Estimation R1 R2 R3
Overall
Exp Mean
Pbo Mean
Mean Diff 𝜶 D6
m=50,s=10
Exp 5 21 6 30
-25.07
-39.00
13.94
0.43
Pbo 10 D7
m=50,s=35 11 25
-26.73
-34.74
8.01
0.15 15 D8
m=35,s=35 16 20
-31.46
-31.52
0.05
0.047 D9
-34.75
-26.88
-7.87
0.14
Con
D10
-38.95
-25.00
-13.96
Note: Analysis by adaptive trimming: 5,000 simulated trials; 1,000 permutations for each trial

25. Clinical Trial Example

26. Clinical Trial Example
A Ph3 RA study
Key inclusion criteria:
Inadequate response to MTX
≥3 erosions
Stable background MTX
≥ 6/68 tender joints
≥ 6/66 swollen joints
hsCRP ≥ 2 x ULN (6mg/L)
Key exclusion criteria:
Prior bDMARD use
Placebo (N=488)
(Background MTX)
Experimental Drug
(Background MTX)
Experimental Drug (N=487)
(Background MTX)
Active Comparator (N=330)
(Background MTX) W0
W12
W24
W52
W56
Randomization
Primary (ACR20)
Follow-up
At W16 or subsequent visits, inadequate responders rescued to Experimental Drug
At W52 patients could enter long-term extension study or 28-day post-treatment follow-up
Patients who discontinued early also entered 28-day post-treatment follow-up

27. Clinical trial example (cont.)
Analysis on Comparator and placebo patients only for illustration purposes
Endpoints: HAQ-DI change from baseline at week 24 in HAQ-DI
Analysis Method: MMRM, mLOCF + ANCOVA, mBOCF + ANCOVA, adaptive trimmed mean (permutation test)

28. Clinical trial example (Cont.)
mBOCF: for patients who discontinue the study or permanently discontinue the study treatment because of an AE, including death, the baseline observation is used as the week 24 observation, indicating no improvement. For patients who receive rescue, the last non-missing observation at or before rescue is used as the week 24 observation
mLOCF: for patients who discontinue the study or permanently discontinue the study treatment for any reason, the last non-missing post-baseline observation before discontinuation is used as the week 24 observation. For patients who receive rescue, the last non-missing observation at or before rescue is used as the week 24 observation
For MMRM and trimmed mean method: observations are censored and considered missing after patients are rescued or permanently discontinue the study treatment for any reason

29. Clinical trial example (cont.)
Patient disposition for the example clinical trial
____________________________________________________________
Placebo (%) Comparator (%)
Number randomized
Number rescued (21.5%) 35 (10.6%)
Discontinuations
AE (3.1%) (2.1%)
Lack of efficacy (3.1%) (0.3%)
All other reasons (4.1%) 13 (4.5%)
Total (10.2%) 23 (17.6%)
Total Missing Data 155 (31.8%) 58 (17.6%)

30. Clinical trial example (cont.)
COMP
N=330
PBO
N=488
MMRM Censored at Discon/Rescue
n (missing %)
272 (17.6%)
333 (31.8%)
Observed mean
-0.709
-0.443
LS mean
-0.452
LS mean diff (95% CI)
p-value
[ , ]
P<0.0001
Adaptive Trimmed Mean
Trimmed mean
-0.878
trimmed mean diff (95% CI)
[-0.565, ]
ANCOVA + mBOCF n
330
488
-0.625
-0.340
-0.620
-0.345
LS mean diff (95% CI)
[-0.351, ]
ANCOVA + mLOCF
-0.640
-0.350
-0.632
-0.355
[-0.354, ]

31. Conclusions
Trimmed mean entails assumptions similar to NRI and BOCF – that are often overlooked
All else equal, trimming reduces power
Trimming can reward / penalize for higher / lower adherence, but specific impact hard to anticipate during study planning because impact differs by reasons for dropout
Original papers claimed trimmed mean was the right answer to the right question. Second paper couched it as a supportive approach. We tend to agree that it is more supportive. However, NRI and BOCF are far from perfect too

32. References
Permutt T, Li F (2017). Trimmed means for symptom trials with dropouts. Pharmaceutical Statistics 16(1): 20-28
Wang M-D et al (2018). An evaluation of the trimmed mean approach in clinical trials with dropout 17(3):
National Research Council. The prevention and treatment of missing data in clinical trials. National Academies Press: Washington, 2010
ICH E9 (R1) addendum on estimands and sensitivity analysis in clinical trials to the guideline on statistical pricipals for clinical trials EMA/CHMP/ICH/436221/ August 2017