1. Sensitivity analyses for missing not at random outcomes in clinical trials
(Invited Session 1.2: Recent Advances in Methods for Handling Missing Data in Clinical Trials)
Ian White
MRC Clinical Trials Unit at UCL
ICTMC, 8th May 2017
ICTMC/SCT 2017 Invited Session 1.2: Recent Advances in Methods for Handling Missing Data in Clinical Trials
Title: Sensitivity analyses for missing not at random outcomes in clinical trials
Abstract: I will describe various approaches for exploring the impact of departures from a missing-at-random assumption on the estimated intervention effect in a clinical trial. For a single incomplete outcome, I will describe a simple mean score approach. For a longitudinal outcome, I will discuss various approaches that have been proposed for handling dropout (monotonic missing data) and describe an extension to multiple imputation by chained equations (MICE) for missing-not-at-random imputations of non-monotonic missing data. In particular, I will show that the sensitivity parameters that are required by MICE are related to parameters that are easier to elicit, and I will illustrate the method in a smoking cessation study.

2. Missing at random (MAR) in a randomised trial says:
observed
missing
There are no systematic differences in outcome between observed and missing participants
or we can explain the systematic differences through observed characteristics
e.g. people who don’t report whether they have quit smoking are as likely to have quit as those who do report
or mental health patients who refuse interview are on average as healthy as those who are interviewed
UNLIKELY – AND UNTESTABLE
 DO SENSITIVITY ANALYSES

3. Aim
Show how to do sensitivity analyses to departures from missing at random
MNAR = missing not at random
Focussing on trial outcomes (not baseline covariates)
Principled sensitivity analysis:
use a model including a sensitivity parameter 𝛿
where 𝛿=0 corresponds to MAR
vary 𝛿 over a plausible range
informed by clinical understanding
see impact on estimated treatment effect

4. Plan Single outcome: mean score method
illustration in a mental health trial
Repeated outcome: the NARFCS approach
definition
difficulties
partial illustration in a smoking cessation trial

5. 1. Single outcome: mean score method
Work with James Carpenter (LSHTM & MRC CTU) and Nick Horton (Amherst College, USA)
In press at Statistica Sinica

6. Single outcome: mean score method
If we had complete data:
analysis (substantive) model: 𝐸 [ 𝑦 𝑖 | 𝑥 𝑖 ] =ℎ 𝛽 𝑆 𝑇 𝑥 𝑖
𝑦 𝑖 is outcome for person 𝑖
ℎ() is inverse link function (typically identity or logit)
𝑥 𝑖 is a covariate vector including 1's, randomised group 𝑧 𝑖 and baseline covariates - we're interested in the component of 𝛽 𝑆 corresponding to 𝑧 𝑖
estimate the analysis model using score equations
𝑖 𝑥 𝑖 𝑦 𝑖 −ℎ 𝛽 𝑆 𝑇 𝑥 𝑖 =0
With incomplete data, we will replace missing values with their expectation under MNAR

7. Single outcome: mean score method
Substantive model: 𝐸 𝑦 𝑖 𝑥 𝑖 ] =ℎ( 𝛽 𝑆 𝑇 𝑥 𝑖 )
Complete data score equations: 𝑖 𝑥 𝑖 𝑦 𝑖 −ℎ 𝛽 𝑆 𝑇 𝑥 𝑖 =0
Single outcome: mean score method
With incomplete data:
missing data occur in 𝑦 𝑖 only
𝑚 𝑖 indicates missingness of 𝑦 𝑖
With incomplete data, replace score by its expectation
means replacing unobserved 𝑦 𝑖 by 𝐸[ 𝑦 𝑖 | 𝑥 𝑖 , 𝑚 𝑖 =1]
we use a pattern-mixture (imputation) model
𝐸 𝑦 𝑖 𝑥 𝑖 , 𝑚 𝑖 =ℎ 𝛽 𝑃 𝑇 𝑥 𝑖 +Δ 𝑥 𝑖 𝑚 𝑖
where 𝛽 𝑃 can be estimated from complete cases
and Δ 𝑥 𝑖 is a user-specified departure from MAR
e.g. Δ 𝑥 𝑖 =𝛿 for all
or Δ 𝑥 𝑖 =𝛿 for all in one arm only
(and vary 𝛿 over a plausible range)

8. Mean score method: details
Substantive model: 𝐸 𝑦 𝑖 𝑥 𝑖 ] =ℎ( 𝛽 𝑆 𝑇 𝑥 𝑖 )
Pattern-mixture model:
𝐸 𝑦 𝑖 𝑥 𝑖 , 𝑚 𝑖 =ℎ 𝛽 𝑃 𝑇 𝑥 𝑖 +Δ 𝑥 𝑖 𝑚 𝑖
Mean score method: details
Get standard errors using a sandwich variance accounting for both stages
Apply small sample correction and degrees of freedom estimation that give results agreeing exactly with standard methods when standard methods are appropriate (i.e. MAR or “missing = failure”)
Can have extra covariates in the pattern-mixture model

9. QUATRO trial
European multicentre RCT to evaluate the effectiveness of adherence therapy in improving quality of life for people with schizophrenia (Gray et al., 2006)
Primary outcome: quality of life measured by the SF-36 MCS scale at baseline and 52-week follow up
Basic results:
Investigators expressed belief that missing values were likely to be lower (worse) than observed values by up to 10 units
Intervention
Control
Total randomised
204
205
Missing outcome
14% 6%
Mean of observed outcomes
40.2
41.3
SD of observed outcomes
12.0
11.5

10. QUATRO trial: sensitivity analysis
MAR
(MAR in Control)
(MAR in Intervention)

11. Ditto for dichotomised outcome
MAR
(MAR in Control)
(MAR in Intervention)

12. Comparison with multiple imputation (MI)
Could equivalently use multiple imputation
Quantitative data: impute under MAR and add 𝛿
Other data: add 𝛿 as offset in GLM
Advantages of mean score method:
not stochastic (always get same answer)
agrees exactly with standard methods in simple cases like MAR or “missing=failure”
But not easy to extend to repeated measures setting

13. 2. Repeated outcome Repeated outcome is harder
Various proposals e.g. Ratitch et al (2013)
I will describe our ongoing work to develop a popular MI approach, MICE, to handle MNAR data
we call this “NARMICE” or “NARFCS”
Work with Daniel Tompsett, Shaun Seaman and Stephen Sutton (University of Cambridge)

14. FCS / MICE method FCS = Fully conditional specification
MICE = Multiple imputation by chained equations
Given incomplete variables 𝑥 1 , 𝑥 2 ,…, 𝑥 𝑝
Define imputation models for each of them, e.g.
𝐸 𝑥 1 𝑥 2 ,…, 𝑥 𝑝 = 𝛼 1 + 𝛽 12 𝑥 2 +…+ 𝛽 1𝑝 𝑥 𝑝
𝑙𝑜𝑔𝑖𝑡(𝐸 𝑥 2 𝑥 1 , 𝑥 3 ,…, 𝑥 𝑝 = 𝛼 1 + 𝛽 21 𝑥 1 +…+ 𝛽 2𝑝 𝑥 𝑝
etc.
Start by imputing in any way
Iteratively re-impute 𝑥 1 , 𝑥 2 ,…, 𝑥 𝑝 : e.g. to re-impute 𝑥 1 :
fit model to individuals with observed 𝑥 1 , using observed and currently imputed values of 𝑥 2 ,…, 𝑥 𝑝
use fitted model to impute missing values of 𝑥 1

15. NARFCS / NARMICE method
Changes FCS / MICE:
Add missingness indicators 𝑚 1 ,.., 𝑚 𝑝 to the procedure
Modify imputation models e.g.
𝐸 𝑥 1 𝑥 2 ,…, 𝑥 𝑝 , 𝑚 1 ,…, 𝑚 𝑝 = 𝛼 1 + 𝛽 12 𝑥 2 +…+ 𝛽 1𝑝 𝑥 𝑝 + 𝛿 11 𝑚 1 + 𝛾 12 𝑚 2 +…+ 𝛾 1𝑝 𝑚 𝑝 etc.
Elicit values of sensitivity parameters 𝛿 11 (and 𝛿 22 , etc.)
Fit model to individuals with observed 𝑥 1 , using observed and currently imputed values of 𝑥 2 ,…, 𝑥 𝑝
this estimates all parameters except 𝛿 11
Use fitted model plus 𝛿 11 𝑚 1 term to impute missing values of 𝑥 1
Leacy, F. P. (2016). Multiple imputation under missing not at random assumptions via fully conditional specification. University of Cambridge, PhD thesis.
BUT the sensitivity parameter 𝛿 11 is very hard to interpret! It’s the expected difference between missing and observed values, conditional on ALL OTHER VARIABLES - including future outcomes

16. Interpreting the sensitivity parameters
We suggest that users are much happier to specify the sensitivity parameters in a marginal model
e.g. the difference between missing and observed values of an outcome, conditional only on age and sex
e.g. parameter of 𝐸 𝑥 𝑘 𝑚 𝑘 , 𝑎𝑔𝑒, 𝑠𝑒𝑥]
we call this a “marginal sensitivity parameter” (MSP)
cf the “conditional sensitivity parameter” (CSP) used by NARFCS
We show that MSPs are typically smaller than CSPs
We show how to link CSPs to MSPs

17. Graph Treatment effect vs MSP
Marginal sensitivity parameter (MSP) – easy to interpret & elicit
Conditional sensitivity parameter (CSP) – used in NARFCS imputation procedure
The idea
First suppose we have just one variable to which we want to attach a sensitivity parameter
Suppose we elicit a range of plausible values of the MSP
Over a range of values of 𝛿:
impute with CSP 𝛿
compute the MSP and the treatment effect
CSP
MSP
Treatment effect -2 -1 1 2
or if we elicit a single value of the MSP, we can search for the corresponding value of the CSP
Extends for multiple sensitivity parameters
Graph Treatment effect vs MSP *

18. Example: the iQuit in Practice trial
602 smokers wanting help in quitting were randomised to “the iQuit system” (a highly tailored, 90 day text messaging system and advice report) or usual care.
Participants self-reported smoking status at 8 weeks and 6 months (binary outcome)
Loss to follow-up:
control arm: 18.2% and 21.5%
intervention arm: 14.0% and 23.4%.
Primary analysis assumed missing = smoking
Naughton F et al. (2014). Randomized controlled trial to assess the short-term effectiveness of tailored web-and text-based facilitation of smoking cessation in primary care (iQuit in Practice). Addiction, 109, 1184–1193.
Naughton F et al. (2014). Randomized controlled trial to assess the short-term effectiveness of tailored web-and text-based facilitation of smoking cessation in primary care (iQuit in Practice). Addiction, 109, 1184–1193.

19. Marginal sensitivity parameter (MSP) – easy to interpret & elicit
Conditional sensitivity parameter (CSP) – used in NARFCS imputation procedure
Method
We are preparing to elicit sensitivity parameters for this trial with a view to performing a sensitivity analysis
We will analyse both outcome times jointly using NARFCS
The CSP (required by NARFCS) at 8 weeks is the difference (or odds ratio) between missing and observed quit fractions, conditional on all other variables including smoking status at 6 months
We plan to elicit instead the difference between missing and observed quit fractions at each time, unconditionally (MSP)
Web tool under development
See screenshot (next)

20. Elicitation in iQuit in practice
“Please use the sliders below to show your beliefs about the probability of quitting for NON RESPONDERS, receiving INTERVENTION, at 8 weeks. Consider all the reasons why a NON RESPONDER would not return the questionnaire.”
Elicitation in iQuit in practice
Step 1: Use this slider to set the position of YOUR best estimate of the most likely (modal) probability of quitting for a NON RESPONDER.
Step 2: Use this slider to show how certain you are about this value.

21. Conclusions
It’s important to do sensitivity analyses for departures from missing at random (or other main assumption)
Mean score method is convenient for single outcome
Stata software rctmiss is available on SSC
Methods for repeated outcomes are harder
NARFCS is a not-at-random version of FCS / MICE
R software within MICE package is nearly available – by Margarita Moreno Betancur (MCRI, Australia) & our group
We can & should elicit interpretable parameters (MSPs) and then find ways to convert these to the parameters needed by the model (CSPs)
Questions raised at CTU:
what % missing requires these methods?
how do you convince trialists that there is a problem with MNAR?
My thoughts – move towards generic tools for elicitation in RCTs?