1. Updates on Regulatory Requirements for Missing Data
Ferran Torres, MD, PhD
Hospital Clinic Barcelona
Universitat Autònoma de Barcelona

2. Documentation http://ferran.torres.name/edu/dia
Power Point presentation
Direct links to guidelines
List of selected relevant references

3. Disclaimer
The views expressed here are those of the author and may not necessary reflect those of any of the following institutions he is related to:
Spanish Medical Agency - AEMPS
EMEA (SAWP; EWP)
Hospital Clinic Barcelona
Autonomous University of Barcelona
The views expressed in this presentation are my personal views, and may not be understood or quoted as being made on behalf of or reflecting the position of any of the institutions …

4. Regulatory guidance concerning MD
1998: ICHE9. Statistical Principles for Clinical Trials
2001: PtC on Missing Data
Dec : Recommendation for the Revision of the PtC on MD
2009: Release for consultation

5. ICH-E9 (3,6) Key points: Potential source of bias
Common in Clinical Trials
Avoiding MD
Importance of the methods
Pre-specification
Lack of universally accepted method for handling
Sensitivity analysis
Identification and description of missingness
These are the key points included in ICHE9, and some of them were also marginally described in E3 and E6

6. Status in early 2000s In general, MD was not seen as a source of bias:
considered mostly as a loss of power issue
little efforts in avoiding MD
Importance of the methods for dealing with:
Available Data Only
Handling of missingness: Mostly LOCF, Worst Case

7. Status in early 2000s
Very few information on the handling of MD in protocols and SAP (little pre-specification)
Lack of Sensitivity analysis, or only one, and no justification
Lack (little) identification and description of missingness in reports

9. PtC on MD Structure Introduction The effect of MD on data analysis
Handling of MD
General recommendations

10. Main Points Avoidance of MD
Bias: specially when MD was related to the outcome
Methods:
Warning on the LOCF
Open the door to other methods:
Multiple imputation, Mixed Models…
Sensitivity analysis

11. Current status in
Missing data remains a problem in protocols and final reports:
Little or no critical discussion on pattern of MD data and withdrawals
None / only one sensitivity analysis
Methods:
Inappropriate methods for the handling of MD
LOCF: Still used as a general approach for too many situations
Methods with very little use in early 2000 are now common (Mixed Models)
MMRM sometimes the only approach in some submissions

13. New Draft PtC 1. Executive Summary 2. Introduction
3. The Effect of MD on the Analysis & the Interpretation
4. General Recommendations
4.1 Avoidance of Missing Data
4.2 Design of the Study. Relevance Of Predefinition
4.3 Final Report
5. Handling of Missing Data
5.1 Theoretical Framework
5.2 Complete Case Analysis
5.3 Methods for Handling Missing Data
6. Sensitivity Analyses
Enlarged, extended

14. Statistical framework
applicability of methods based on a classification according to missingness generation mechanisms:
missing completely at random (MCAR)
missing at random (MAR)
missing not at random (MNAR)
We have included the statistical classification of the MD according to their generation mechanism as well as its implication on the applicability of the different methods.

15. Options after withdrawal
> Worse 36 32 28 24 20 16 12 8 4
We tackled any kind of missing but probably missings due to withdrawals are the most relevant and worrying
< Better
Time (months)

16. Options after withdrawal
Ignore that information completely: Available Data Only approach
To “force” data retrieval?:
“Pure” estimates valid only when no treatment alternatives are available
Otherwise the effect will be contaminated by the effect of other treatments
Single Imputation methods
MAR methods:
Mixed-effect models for repeated measures (MMRM)
MNAR methods
the guideline sets the generals principles and describes the main considerations for the methods of handling MD, but unfortunately it is not a cooking recipe
To force: but, up to what extent? Outcomes will be somehow contaminated when there are alternatives
MNAR methods: for which there is still little experience in regulatory submissions and their role should be probably focused to sensitivity analysis

17. Single imputation methods
LOCF, BOCF and others
Many problems described in the previous PtC
Their potential for bias depends on many factors
including true evolutions after dropout
Time, reason for withdrawal and proportion of missingness in the treatment arm
they do not necessarily yield a conservative estimation of the treatment effect
The imputation may distort the variance and the correlations between variables
There is very little innovation in this part
- Avoid the overuse of LOCF

18. MMRM (and others MAR) MAR assumption MD depends on the observed data
the behaviour of the post drop-out observations can be predicted with the observed data
It seems reasonable and it is not a strong assumption, at least a priori
In RCT, the reasons for withdrawal are known
Other assumptions seem stronger and more arbitrary
MAR methods are extensively treated in this update
In RCT, the reason are always recorded, so, the assumptions under this methods work can be somehow assessed.

19. However…
It is reasonable to consider that the treatment effect will somehow cease/attenuate after withdrawal
If there is a good response, MAR will not “predict” a bad response
=>MAR assumption not suitable for early drop-outs because of safety issues
In this context MAR seems likely to be anti-conservative
Imagine the case of a very effective but also relatively highly toxic treatment.
Early drop-outs because of safety will not penalise the final treatment effect since the observed data is favourable

20. The main analysis: What should reflect ?
A) The “pure” treatment effect:
Estimation using the “on treatment” effect after withdrawal
Ignore effects (changes) after treatment discontinuation
Does not mix up efficacy and safety
B) The expected treatment effect in “usual clinical practice” conditions
People who is very much in favour say: …
Imputation Using Dropout Reason (IUDR)
Good sensitivity analyses
Differential imputation according to withdrawals
Penalising treatment related withdrawals (i.e., lack of efficacy, safety)

21. MAR
MMRM aims to estimate the treatment effect that would be seen if all patients had continued on the study as planned.
In that sense MMRM results could be seen as not fully compliant with the ITT principle
Regulatory assessment is focused on what could be expected "on average" in a population, where not all patients have complied with the assigned treatment  for the full duration of the trial

22. Description of MD Detailed description (numerical and graphical):
Pattern of MD
Rate and time of withdrawal
By reason, time/visit and treatment
Some withdrawals will occur between visits: use survival methods
Outcome
By reason of withdrawal and also for completers
These data could be highly informative to assess the potential bias and the adequacy of the assumptions for the handling of MID

23. General recommendations
Sensitivity analysis (there is a new separate section)
Avoidance of MD
Design
Relevance of predefinition (avoid data-driven methods )
detailed description
and justification of absence of bias in favour of experimental treatment
Final Report
Detailed description of the planned and amendments of the predefined methods
Emphasize, stress, highlight

24. Sensitivity Analyses One specific section
a set of analyses showing the influence of different methods of handling missing data on the study results
Pre-defined and designed to assess the repercussion on the results of the particular assumptions made in the handling of missingness
Responder analysis
Sensitivity analyses may give robustness to the conclusions
Responder analysis:
Commonly, the primary analysis of a continuous variable is supported by a responder analysis. How missing data are going to be categorised in this analysis should be pre-specified and justified. Though sub optimal from a statistical perspective, in some cases… missing data as failures and conducting a responder analysis could be the only viable option.

25. Concluding Remarks Avoid and foresee MD Sensitivity analyses
Methods for handling:
No gold standard for every situation
In principle, “almost any method may be valid”:
=>But their appropriateness has to be justified
MNAR

26. The regulatory view is sometimes difficult to understand and probably too conservative

27. But the aim is to avoid a free lunch for everything and to set some reasonable rules for that

28. Basically, to predefine, justify, discuss and to make clear the analysis and assumptions

29. … in the end, to avoid any bias, basically that favouring the experimental arm