MISSING DATA DSBS Meeting 28 May 2009 Kristian Windfeld, Genmab

ICH E9 (1998) on missing values Try to avoid – by design (2.3) Frequency and type must be documented in the CTR (5.2) Imputation techniques (LOCF, …, complex math. models…) (5.2.1) Are considered protocol violations (5.2.2) Assess pattern of occurrence among treatment groups (5.2.2) Source of bias (5.3) Pre-specify methods for handling in protocol (5.3) Analyze sensitivity of trial results to missing value handling method (5.3)

CPMP PtC on missing data (2001) Effect of missing values on analysis & interpretation: Power/variability (smaller sample size => smaller power but plausibly less variability => more power) Bias – unless not related to unobserved real value (which cannot be verified)

CPMP PtC on missing data (2001) Handling of missing data Complete case analysis Exploratory trials or as supportive analysis Imputation LOCF (”widely used”…”likely to be accepted if measurements constant over time”…”may provide an acceptably conservative approach”) best/worst case imputation multiple imputation mixed models

CPMP PtC on missing data (2001) Recommendations Avoid missing data Pre-specify handling method in statistical section of protocol with justification Approach must be conservative Approach may be updated in a SAP if unforeseen problems occur Analyze missing values (number, timing, pattern, reason)! Analyze sensitivity of results to missing value handling method => Everybody continued to do ITT with LOCF…

CHMP: Recommendation for revision of PtC on missing data (2007) In many MAAs the handling of missing data is poor Little or no discussion on missing data pattern… Only one sensitivity analysis… or none with no justification Misconception that LOCF is necessary and sufficient… Need for cautionary note on mixed models and multiple imputation methods – their use still controversal

New EMEA draft guideline GUIDELINE ON MISSING DATA IN CONFIRMATORY CLINICAL TRIALS (CPMP/EWP/1776/99 Rev. 1) Released for consultation 23 april – 31 oct 2009

New EMEA draft guideline Not much news… but some additional points: Missing values violate ITT principle (collect data regardsless of protocol compliance) Missing value taxonomy (MCAR, MAR, MNAR) – but since unverifiable => use conservative method Use methods not assuming MCAR/MAR, ”pattern mixture, selection, and shared parameter models” mentioned Specific suggestions for sensitivity analyses

Example: Missing ACR20 values from RA trial Missing Value Handling Placebo700 mg NACR20N 700mg/placebo OR Complete case analysis (no imputation) 4613%4760%10 0 (non-response) Imputation 5511%5749%7.8 1 (response) Imputation 5527%5767%5.5 Is non-response imputation conservative?

Key message We need to start taking missing values seriously!

Imagine a news headline… ”Licensing application for obliviximab rejected because of the lack of analysis of the missing data…” Do you want to be responsible?

Discussion questions Why don’t we take the guidelines seriously when it comes to missing value handling? How many sensitivity analyses are needed and should be planned? When is the missing values influence so great that trial results become non-interpretable? Is the potentially inflated precision resulting from single imputation methods a problem and if so, how may this be addressed?