Multiple imputation using ICE: A simulation study on a binary response Jochen Hardt Kai Görgen 6 th German Stata Meeting, Berlin June, 27 th 2008 Göteborg University University of Mainz, Bernsteincenter for Computational Neuroscience, Berlin

Almost all sociological / medical data have missings - typically in the range of.5 to 5 % in a variable Many statistical procedures can only use cases without missings What we already know about missing substitution: 1)With a small amount of missings everything is easy 2)Large samples are easy

Missingness at random A very simple example - Analysis of complete cases - Imputation of means - Singular regression imputation - Multiple imputation: hotdeck - Multiple imputation: chained equations A not so simple example Multiple imputation by chained equations in real data Overview

There is a distinction in the literature about data being missing completely at random (MCAR), missing at random (MAR) or being missing not at random (Rubin, 1996). MCAR means that the pattern of missings is totally at random, not depending on any variable in or not in the analysis. MAR is an intuitively somewhat misleading label, because it allows strong dependencies in the pattern of missings. If, for example, in a set of variables all data for men are missing and for women are non- missing, the dataset is still MAR as long as gender is included as a variable. The formal definition is that missings are at random given all information available in the dataset. Background I

MCAR usually does not apply to data in social sciences MAR seems quite plausible for many datasets. But the definition has the disadvantage that it can never be tested on any given dataset – always it is possible that some unobserved variables - at least partitially - cause the pattern of missing. MNAR means that there is such an unknown process in the data that creates the missings. E.g. for socially undesirable behaviour, such as lying, stealing or betraying, it is plausible to assume that missing values rather reflect higher than lower levels of such behaviour, but an exact modelling of the answering process is mostly not possible. One of the most prominent question for MNAR is the one about income, which has high rates of missings, usually in the range of 20 % - 50 %. Background II

reg Y X, both standard distributed continuous variables Y = 1*X + 1*error, n = 50, i = 3%, 8%, 13%…. 68% of X are set missing, for each I: 200 replications were made A very simple example: y x

Works ok but waste of information, particularly in multivariate analyses The old solution: take only the cases without missings. Percent missings in x Standard deviation for beta Estimate for beta ± sd

Quite stable estimate, stronger increase in sd than in complete case analysis The 2nd solution: mean substitution

Overestimation of the effect when response is included The 3rd solution: substitution by regression

Hotdeck Imputation s Typo: 1 of course

Considerably more variance due to imputation, break-down at about 50 % missings (m = 5, 4 variables) Number 4: Multiple imputation - hotdeck

Multiple Imputation by Chained Equations: ICE s

Multiple Imputation a random subset of the data is drawn A value for each missing of var X 1 is estimated via (linear, logistic, ordered, etc) regression The closest observed values to that estimate are chosen and replace the missings The program switches to X 2 …….. Cycled over ten times Finish when m datasets are created

Multiple Imputation: Analysis in each dataset a (regression) analysis is performed Results are combined due to Rubins rule (a) parameters (b) variances within between total

Stable estimates with small variances (m = 5, 4 variables) No 5 finally: Multiple Imputation on Chained Equations - ICE

Analysis of complete cases: not bad when only few variables Imputation of means not bad for continous variables don‘t impute the mode take the mean for categorical variables, too no inflation of ß‘s when no replacement in response Regression imputation don‘t include response into model Multiple imputation: Hotdeck Stata‘s version is not recomendable Multiple Imputation by Chained Equations very good Let‘s have a look onto a not so simple example Preliminary summary on the very simple example

Var ß sd X1: maternal love Response: Lifetime suicide attempt 0 = no (83 %) 1 = yes (17 %) N = 505 One binary response (Suicide attempts) is predicted by 20 continous variables plus 5 discrete Variables:

Percent missing in x ICE estimate for beta, 4 variables in the model, CMAR

ICE estimate for beta, 4 variables in the model, CMAR Percent missing in x

ICE estimate for beta, 4 variables in the model, CMAR Percent missing in x

ICE estimate for beta, 11 variables in the model, CMAR Percent missing in x

ICE estimate for beta, 25 variables in the model, CMAR Percent missing in x

The same done with MICE in R Percent missing in x estimate for beta, 11 variables in the model, CMAR

single regression substitution estimate for beta, CMAR 10 variables in the model (response excluded) Percent missing in x

mean substitution imputation estimate for beta, CMAR Percent missing in x

ICE estimate for beta, 11 variables in the model, NMAR Percent missing in x

Single regression imputation 10 variables in the model,NMAR Percent missing in x

All non-linear effects are downward biased by any method. The example shows an interaction coefficient estimated with ICE, 11 variables in the model, CMAR Percent missing in x

Summary - In large samples we can substitute considerable higher proportions of missings than in small ones. - Multiple imputation with ICE performs well in all situations (as far as we examined) - Having more variables in the imputation model leads to better estimates, i.e.smaller sd’s. - With binary responses, ICE may report extreme sd’s when the number of variables grows high, or the number of cases low. Then we have gone too far. - Single regression imputation performs quite well under certain conditions - Non-linear effects get lost with all methods