1. Does cognitive ability in childhood predict fertility
Does cognitive ability in childhood predict fertility? An example of the use and validity of the Missing Data Strategy
Brian Dodgeon, Tarek Mostafa & George B. Ploubidis

2. Outline -major causes of missingness perhaps not included in models?
‘Missing at Random’ assumption underlies many methods for dealing with missing data
But can we rule out possibility that NCDS patterns of missingness are MNAR?
-major causes of missingness perhaps not included in models?
We use an example to test how MI performs when the model is intentionally mis-specified
Sensitivity analysis to compare MAR and MNAR scenarios

3. Example research project
Does cognitive function age 11 predict childlessness at age 42?
At age 11 we have N=14095 cohort members who did cog. tests
By age 42 we have only N=11419 cohort members for whom we know whether or not they’re childless (not all present at age 11)
We control for important childhood predictors:
Birthweight
Breastfed or not
Parental social class & education
Mother smoking prior to pregnancy
Mother working before child aged 5

4. Example research project – Preliminary analysis
First using Complete Case Analysis (N=6947), we see a U-shaped pattern of childlessness at age 42.
Those with lowest and highest scores most likely childless
So we divide age 11 cog test results into 3 categories:
Low: 1SD or more below mean
Middle mean +- 1 SD
High 1SD or more above mean
Both low and high categories have positive assoc’n with childlessness age 42, but is this result biased?
(Perhaps those with low cog more likely to drop out by 42?)

5. Example research project – exploring missingness
We attempt four MI specifications for dealing with missing values, using a set of auxiliary variables which predict missingness:
Mean replacement (N=11419)
Multiple imputation (MI) with outcome at 42 not imputed(N=10280)
MI with outcome imputed (N=15800)
MI outcome imputed with vars up to age 16 (N=15800)
(predictors of missingness from sweeps > age 16 not used)

6. Objectives We give examples of how the identified predictors of wave-
specific non-response should be used
Objective 2
We intentionally mis-specify those same approaches as a sensitivity analysis
Abandon best predictors of non-response (ie now MNAR?)
See how results hold up compared with Objective 1

7. Cognition at age 11 by childlessness at age 42
Probit coeffs/95% CIs using distinct missing data strategies – Objective 1
CCA assumes cases not present are Missing Completely at Random. But this assumption ignores known patterns of attrition bias.
We have a significant positive association with childlessness at 42, for both high and low cognition, compared to ‘middle’ range.
Probit coeff (95% CI)
Low Cog ability
High Cog ability
N=6947
MCAR
Complete Case Analysis

8. Cognition at age 11 by childlessness at age 42
Probit coeffs/95% CIs using distinct missing data strategies – Objective 1
Using the simple imputation method of mean replacement (or ‘missing’ category for categorical variables), we gain statistical power (CIs are smaller). But mean replacement ignores more subtle patterns of variation.
Probit coeff (95% CI)
Low Cog ability
High Cog ability
N=6947
MCAR
Complete Case Analysis
N=11419
Mean replacement/
missing category

9. Cognition at age 11 by childlessness at age 42
Probit coeffs/95% CIs using distinct missing data strategies – Objective 1
Multiple Imputation by Chained Equations (MICE) models more complex patterns of variation. By not imputing the outcome (chidlessness) we have N=10280, the number of cases for whom we have an outcome.
Probit coeff (95% CI)
Low Cog ability
High Cog ability
N=6947
MCAR
Complete Case Analysis
N=11419
Mean replacement/
missing category
N=10280
MI Outcome not imputed

10. Cognition at age 11 by childlessness at age 42
Probit coeffs/95% CIs using distinct missing data strategies – Objective 1
By imputing the outcome (childlessness at 42), we get more statistical power (N=15800).
Probit coeff (95% CI)
Low Cog ability
High Cog ability
N=6947
MCAR
Complete Case Analysis
N=11419
Mean replacement/
missing category
N=10280
MI Outcome not imputed
N=15800
MI Outcome Imputed

11. Cognition at age 11 by childlessness at age 42
Probit coeffs/95% CIs using distinct missing data strategies – Objective 1
Using MI with outcome imputed, but restricting the predictors of missingness to those variables in waves up to age 16, we maximise the statistical power (N=16013), obtaining smaller confidence intervals than with the other methods.
Probit coeff (95% CI)
Low Cog ability
High Cog ability
N=6947
MCAR
Complete Case Analysis
N=11419
Mean replacement/
missing category
N=10280
MI Outcome not imputed
N=15800
MI Outcome Imputed
N=16013
MI Outcome
Imputed
with vars up to age 16

12. Discussion of results (Objective 1)
The imputation process gives us more stat. power and produces similar result to Complete Case Analysis
We prefer the ‘MI with outcome imputed’ approach
Our ‘benchmark’ for Objective 2 will be MI with outcome
imputed, with predictors of non-response from waves up to age 16
This approach embodies fewer assumptions, since population up to age 16 is still high (about 10% reduction from birth population, taking into account deaths)

13. Cognition at age 11 by childlessness at age 42
Probit coefficients/95% CIs – Objective 2
Probit coeff (95% CI)
N=16013
MI Outcome
Imputed
with vars up to age 16
Low Cog ability
High Cog ability

14. Cognition at age 11 by childlessness at age 42
Probit coefficients/95% CIs – Objective 2
Probit coeff (95% CI)
N=16013
MI Outcome
Imputed
with vars up to age 16
N=16086
MI removing most importnt
predictor of
response
At age 11
Low Cog ability
High Cog ability

15. Cognition at age 11 by childlessness at age 42
Probit coefficients/95% CIs – Objective 2
Probit coeff (95% CI)
N=16013
MI Outcome
Imputed
with vars up to age 16
N=16086
MI removing most importnt
predictor of
response
At age 11
N=16082
MI removing 2 most important
predictors of
up to age 16
Low Cog ability
High Cog ability

16. Cognition at age 11 by childlessness at age 42
Probit coefficients/95% CIs – Objective 2
Heckman Probit is a selection model with MNAR assumption
Probit coeff (95% CI)
N=16013
MI Outcome
Imputed
with vars up to age 16
N=16086
MI removing most importnt
predictor of
response
At age 11
N=16082
MI removing 2 most important
predictors of
up to age 16
N=12038
Heckman
Probit
Low Cog ability
High Cog ability

17. Conclusion
In all scenarios the non-linear trend is confirmed
(lowest and highest cognition at age 11 predicts childlessness)
Mis-specified MI models return results with a similar substantive interpretation
Heckman selection returns somewhat different results
But, Heckman sensitive even to subtle departures from assumptions
If Heckman is correct, some form of selection is not captured by MI, but most scenarios (even MNAR) do not conform with Heckman results

18. Future work
Formal sensitivity analysis with pattern mixture models to further probe the MAR assumption (Carpenter & Kenward, 2012)
Directed Acyclic Graphs – M Graphs to identify “colliders” among the predictors of response
Test the performance of different sets of identified predictors of response
Carpenter, J.R & Kenward, M.G (2012) Multiple Imputation and its Application. Wiley, Chichester.