Download presentation
Presentation is loading. Please wait.
1
Model Selection in Semiparametrics and Measurement Error Models Raymond J. Carroll Department of Statistics Faculty of Nutrition and Toxicology Texas A&M University http://stat.tamu.edu/~carroll
2
Outline Problem motivating the work Measurement error in nutritional epidemiology Power of model selection/averaging Difficulties with BIC Semiparametric formulation The Hjort and Claeskens asymptotics Major results on semiparametric efficiency
3
Co-authors, Nutritional Epidemiology Victor Kipnis National Cancer Institute Laurence Freedman, Gertner Institute (Israel) and National Cancer Institute
4
Co-author, Semiparametrics Gerda Claeskens, University of Leuvan
5
Papers Seemingly Unrelated Measurement Error Models, Biometrics, 2006, Victor Kipnis and Larry Freedman Post-Model Selection Inference in Semiparametric Models, with Gerda Claeskens This paper has been under review for 6 months, at a journal that just asked me to review a paper in 6 weeks.
6
Seemingly Unrelated Measurement Error Models (SUMEM) Problem: Understand the properties of food frequency questionnaires (FFQ), the most used means of estimating nutrient intakes Issue: True intake cannot be measured
7
SUMEM Notation: Y = FFQ X = true intake (Not Observable) Z = other covariates Goals: Estimate two important properties For pre-study sample sizes post-study relative risk and power
8
The OPEN Study I will use data from the OPEN Study First large biomarker study for nutritional epidemiology Biomarkers available: Protein (urinary nitrogen) Energy/Calories (Doubly labeled water) Approximately 200 women in the study
9
SUMEM: Model for Protein Basic variance components model: Error Model for Biomarker:
10
SUMEM: Model for Protein More Data: Along with Protein, we also measure Energy (calories) Correlated: Energy = Protein + Fat + Carbohydrates + Alcohol Combine: We try to combine energy with protein
11
SUMEM: Model for Protein Variance components model with Energy: Error Model for Biomarker: Note how this model predicts FFQ protein using true energy
12
SUMEM: More pain, no gain Variance components model with Energy: No Gain: One can show that if you fit this measurement error model, Marginal protein fit unaffected, identical estimates of correlation and attenuation Note how this model predicts FFQ protein using true energy
13
SUMEM: More pain, no gain Variance components model with Energy: Intuition: To predict FFQ Protein: If I tell you true protein intake, why should true energy intake matter? Note how this model predicts FFQ protein using true energy
14
SUMEM: More pain, more gain? Variance components model with Energy: A reduced model: Full Model Reduced Model
15
OPEN Biomarker Study Results Variance Reduction: As if the sample size increased by a factor of 3 (s.e. from Burnham & Anderson) ModelEstimateStd. err. CorrelationFull0.2820.088 Reduced0.2500.048 AttenuationFull0.1290.041 Reduced0.1140.024
16
OPEN: Model Selection AIC and Weights: Let be the likelihood ratio chi-squared statistic, full vs. reduced Let d be the difference in degrees of freedom The AIC weight to the reduced model is We can do a weighted average of the full and reduced model estimates
17
OPEN: Model Selection BIC and Weights: Let be the likelihood ratio chi-squared statistic, full vs. reduced Let d be the difference in degrees of freedom The BIC weight to the reduced model is
18
OPEN: Model Selection In OPEN: the AIC weight to the reduced model is We can do a weighted average of the full and reduced model estimates In OPEN, this is obviously essentially the reduced model
19
OPEN: Bootstrapping Bootstrap: resample the individuals Note how the bootstrap distribution of the AIC weights is nothing like what one would expect
20
OPEN: Bootstrapping Note how the bootstrap distribution of the attenuation is much more variable than the Burnham and Anderson calculations suggest
21
OPEN: Bootstrapping Note how the bootstrap distribution of the correlation is much more variable than the Burnham and Anderson calculations suggest
22
OPEN Type Simulation Setup: We used OPEN parameters, but put in a full model n = 200, as in OPEN BIC is supposed to be a consistent model selector, hence should have weights near 0. AIC is not a consistent model selector
23
Full Model Simulation The weights should be near zero, since the full model holds. Note how BIC is an utter disaster: it almost always selects the reduced model
24
Full Model Simulation Correlation: Fitting the reduced model here results in a 30% bias This would mean a study would have only 55% of the sample size it needs to attain a desired power Bias here is important!
25
Full Model Simulation Attenuation: Fitting the reduced model here results in a 30% bias
26
Full Model Simulation Correlation: Fitting the reduced model here results in a 25% bias This would mean a study would have only 60% of the sample size it needs to attain a desired power Bias here is important!
27
Full Model Simulation For AIC, the Burnham and Anderson standard error estimates are OK, but coverage is somewhat low For BIC, it is so badly biased that the coverage probabilities are < 50% in all cases
28
Summary of Numerical Work BIC: seriously biased in the full model holds Seems to love the reduced model From the Biometrics article
29
Oracle Estimators BIC: This is an example of an oracle estimator Leeb and Poetscher state
30
Oracle Estimators Leeb and Poetscher go on to state This seems to be too good to be true, and it is It is a delusion to believe it carries statistical meaning It is remarkable that some of the lessons learned from Hodges’ counterexample seem not to have been received in the model selection literature
31
Asymptotic Framework I will next describe some local-alternative asymptotics for semiparametric models These go some way to understanding the problem with oracle estimators Semiparametric generalization of the work of Hjort and Claeskens
32
Semiparametric Framework Likelihood Formulation Example: Partially Linear Model
33
Semiparametric Framework Example: Heteroscedastic Linear Model
34
Semiparametric Framework Likelihood Formulation Two models: Full model Reduced model
35
Semiparametric Framework Local misspecification assumption Intuitively: even an oracle will have to think a bit to distinguish between the full and reduced models Practically: the models are not extremely different
36
Semiparametric Framework Local misspecification assumption Hjort and Claeskens: parametric models We deal with semiparametric models
37
Profile Methods Likelihood Formulation For every, do local likelihood nonparametric regression to obtain Then maximize
38
Semiparametric Information Bound The semiparametric information is the semiparametric version of Fisher information Many references, good one is Murphy and van der Vaart, JASA, 2002
39
Semiparametric Information Bound Standard Result: For any function, at the full model
40
Semiparametric Information Bound Non-Standard Result: we have to compute the limit distribution of the reduced model estimate at the full (local) model Easy using contiguity arguments (LeCam’s 1 st, 2 nd and 3 rd Lemma)
41
Semiparametric Information Bound Non-Standard Result: Note asymptotic bias
42
Semiparametric Information Bound Model Averaging Result: Consider a model averaged estimator with weights for the reduced model depending on the profile likelihood, e.g., BIC, AIC, etc. Recall that Let D be the limiting distribution of
43
Semiparametric Information Bound Model Averaging Result: Then the model average estimator has limiting distribution This is exactly the same result as in Hjort and Claeskens, except their Fisher information matrix is replaced by the semiparametric information bound
44
Semiparametric Information Bound Inference: Hjort and Claeskens propose a method for inference that is asymptotically correct. In numerous simulations and examples for AIC, we have found that it is essentially the same as full- model inference BIC: In this local misspecification framework, BIC is an inconsistent model selector: it selects the reduced model with probability 1.
45
Summary SUMEM: Model averaging can, in real life, result in estimators that decrease variance by 2/3 while retaining robustness Very important in measurement error problems, where information is sparse Inference: The Burnham and Anderson standard error formulae are reasonably precise Because of non-normal limit distributions, coverage probabilities are much less than nominal
46
Summary BIC and Oracles: Simply disastrous in our example Lack of uniform consistency means the: “it sounds too good to be true” is correct Bootstrap: Does not work Not new, but striking numerics Semiparametrics: In the local framework, same as parametric methods with the semiparametric information bound replacing the Fisher information (I contiguity)
47
Burnham and Anderson Formula You want to estimate a standard errror for a model-average estimate of
48
Bootstrap Problem First two terms have same limit as the limit in the data world. 3 rd term is not 0, hence inconsistency
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.