Practical Model Selection and Multi-model Inference using R Presented by: Eric Stolen and Dan Hunt

Indigo snake home range radio telemetry study of eastern indigo snake home range sizes in central FL Response variable was ln(home range) (95% fixed-kernel estimate) sex, landcover type, and length of time (weeks) General linear model Interested in effect sizes and predictions

Indigo snake home range Science questions: –“Is there evidence for a difference in home range size between habitats?” –“Is there evidence for a difference in home range size between sexes?” –“Is there an effect of length of time tracked? –“Is the effect of habitats type different between the sexes (is there an interaction term)?” –“Does the data support estimating an effect of habitats type with 3 levels or 2 levels?”

Models – Indigo Snake example SEX land cover type 3 levels (lc1) land cover type 2 levels(lc2) weeks SEX + lc1 SEX + lc2 SEX + weeks lc1 + weeks lc2 + weeks SEX + lc1 + weeks SEX + lc2 + weeks SEX + lc2 + SEX * lc1 SEX + lc2 + SEX * lc2 SEX + lc2 + weeks + SEX * lc1 SEX + lc2 + weeks + SEX * lc2

Models – Indigo Snake example SEX land cover type 3 levels (lc1) land cover type 2 levels(lc2) weeks SEX + lc1 SEX + lc2 SEX + weeks lc1 + weeks lc2 + weeks SEX + lc1 + weeks SEX + lc2 + weeks SEX + lc1 + SEX * lc1 SEX + lc2 + SEX * lc2 SEX + lc1 + weeks + SEX * lc1 SEX + lc2 + weeks + SEX * lc2

Models – Indigo Snake example SEX land cover type 3 levels (lc1) land cover type 2 levels(lc2) weeks SEX + lc1 SEX + lc2 SEX + weeks lc1 + weeks lc2 + weeks SEX + lc1 + weeks SEX + lc2 + weeks SEX + lc1 + SEX * lc1 SEX + lc2 + SEX * lc2 SEX + lc1 + weeks + SEX * lc1 SEX + lc2 + weeks + SEX * lc2

Models – Indigo Snake example SEX land cover type 2 levels(lc2) weeks SEX + lc2 SEX + weeks lc2 + weeks SEX + lc2 + weeks SEX + lc2 + SEX * lc2 SEX + lc2 + weeks + SEX * lc2

Models – Indigo Snake example SEX land cover type 2 levels(lc2) weeks SEX + lc2 SEX + weeks lc2 + weeks SEX + lc2 + weeks SEX + lc2 + SEX * lc2 SEX + lc2 + weeks + SEX * lc2

Models – Indigo Snake example SEX land cover type 2 levels(lc2) weeks SEX + lc2 SEX + weeks lc2 + weeks SEX + lc2 + weeks SEX + lc2 + SEX * lc2 SEX + lc2 + weeks + SEX * lc2

I-T mechanics AICc i = -2*log e (Likelihood of model i given the data) + 2*K (n/(n-K-1)) or = AIC + 2*K*(K+1)/(n-K-1) (where K = the number of parameters estimated and n = the sample size)

I-T mechanics AICc min = AICc for the model with the lowest AICc value  i = AICc i – AICc min

Model Probability (also Bayesian posterior model probabilities) evidence ratio of model i to model j = w i / w j I-T mechanics

Least Squares Regression AIC = n log e (   ) + 2*K (n/(n-K-1)) Where    RSS / n (explain offset for constant part)

I-T mechanics Counting Parameters: K = number of parameters estimated Least Square Regression K = number of parameters + 2 (for intercept & 

I-T mechanics Counting Parameters: K = number of parameters estimated Logistic Regression K = number of parameters + 1 (for intercept 

I-T mechanics Counting Parameters: Non-identifiable parameters

Comparing Models

Evidence Ratio = 3.03

Comparing Models Evidence Ratio =4.28 ( ) / ( )

Mathematical details What types of models can be compared within a single I-T analysis? –Data must be fixed (including response) –Must be able to calculate maximum likelihood –(ways to deal with quasi-likelihood) –Models do not need to be nested –In some cases AIC is additive

Model Fitting Preliminaries Understanding the data/variables Avoid data dredging! safe data screening practices Detect outliers, scale issues, collinearity Tools in R

–Generalized linear models lm glm –Packages Design Package –FE Harrell Regression Modeling Strategies with Applications to Linear Models, Logistic Regression, and Survival Analysis. Springer. CAR package –Fox, J An R and S-plus Companion to Applied Regression. Sage Publications.

Tools in R –Model formula Ex) –Output summary(model4) model4$aic Model4$coefficients model4 <- glm(help~age2 + sex + mom_dad + suburb + brdeapp + matepp + density + I(density^2), family=binomial,data=choices)

Tools in R Fitting the model set – –R program does the work Trouble-shooting Export results

Model Checking –Global model must fit –Models used for inference must meet assumptions, –Look for numerical problems Tools in R

Interpretation of models for inference Case 1: One or a few models best models Examining model parameters and predictions –Effects –Prediction graphing results –nomograms –Presenting Results Anderson, D. R., W. A. Link, D. H. Johnson, and K. P. Burnham Suggestions for presenting the results of data analysis. Journal of Wildlife Management 65:

Model selection uncertainty Model-average prediction Model-average parameter estimates Multi-model Inference

Model Averaging Predictions

Model-averaged prediction Model Averaging Predictions

Prediction from model i Model Averaging Predictions

Weight model i Model Averaging Predictions

Model-averaged parameter estimate Model Averaging Parameters

Unconditional Variance Estimator

Snake Example