General Linear Models; Generalized Linear Models Hal Whitehead BIOL4062/5062

Transformations Analysis of Covariance General Linear Models Generalized Linear Models Non-Linear Models

Common Transformations Logarithmic: X’=Log(X) –Most common, morphometrics, allometry Squareroot: X’=√X –Counts, Poisson distributed –X’=√(X+0.5) if counts include zeros Arcsine-squareroot: X’=arcsine(√X) –Proportions (or percentages /100) Box-Cox –General transformation

Regression and ANOVA Multiple regression: Y = β 0 + β 1 ·X 1 + β 2 ·X 2 + β 3 ·X 3 + … + Error {X’s are continuous variables} ANOVA: Y = γ 0 + γ 1 ( Z 1 )+ γ 2 ( Z 2 ) + γ 3 ( Z 3 ) + … + Error {Z’s are categorical variables, defining groups}

Analysis of Covariance (mixture of ANOVA and regression) Y = β 0 +β 1 ·X 1 +β 2 ·X 2 +…+γ 1 ( Z 1 )+γ 2 ( Z 2 )+... +Error {X’s are continuous variables} {Z’s are categorical variables, defining groups} Important assumption: Parallelism: β’s the same for all groups Estimate β’s and γ’s using least squares

Analysis of Covariance Data: –Catch rates of sperm whales (per whaling day) by Yankee whalers from logbooks of Yankee whalers off Galapagos Islands Questions: –Was there a significant change in catch rate over this period? –Was there a significant seasonal pattern?

Analysis of Covariance Model: Catch (m,t) = β 0 + β 1 ·t + γ(m) + Error t = [continuous] m = Jan-Feb, Mar-Apr, …, Nov-Dec

Analysis of Covariance Model: Catch (m,t) = β 0 + β 1 · t + γ(m) + Error Parameter estimates: β 0 = [constant] β 1 = [change/yr] γ(Jan-Feb)= γ(Mar-Apr)= γ(May-Jun) = γ(Jul-Aug)= γ(Sep-Oct) = γ(Nov-Dec) = 0.000

Analysis of Covariance Model: Catch (m,t) = β 0 + β 1 · t + γ(m) + Error Analysis of Variance Table: SourceSSdf MS F-ratioP YEAR MONTH Error

Analysis of Covariance Durbin-Watson D Statistic: First Order Autocorrelation: 0.034

General Linear Model: Analysis of Covariance plus Interactions Y = β 0 + β 1 ·X 1 + β 2 ·X 2 + … + γ 1 ( Z 1 ) + γ 2 ( Z 2 ) + … + β 12 ·X 1 ·X 2 + … + γ 12 ( Z 1, Z 2 ) + … + α 12 ( Z 1 )·X 1 + … + Error {X’s are continuous variables} {Z’s are categorical variables, defining groups}

Characteristics of General Linear Models The response Y has a normal distribution with vector mean μ and SD σ 2. A coefficient vector (b=[β’s, γ’s, α’s]) defines a linear combination of the predictors (X’s). The model equates the two as: μ = X·b

General Linear Models Coefficients (β’s, γ’s, α’s), and fit of model (σ² or r²) estimated using least squares Subsets of predictor variables may be selected using stepwise methods, etc. Beware: –Collinearity –Empty or nearly-empty cells (combinations of categorical variables with few units)

General Linear Model Data: –Movements of sperm whales (displacement per 12-hr) off Galapagos Islands with year, clan, and shit rate Questions: –Are movements of sperm whales affected by year, clan, shit rate or combinations of them?

General Linear Model Potential X variables: Year (Categorical: 1987 and 1989) Clan (Categorical: ‘Plus-one’ and ‘Regular’) Shit-rate (Continuous, Arcsine-Squareroot transform) Year*Clan Year*Shit-rate Clan*Shit-rate

General Linear Model X variables selected by stepwise selection (P-to-enter = 0.15/ P-to-remove = 0.15) BackwardForwardYearClanShit-rateYear*ClanYear*Shit-rate Clan*Shit-rate

General Linear Model Backward Y =c + Clan + Year*Clan Forward Y =c + Shit-rate*Clan

General Linear Model Why two “best models”? Backward Y =c + Clan + Year*Clan Forward Y =c + Shit-rate*Clan

General Linear Model Which is “best”? Backward Y =c + Clan + Year*Clan Forward Y =c + Shit-rate*Clan r²= d.f. r²= d.f.

General Linear Models The response Y has a normal distribution with vector mean μ and SD σ 2. A coefficient vector (b=[β’s, γ’s, α’s]) defines a linear combination of the predictors (X’s). The model equates the two as: μ = X·b

Generalized Linear Models The response Y has a distribution that may be normal, binomial, Poisson, gamma, or inverse Gaussian, with parameters including a mean µ. A coefficient vector (b=[β’s, γ’s, α’s]) defines a linear combination of the predictors (X’s). A link function f defines the link between the two as : f(μ) = X·b

Generalized linear models Examine assumptions using residuals Examine fit using “deviance”: –a generalization of the residual sum of squares –twice difference of log-likelihoods of model in question and full model –fits of different models can be compared –Related to AIC

Generalized Linear Models: can fit non-linear relationships using ‘link functions’ and can consider non- normal errors MATLAB: glmdemo

Proportion of sexually-mature animals at different weights MATLAB: glmdemo

Two problems with linear regression: 1) probabilities 1 2) clearly non-linear MATLAB: glmdemo

Polynomial Regression better, but also: 1) probabilities 1 2) inflections are not real MATLAB: glmdemo

Instead fit “logistic regression” using generalized linear model and binomial distribution MATLAB: glmdemo Y= 1/(1+e β0+β1 · X )

Compare two generalized linear models MATLAB: glmdemo Y= 1/(1+e β0+β1 · X ) Y= 1/(1+e β0+β1 · X +β2 · X · X ) Difference in deviance =0.70; P=0.40

Examine assumptions using residuals MATLAB: glmdemo

Making predictions: MATLAB: glmdemo

Non-linear models, e.g. Y = c + EXP(ß 0 + ß 1 · X) + E Y = ß 0 + ß 1 · X · [X>X K ] + E More general than generalized linear models But harder to fit: –iterative process –may not converge –non-unique solution –harder to compare

Summary: Methods with One Dependent Variable Simple Linear Regression One-way ANOVA Multiple Linear Regression Multi-way ANOVA Analysis of Covariance General Linear Model Generalized Linear Model Non-Linear Model Increasing Complexity