1. Generalized Linear Models (GLMs) & Categorical Data Analysis (CDA) in R Hong Tran, April 21, 2015

2. Laboratory for Interdisciplinary Statistical Analysis Collaboration: Visit our website to request personalized statistical advice and assistance with: Designing Experiments Analyzing Data Interpreting Results Grant Proposals Software (R, SAS, JMP, Minitab...) LISA statistical collaborators aim to explain concepts in ways useful for your research. Great advice right now: Meet with LISA before collecting your data. All services are FREE for VT researchers. We assist with research—not class projects or homework. LISA helps VT researchers benefit from the use of Statistics www.lisa.stat.vt.edu LISA also offers: Educational Short Courses: Designed to help graduate students apply statistics in their research Walk-In Consulting: Available Monday-Friday from 1-3 PM in the Old Security Building (OSB) for questions <30 mins. See our website for additional times and locations.

3. Outline  1.What is CDA?  2.Contingency Table  3.Measures of Association  4.Test of Independence  5.What is GLM? When should we use it?  6.How to evaluate the GLM models?  7.Logistic Regression  8.Poisson Regression

4. What is CDA? Dependent Variable (Y) Independent Variables (X) Model Continuous (Normal)ContinuousLinear Regression ContinuousCategoricalANOVA ContinuousMixedANCOVA Categorical CDA

5. Contingency Table  I rows for categories in X  J rows for categories in Y  Values in cell=possible outcomes

6. Example 1 of Contingency Table  the relationship between smoking and epidermoid/undifferentiated pulmonary carcinoma (cancer)  Cohort study conducted  2x2 contingency table  Does smoking increase the risk of having epidermoid/undifferentiated pulmonary carcinoma?

7. Generating Contingency Table in R  Input the 2×2 table in R as a 2×2 matrix  Change the matrix to table using the function as.table(), because some functions are happier with tables than matrices

8. Measure of Association  Continuous Variables-Pearson Correlation Coefficient  Ordinal Variables-Pearson Correlation Coefficient  Nominal Variables-Phi Coefficient and Cramer’s V

9. Pearson Correlation

10. Pearson Correlation Example 2  mtcars in R  1974 Motor Trend US magazine mpg: miles per gallon wt: weight drat: rare axle ratio

11. Phi Coefficient  measures the association between two binary variables.  Its value ranges from -1 to +1, where +1/-1 indicates perfect positive association/negative association, 0 indicates no association.  The square of the phi coefficient is related to the chi-squared statistic for a 2×2 contingency table.

12. Cramer’s V  Cramer’s V measures the association between two nominal variables.  It varies from 0 (no association) to 1 (complete association) and can reach 1 only when the two variables are equal to each other.

13. Measures of Association Comments:  1, When the two variables are binary, Cramer’s V is the same as Phi Coefficient  2, In R, under library(psych), use function phi() for Phi Coefficient  3, In R, under library(vcd), use function assocstats() for Cramer’s V

14. Test of Independence  Large Sample Size Chi-square Test  Small Sample Size Fisher’s Exact Test

15. Test of Independence (Chi-square Test)

16. Back to Example 1 CasesControlTotal Smoke18/31313/31331/313 Non-smoker46/313236/313282/313 Total64/313249/3131

17. Test of Independence (Chi-square Test)

18. Test of Independence (Fisher’s Exact Test)  When any of the expected counts fall below 5, Chi-square test is not appropriate. Instead, we use Fisher’s Exact Test.  Example 3: The following data are from a Stanford University study of the effectiveness of the antidepressant Celexain the treatment of compulsive shopping. WorseSameBetter Celexain237 Placebo282

19. Test of Independence  Chi-Square Test Use R function chisq.test()  Fisher’s Exact Test Use R function fisher.test()

20. Generalized Linear Models  When the response variables are not continuous, not normally distributed  Count numbers: 1, 2, 3,…  Binary: 0 and 1

21. Comparison General Linear ModelGeneralized Linear Model Special cases ANOVA, ANCOVA, MANOVA, MANCOVA, linear regression, mixed model Linear regression, logistic regression, Poisson regression Function in R lmglm Typical method estimation Least SquareMaximum Likelihood

22. Ordinary Linear Regression  Ordinary Linear Regression (OLR) investigates and models the linear relationship between independent variables and dependent variables that are continuous.  The simplest regression is Simple Linear regression, which models the linear relationship between a single independent variable and a single dependent variable.  Simple Linear Regression Model:

23. Assumptions in OLR

24. GLM Model  function is called the link function because it connects the mean and the linear predictor  Dependent variable’s distribution must come from the Exponential Family of Distributions  Includes Normal, Bernoulli, Binomial, Poisson, Gamma, etc. 3 Components  Random: Identifies dependent Y and its probability distribution  Systematic: Independent variables in a linear predictor function  Link function: Invertible function.that links the mean of the dependent variable to the systematic component.

25. Response Distribution

26. Types of GLMs

27. GLM and OLR

28. Model Evaluation: Deviance  Deviance: measures how close the predicted values from the fitted model match the actual values from the raw data. Definition: Deviance = -2[log-likelihood(proposed model)-log- likelihood(saturated model)]  A saturated model is a model that fits the data perfectly, so its log- likelihood is the maximum. It has as many parameters as observations and hence it provides no simplification at all.  The deviance has a chi-squared asymptotic null distribution.  The degree of freedom is n-p, where n is the number of observations and p is the number of model parameters.  Smaller deviance, the better the model

29. Inference in GLM  Goodness of Fit test ─ The null hypothesis is that the model is a good alternative to the saturated model. ─ Deviance is the Likelihood Ratio Statistic  Likelihood Ratio test - Allows for the comparison of one model to another model by looking at the difference in deviance of the two models. -Null Hypothesis: the predictor variables in Model 1 that are not found in Model 2 are not significant to the model fit. -Alternative Hypothesis: the predictor variables in Model 1 that are not found in Model 2 are significant to the model fit. ─ LRS is distributed as Chi-square distribution. ─ Simpler models have larger deviance.

30. Model Comparison in GLM  Two additional measures for model comparison are: ─ AkaikeInformation Criterion (AIC) Penalizes model for having many parameters AIC=-2logLikelihood+2*p where p is the number of parameters in the model The smaller AIC, the better the model ─ Bayesian Information Criterion (BIC) BIC=-2logLikelihood+ln(n)*p where p is the number of parameters in the model and n is the number of observations Usually stronger penalization for additional parameter than AIC The smaller BIC, the better the model

31. Summary  Setup of GLM  Inference in GLM  Deviance and Likelihood Ratio Test ─ Test goodness of fit for the proposed GLM model ─ Test the significance of a predictor variable or set of predictor variables in the model  Model Comparison in GLM ─ AIC ─ BIC

32. Logistic Regression  Logistic regression is a regression technique for predicting the outcome of a binary dependent variable. Example: y=1-Success, 0-Failure  Random Component: the dependent variable follows a Bernoulli distribution ─ Probability of Success: ─ Probability of Failure: 1- ─ The probability of obtaining y=1 or y=0 is given by Bernoulli Distribution: ─ Mean(Y): μ=

33. Logistic Regression

35. Steps for Logistic Regression in R  1.Create a single vector of 0’s and 1’s for the response variable.  2.Use the function glm() family=binomial to fit the model.  3.Test for goodness of fit and significance of predictors.  4.Interpretation

36. Poisson Regressions  Poisson regression is a regression technique for predicting the outcome of a count dependent variable.  Dependent variable measures the number of occurrences in a given time frame.  Outcomes equal to 0,1,2,…  Examples: Number of penalties during a football game. Number of customers shop at a grocery store on a given day. Number of car accidents at an intersection during a period of time.

37. Poisson Regression

39. Steps for Poisson Regression in R  1.Input data where y is a column of counts.  2.Use the function glm() family=poisson to fit the model.  3.Test for goodness of fit and significance of predictors.