Power and Multiple Regression

Relationship between Power and hypothesis testing Accept Null Hypothesis Reject Null Hypothesis Null Hypothesis is true Correct decision Type I error( alpha typically set to 5%) Null Hypothesis is False Type II error (aka Beta) Correct decision: Probability of making this decision correctly is defined as Power

Why is Power important? Insure sample is large enough to detect effect of interest

Requirements to estimate Power Alpha Effect size of interest Sample size

Requirements to estimate Power Effect size of interest Determined by theory or intuition Are men heavier than women? What is an “important” difference? Two kilograms? Twenty kilograms?

Requirements to estimate Power Alpha Risk of committing Type I error (rejecting null hypothesis when it is true) vs Type II error (accepting null hypothesis when false)

Power in Stata Powerreg command: gives power estimates for changes in R2 Example: Predicting student’s weight based on height Requirements: Alpha: We will use conventional .05 Effect size: What impact do we expect the use height to have on R2? This is based on prior research, experience, intuition or theory Let’s use .20 or a 20% increase in r-square Sample size: 22 students in class

Power in Stata

Power in Stata When choosing effect size of interest you are choosing the change in R- square of interest In most instances you will not have strong a priori reasons for a specific effect size or change in R-square For the assignment 5% is reasonable

Power for Logistic Regression Use powerlog in Stata Gives power estimates for changes in predicted probabilities Requirements: Alpha: Effect size: Sample size:

Power for Logistic Regression Use powerlog in Stata Gives power estimates for changes in predicted probabilities Requirements: Alpha: Conventional .05 Effect size: Predicted probability at mean of independent variable (P1) Predicted probability at mean + 1 SD of independent variable (P2) Sample size: To be determined by program

Power for Logistic Regression Example: Power for baseball team making playoffs based on ERA Gives power estimates for changes in predicted probabilities Requirements: Alpha: Conventional .05 Effect size: Predicted probability at mean of ERA = . 25 (P1) Predicted probability at mean + 1 SD of independent variable = .13 (P2) Sample size: To be determined by program

Obtaining predicted probabilities for independent variables If you have the data Use summarize command to obtain standard deviation Use margins command to produce predicted probabilities E.g. margins, at independent_variable=(1, 2,3…)) If you don’t have data Use published research Make educated guess

Power for Logistic Regression

Power for Logistic Regression using .025 to assume two tailed test

Power for Logistic Regression using different predicted probabilities

For Assignment If using OLS regression If using Logistic Regression Estimate power for bivariate model (you can add additional variables for the rest of assignment) Estimate power for a r-square and an increase in r-square Use an r-square consistent with prior research, or Use a change of five percentage points If using Logistic Regression Use power analysis for bivariate model (you can add additional variables for the rest of assignment) Use two predicted probabilities to estimate power Determine if sample is sufficient for desired power (e.g. .8)