Effect Sizes & Power Analyses for k-group Designs Effect Size Estimates for k-group ANOVA designs Power Analysis for k-group ANOVA designs Effect Size Estimates for k-group X 2 designs Power Analysis for k-group X 2 designs

k-BG Effect Sizes When you have more than 2 groups, it is possible to compute the effect size for “the whole study”. Include the F-value, the df (both for the effect and the error), and click the button for the type of design you have (BG or WG) However, this type of effect size is not very helpful, because: -- you don’t know which pairwise comparison(s) make up the r -- it can only be compared to other designs with exactly the same combination of conditions

k-BG Effect Sizes Just as RH: for k-group designs involve comparing 2 groups at a time (pairwise comparisons)… The most useful effect sizes for k-group designs are computed as the effect size for 2 groups (effect sizes for pairwise comparisons) Since you won’t have F-values for the pairwise comparisons, you will use Computator to complete a 2-step computation Using info from the SPSPS output d = (M1 - M2 ) /  MSerror d² r =  d² + 4

Descriptives outcome variable -- larger scores are better no therapy weekly therapy daily therapy Total NMeanStd. Deviation Pairwise effect sizes computation for k-BG designs For no therapy vs. weekly therapy … For a BG design be sure to press

k-WG Effect Sizes Just as RH: for k-group designs involve comparing 2 groups at a time (pairwise comparisons)… Effect sizes for k-group designs are computed as the effect size for 2 groups (effect sizes for pairwise comparisons) Since you won’t have F-values for the pairwise comparisons, you will use Computator to complete a 3-step computation Using info from the SPSPS output d = (M1 - M2 ) /  (MSerror * 2) dw = d * 2 dw² r =  dw² + 4

Pairwise effect sizes computation for k-WG designs For no intake vs. mid … For a WG design be sure to press Descriptive Statistics INTAKE MID FINAL MeanStd. DeviationN

Determining the power you need.. For a 2-condition design... the omnibus-F is sufficient -- retain or reject, you’re done ! you can easily determine the sample size needed to test any expected effect size with a given amount of power For a k-condition design … the power of the omnibus-F - isn’t what matters ! a significant omnibus-F only tells you that the “two most different” means are significantly different follow-up (pairwise) analyses will be needed to test if the pattern of the mean differences matches the RH: you don’t want to have a “pattern of results” that is really just a “pattern of differential statistical power” you need to assure that you have sufficient power for the smallest pairwise effect needed to test your specific RH:

k-group Power Analyses As before, there are two kids of power analyses;;; A priori power analyses conducted before the study is begun start with r & desired power to determine the needed N Post hoc power analysis conducted after retaining H0: start with r & N and determine power & Type II probability

Power Analyses for k - BG designs Important Symbols S is the total # of participants in that pairwise comp n = S / 2 is the # of participants in each condition of that pairwise comparison N = n * k is the total number or participants in the study Example the smallest pairwise effect size for a 3-BG study was.25 with r =.25 and 80% power S = 120 for each of the 2 conditions n = S / 2 = 120 / 2 = 60 for the whole study N = n * k = 60 * 3 = 180

Power Analyses for k - WG designs Important Symbols S is the total # of participants in that pairwise comp For WG designs, every participant is in every condition, so… S is also the number of participants in each condition Example the smallest pairwise effect size for a 3-WG study was.20 with r =.20 and 80% power S = 191 for each condition of a WG design n = S = 191 for the whole study N = S = 191

Combing LSD & r … Cx Tx1 mean M dif r M dif r Cx 20.3 Tx Tx * *.41 * indicates mean difference is significant based on LSD criterion (min dif = 6.1) Something to notice … The effect size of Cx vs. Tx1 is substantial (Cohen calls.30 “medium effect”), but is not significant, suggesting we should check the power of the study for testing an effect of this size.

k-group Effect Sizes When you have more than 2 groups, it is possible to compute the effect size for “the whole study”. Include the X², the total N and click the button for df > 1 However, this type of effect size is not very helpful, because: -- you don’t know which pairwise comparison(s) make up the r -- it can only be compared to other designs with exactly the same combination of conditions

Pairwise Effect Sizes Just as RH: for k-group designs involve comparing 2 groups at a time (pairwise comparisons)… The most useful effect sizes for k-group designs are computed as the effect size for 2 groups (effect sizes for pairwise comparisons) The effect size computator calculates the effect size for each pairwise X² it computes

k-group Power Analyses As before, there are two kinds of power analyses;;; A priori power analyses conducted before the study is begun start with r & desired power to determine the needed N Post hoc power analysis conducted after retaining H0: start with r & N and determine power & Type II probability

Power Analyses for k -group designs Important Symbols S is the total # of participants in that pairwise comp n = S / 2 is the # of participants in each condition of that pairwise comparison N = n * k is the total number or participants in the study Example the smallest pairwise X² effect size for a 3-BG study was.25 with r =.25 and 80% power S = 120 for each of the 2 conditions n = S / 2 = 120 / 2 = 60 for the whole study N = n * k = 60 *3 = 180