Slides to accompany Weathington, Cunningham & Pittenger (2010), Chapter 11: Between-Subjects Designs 1

Objectives t-test for independent groups Hypothesis testing Interpreting t and p Statistical power 2

t-test for Independent Groups Basic inferential statistic Ratio of two measures of variability = Difference between two group means Standard Error of the difference between group means Allows us to consider effect, relative to error 3

Standard Error of the Difference between Means 4

t-test Larger |t-ratio| = greater difference between means Based on this we can decide whether to reject H o –Usually H o = µ 1 = µ 2 Sampling error may account for some difference, but when t is “large” enough… 5

Hypothesis Testing: t-tests Based on estimates of probability When α =.05, there is a 5% chance of rejecting H o when we should not ( Type I error ) –See Figure 11.2 (each tail = 2.5%) –Region of rejection If t falls within the shaded ranges, we reject H o because probability is so low 6

Figure

Hypothesis Testing Steps 1.State H o and H 1 –Before collecting or examining the data 2.Identify appropriate statistical test(s) –Based on hypotheses –Often multiple approaches are possible –Depends on how well data meet the assumptions of specific statistical tests 8

Hypothesis Testing Steps 3.Set the significance level (α) –α = p(Type I error) Risk of false alarm You control –1 – α = p(Type II error) Risk of miss Careful, you might “overcontrol” 9

Hypothesis Testing Steps 4.Determine significance level for t-ratio –Use appropriate table in Appendix B, df for the test and your selected alpha (α) level to determine t critical –If your observed |t ratio| > t critical reject H o –If your observed p-level is less than α you can also reject H o 10

Hypothesis Testing Steps 5.Interpreting t-ratio –Is it statistically significant? –Is it practically/clinically significant? Does the effect size matter, really? Book mentions d-statistic 11

Hypothesis Testing Steps 5.Interpreting t-ratio –Magnitude of the effect Degree of variance accounted for by the IV Omega squared = % of variance accounted for by IV in the DV –Is there cause and effect? Typically requires manipulated IV, randomized assignment, and careful pre- / post- design 12

Correct Interpretation of t and p If you have a significant t-ratio: = statistically significant difference between two groups = IV affects DV = probability of a Type I error is α 13

Errors in p Interpretation Changing α after analyzing the data –Unethical –We cannot use p to alter α Kills your chances of limiting Type I error risk p only estimates the probability of obtaining at least the results you did if the null hypothesis is true, and it is based on sample statistics  not fully the case for α 14

Errors in p Interpretation Stating that p = odds-against chance –p =.05 does not mean that the probability of results due to chance was 5% or less –p is not the probability of committing a Type I error –Recommended interpretation: If p is small enough, I reject the null hypothesis in favor of the alternative hypothesis. 15

Errors in p Interpretation Assuming p = probability that H 1 is true (i.e., that the results are “valid”) –p does not confirm the validity of H 1 –Smaller p values do not indicate a more important relationship between IV and DV Effect size estimates are required for this 16

Errors in p Interpretation Assuming p = probability of replicating results –The probability of rejecting H o is not related to the obtained p-value A new statistic, p rep is getting some attention for this purpose (see Killeen, 2005) 17

Statistical Tests & Power β = p(Type II error) or p(miss) 1 – β = p(correctly rejecting false H o ) = power Four main factors influence statistical power 18

Power: Difference between µ Power increases when the difference between µ of two populations is greater 19

Power: Sample Size Issue of how well a statistic estimates the population parameter (Fig. 10.5) Larger N  smaller SEM As SEM decreases  overlap of sampling distributions for two populations decreases  power increases Don’t forget about cost 20

Power: Variability in Data Lots of variability  variance in the sampling distribution and greater overlap of two distributions Reducing variability reduces SEM  overlap decreases  power goes up Techniques: Use homogeneous samples, reliable measurements 21

Power: α Smaller α  lower Type I probability  lower power As p(Type I) decreases, p(Type II) increases (see Figure 11.6) As α increases, power increases –Enlarges the region of rejection 22

23

Estimating Sample Size Based on power Tables in Appendix B can give you estimates for t-ratios –Effect size is sub-heading Cost / feasibility considerations Remember that sample size is not the only influence on statistical power 24

What is Next? **instructor to provide details 25