PSY 1950 t-tests, one-way ANOVA October 1, 2008

vs

History of the t-test William Gosset –Statistician, brewer at Guinness factory Which variety of barley is best? –Small samples, no known population  –Student. (1908). The probable error of a mean. Biometrika, 6, 1–25. += +=

From z to t One sample z-test Null hypothesized  Known  2 sample mean - population mean standard error One sample t-test Null hypothesized  Unknown  2 sample mean - population mean estimated standard error Use s 2 for  2

The Sampling Distribution of s 2 s 2 is unbiased estimator of  2 –mean s 2 =  2 But sampling distribution of s 2 is positively skewed, especially for small samples Because of this, odds are that an individual s 2 underestimates  2, especially for small samples Thus, on the average, t > z, especially for small samples Can’t use z-distribution to determine p for t Must devise new distribution that takes into account sample size

df = n - 1

Psychologists are Naughty Brewers Pearson to Student/Gosset in 1912: “only naughty brewers take n so small that the difference is not on the order of the probable error!”

Assumptions 1.Normality (of population, not sample) 2.Independence of observations (within sample)

Tails Two-tailed test –p <. 025 in both tails –Conservative, conventional One-tailed test –p <.05 in predicted tail –A priori, justifiable directional hypothesis? The one-and-a-half tailed test –p <. 05 in predicted tail –p <. 025 in unpredicted tail –Un-ignorable “wrong-tailed” result? The lopsided test –p <. 05 in predicted tail –p <. 005 in unpredicted tail

From 1-sample t to 2-sample t One sample t-test Null hypothesized  Unknown  2 sample mean - population mean estimated standard error Use s 2 for  2 Two sample t-test Null hypothesized  =  1 -  2 Unknown  2 =   2 2 sample mean dif - population mean dif estimated standard error Use s 1 2 and s 2 2 for  1 2 and  2 2

Standard Error of the Difference Between Means Variances add: the variance of x minus y = the variance of x plus the variance of y –Only true if x and y are uncorrelated

Assumptions 1.Normality (of populations, not samples) 2.Independence of observations (within and between samples) Dependence due to groups Sampling Shared history Social interaction Dependence due to time/sequence e.g., psychophysical variables Dependence due to space e.g., city blocks 3.Homogeneity of variance (of populations, not samples) –Okay so long as one variance isn’t more than 4 times the other, and samples sizes are approximately equal

ANOVA Analysis of variance –Comparing variance between sample means with variance within samples means Variance within = noise Variance between = noise + possible signal Omnibus test –Are there any differences in means between populations? –H 0 :  1 =  2 =  3 … –H 1 : at least one population mean is different from another F-ratio = Variance between /Variance within –Variance between /variance within > 1  reject H 0 –Variance between /variance within ≤ 1  retain H 1

ANOVA Example: 0,1,2;1,2,3;2,3,4

Assumptions 1.Normality (of populations, not samples) 2.Independence of observations (within and between samples) 3.Homogeneity of variance (of populations, not samples) –Okay so long as one variance isn’t more than 4 times another, and samples sizes are approximately equal

Crawford, J. R., & Howell, D. C. (1998). Comparing an individual’s test score against norms derived from small samples. The Clinical Neuropsychologist, 12,

Why develop new statistics? Clinicians often compare an individual’s score to a normative sample that is treated like a population Sometimes normative sample is small –Instruments with poor normative data –Demographic considerations decrease n –Local norms are expensive to collect –Case studies can have small comparison groups

What’s wrong with the z score? Z-scores assume that normalized sample is a population With small n, sampling distribution of variance can be skewed Leads to a greater likelihood of underestimating population SD and overestimating z mpling_dist/index.html

Why use the modified t statistic? T-statistic allows clinicians to use a small normative sample to estimate population SD Formula is almost the same as z-score formula but allows for wider tails t = [X 1 – X M2 ] / [s 2 √[(N 2 + 1) / N 2 ]]

When should modified t statistic be used? Difference is “vanishingly small” when sample size is greater than 250, and not necessarily large even with smaller samples Modified t-test should be used with a sample size of less than 50 Shouldn’t be used when normative data are skewed