1 Paired Differences Paired Difference Experiments 1.Rationale for using a paired groups design 2.The paired groups design 3.A problem 4.Two distinct ways to estimate μ 1 – μ 2 5.Formal statement – large sample test 6.Formal statement – small sample test 7.Examples

2 Paired Differences Rationale for using a paired groups design The basic problem: When you measure two samples of cases that have been treated differently, the differences between the two resulting sets of scores will be produced by either or both of two types of effect: * the treatment * everything else that matters

3 Paired Differences Rationale for using a paired groups design We’re not interested in the effect on performance of “everything else that matters”. We want to know whether the treatment effect is real. But suppose that the variability due to “everything else that matters” is much larger than the variability due to the treatment. In that case, we may not be able to detect the signal (treatment effect) because of the noise (error variability – that is, variability due to everything else that matters).

4 Paired Differences Rationale for using a paired groups design We have to do something to reduce that part of the difference between the groups that is due to “everything else that matters.” We do this by matching or testing the same people twice. Both approaches remove the effects of nuisance variables.

5 Paired Differences Rationale for using a paired groups design * If the two samples of cases are more alike in things that matter, then the contribution of the treatment to any difference between the means is proportionally larger. * That is, if the contribution of the treatment (to the difference between group means) stays the same, but the contribution of other differences between the groups goes down, then we have a more sensitive test.

6 Paired Differences Total variability in the data set Variability due to the treatment effect Variability due to all other causes Here, most of the variability in the data set is produced by things other than the treatment effect.

7 Paired Differences Total variability in the data set Numerator of Z or t- test Denominator of Z or t- test

8 Paired Differences Total variability in the data set Variability due to the treatment effect Variability due to all other causes Here, variability due to treatment effect is the same, but variability due to other causes has decreased.

9 Paired Differences Total variability in the data set Numerator of Z or t- test Denominator of Z or t- test

10 Paired Differences The paired groups design One way to reduce variability due to “everything else that matters” is to use the paired groups design. * Matched pairs: select people in pairs matched on some relevant variable (e.g., IQ), then randomly assign one to each condition. * Repeated measures: every person gets both treatments, so acts as their own control.

11 Paired Differences The paired groups design Suppose that for each person with IQ = 110 in the treatment condition we have a person with IQ = 110 in the control condition. Similarly with all other IQs represented in the treatment condition – each has a matched-IQ case in the control condition. * Now, if we subtract score for one member of pair from score for other member, the effect of IQ cannot contribute to that difference.

12 Paired Differences A problem When we match pairs or used repeated measures on the same people, we violate one of the assumptions of the independent groups tests of difference between two population means * The statistical test is based on the assumption that the observations in one group are independent of the observations in the other group.

13 Paired Differences A problem Why is that assumption a problem? * Because here, once we have selected Group 1, we do not then independently select Group 2. * As a direct result the sample mean difference X 1 – X 2 is not a good estimator of the population mean difference μ 1 – μ 2.

14 Paired Differences Two distinct ways to estimate μ 1 – μ 2 1. Choose a random sample from Population A. Independently choose a random sample from Population B. Compute the means for each sample and find the difference between these means. 2. Choose a random sample from Population A and a matching sample from Population B. Find the difference between each score in sample A and its matched score in sample B. Compute the mean of these differences.

15 Paired Differences Two distinct ways to estimate μ 1 – μ 2 In the first case, we work with a sampling distribution based on differences between independent sample means. * Anything that could make one sample mean different from the other will contribute to the variability (σ X 1 -X 2 ) of that sampling distribution. * With a more variable sampling distribution, we need a larger sample difference to be confident that the inferred population difference is real.

16 Paired Differences Two distinct ways to estimate μ 1 – μ 2 In the second case, because of matching, many of the (random) things that could drive sample means apart are eliminated from the differences X 1i – X 2i. * As a result, the variability in the sampling distribution (σ D ) has fewer sources. * So we can infer a real population difference with a smaller sample difference (samples are less likely to be different just by chance).

17 Paired Differences Paired groups test: large samples H O : μ D = D O H A : μ D < D O H A : μ D ≠ D O or μ D > D O (D O = historical value of the difference between the population means.) Test statistic:Z = X D – D O σ D ∕√n D

18 Paired Differences Paired groups test: large samples Rejection region: Z Z α/2 or Z > Z α Assumptions: 1. Distribution of differences is normal. 2. Difference scores are randomly selected from the population of differences (between matched pairs or repeated measures).

19 Paired Differences Important note Z = X D – D O σ D ∕√n D Notice that the numerator does not have an X 1 and an X 2. It just has X D. * We begin by finding the differences between each pair of observations. From then on, we work only with these difference scores.

20 Paired Differences Paired groups test, small samples H O : μ D = D O H A : μ D < D O H A : μ D ≠ D O or μ D > D O (D O = historical value of the difference between the population means.) Test statistic:t = X D – D O s D ∕√n D

21 Paired Differences Paired groups test – small samples Rejection region: t t α/2 or t > t α Assumptions: 1. Distribution of differences is normal. 2. Difference scores are randomly selected from the population of differences (between matched pairs or repeated measures).

22 Paired Differences Example 1 A psychologist was studying the effectiveness of several treatments to help people quit smoking. In one treatment, smokers heard a lecture about the effects of cigarette smoke on the human body, accompanied by graphic slides of those effects. In the other treatment, smokers had daily phone conversations with a therapist who encouraged them not to smoke that day. To control for effects of age and sex, the psychologist assigned people to the experimental groups in pairs matched on those variables. The data, in the form of number of hours without a cigarette appear on the next slide

23 Paired Differences Example 1 PairL+SDEDiffDiff ∑ = 36∑ = 1694 Notice the negative signs!! Without negative signs, Σ would be 106

24 Paired Differences Example 1 s D 2 = 1694 – (36) 2 = s D = √ = 14.79

25 Paired Differences Example 1 H O : μ D = D O H A : μ D ≠ D O Test statistic:t = X D – D O s D ∕√n D

26 Paired Differences Example 1 Rejection region: t crit = t 7,.025 = t obt = 4.5 – 0 = 4.5 = /√ Decision: do not reject H O – no evidence treatment effects differ

27 Paired Differences Example 2 Tetris is a computer game requiring some spatial information-processing skills and good eye-hand coordination, either or both of which may improve with practice. Six people who had never previously played Tetris were tested on the game at the beginning (Test 1) and at the end (Test 2) of a 2-week period during which they played Tetris for one hour each day. Their Tetris scores on the two testing sessions appear on the next slide.

28 Example 2 a. Did the subjects’ Tetris scores improve significantly from Test 1 to Test 2 (α =.05)? b. Is the variance of the subjects’ Test 2 scores significantly different from 400,000, the variance among the population of experts at Tetris (α =.05)? Paired Differences

29 Paired Differences Example 2a SubjectTest 1Test 2Diff D D  ∑= 8015 D 2  ∑= S D =

30 Paired Differences Example 2a Rejection region: t crit = t 5,.05 = t obt = – 0 = /√8 Decision: Reject H O – scores improved significantly