1. EXPERIMENTAL DESIGN Random assignment Who gets assigned to what? How does it work What are limits to its efficacy?

2. RANDOM ASSIGNMENT Equal probability of assignment to each condition (treatment, control, etc.) –or fixed, known probability if other design conditions are included Use of random number table, computer- generated random number to make assignments

3. WHO GETS ASSIGNED Primary units (such as students, patients, or clients) assigned individually without additional personal information used Assignment within personal or demographic categories- gender, psychological diagnosis, etc. Multiple levels of assignment- pools used for selection

4. How Randomization Works Distributes various causal conditions, variables equally across assignment conditions Generates random differences in initial conditions, pretest scores whose variance can be estimated in probability Creates individual variation (“error”) that is independent of treatment

5. Limits of Efficacy Randomization does not last forever- groups begin to change over time in unknown ways History is uncontrolled Maturation is uncontrolled over long periods of time Testing effects are not controlled Mortality effects are not controlled

6. TWO GROUP MEANS TESTS

7. Two independent groups experiments Randomization distributions. 6 scores (persons, things) can be randomly split into 2 groups 20 ways: 1 2 3  4 5 6 1 2 4  3 5 6 1 2 5  3 4 6 1 2 6  3 4 5 1 3 4  2 5 6 1 3 5  2 4 6 1 3 6  2 4 5 1 4 5  2 3 6 1 4 6  2 3 5 1 5 6  2 3 4 2 3 4  1 5 6 2 3 5  1 4 6 2 3 6  1 4 5 2 4 5  1 3 6 2 4 6  13 5 2 5 6  1 3 4 3 4 5  1 2 6 3 4 6  1 2 6 3 5 6  1 2 4 4 5 6  1 2 3

8. Two independent groups experiments Differences between groups can be arranged as follows: -3 -1 1 3 -5 -3 -1 1 3 5 -9 -7 -5 -3 -1 1 3 5 7 9 look familiar?

9. t-distribution Gossett discovered it similar to normal, flatter tails different for each sample size, based on N-2 for two groups (degrees of freedom) randomization distribution of differences is approximated by t-distribution

10. t-distribution assumptions NORMALITY –(W test in SPSS) HOMOGENEITY OF VARIANCES IN BOTH GROUPS’ POPULATIONS –Levene’s test in SPSS INDEPENDENCE OF ERRORS –logical evaluation –Durbin-Watson test in serial data

11. Null hypothesis for test of means for two independent groups H 0 :  0 -  1 =0 H 1 :  0 -  1  0. fix a significance level, . Then we select a sample statistic. In this case we choose the sample mean for each group, and the test statistic is the sample difference d = y 0 – y 1.

12. Null hypothesis for test of means for two independent groups t = d / s d __________________________________________ = (y 0 – y 1 )/  {{ [(n 0 –1)s 2 0 + (n 1 – 1)s 2 1 ] / (n 0 + n 1 –2)} { 1/n 0 + 1/n 1 } The variance of a difference of two scores is: s 2 (y 1 -y 2 ) = s 2 1 + s 2 2 -2r 12 s 1 s 2

13. Standard deviation of differences s 2 (y 1 -y 2 ) = s 2 1 + s 2 2 -2r 12 s 1 s 2 Example, s 2 1 = 100, s 2 2 = 144, r 12 =.7 s 2 (y 1 -y 2 ) = 100 + 144 -2(.7)(10)(12) = 244 - 168 = 76 s (y 1 -y 2 ) = 8.72

14. Standard deviation of differences s 2 (y 1 -y 2 ) = s 2 1 + s 2 2 -2r 12 s 1 s 2 Example, s 2 1 = 100, s 2 2 = 144, r 12 =0 n 1 = 24, n 2 =16 s 2 (y 1 -y 2 ) = 100 + 144 = 244 s (y 1 -y 2 ) = 15.62

15. 0 SD=15.62 t-distribution, df=24+16-2 = 38

16. Standard error of mean difference score standard error of the sample difference. It consists of the square root of the average variance of the two samples,  d 2 =[(n 0 –1)s 2 0 + (n 1 – 1)s 2 1 ] / (n 0 + n 1 –2) divided by the sample sizes ( 1/n 0 + 1/n 1 )  d 2 =  d 2 / ( 1/n 0 + 1/n 1 ) Same concept as seen in sampling distribution of single mean

17. Example Willson (1997) studied two groups of college freshman engineering students, one group having participated in an experimental curriculum while the other was a random sample of the standard curriculum. One outcome of interest was performance on the Mechanics Baseline Test, a physics measure (Hestenes & Swackhammer, 1992). The data for the two groups is shown below. A significance level of.01 was selected for the hypothesis that the experimental group performed better than the standard curriculum group (a directional test): GroupMeanSDSample size Exper471575 Std Cur371650 __________________________________________ t = (47 – 37) /  [(74 y 15 2 ) + (49 y 16 2 ) / (75 + 50 – 2)][1/75 + 1/50] _______________________________ = (10) /  [(16650 + 12554) / (123)][1/75 + 1/50] = 1.947 The t-statistic is compared with the tabled value for a t-statistic with 123 degrees of freedom at the.01 significance level, 2.358. The observed probability of occurrence is 1 - 0.97309 =.02691, greater than the intended level of significance. The conclusion was that the experimental curriculum group, while performing better than the standard, did not significantly outperform them.

18. 1416 not

19. experimentwise error probability of a Type I error in any of the tests, called the experimentwise error Rough approximation: < k  Example, if we run 3 t-tests at p=.05, experimentwise error rate <.15 limit by setting experimentwise error to some value, like.05, then  =.05/k Called Bonferroni correction (when calculated exactly) = 1 - (1-  ) k

20. Confidence interval around d d  t   {{[(n 0 –1)s 2 0 + (n 1 – 1)s 2 1 ] / (n 0 + n 1 –2)} { 1/n 0 + 1/n 1 } Thus, for the example, using the.01 significance level the confidence interval is  2.358 (5.136) = (-2.11, 22.11). Thus, the population mean difference is somewhere between about -2 and 22 This includes 0 (zero) so we do not reject the null hypothesis.

21. Wilcoxon rank sum test for two independent groups. While the t-distribution is the randomization distribution of standardized differences of sample means for large sample sizes, for small samples it is not the best procedure for all unknown distributions. If we do not know that the population is normally distributed, a better alternative is the Wilcoxon rank sum test.

22. Wilcoxon rank sum test for two independent groups. Heart Rejected?Survival days Yes624, 46, 64, 1350, 280, 10, 1024, 39, 730, 136 No15, 3, 127, 23, 1, 44, 551, 12, 48, 26 Ranks for data aboveSum Yes17, 10, 12, 20, 15, 3, 19, 8, 18, 14136 No5, 2, 13, 6, 1, 9, 16, 4, 11, 774 Test StatisticsRANKDAY Mann-Whitney U19.000 Wilcoxon W74.000 Z-2.343 Asymp. Sig. (2-tailed).019 Exact Sig. [2*(1-tailed Sig.)].019

23. Confidence interval for S Confidence interval for S. While S (or U) may not be an obvious statistic to think about, both have the same standard deviation ____________ s S =  n 1 n 2 (n + 1)/12 so that for the asymptotic normality condition (with n 1 and n 2 at least 8 each), for alpha =.05. S  1.96 s S gives a 95% confidence interval. For the data above s S = 13.23, and the 95% confidence interval is 74  25.93 = (48.07, 99.93).

24. Correlation representation of the two independent groups experiment r 2 pb t 2 =  (1 – r 2 pb )/ (N-2) t 2 r 2 pb =  t 2 + N - 2 N=n 1 + n 2

25. Correlation representation of the two independent groups experiment t r pb =  t 2 + N - 2 1/2

26. x y e r pb Path model representation of two group experiment

27. Test of point biserial=0 H 0 :  pb = 0 H 1 :  pb  0 is equivalent to t-test for difference for two means.

28. Fig. 6.4: Scatterplot of ranks of days of survival for persons who experienced tissue rejection (1) or not (0) 0 1 NO YES REJECTION

29. Dependent groups experiments d = y 1 – y 0 for each pair. Now the hypotheses about the new scores becomes H 0 :  = 0 H 1 :   0 The sample statistic is simply the sample difference. The standard error of the difference can be computed from the standard deviation of the difference scores divided by n, the number of pairs

30. Dependent groups experiments _________________ s d =  [s 2 0 + s 2 1 –2r 12 s 0 s 1 ]/n Then the t-statistic is _ t = d / s d

31. Dependent groups experiments In a study of the change in grade point average for a group of college engineering freshmen, Willson (1997) recorded the following data over two semesters for a physics course: Variable N Mean Std Dev PHYS1 128 2.233333 1.191684 PHYS2 128 2.648438 1.200983 Correlation Analysis: r 12 =.5517 To test the hypothesis that the grade average changed after the second semester from the first, for a significance level of.01, the dependent samples t- statistic is ________________________________________ t = [2.648 – 2.233]/ [ 1.192 2 + 1.201 2 – 2 (.5517) x 1.192 x 1.201]/128 =.415 /.1001 = 4.145 This is greater than the tabled t-value  t(128) = 2.616. Therefore, it was concluded the students averaged higher the second semester than the first.

32. Nonparametric test of difference in dependent samples. sign test. A count of the positive (or negative) difference scores is compared with a binomial sign table. This sign test is identical to deciding if a coin is fair by flipping it n times and counting the number of heads. Within a standard error of.5n 1/2 the number should be equal to n/2.As n becomes large, the distribution of the number of positive difference scores divided by the standard error is normal. An alternative to the sign test is the Wilcoxon signed rank test or symmetry test

33. Summary of two group experimental tests of hypothesis Table is a compilation of last two chapters: –sample size –one or two groups –normal distribution or not –known or unknown population variance(s)