Independent-Samples t test Used to test for a difference between two groups when using a between-subjects design with independent samples To test for differences between the two groups when using a between-subjects design with independent samples, we use the independent-samples t test.

Single-Sample z-Test t-Test Randomly selected sample DV normally distributed DV measured using ratio or interval scale mean of the population is known SD of the population is known Randomly selected sample DV normally distributed DV measured using ratio or interval scale mean of the population is known SD of the population is not known and must be estimated Let’s consider the assumptions for an independent-samples t-test and how they compare to the other tests we’ve talked about Z-test assumptions Single-sample t-test Review the assumptions… With the z-test and the single-sample t-test, we had one group and compared this group against a known population

Single-Sample Independent-Samples t-Test t-Test Randomly selected sample DV normally distributed DV measured using ratio or interval scale mean of the population is known SD of the population is not known and must be estimated Randomly selected sample DV normally distributed DV measured using ratio or interval scale Homogeneity of variance With the independent-samples t-test, you are comparing 2 groups Do you think an assumption will be made about the sample being randomly selected? Yes Normally distributed? Yes, because you are still making inferences about the populations that these 2 groups come from. You’re asking “Are they from the same population or different populations?” Those inferences require you to use t tables that are based on the normal distribution. What about having to be measured on an interval or ratio scale? Well, a t test compares the mean of 2 groups. Can you really get a mean when using a nominal or ordinal scale? NO So, this assumption remains. We drop the assumption about the pop. mean because we are no longer comparing to a population whose mean is known. We no longer talk about estimating the SD of the population because that point is made to contrast the single sample t-test with the z-test. We will be estimating variability (using s2), but now we will be doing it for more than one population. And we will be assuming that the variance in each of those populations is roughly the same.

General Model for z-Test and Single-Sample t-Test Original Population Sample H0 Treated Population HA Looking back at the model for the z-test or for the single-sample t test, we were trying to determine if our sample came from the original population or if it represented a different, treated population. To say that the sample came from the original population would mean that even if everyone in the population got the treatment, the population mean would not be significantly different from what it was before treatment. (Supporting the null hypothesis that the treatment did not have an effect.) If the sample represents a different, treated population, this implies that if everyone in the original population were to get the treatment our sample received, the population mean on our DV would be different from what it was when the population was untreated. (Supporting the alternative hypothesis that the treatment did have an effect.)

General Model for Independent-Samples t-Test Population A Sample A H0 Population B Sample B For an independent-samples t test, we are trying to determine whether both of our samples come from one population or from two different populations (whose means are different from each other). The null hypothesis says that they come from one population, so there’s no difference between their means. The null hypothesis is written as H0: 1 - 2 = 0 This formula could be reworked to show that 1 = 2 , but it will be represented as 1 - 2 = 0 when we work with it for reasons that will become clear soon. This null hypothesis says that whether or not we administer the manipulation, both groups will have the same level of the DV, and even if the manipulation were administered to the entire population, the outcome would be the same as it would be if you didn’t administer the manipulation. H0: 1 - 2 = 0

General Model for Independent-Samples t-Test Population A Sample A HA Population B Sample B The alternative hypothesis says that our two samples come from 2 different populations, implying that the population means are different. It is written as HA: 1 - 2  0. Again this could be represented as 1  2 but we will use 1 - 2  0 as our alternative hypothesis, and it will be considered incorrect to say 1  2 when asked for HA The alternative hypothesis amounts to saying that if we administer the manipulation, the group that receives it will score differently on the DV than will the control group that doesn’t receive the manipulation. Thus, if the manipulation were administered to the entire population, the population mean would be different than it was before you administered the manipulation. HA: 1 - 2  0

Sampling Distribution of the Difference Between the Means Remember, the independent-samples t test helps us determine whether the mean of one sample differs from the mean of the second sample. The t table gives us information based on the theoretical t distribution. For the z test and the one-sample t test, our theoretical t distribution was derived by taking a sample of N scores from the population, calculating the mean, plotting the frequency of each of these means, returning the scores to the population, and repeating an infinite # of times. For the independent-samples t test, we are not interested in the mean of a sample of scores; instead, we interested in the difference between the means of 2 samples of scores. μ

Sampling Distribution of Difference Between the Means To create this sampling distribution: Select 2 random samples from one population Each sample is the same size as the N of our groups Compute the sample mean for each sample Subtract one sample mean from the other and plot the difference Do this an infinite # of times To get the sampling distribution to use in the independent-samples t test, we would select 2 samples from the population (each with N1 and N2 scores), calculate the difference between the means, plot the difference between the means, return the scores to the population, and repeat an infinite # of times The resulting distribution is called the: sampling distribution of difference between the means With the single-sample t test, we compared t-obtained against t-crit to determine the probability of obtaining our sample mean when the null hypothesis was true. Now, when we’re comparing our t-obtained against t-crit, we are determining the probability of obtaining our observed difference between the sample means when the null hypothesis is true.

Standard Error of the Difference between the Means The average distance between the mean of the sampling distribution (of the difference between the means) and all of the differences between the means plotted in the sampling distribution of the differences between the means. How much difference should you expect between the sample means even if your treatment has no effect? As with previous definitions of standard error, this one is still the average distance between the mean of the sampling distribution (which is equal to the mean of the population from which the sampling distribution was taken), and the scores that make up the sampling distribution. In this case, the scores that make up the sampling distribution are the differences between the mean of sample 1 and the mean of sample 2. This could be stated more simply as: the average distance between the mean and all the scores in the sampling distribution of the difference between the means. The standard error of the difference between the means is the answer to the question at the bottom.

Independent-Samples t-Test t Tests Formulas Single-Sample t-Test Independent-Samples t-Test Here you see the general description of the formulas for one-sample versus independent-samples t tests. The formula for the t tests have similar forms, but instead of the sample mean, you have the difference between the two samples’ means, and instead of the population mean, you have the difference between the two populations’ means.

Independent-Samples t-Test Formula Definitional Formulas Single-Sample t-Test Independent-Samples t-Test Here are the definitional formulas for the one- vs. independent-sample t tests. The independent-samples t test uses the difference between 2 sample means to estimate the difference between 2 population means. In each of the t-obtained formulas, the standard error in the denominator tells us how much difference is reasonable to expect between the sample mean and the population mean if the treatment does not have an effect. In the single-sample t formula, the standard error measures the amount of error expected for a sample mean (i.e., how far we should expect the sample mean to be from the population mean, on average, if the treatment has no effect). In the independent-samples t formula, the standard error measures the amount of difference that is expected between the two components of the numerator due to sampling error. The two components of the numerator are: (1) the difference between the sample means (sample mean 1 – sample mean 2) and (2) the difference between the population means (population mean 1 – population mean 2).

Single-Sample t-Test Step 1: Step 2: Step 3: Estimated variance of the population (definitional formula) Step 3: These are the formulas for hand-calculating the single-sample t. Single-sample t-test: Step 1  calculate the estimated variance of the population Step 2  calculate the estimated standard error of the mean Step 3  calculate t Estimated standard error of the mean

Single-Sample Independent-Samples t-Test t-Test Step 1: calculate the estimated variance of the population calculate the estimated variance of the population for each group Breaking down the previous slide, we start with Step 1: Calculate the estimated variance of the population for each group Single-sample t-test had to estimate the variance of the population Independent-samples t-test Now have to estimate the variance of the population for each group Because this is an estimate of the population, it is not likely that the variance for either group is exactly the same as the true variance in the population Why? Because of sampling error, as usual The actual variance of the population may be closer to one of the estimates than the other (the estimated variance based on either group 1 or group 2). Because we don’t know which of the two estimates is closest to the actual population variance, we take an average of our 2 estimates. However, it’s not a simple average (variance 1 + variance 2) / 2; instead, it’s a weighted average. The reason we want a weighted average is because our sample sizes may not be equal, so we want to weight the variances according the sample size (so that the result accurately reflects the contribution of each group).

Pooled Variance Step 1a: Calculate the pooled variance This is the formula for calculating the pooled variance. Click on the speaker in the powerpoint for an explanation of why we want to use the pooled variance. 14

Pooled Variance  Equal Sample Sizes SS1 = 50 SS2 = 30 n1 = 6 n2 = 6 What would the pooled variance be? If the sample sizes for the 2 groups are equal, then you will get the same answer if you add the variances for the 2 samples and divide by 2 as you would if you used the pooled variance formula. Average of s12 and s22 15

Pooled Variance  Unequal Sample Sizes SS1 = 20 SS2 = 48 n1 = 3 n2 = 9 But what if the size of your two samples is NOT the same? Will the pooled variance be the same as the simple average variance? Why or why not? Calculate the pooled variance. Compare the result to the result you would get using simple averaging. If the sample sizes for the 2 groups are NOT equal, you will need to use the formula for the pooled variance. Otherwise, you will get a BIASED estimate for the variance, meaning that it will not accurately reflect the contribution of each group’s variance. Average of s12 and s22 16

Single-Sample Independent-Samples t Test t Test Step 2: calculate the estimated standard error of the mean calculate the estimated standard error of the difference between the means (standard error of the difference) Step 2  For the single-sample t test, we calculated the estimated standard error of the mean For the independent-sample t test, we calculate the estimated standard error of the difference between the means. This is generally referred to as the standard error of the difference.

Single-Sample Independent-Samples t-Test t-Test Step 2: calculate the estimated standard error of the mean calculate the estimated standard error of the difference between the means (standard error of the difference) The formula for the standard error of the difference can be rewritten as shown here.

Single-Sample Independent-Samples t-Test t-Test Step 3: calculate tobt Step 3 is to calculate t obtained

Single-Sample Independent-Samples t-Test t-Test Step 3: calculate tobt The value for µ 1 – µ 2 = 0 comes from the null hypothesis. Because the null hypothesis sets the population mean difference equal to 0, this part of the equation drops out.

Hypothesis Testing with Two Independent Samples If the assumptions are met, then we proceed with testing our hypotheses just like we did before

Step 1. State the hypotheses (two-tailed) A. Is it a one-tailed or two-tailed test? Two-tailed B. Research hypotheses Alternative hypothesis: There is a difference between the control group and the experimental group. Null hypothesis: There is no difference between the control group and the experimental group. C. Statistical hypotheses: HA: 1 - 2  0 which is equivalent to 1  2 H0: 1 - 2 = 0 which is equivalent to 1 = 2 For a two-group experiment, the alternative hypothesis is that there is a difference between the control group and the experimental group The null hypothesis is that there is NO difference between the control group and the experimental group What about the statistical hypotheses? Think about what you’re trying to determine  Is there a difference between the mean of the experimental group and the mean of the control group? So when you write the statistical hypotheses, you should indicate that you’re looking at the difference between these 2 groups The HA says that there IS a difference between the groups, so your difference is NOT 0 The H0 says that there is NOT a difference, so your difference equals 0 You want to write the hypotheses indicating that you are comparing the difference between the means of the populations (µ) instead of the difference between the means of the samples. Although we are comparing the sample means, we are using them to infer about the difference between the populations. At the right on each line, you see that you can re-arrange the variables to indicate that the population means do not or do equal each other. We don’t write it this way because the way it is written at the left reflects the way in which the sampling distribution is created, from the difference between the means.

The HA and H0 Hypotheses The HA says that there is a difference between the groups, so your difference is NOT zero The H0 says that there is NOT a difference, so your difference equals zero You can put the control group or the experimental group as group 1 in your equations, but you HAVE TO BE CONSISTENT You should substitute abbreviated names based on the conditions instead of 1 and 2 as subscripts To reiterate, The HA says that there is a difference between the groups, so your difference is NOT 0. The H0 says that there is NOT a difference, so your difference equals 0. You can put the control group or the experimental group as group 1 in your equations, but you HAVE TO BE CONSISTENT You should substitute abbreviated names based on the conditions for 1 and 2 as subscripts

Step 1. State the hypotheses. A. Is it a one-tailed or two-tailed test? One-tailed Now let’s look at the hypotheses for one-tailed tests.

Step 1. State the hypotheses (one-tailed) B. Research hypotheses Alternative hypothesis: The experimental group will perform better than the control group. The experimental group’s scores will be lower than the control group’s score. Null hypothesis: The experimental group will perform the same as or worse than the control group. The experimental group’s scores will be the same as or higher than the control group’s scores. C. Statistical hypotheses: HA: experimental - control > 0 experimental - control < 0 H0: experimental - control < 0 experimental - control > 0 Here we are going to see how to write the hypotheses for a two-sample t test when the hypotheses are one-tailed. Think about what you’re trying to determine  Is the mean of the experimental group higher (or in the case of a hypothesis in the other direction, lower) than the mean of the control group? In black, there is a template for writing the hypothesis in which the experimental group is expected to have higher scores than the control group. In red, the experimental group is expected to have lower scores than the control group. When you write the statistical hypotheses, you should indicate that you’re looking at the difference between these 2 groups In black the HA says that the scores of the experimental group minus the scores of the control group equal a positive number (because the experimental group mean is higher than the control group mean) In red, the HA says that the scores of the experimental group minus the scores of the control group equal a negative number (because the experimental group mean is lower than the control group mean) In black the H0 says that the scores of the experimental group minus the scores of the control group do not equal a positive number; rather, they are either zero or negative (because the experimental group mean is lower than the control group mean) In red, the H0 says that the scores of the experimental group minus the scores of the control group do not equal a negative number; rather, they are either zero or positive (because the experimental group mean is higher than the control group mean) As always, whatever direction the HA indicates, the H0 has to cover the other two possibilities (the three possibilities being greater than, equal to, or less than).

Step 1. State the hypotheses. A. Is it a one-tailed or two-tailed test? One-tailed B. Research hypotheses Alternative hypothesis: Participants who eat peppermint will score higher than those who don’t eat peppermint on the digit recall test. Null hypothesis: Participants who eat peppermint will score the same as or lower than those who don’t eat peppermint on the digit recall test. C. Statistical hypotheses: HA: peppermint - no peppermint > 0 H0: peppermint - no peppermint < 0 Now let’s come up with the hypotheses for our experiment with the peppermint and digit span scores.

Step 2. Set the significance level   = .05. Determine tcrit. Factors that must be known to find tcrit 1. Is it a one-tailed or a two-tailed test? one-tailed 2. What is the alpha level? .05 3. What are the degrees of freedom? df = ?

Independent-Samples t-Test Degrees of Freedom Single-Sample t-Test Independent-Samples t-Test df = (n – 1) df = (n1 – 1) + (n2 – 1) = n1 + n2 – 2 For the one-sample t test, df = n -1 So what do you think it would be for 2 samples? For the independent samples t test there are two groups, each of which is constrained by the fact that the sum of deviations from the mean of that group = 0. Therefore the df for this test are:  (n1 - 1) + (n2 - 1)  or n1 + n2 - 2

Step 3. Select and compute the appropriate statistical test. Step 1a: Step 3: Step 2: So when we arrive at Step 3, it’s time to select and compute the appropriate statistical test. There are several sub-steps to follow once you recognize that you are going to use a two-independent-samples t test: Step 1  estimate the variance of each population using the sample variance (s-squared) Step 1a  calculate the pooled variance using the SSs and degrees of freedom obtained in Step 1 when calculating the sample variance for each group. Step 2  calculate the estimated standard error of the difference between the means Step 3  calculate t 29

Step 4. Make a decision. tcrit = ??? Determine whether the value of the test statistic is in the critical region. Draw a picture. tcrit = ???

Step 4. Make a decision. +tcrit -tcrit If +tobt > +tcrit OR if -tobt < -tcrit  Reject Ho If -tcrit < tobt < +tcrit  Retain Ho +tcrit -tcrit Remember, the critical region is in the tail of the distribution. If the positive value of t-obtained is larger than a positive critical value of t, or If the negative value of t-obtained is smaller than a negative critical value of t, then reject the null hypothesis. If the value of t-obtained falls above the negative or below the positive critical values of t, then retain the null hypothesis. REJECT H0 REJECT H0 RETAIN H0

Step 5. Report the statistical results. Reject H0: t(df) = tobt, p < .05 Retain H0: t(df) = tobt, p > .05 If the probability (p) of being wrong if you reject the null hypothesis is less than .05, then you reject the null, because you are willing to accept up to 5% chance (or whatever you’ve set for alpha) of being wrong when you reject the null (i.e., up to 5% chance of making a Type I error). If the probability (p) of being wrong if you reject the null hypothesis is greater than .05, then you retain the null, because you are not willing to accept more than a 5% chance (or whatever you’ve set for alpha) of being wrong when you reject the null (i.e., more than a 5% chance of making a Type I error).

Step 6: Write a conclusion. State the relationship between the IV and the DV in words, ending with the statistical results. General format: Members of the first group (M = xx.xx) did/did not score lower/higher/differently than members of the second group (M = xx.xx), t(df) = tobt, p < > .05.

Hypothesis Testing with Two Independent Samples An Example

An Example Research Question: Are students who calculate statistics by hand better able to select the appropriate statistical test to use than students who do not calculate statistics by hand (who use SPSS)? Assume that past research has consistently shown that students who calculate statistics by hand are better, so we decide to generate a directional (one-tailed) hypothesis. Follow along on Worksheet#1

Step 1. State the hypotheses. A. Is it a one-tailed or two-tailed test? One-tailed B. Research hypotheses Alternative hypothesis: Students who calculate statistics by hand are better able to select the appropriate statistical test to use than students who do not calculate statistics by hand. Null hypothesis: Students who calculate statistics by hand are not better (i.e., are no different from or are less able) to select the appropriate statistical test than students who do not calculate stats by hand. C. Statistical hypotheses: HA: hand - SPSS > 0 H0: hand - SPSS < 0 What is the alternative hypothesis for a two-group experiment? What is the null hypothesis for a two-group experiment? What about the statistical hypotheses? Think about what you’re trying to determine  Is there a difference between the mean of the experimental group and the mean of the control group? So when you write the statistical hypotheses, you should indicate that you’re looking at the difference between these 2 groups The alternative hypothesis is that hand calculating results in higher scores than using SPSS. The null is that hand-calculating does not result in higher scores than using SPSS; rather, it results in the same or lower scores.

Step 2. Set the significance level   = .05. Determine tcrit. 1. Is it a one-tailed or a two-tailed test? one-tailed 2. What is the alpha level? .05 3. What are the degrees of freedom? n1 = 5; n2 = 5 df = (n1 – 1) + (n2 – 1) = (5 -1) + (5-1) = 4 + 4 = 8 tcrit = 1.860 Factors that must be known to find tcrit: 1. Is it a one-tailed or a two-tailed test? one-tailed 2. What is the alpha level? .05 3. What are the degrees of freedom based on the sample? df = ?

Step 3. Select and compute the appropriate statistic. Independent-Samples t-Test

Hand-CalculationScores (X1) 8 7 10 9 ∑X1= n = ∑(X – )2 = You have a set of raw scores for Group 1. First find the mean of that set of scores. Then find the deviation of each score from that mean. Then square each deviation. 39

Hand-CalculationScores (X1) 8 8.4 -.4 .16 7 -1.4 1.96 10 1.6 2.56 9 .6 .36 ∑X1= 42 n = 5 ∑(X – )2 = 5.2 You have a set of raw scores for Group 1. First find the mean of that set of scores. Then find the deviation of each score from that mean. Then square each deviation. 40

X – (X – )2 6 5 7 8 ∑X2= n = ∑(X – )2 = SPSS Scores (X2) Do the same for the other group. 41

SPSS Scores (X2) X – (X – )2 6 6.4 -.4 .16 5 -1.4 1.96 7 .6 .36 8 1.6 2.56 ∑X2= 32 n = 5 ∑(X – )2 = 5.2 Do the same for the other group. 42

1) calculate the estimated variance of each population Step 1  calculate the estimated variance of each population 43

1a) calculate the pooled variance Step 1  calculate the estimated variance of each population Step 1a  calculate the pooled variance 44

2) calculate the estimated standard error of the difference between the means Step 2  calculate the estimated standard error of the difference between the means 46

3) calculate tobt Step 3  calculate t 48

Step 4. Make a decision. tcrit = + 1.860 If +tobt > +tcrit  Reject Ho If tobt < +tcrit  Retain Ho tcrit = + 1.860 The critical region is in the positive tail of the distribution when the test is one-tailed and the expected result is an increase in scores on the DV. tobt = 2.77

Step 5. Report the statistical results. t(8) = 2.77, p < .05

Step 6: Write a conclusion. State the relationship between the IV and the DV in words: Students who calculate statistics by hand (M = 8.4) are significantly better at selecting the appropriate statistical test to use than students who do not calculate statistics by hand (M = 6.4), t(8) = 2.77, p < .05. Start with a description of the experimental group and immediately indicate its mean score. Next, describe the direction of the difference between that group and the other group and indicate the outcome on which the groups are different (in this case, selecting appropriate statistical tests). Once you’ve named the comparison group, state its mean. End with the statistical results, separated from the rest of the sentence by a comma.

Step 7. Compute the estimated d. Just like we did for the one-sample t-test, we can calculate Cohen’s estimated d to learn how many standard deviations difference there are between the two groups. Estimated d = mean difference / pooled standard deviation (as you see from the formula, for the denominator, you find the square root of the pooled variance).

Step 7. Compute the estimated d.

Percentage of Variance Explained (r2) Describing the Strength of the Relationship We can also calculate r squared using our t obtained from the independent-samples t test. We’ll use the same formula as we did for the one-sample t test.

Step 8. Compute r2 and write a conclusion.

Step 8. Compute r2 and write a conclusion. Approximately 49% of the variance in students’ ability to select the appropriate statistical test can be accounted for by their group membership (i.e., the way they learned statistics). The way students learned statistics can account for 48.96% of the variance in their ability to select the appropriate statistical test. The general format: Approximately X% of the variance in the DV can be attributed to/accounted for by the IV. The IV can account for X% of the variance in the DV.