1. Significance and t testing
FSE 200

2. Salkind, Chapter 9
Significance

3. What You Will Learn in Chapter 9
What significance is and why it is important
Significance versus meaningfulness
Type I error
Type II error
How inferential statistics works
How to determine the right statistical test

4. The Concept of Significance
Any difference between groups that is due to a systematic influence rather than chance
Must assume that all other factors that might contribute to differences are controlled

5. If Only We Were Perfect…
Significance level
The risk associated with not being 100% positive that what occurred in the experiment is a result of what you did or what is being tested
The goal is to eliminate competing reasons for differences as much as possible

6. The Null Hypothesis and Your Action
Statistical significance
The degree of risk you are willing to take that you will reject a null hypothesis when it is actually true

7. The Null Hypothesis and Your Action

8. Type I Errors (Level of Significance)
The probability of rejecting a null hypothesis when it is true
Conventional levels are set between .01 and .05
Usually represented in a report as
p < .05

9. Type II Errors
The probability of rejecting a null hypothesis when it is false
As your sample characteristics become closer to the population, the probability that you will accept a false null hypothesis decreases

10. Different Types of Errors

11. Significance Versus Meaningfulness
A study can be statistically significant but not very meaningful
Statistical significance can be interpreted only in terms of the context in which it occurred
Statistical significance should not be the only goal of scientific research
Significance is influenced by sample size…we’ll talk more about this later

12. How Inference Works
A representative sample of the population is chosen
A test is given, and means are computed and compared
A conclusion is reached as to whether the scores are statistically significant
Based on the results of the sample, an inference is made about the population

13. Deciding Which Test to Use
Determining which statistical test to use

14. Test of Significance 1. A statement of null hypothesis
2. Set the level of risk associated with the null hypothesis
3. Select the appropriate test statistic
4. Compute the test statistic (obtained) value
5. Determine the value needed to reject the null hypothesis using appropriate table of critical values
6. Compare the obtained value to the critical value
7. If obtained value is more extreme, reject null hypothesis
8. If obtained value is not more extreme, accept the null

15. The Picture Worth a Thousand Words
Making decisions about the null hypothesis

16. Tests between the means of different groups
Salkind, Chapter 10
Tests between the means of different groups

17. What You Will Learn in Chapter 10
When to use a t test
How to compute the observed t value
How to use the TTEST function
How to use the t test Toolpak tool to compute the t value
Interpreting the t value and what it means

18. t Tests for Independent Samples
Determining the correct test statistic

19. Computing the Test Statistic
Numerator is the difference between the means
Denominator is the amount of variation within and between each of the two groups

20. Degrees of Freedom Degrees of freedom approximate the sample size
Degrees of freedom can vary based on the test statistic selected
For this procedure…
n1 – 1 + n2 – 1 or n1 + n1 -2

21. So How Do I Interpret… t(58) = –.14, p > .05
t represents the test statistic used
58 is the number of degrees of freedom
–.14 is the obtained value
p > .05 indicates the probability

22. TTEST Function TTEST does not compute t value
It returns the likelihood the resulting t value is due to chance

23. TTEST Function
Data for Using the TTEST Function

24. TTEST Function
Using TTEST to compute the probability of a t value

25. Special Effects…
Effect size is a measure of how different two groups are from one another
Standardized difference between two group means
Jacob Cohen

26. Computing Effect Size Small = 0.0 – .20 Medium = .20 – .50
Large = .50 and above

27. Acknowledgement
The majority of the content of these slides were from the Sage Instructor Resources Website