Significance and t testing FSE 200

Salkind, Chapter 9 Significance

What You Will Learn in Chapter 9 What significance is and why it is important Significance versus meaningfulness Type I error Type II error How inferential statistics works How to determine the right statistical test

The Concept of Significance Any difference between groups that is due to a systematic influence rather than chance Must assume that all other factors that might contribute to differences are controlled

If Only We Were Perfect… Significance level The risk associated with not being 100% positive that what occurred in the experiment is a result of what you did or what is being tested The goal is to eliminate competing reasons for differences as much as possible

The Null Hypothesis and Your Action Statistical significance The degree of risk you are willing to take that you will reject a null hypothesis when it is actually true

The Null Hypothesis and Your Action

Type I Errors (Level of Significance) The probability of rejecting a null hypothesis when it is true Conventional levels are set between .01 and .05 Usually represented in a report as p < .05

Type II Errors The probability of rejecting a null hypothesis when it is false As your sample characteristics become closer to the population, the probability that you will accept a false null hypothesis decreases

Different Types of Errors

Significance Versus Meaningfulness A study can be statistically significant but not very meaningful Statistical significance can be interpreted only in terms of the context in which it occurred Statistical significance should not be the only goal of scientific research Significance is influenced by sample size…we’ll talk more about this later

How Inference Works A representative sample of the population is chosen A test is given, and means are computed and compared A conclusion is reached as to whether the scores are statistically significant Based on the results of the sample, an inference is made about the population

Deciding Which Test to Use Determining which statistical test to use

Test of Significance 1. A statement of null hypothesis 2. Set the level of risk associated with the null hypothesis 3. Select the appropriate test statistic 4. Compute the test statistic (obtained) value 5. Determine the value needed to reject the null hypothesis using appropriate table of critical values 6. Compare the obtained value to the critical value 7. If obtained value is more extreme, reject null hypothesis 8. If obtained value is not more extreme, accept the null

The Picture Worth a Thousand Words Making decisions about the null hypothesis

Tests between the means of different groups Salkind, Chapter 10 Tests between the means of different groups

What You Will Learn in Chapter 10 When to use a t test How to compute the observed t value How to use the TTEST function How to use the t test Toolpak tool to compute the t value Interpreting the t value and what it means

t Tests for Independent Samples Determining the correct test statistic

Computing the Test Statistic Numerator is the difference between the means Denominator is the amount of variation within and between each of the two groups

Degrees of Freedom Degrees of freedom approximate the sample size Degrees of freedom can vary based on the test statistic selected For this procedure… n1 – 1 + n2 – 1 or n1 + n1 -2

So How Do I Interpret… t(58) = –.14, p > .05 t represents the test statistic used 58 is the number of degrees of freedom –.14 is the obtained value p > .05 indicates the probability

TTEST Function TTEST does not compute t value It returns the likelihood the resulting t value is due to chance

TTEST Function Data for Using the TTEST Function

TTEST Function Using TTEST to compute the probability of a t value

Special Effects… Effect size is a measure of how different two groups are from one another Standardized difference between two group means Jacob Cohen

Computing Effect Size Small = 0.0 – .20 Medium = .20 – .50 Large = .50 and above

Acknowledgement The majority of the content of these slides were from the Sage Instructor Resources Website