1. Hypothesis Testing

2. Outline The Null Hypothesis Type I and Type II Error
Using Statistics to test the Null Hypothesis
The Logic of Data Analysis

3. Research Questions and Hypotheses
Non-directional:
No stated expectation about outcome
Example:
Do men and women differ in terms of conversational memory?
Hypothesis:
Statement of expected relationship
Directionality of relationship
Women will have greater conversational memory than men

4. Grounding Hypotheses in Theory
Hypotheses have an underlying rationale:
Logical reasoning behind the direction of the hypotheses (theoretical rationale – explanation)
Why do we expect women to have better conversational memory?
Theoretical rationale based on:
1. Past research
2. Existing theory
3. Logical reasoning

5. The Null Hypothesis Null Hypothesis - the absence of a relationship
E..g., There is no difference between men’s and women’s with regards to conversational memories
Compare observed results to Null Hypothesis
How different are the results from the null hypothesis?
We do not propose a null hypothesis as research hypothesis - need very large sample size / power
Used as point of contrast for testing

6. Hypotheses testing When we test observed results against null:
We can make two decisions:
1. Accept the null
No significant relationship
Observed results similar to the Null Hypothesis
2. Reject the null
Significant relationship
Observed results different from the Null Hypothesis
Whichever decision, we risk making an error

7. Type I and Type II Error 1. Type I Error 2. Type II Error
Reality: No relationship
Decision: Reject the null
Believe your research hypothesis have received support when in fact you should have disconfirmed it
Analogy: Find an innocent man guilty of a crime
2. Type II Error
Reality: Relationship
Decision: Accept the null
Believe your research hypothesis has not received support when in fact you should have rejected the null.
Analogy: Find a guilty man innocent of a crime

8. Potential outcomes of testing
Decision
Accept Null Reject Null R
E No
A Relationship L
I Relationship T Y 1 2 3 4

9. Potential outcomes of testing
Decision
Accept Null Reject Null R
E No
A Relationship L
I Relationship T Y
Correct
decision 2 3 4

10. Potential outcomes of testing
Decision
Accept Null Reject Null R
E No
A Relationship L
I Relationship T Y 1 2 3
Correct
decision

11. Potential outcomes of testing
Decision
Accept Null Reject Null R
E No
A Relationship L
I Relationship T Y 1
Type I Error 3 4

12. Potential outcomes of testing
Decision
Accept Null Reject Null R
E No
A Relationship L
I Relationship T Y 1 2 4
Type II Error

13. Potential outcomes of testing
Decision
Accept Null Reject Null R
E No
A Relationship L
I Relationship T Y
Correct
decision
Type I Error
Correct
decision
Type II Error

14. Function of Statistical Tests
Statistical tests determine:
Accept or Reject the Null Hypothesis
Based on probability of making a Type I error
Observed results compared to the results expected by the Null Hypotheses
What is the probability of getting observed results if Null Hypothesis were true?
If results would occur less than 5% of the time by simple chance then we reject the Null Hypothesis

15. Start by setting level of risk of making a Type I Error
How dangerous is it to make a Type I Error:
What risk is acceptable?:
5%?
1%?
.1%?
Smaller percentages are more conservative in guarding against a Type I Error
Level of acceptable risk is called “Significance level” :
Usually the cutoff - <.05

16. Conventional Significance Levels
.05 level (5% chance of Type I Error)
.01 level (1% chance of Type I Error)
.001 level (.1% chance of Type I Error)
Rejecting the Null at the .05 level means:
Taking a 5% risk of making a Type I Error

17. Steps in Hypothesis Testing
1. State research hypothesis
2. State null hypothesis
3.Set significance level (e.g., .05 level)
4. Observe results
5. Statistics calculate probability of results if null hypothesis were true
6. If probability of observed results is less than significance level, then reject the null

18. Guarding against Type I Error
Significance level regulates Type I Error
Conservative standards reduce Type I Error:
.01 instead of .05, especially with large sample
Reducing the probability of Type I Error:
Increases the probability of Type II Error
Sample size regulates Type II Error
The larger the sample, the lower the probability of Type II Error occurring in conservative testing

19. Statistical Power The power to detect significant relationships
The larger the sample size, the more power
The larger the sample size, the lower the probability of Type II Error
Power = 1 – probability of Type II Error

20. Statistical Analysis Statistical analysis:
Examines observed data
Calculates the probability that the results could occur by chance (I.e., if Null was true)
Choice of statistical test depends on:
Level of measurement of the variables in question:
Nominal, Ordinal, Interval or Ratio

21. Logic of data analysis Univariate analysis Bivariate analysis
One variable at a time (descriptive)
Bivariate analysis
Two variables at a time (testing relationships)
Multivariate analysis
More than two variables at a time (testing relationships and controlling for other variables)

22. Variables Dependent variable: Independent variables:
What we are trying to predict
E.g., Candidate preference
Independent variables:
What we are using as predictors
E.g., Gender, Party affiliation

23. Testing hypothesis for two nominal variables
Variables Null hypothesis Procedure
Gender
Passing is not Chi-square
related to gender
Pass/Fail

24. Testing hypothesis for one nominal and one ratio variable
Variables Null hypothesis Procedure
Gender
Score is not T-test
related to gender
Test score

25. Testing hypothesis for one nominal and one ratio variable
Variable Null hypothesis Procedure
Year in school
Score is not
related to year in ANOVA
school
Test score
Can be used when nominal variable has more than two categories and can include more than one independent variable

26. Testing hypothesis for two ratio variables
Variable Null hypothesis Procedure
Hours spent
studying Score is not
related to hours Correlation
spent studying
Test score

27. Testing hypothesis for more than two ratio variables
Variable Null hypothesis Procedure
Hours spent
studying Score is not positively
related to hours
Classes spent studying and Multiple
missed not negatively related regression
to classes missed
Test score

28. Commonality across all statistical analysis procedures
Set the significance level:
E.g., .05 level
Means that we are willing to conclude that there is a relationship if:
Chance of Type I error is less than 5%
Statistical tests tell us whether:
The observed relationship has less than a 5% chance of occurring by chance

29. Summary of Statistical Procedures
Variables
Procedure
Nominal IV, Nominal DV
Chi-square
Nominal IV, Ratio DV
T-test
Multiple Nominal IVs, Ratio DV
ANOVA
Ratio IV, Ratio DV
Pearson’s R
Multiple Nominal IVs, Ratio DV with ratio covariates
ANCOVA
Multiple ratio
Multiple Regression