1. Significance testing Introduction to Intervention Epidemiology
Tunis, 3 November 2014
Dr. Ibrahim Saied
Epidemiologist, Ministry of Health and Population - Egypt

2. Objectives Building sound concepts about: Null hypothesis P value
Significance testing
Confidence interval
By the end of this presentation the participants are aimed to have sound concepts about:
No point no five

3. Association Vs Significance test
RR=1
P = 0.03
RR=5 X ?
Problem ?
Factor B
Factor A
From engagement to marriage
I have a problem
Significance testing
Calculation or Estimation of Association
What about factor A
Also what about factor B

4. The aim of a statistical test+
To reach a scientific decision (“yes” or “no”) on a difference (or effect), on a probabilistic basis, on observed data.
Which group is taller,
female or male?
Subjective
Objective To

5. Null Hypothesis

6. Null vs. alternative hypotheses
Null hypothesis (H0): “There is no difference (no effect).” RR=1 ; OR=1
Alternative hypothesis (H1): “There is a difference (effect).”; indicates that the null hypothesis is not true. RR ≠ 1 ; OR ≠ 1
→ The intention of a significance test is to reject the null hypothesis!

7. Even in law
The accused person has to be considered innocent until proved otherwise.

8. Null hypothesis
These
squares
have no
white
elephants

9. Alternative hypothesis
These
squares
have
white
elephants

10. Right decisions and types of errors
Truth
H0 is true
H0 is false
H0 is not rejected
Correct
decision
Type II error
( error)
Decision
H0 is rejected
Correct
decision
Type I error
(-error)
Power

11. P-value

12. Probability of getting our result (observation) due to chance
p-value
Probability of getting our result (observation) due to chance
Chance
Results
Similarity
25 %
15 %
3 %
4 %
P value
Help to reject null hypothesis 8

13. How to interpret P value
Statistical expression
0.05
5 %
Significant
Not significant
0.01
0.04
0.05
0.06
0.15

14. Significance Tests

15. Some types of significance tests
According to type of variables:
Variable 1
Variable 2
Test
Categorical
Chi square
Numerical
t-test
Correlation

16. Test decision >> Scenario 1
0.17
p value
Expected
Observed
Not Reject
Null Hypothesis
Short distance

17. Test decision >> Scenario 2 Reject Null Hypothesis
0.03
p value
Expected
Observed
Reject Null Hypothesis
Long distance

18. How to read test results?
Test Value
P Value
Sig.
Test
Chi =
0.032
t =
r =

19. Example from SPSS

20. Example from SPSS (Cont’d)
Chi-Square = 0.487,  p = .485.
When reading this table we are interested in the results of the "Pearson Chi-Square" row. We can see here that χ(1) = 0.487, p = This tells us that there is no statistically significant association between Gender and Preferred Learning Medium; that is, both Males and Females equally prefer online learning versus books.
This tells us that there is no statistically significant association between Gender and Preferred Learning Medium; that is, both Males and Females equally prefer online learning

21. Confidence interval

22. Confidence interval
Range of values, on the basis of the sample data, in which the population value (or true value) may lie.
Example:
A 95% CI includes the true value with a certainty of 95%.

23. Confidence Interval
A confidence interval represents the range of effects that are compatible with the data.
CI provides
Precision of the point estimate
Direction of the effect (risk factor, protective factor)
Magnitude of the measured effect
How reliable are the information (parameters) one obtains from the data?

24. Wide Confidence interval
Parameter *
Small Sample
Sample value 2
95% of values * 1
Wide Confidence interval

25. Narrow Confidence interval
Parameter *
Large
Sample
Sample value 2 * 1
95% of values
Narrow Confidence interval

26. Confidence interval (CI)
Frequently used formulation:
If the data collection and analysis could be replicated many times, the CI should include within it the true value of the measure 95% of the time.

27. (lower limit ; upper limit) Point estimate ± “deviation”
Structure of CIs
(lower limit ; upper limit)
Point estimate ± “deviation”
“deviation” depends on
Sample size
Level of confidence
Variability of data
“deviation” comprises the Standard error

28. precision of the estimates increases
Confidence Interval
Sample size increases 
RR=4 ( ) RR=4.5 ( )
High data variability  large CI
High confidence level  large CI
precision of the estimates increases
Sample of 25
Sample of 85

29. CIs and statistical significance
If the null hypothesis (RR=1) is included within the CI
→ not significant RR=2 ( )
(0.8 , 0.9 , 1 , 2 , 3 , 4)
If the null hypothesis (RR=1) is not included within the CI
→ significant RR=2 ( )
where significance level=(1- confidence level)
(1.8 , 1.9 , 2 , 3 , 4 , 5 , 6)

30. Chlordiazepoxide use and risk of congenital heart disease
Example
Chlordiazepoxide use and risk of congenital heart disease
Chlo. use
No Chlo. use
Cases 4
386
Controls
1250
OR = (4 x 1250) / (4 x 386) = 3.2
p = 0.08 (not significant)
From Rothman K

31. So chlordiazepoxide use is safe?
”The confidence interval includes 1 so the association is not significant”

32. Summary Confidence interval RR = 3 (2 – 8) Test of significance
P = 0.03
Null Hypothesis
RR = 1
Association
RR = 5 Go
Problem

33. Conclusions Not all associations found to be statistically significant
Significance testing evaluates only the role of chance as alternative explanation of observed difference or effect
Confidence intervals are more informative than p-values

34. Recommendations Finding the proper test is an important step
Use confidence intervals to describe your results! (More than one dimension)
Report p-values precisely!
Say “0.002” not just saying “less than 0.05”
Revise and check your results
Interpret with caution associations that achieve statistical significance!
Always look at the raw data (2x2-table). How many cases can be explained by the exposure?

35. Suggested reading
KJ Rothman, S Greenland, TL Lash, Modern Epidemiology, Lippincott Williams & Wilkins, Philadelphia, PA, 2008
SN Goodman, R Royall, Evidence and Scientific Research, AJPH 78, 1568, 1988
SN Goodman, Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy, Ann Intern Med. 130, 995, 1999
C Poole, Low P-Values or Narrow Confidence Intervals: Which are more Durable? Epidemiology 12, 291, 2001

36. Significance testing Introduction to Intervention Epidemiology
Tunis, 3 November 2014
Dr. Ibrahim Saied
Epidemiologist, Ministry of Health and Population - Egypt