1. Computing in Archaeology Session 10. Statistical tests of significance © Richard Haddlesey www.medievalarchitecture.net

2. Aims To understand what we mean by statistical significance and archaeological significance To understand what we mean by statistical significance and archaeological significance To test significance (through the Null hypothesis and Chi-squared testing) To test significance (through the Null hypothesis and Chi-squared testing) Key text: Fletcher & Lock (2 nd Ed) 2005. Digging Numbers. Oxford. 63-5, 128-38 Key text: Fletcher & Lock (2 nd Ed) 2005. Digging Numbers. Oxford. 63-5, 128-38

3. Is it significant? What do we mean by significant?

4. Choosing a test

5. positively skewed distribution normal distribution

6. positively skewed distribution normal distribution parametric test

7. positively skewed distribution non-parametric test normal distribution parametric test

8. Hypothesis testing Before we can test significance we must formulate two hypotheses Before we can test significance we must formulate two hypotheses So what do we mean by hypothesis testing? So what do we mean by hypothesis testing? Theories abound in archaeology although many of them cannot be tested in any way let alone in the formal way described throughout this lecture Theories abound in archaeology although many of them cannot be tested in any way let alone in the formal way described throughout this lecture

9. Hypothesis testing A test must be repeatable, not just by you, but by anyone who has access to the data set A test must be repeatable, not just by you, but by anyone who has access to the data set A hypothesis, therefore, must represent a quantifiable relationship and it is this relationship which is tested formally A hypothesis, therefore, must represent a quantifiable relationship and it is this relationship which is tested formally We could say that all hypotheses are theories whereas not all theories are hypotheses We could say that all hypotheses are theories whereas not all theories are hypotheses

10. Example hypothesis In order to illustrate the logic of a hypothesis test consider testing the hypothesis that at least 40% of all bronze spearheads come from burials In order to illustrate the logic of a hypothesis test consider testing the hypothesis that at least 40% of all bronze spearheads come from burials Fletcher & Lock, 63

11. Step 1 – formulate two hypotheses

12. null hypothesis (H 0 )

13. Step 1 – formulate two hypotheses null hypothesis (H 0 ) alternative hypothesis (H 1 )

14. Step One: H 0 & H 1 This should be done so that one and only one must be true This should be done so that one and only one must be true In this case we would have: In this case we would have: H 0 : proportion of bronze spears from burials is 40%H 0 : proportion of bronze spears from burials is 40% H 1 : proportion of bronze spears from burials is <40%H 1 : proportion of bronze spears from burials is <40%

15. Step 1 – formulate two hypotheses null hypothesis (H 0 ) alternative hypothesis (H 1 ) Step 2 – take measurements

16. Step Two Take a suitable measurement or observation from which a test statistic and its associated probability (step 3) can be calculated Take a suitable measurement or observation from which a test statistic and its associated probability (step 3) can be calculated Here we have a sample of 20 bronze spearheads 7 of which have been found in burials (this is the observed result) Here we have a sample of 20 bronze spearheads 7 of which have been found in burials (this is the observed result)

17. So far so good! So far so good!

18. Step 1 – formulate two hypotheses null hypothesis (H 0 ) alternative hypothesis (H 1 ) Step 2 – take measurements Step 3 – calculate test statistic

19. Step 3: the difficult bit Here we calculate a test statistic which can then be tested for significance in step 4. Here we calculate a test statistic which can then be tested for significance in step 4. The test statistic allows for the calculation of the probability of the observed result which is often called the p-value The test statistic allows for the calculation of the probability of the observed result which is often called the p-value

20. Step 3: continued If H 0 is true and at least 40% of all bronze spearheads do come from burials what is the probability of a sample of 20 containing 7 from burials? If H 0 is true and at least 40% of all bronze spearheads do come from burials what is the probability of a sample of 20 containing 7 from burials? P (burial)=0.40 and so P (not burial) =0.60 P (burial)=0.40 and so P (not burial) =0.60 P (not burial for 1 st & 2 nd )=(0.60)(0.60) =(0.60) 2 P (not burial for 1 st & 2 nd )=(0.60)(0.60) =(0.60) 2 hence P (not burial for 13 (20-7))=(0.60) 13 =0.0013 hence P (not burial for 13 (20-7))=(0.60) 13 =0.0013 The p-value (probability of the observed result) is 0.0013 or 0.13% The p-value (probability of the observed result) is 0.0013 or 0.13%

21. Step 1 – formulate two hypotheses null hypothesis (H 0 ) alternative hypothesis (H 1 ) Step 2 – take measurements Step 3 – calculate test statistic Step 4 – calculate significance

22. Step 4: testing the hypotheses Remember that the null hypothesis is being tested. The significance of the test statistic will determine whether the Null Hypothesis is accepted or rejected Remember that the null hypothesis is being tested. The significance of the test statistic will determine whether the Null Hypothesis is accepted or rejected There are set conventions for significance testing and these will guide our discussion There are set conventions for significance testing and these will guide our discussion

23. Step 4: continued Common significance levels used in the social sciences are: Common significance levels used in the social sciences are: p<0.10 reject at the 10% levelp<0.10 reject at the 10% level p<0.05 reject at the 5% levelp<0.05 reject at the 5% level p<0.01 reject at the 1% levelp<0.01 reject at the 1% level p<0.001 reject at the 0.1% levelp<0.001 reject at the 0.1% level The 5% level is often used within archaeology The 5% level is often used within archaeology

24. Step 4: continued If p<0.05 (5%) reject H 0 at the 5% level and conclude that there is significant evidence to show that the percentage of bronze spearheads from burials is less than 40% (in other words if H 0 is rejected H 1 must be accepted If p<0.05 (5%) reject H 0 at the 5% level and conclude that there is significant evidence to show that the percentage of bronze spearheads from burials is less than 40% (in other words if H 0 is rejected H 1 must be accepted

25. What does this mean? We can now conclude that we are 95% certain that the percentage of bronze spearheads from burials is less than 40% We can now conclude that we are 95% certain that the percentage of bronze spearheads from burials is less than 40% If, however, the p-value was greater than 0.05, the conclusion would have been to reject H 0 at the 5% level and accept H 1 If, however, the p-value was greater than 0.05, the conclusion would have been to reject H 0 at the 5% level and accept H 1

27. Confidence interval 90% 95% 99%

28. Confidence interval 90% 95% 99% Probability p=0.10

29. Confidence interval 90% 95% 99% Probability p=0.10 p=0.05

30. Confidence interval 90% 95% 99% Probability p=0.10 p=0.05 p=0.01

31. p<0.05 reject at the 5% level

32. p<0.10 reject at the 10% level

33. p<0.05 reject at the 5% level p<0.10 reject at the 10% level p<0.01 reject at the 1% level

34. Chi-squared test The Chi-squared Test was developed by Karl Pearson in 1900 to test if a contingency table provides significant evidence of an association between two variables The Chi-squared Test was developed by Karl Pearson in 1900 to test if a contingency table provides significant evidence of an association between two variables It can be used for both nominal and ordinal levels, though it is better suited to nominal data It can be used for both nominal and ordinal levels, though it is better suited to nominal data

35. sample: 40 spearheads variables: material – iron/bronze loop – yes/no First we need to display the data in a contingency table Chi-squared test

36. Chi-squared explained Its a method of comparing the observed frequencies (the data) with those expected under the null hypothesis of no association between two variables Its a method of comparing the observed frequencies (the data) with those expected under the null hypothesis of no association between two variables

37. bivariate frequency table No loop Loop Iron20020 Bronze91120 291140 Is there any association between the two variables? How strong is the association between the two variables?

38. expected frequency (E) = (row total)(column total) (overall total) No loop Loop Iron20020 Bronze91120 291140

39. expected frequency (E) = (row total)(column total) (overall total) No loop Loop Iron20020 Bronze91120 291140

40. expected frequency (E) = (20)(11) = 5.5 (40) No loop Loop Iron20 0 (5.5) 20 Bronze91120 291140

41. No loop Loop Iron 20 (14.5) 0 (5.5) 20 Bronze 9 (14.5) 11 (5.5) 20 291140

43. No loop LoopIron 20 (14.5) 0 (5.5) 20 Bronze 9 (14.5) 11 (5.5) 20 291140

44. No loop Loop Iron 20 (14.5) 0 (5.5) 20 Bronze 9 (14.5) 11 (5.5) 20 291140 degrees of freedom (d.f.) = (r-1)(c-1)

45. No loop Loop Iron 20 (14.5) 0 (5.5) 20 Bronze 9 (14.5) 11 (5.5) 20 291140 degrees of freedom (d.f.) = (r-1)(c-1) = (2-1)(2-1) = (1)(1) = 1

46. Critical values of the χ 2 distribution

47. d.f. = 1

48. Critical values of the χ 2 distribution d.f. = 1 χ 2 = 15.18

49. Cramers V statistic Cramers V statistic can be calculated to measure the strength of association Cramers V statistic can be calculated to measure the strength of association This gives us a value V between 0 and 1 with values close to 1 indicating a strong relationship This gives us a value V between 0 and 1 with values close to 1 indicating a strong relationship

50. Cramers V statistic Where: n = total of all frequencies (40) m = the smaller of (c-1) and (r-1)

51. Cramers V statistic V= 15.18/(40)(1) 0.3795 = 0.62

52. Summary