1. University of Warwick, Department of Sociology, 2012/13 SO 201: SSAASS (Surveys and Statistics) (Richard Lampard) Week 1-2 Wedding Attitude by Sex Cross-tabulation Example

2. First version of cross-tabulation Wedding more celebration than commitment? by Sex MaleFemaleTotal Agree strongly118167285 9.7%10.7%10.3% Agree 5227101232 42.8%45.7%44.4% Neither 203234437 16.6%15.0%15.7% Disagree 265335600 21.7%21.5%21.6% Disagree strongly 6867135 5.6%4.3%4.9% Can't choose 231942 1.9%1.2%1.5% Not answered212344 1.7%1.5%1.6% Total 122015552775

3. Commentary on first version The attitude measure is a nominal variable (i.e. a set of categories), but the first five categories constitute an ordinal variable (i.e. ordered categories) It is tempting to view the first five categories as an interval-level variable (i.e. a scale) (Pearson) chi-square treats the measure as nominal, and has a statistically non-significant value (7.628, 6 d.f., p=0.267 > 0.05) However the Linear-by-Linear Association statistic treats the variable as interval-level (N.B. the last two categories mean that it isn’t!), and is significant (4.870, 1 d.f., p=0.027 < 0.05)

4. Second version of cross-tabulation Wedding more celebration than commitment? by Sex MaleFemaleTotal Agree strongly 118167285 10.0%11.0%10.6% Agree 5227101232 44.4%46.9%45.8% Neither 203234437 17.3%15.5%16.3% Disagree 265335600 22.5%22.1%22.3% Disagree strongly 6867135 5.8%4.4%5.0% Total 117615132689

5. Commentary on second version When we render the attitude measure properly ordinal by removing the last two categories, the Linear-by-Linear Association statistic ‘unfortunately’ becomes non- significant (p=0.071 > 0.05) However, this statistic doesn’t treat the variable as ordinal, but as interval-level (using the codes for the categories as a scale).

6. Third version of cross-tabulation Wedding more celebration than commitment? by Sex MaleFemale Total Agree strongly/Agree 640877 1517 54.4%58.0% 56.4% Neither/Disagree 468569 1037 39.8%37.6% 38.6% Disagree strongly 6867 135 5.8%4.4% 5.0% Total 117615132689

7. Commentary on third version However, if we consider the possibility that, substantively, there is not much difference between agreeing and agreeing strongly, and that there is not much different between answering ‘Neither’ and answering ‘Disagree’, then we can reduce the measure to three categories. If these are coded 1, 2 and 3, and then treated as interval-level, then the Linear-by-Linear Association statistic become significant (p=0.033 < 0.05)

8. Conclusion As is sometimes the case, our findings are ambiguous here. Clearly the basic chi-square statistic ignores the apparent trend in the percentages in the table, and is not taking proper account of the variable’s ordinality. But if we tinker with the categorisation in a way that simply reflects the observed pattern in the cross-tabulation, we can be viewed on one hand as teasing out a ‘real’, substantively plausible underlying relationship, or on the other hand as (inappropriately) nudging the p-value in the direction of significance, and hence increasing the risk of a Type I error (i.e. a false positive). Either way, with p-values around the conventional 5% (0.05) significance level, we should perhaps be cautious about viewing a finding as either ‘significant’ or ‘non-significant’…