University of Warwick, Department of Sociology, 2012/13 SO 201: SSAASS (Surveys and Statistics) (Richard Lampard) Survey Design: Some Implications for Secondary Analysis (Week 10)

Weighting Sample designs often do not select respondents with a single, consistent probability of selection. Unequal selection probabilities need to be taken into account at the analysis stage, to allow the results to be properly generalisable to the population of interest.

Complex Samples and Sampling Error Weighting can increase the expected amount of sampling error. Stratification can reduce the expected amount of sampling error. Clustering within samples can increase the expected amount of sampling error.

But... Secondary analysts frequently treat samples as if they were simple random samples. Hence the resulting analyses may under-estimate (or over-estimate) the likely amount of sampling error. In consequence the p-values may not be as reliable as they should be.

The solution... Recent versions of SPSS (and various other forms of software) allow one to incorporate these aspects of sample design into the analysis, and thus take account of their effect on the standard errors of estimates in an appropriate way.

Merging files Sometimes the design of a survey and/or a research focus means that a secondary analyst needs to merge data from two or more files. For example, information on aspects of individuals’ life histories may be included in separate files. Sometimes information about household members needs to be matched by saving subsets of members and matching them (e.g. matching individuals with their resident partners).

Example In the case of a repeated survey like the British Social Attitudes survey, different years which contain some of the same questions can be matched to facilitate examinations of social change.