Secondary survey data Paul Lambert, University of Stirling Presentation to the Scottish Civil Society Data Partnership Project (S-CSDP), Webinar 1 on ‘Dealing with data: Defining ‘big data’ and research opportunities in studying civil society with secondary and administrative data’ 3 Mar 2016

S-CSDP, 3 Mar S-CSDP workshop of 25 Feb 2016 Intros. to secondary survey data Lab session with guidance and example files Café conversations on particular apsects of SSDA WorkshopOne/ What is ‘secondary survey data’? Access to deliberately collected research survey data for further analysis Typically originating with publicly funded survey instruments (Govt/HE) High quality (Large scale; ambitious, complex designs; best available standards in sampling, interview mode, data curation, documentation…. ) Efficient (Online access at no/low cost; replicability/transparency; ethical; fairly easy to use [after training]) [e.g. Hakim 1982; Kiecolt & Nathan 1985; Dale et al. 1988; Rose 2000; Dale 2006; Freese 2007; Treiman 2009; Purdham & Elliot 2015; ESRC 2015; UKDA c1967: ] Is secondary survey data ‘big data’? Yes! According to its scale, complexity, comprehensiveness, growth, content But not in sense that it was deliberately collected, and isn’t a new idea…

S-CSDP, 3 Mar In the UK, the ‘Understanding society’ (UKHLS) survey exemplifies many of these impressive developments ( Very large sample size and boost samples from subgroups of interest (e.g. ethnic minorities) Vast breadth of data: socio-economic; leisure and lifestyle; attitudes and values; health and biosocial Annual ‘household panel’ design covering individuals in 40k households Commenced 2009, (incorporating BHPS subset from 1991) What’s exciting in the world of secondary survey data? Recent expansions of high quality resources New stats. analytical possibilities New depths to data resources (e.g. linked admin data; qualitative data) Improved data access & support Accumulation of more and more data and more comparison points (e.g.

If secondary survey data is so great, why isn’t it used a bit more? Lack of control over question wording /definition of measures Time frame for preparation, documentation, permission & analysis can be too slow for some needs Some people wrongly believe that by-product ‘big data’ will supplant secondary survey data (e.g. Savage and Burrows 2007) Secondary data is in general a little bit frightening – it’s large and complex, requiring technical skills that seem inhibiting, and it’s not as comforting to work with as something you collected yourself… S-CSDP, 3 Mar 20164

Analysing survey data: Conventionally download data to machine, then use software to study it (see ) S-CSDP, 3 Mar (i) Somewhere on your computer, you typically have a copy of a data file from the original survey (& its documentation) (ii) Your next step ordinarily is to access a software package that will be able to open and then do things to the data (iii) If you are good, you will use separately saved ‘command files’ to run processes through the software on the data, generating subsequent outputs

Secondary analysis without microdata? S-CSDP, 3 Mar Image from NESSTAR, a tool for online analysis of large-scale archived surveys (example within lab: kshopOne/ kshopOne/ Image from OECD ‘gender data portal’ ( access to bespoke secondary macrodatahttp://

Plenty of help out there on secondary survey data S-CSDP, 3 Mar There’s no shortage of things to tell you how to do secondary survey research…! Numerous free online resources (see lab – e.g. UCLA statistics on software; UK Data Service on data) Numerous short training courses (e.g. NCRM, More extended training courses may cost more but can also be good value for money (e.g. subscriptions to MSc courses or modules; Essex Summer School, ) Statistical analysis of complicated secondary survey data… …a challenging business involving both ‘data management’ and ‘data analysis’ Data management: Preparing data for analysis, e.g. dealing with metadata, processing missing data, linking datasets Data analysis: choosing suitable statistical techniques and means of summarising their results for various guides…

S-CSDP, 3 Mar UKDA in circa 1976, from: Data providers are important actors! E.g. …Despite the wealth and expansion to new data sources, the core tasks of data providers continue to cover organising acquisitions, and providing prospective users with routes to the data… Quality variations in secondary survey data analysis? There are some outstanding, & some more questionable, submissions to acquisitions Variations in how users exploit the data – suitable statistical techniques, suitable controls for relevant social mechanisms?

S-CSDP - Promoting the use of secondary survey data for research on and by civil society Idealistic view: secondary survey datasets are often the highest quality empirical evidence we have Pragmatic view: The data are there and ought to be used! They are cheap and (fairly) easy to use (e.g. guidance above)! They provide best practice templates for analysis of own data! S-CSDP, 3 Mar Dale, A. (2006). Quality Issues with Survey Research. International Journal of Social Research Methodology, 9(2), Dale, A., Arber, S., & Procter, M. (1988). Doing Secondary Analysis. London: Unwin Hyman Ltd. ESRC. (2015). Secondary Data Analysis Initiative. Retrieved 1 February, 2016, from research/secondary-data-analysis-initiative/ Kiecolt, K. J., & Nathan, L. E. (1985). Secondary Analysis of Survey Data. Thousand Oaks, Ca.: Sage. Freese, J. (2007). Replication Standards for Quantitative Social Science: Why Not Sociology? Sociological Methods and Research, 36(2), Hakim, C. (1982). Secondary Analyis in Social Research : A guide to data sources and methods with examples. London: Allen and Unwin. Purdham, K., & Elliot, M. (2015). The changing social science data landscape. In P. Halfpenny & R. Procter (Eds.), Innovations in Digital Research Methods (pp ). London: Sage. Rose, D. (Ed.). (2000). Researching Social and Economic Change: The Uses of Household Panel Studies. London: Routledge. Treiman, D. J. (2009). Quantitative Data Analysis: Doing Social Research to Test Ideas. New York: Jossey Bass. References cited