Pilot Project Increasing the Use of Large Scale Surveys in Undergraduate Dissertations in the Social Sciences Mark Brown and Jo Wathan School of Social Science University of Manchester

What were presenting today: Motivation and rationale for the project About the pilot Taking it forward: from pilot to credit- bearing course

Context British students are usually encouraged to collect their own data for final year undergraduate projects…This contrasts with the USA where most sociology students conduct secondary analysis of large-scale survey data… The lack of use of secondary analysis in Britain is surprising given that such a large number of high quality national surveys are readily available and that expertise in the analysis and data management of large surveys is in great demand by employers. Sara Arber in Nigel Gilbert (2001)Researching Social Life

What are the constraints? Lack of confidence in quantitative skills Perceived lack of relevance/interest of quantitative data/analysis Lack of knowledge about the resources available

Reason for Optimism There is a vast amount of relevant social data on topics sociology students are interested in. Its become much more accessible and user friendly – teaching datasets, user-guides, workbooks Students are increasingly aware of the value of attaining quant skills for their CVs The numerical skills required are below those needed for GCSE maths

Let the resources speak for themselves Our conviction … If… students knew what was available (through ESDS), and could engage hands-on with the materials then… resources would sell themselves. Students would enjoy the learning process, and want to do more. That learning and teaching data analysis (SPSS training) is enhanced if students see it as part of the research process that starts with a research question ESDS (and related resources) serve learning and teaching in all parts of that process: ESDS (and related resources) serve learning and teaching in all parts of that process: Searching for datasets Searching for datasets Data evaluation: is it fit for purpose? Data evaluation: is it fit for purpose? How to analyse data. How to analyse data.

The pilot project Jo ….

The pilot To increase use of secondary analysis in dissertations Take on <20 sociology 2 nd years in yr 2 Provide basic training in group setting Provide basic training in group setting For those continuing to a dissertation one to one support For those continuing to a dissertation one to one support Production of 3 topic workbooks for dissemination through ESDS Government External evaluation

Pathways through the project Yr 2 Summer vac Xmas W1W2W3W4W5 WORKSHOPS All Dissertation based on secondary analysis £100 Yr 3: Sem 1Yr 3: Sem 2 Research training module Dissertation project proposal DROP IN SUPPORT +£100 when submitted (+ enter for £500 prize) Dissertation deadline

Intake What was the highest qualification you gained in maths at school? GCSE (or equivalent)13 Grade A 3 Grade B 8 Grade C 1 Grade D 1 A level (or equivalent1 Grade A 0 Grade B 0 Grade C 1 Grade D 0 AgreeDisagree Not sure I lack the knowledge about the datasets available to me833 I lack the SPSS skills required635 I lack the statistical skills and understanding required725 I would struggle with the interpretation of the output644 I do not find survey analysis interesting257 Survey analysis will not answer my questions of interest248

Topics and data for dissertation proposals Class homogamy SARs Attitudes to crime BCS Background to understanding gender differences in markets LFS Sexual health (esp. young) Sexual attitudes and behaviour Intergenerational differences in attitudes BSAS /YPSAS Relationship between poverty and crime BCS Background to study into the girl guide movement Young people social attitudes

Workshops Year 2: Research benefit of secondary analysis & using online data – one off Then in year 3, fortnightly 3 hour sessions from 28/9. Mixture of presentation, group discussion and hands-on. Based on BCS exemplar workbooks. Year 3/1:Thinking about your research question & locating data in ESDS Year 3/2:frequencies and bar charts, weighting Year 3/3:recodes and select if Year 3/4: crosstabs, continuous data and reporting

Workbooks Available at Based on British Crime Survey teaching dataset 2002 Chosen after consulting on topics of interest

Attendance 14 original participants Lowest participation in workshop 11 (plus apology) 9 attended all 5 workshops Biggest drop at deadline for submitting dissertation proposal

Did they the workshops useful?

Will they be using secondary analysis in their dissertations?

Before and after affects Subjective change Conducted anonymously More honest? More honest? Not possible to look at individual change Not possible to look at individual change Confidence measured on scale out of 5: 5 =very able to = unable to

Locating data

Evaluating the suitability of a survey dataset

Open SPSS and run some frequency tables and graphs for single variables

Summary statistics (mean, median etc...)

Subsetting

Carrying out a simple recode

Interpret a crosstab

Did it work? High level of enthusiasm amongst students Students focused on potential for their dissertations Monetary reward became secondary for most but not all Students found it helpful and improved their skills Required one-to-one support Format not ideal Five dissertations submitted with a secondary analysis component

Back to Mark

Building from the pilot Recognition that pilot model (with financial incentive) is unsustainable Major objective was to bring training within the formal curriculum – develop a credit- bearing course

Incorporating within the curriculum Manchester Sociology (BSocSc) Research Design Secondary Analysis of Social Surveys Qualitative Methods Quantitative Methods Life Stories and Biographical research or compulsory Year 2 Teaching in Research Methods

Course content: From research question to data analysis. Built around the resources of ESDS TOPICPRACTICAL RESOURCE FOR PRACTICAL 1 Survey data: a goldmine for Social Scientists What data is out there on my topic? ESDS Data Search Data Archive Question Bank 2 The right question the right data Is it fit for purpose? evaluating a survey dataset ESDS data pages Survey home pages NESTAR 3 Analysing data 1 First steps in data analysis ESDS Teaching Dataset (BCS 2002) in SPSS Exercises available in themed workbooks 4 Analysing data 2 Filtering and recoding 5 Analysing data 3 Controls and statistical significance

Assessment Part 1: Finding and evaluating data for a research question of choice TOPICPRACTICAL RESOURCE FOR PRACTICAL 1 Survey data: a goldmine for Social Scientists What data is out there on my topic? ESDS Data Search Data Archive Question Bank 2 The right question the right data Is it fit for purpose? evaluating a survey dataset ESDS data pages Survey home pages NESTAR 3 Analysing data 1 First steps in data analysis ESDS Teaching Dataset (BCS 2002) in SPSS Exercises available in themed workbooks 4 Analysing data 2 Filtering and recoding 5 Analysing data 3 Controls and statistical significance

Assessment Part 2: Data Analysis TOPICPRACTICAL RESOURCE FOR PRACTICAL 1 Survey data: a goldmine for Social Scientists What data is out there on my topic? ESDS Data Search Data Archive Question Bank 2 The right question the right data Is it fit for purpose? evaluating a survey dataset ESDS data pages Survey home pages NESTAR 3 Analysing data 1 First steps in data analysis ESDS Teaching Dataset (BCS 2002) in SPSS Exercises available in themed workbooks 4 Analysing data 2 Filtering and recoding 5 Analysing data 3 Controls and statistical significance

Encouraging statistics First year: 6 students This year: 21 students (third of compulsory class) Positive student feedback – enjoyment as well as attainment Will monitor to see if year 2 interest converts to a quantitative dissertation in year 3.

Student feedback Agree Mostly Agree Neither Agree or Disagree Mostly disagree Disagree The material was intellectually stimulating On-line material significantly enhanced my learning The study materials were helpful

In conclusion Excellent resources now exist to support methods teachers who want to introduce more hands-on learning with real data (for both classroom and self-study). Materials easily customised to meet needs of different classes (by level and topic) Our experience suggests we should be less defensive about selling quantitative methods to students: Challenge the stereotype that sociology students must find it boring, irrelevant, difficult Challenge the stereotype that sociology students must find it boring, irrelevant, difficult When learning is enriched by engagement with ESDS resources and real survey data, sociology students show interest and confidence in considering a dissertation based on the secondary analysis of survey data. When learning is enriched by engagement with ESDS resources and real survey data, sociology students show interest and confidence in considering a dissertation based on the secondary analysis of survey data.