A Realist Methodology Incorporating Large Quantitative Data Sets Andrew Brown, Andy Charlwood, Christopher Forde and David Spencer Presentation prepared for Leeds Social Science Institute Research Methods Seminar Series, 24 th May 2007

Background  Ongoing project on job quality, initiated by Brown, Charlwood, Forde and Spencer (2006) Changing Job Quality in Great Britain, DTI  Here we will present our emerging methodology as it relates to large quantitative data sets  We draw on some of our previous work in realist methodology  Then we will give an example using job satisfaction data

Large Quantitative Data Sets We focus on 2 data sets – WERS and the BHPS (1) The Workplace Employment Relations Survey  Conducted in 1980, 1984, 1990, 1998, 2004  Comprises 2 types of survey  Cross section: a representative sample of approx workplaces in Great Britain  A ‘panel’ survey: looking at change in establishments between the most recent survey and the last one

Large Quantitative Data Sets (1) WERS (continued) The cross-sectional survey itself comprises 3 elements:  Survey of the most senior manager responsible for employee relations  Survey of the most senior worker rep. in the establishment  Survey of 25 workers in each workplace  Data from each survey element can be matched together  Offers detailed information on personal, job and workplace characteristics  Also includes range of subjective measures of job quality

Large Quantitative Data Sets (2) British Household Panel Survey  Annual survey of each adult (16+) member of a nationally representative sample of more than 5000 households  The same individuals are re-interviewed in successive waves  Contains data on personal and job characteristics, basic workplace data and a range of job quality measures

Suggested Realist Methodology How should the data sets be used?  Large quantitative data sets may reveal patterns across the system as a whole (e.g. the nation)  System-wide patterns cannot be explained by random contingencies  This basic insight informs the following realist methodology

Suggested Realist Methodology 1) Observe system-wide pattern using large quantitative data set 2) Use theory to help hypothesise the real cause of the pattern 3) Use qualitative research to help develop, test and substantiate hypotheses

Job Satisfaction Example OBSERVED PATTERN:  Both WERS and BHPS reveal U-shaped pattern of job satisfaction in earnings

Job Satisfaction Example

THEORY TO BE USED:  According to theories of objective well- being (e.g. Sen) individuals or classes may adapt their norms and expectations to their objective situation  Therefore subjective reports of job satisfaction may mask objectively low quality jobs

Job Satisfaction Example HYPOTHESIS TO EXPLAIN U-SHAPE:  Low earnings: low norms and expectations matched by actual job  Medium level earnings: High norms and expectations not matched by job  High earnings: High norms and expectations matched by job

Job Satisfaction Example QUALITATIVE RESEARCH ON HYPOTHESIS:  The low-paid are the critical portion of earnings distribution for our hypothesis  Three relevant studies:  Edwards and Burkitt (2001) use focus groups  reveal low norms and expectations amongst low-paid

Job Satisfaction Example  Walters (2005) and Tomlinson (2006) both use in depth interviews  reveal that low-paid, part-time women have low benchmark level of norms and expectations  so they report ‘high’ job satisfaction but are not truly satisfied

Conclusion  Available evidence supports our interpretation of U-shape  More qualitative research required  Important implications for policy, especially towards the low-paid  Our realist methodology, mixing quantitative and qualitative methods, may have broad potential