3. Quality in qualitative research ESRC research methods festival July 2008 Jane Lewis, NCB

4. Coverage of talk  why is quality so difficult in qualitative research?  NatCen’s work to develop a framework  some subsequent developments

5. Why is quality so difficult?  diversity of theoretical approaches  diversity of methods  importance of non-standardised, unstructured, flexible approaches  concerns about ‘methodolatry’  diversity of positions on key concepts

6. Range of positions on quality criteria Accept ‘traditional’ criteria Alternative criteria Reject criteria Parallel but modified criteria Guiding principles and ideas

7. Our work to develop a framework  literature review and review of existing frameworks  consultation  developed framework  workshop  refined, tested, refined

8. Structure and applicability  scoped applicability esp in terms of –theoretical positions –methods  4 guiding principles  18 appraisal questions  for each, a series of quality indicators

9. Some developments since: Dixon-Woods et al (2007)  used CO and CASP frameworks + unprompted judgement to determine inclusion in systematic review  frameworks produced less agreement  frameworks sensitised assessors to methodological issues  tipped balance to sound but less insightful  reproducability not validity

10. Some developments since: Walsh and Downe (2006)  reviewed 8 frameworks  synthesised and removed ‘redundant’ criteria  12 essential criteria, 53 specific prompts

11. Some developments since: Daly et al (2007) hierarchy of evidence Conceptual studies Descriptive studies Single case study Generalizable studies

12. Conclusions  a fair amount of common ground  reducing complexity is difficult  different needs of specialists vs non- specialists  different requirements for assessing quality vs eligibility  theoretical contribution needs to be based on methodological rigour

13. References Spencer L, Ritchie J, Lewis J and Dillon L (2003) Quality in Qualitative Evaluation: A framework for assessing research evidence London: Cabinet Office http://www.gsr.gov.uk/professional_guidance/quality.asp http://www.gsr.gov.uk/professional_guidance/quality.asp Dixons-Woods M, Sutton A, Shaw R, Miller T, Smith J, Young B, Bonas S, Booth A and Jones D (2007) ‘Appraising qualitative research for inclusion in systematic reviews: a quantitative and qualitative comparison of three methods’ in J Health Serv Res Policy vol 12 no 1 Jan 2007 Walsh D and Downe S (2006) ‘Appraising the quality of qualitative research’ in Midwifery vol 22 Daly J, Willis K, Small R, Green J, Welch N, Kealy M and Hughes E (2007) ‘A hierarch of evidence for assessing qualitative health research’ in Journal of Clinical Epidemiology vol 60

14. Jane Lewis : jlewis@ncb.org.uk