Mixed methods in medical research Robert Pool Centre for International Health, Hospital Clinic, University of Barcelona London School of Hygiene & Tropical Medicine

THE BACKGROUND

Social science & medicine Increasing demand for qualitative social science in medical research programmes due to: –Realisation that illness is social as well as biological (adherence, acceptability, treatment seeking); the need to understand beliefs in order to change behaviour, fashionable –Better interdisciplinary communication Consultation, collaboration, integration Cooperation depends on suppressing/ignoring underlying differences

Why mix methods? 1.To triangulate (get closer to the truth) 2.To develop (sequential use) 3.To complement (examine different aspects) 4.To broaden (discover wider context, new perspectives)

THE PROJECT

Microbicides Development Programme Objective: To determine the efficacy and safety of two concentrations of “PRO 2000/5 Gel” compared to placebo in preventing vaginally acquired HIV infection Primary outcome: acquisition of HIV infection Sample: 10,000 women Process: Feasibility  Pilot  Phase III

Study sites

Quantitative trial data Women followed up for 12 months Regular clinic visits (every 4 weeks) for: –condom & gel distribution –applicator returns –clinical exams –lab tests (STDs, HIV & pregnancy) –clinical & behavioural interviews (CRF)

Issues Interpreting the trial result depends on the accuracy of sexual behaviour and adherence data  But self-reporting is unreliable

Key questions How do you know that participants understand the categories and questions in the way you intend? How do you ask sexual behaviour questions across different cultures & vernaculars in a standardised way that enables meaningful comparison? How do you know that your sexual behaviour data are accurate? How do you know whether women have used the product (properly)?

The social science Qualitative social science integrated to address these issues Random subsample (100 women at each site) In-depth interviews with women & partners Coital diaries Focus group discussions Ethnography

Explore & clarify standard categories & questions Almost all the central concepts in sexual behaviour research are ambiguous: –marriage, regular partner, casual partner –sex acts, condom use, etc. Get emic meanings

THE TRIANGULATION PROCESS Women Case Record Form Coital Diary 4 weeks before CRF Comparison Form 1.Interview 2.Comparison & probing inconsistencies ID I T&T Nvivo Summaries Nvivo T&T ID I 1.Interview 2.Comparison with women Male partners Women, community Summaries T&T Nvivo FGDs

In-depth interview guide

Consistency of CRF data In 53% (254/482) some inconsistency Mainly under-reporting of sex acts and over-reporting of gel and condom use on CRF

Two main sets of reasons for the inaccuracies in the CRF: –The participant forgetting or not understanding. –The interviewer not asking, not listening, or not understanding Confirmed by recording of CRF interviews Training, memory aid, short recall periods, clarification of terminology

Getting closer to the truth You can get more accurate data on sexual behaviour (get closer to the truth) if you: Clarify key concepts & categories Need to identify and solve inconsistencies during the study Ensure continuous feedback between different teams, methods and data sets Engage in dialogue with participants

THE ISSUES

1. How do you deal with inaccuracies once you’ve found them? Unresolved inconsistencies (uncertainty)? Resolved inconsistencies: –change the database? –extrapolate?

2. What do you do with key categories that remain ambiguous? Sex acts/rounds/days

3. How do you take into account the influence of the research process on the data? On behaviour (condom use, practices) On symptom perception (itching) On how people categorise their experience (definition of sex acts)

4. How do you deal with “ethics”? The misplaced application of clinical trial ethics to social science research: Flexibility of questions and “instruments” (open interviews, CD) Flexibility of procedures (recording CRF interview, follow up women) Observation

Conclusion Mixed models rather than mixed methods Combine emic and etic approaches Follow up and solve inconsistencies during the study Reform “ethics” procedures

How do we decide which data are most accurate? Case Record Form 2 sex acts in the last week Coital DiaryTicked “sex” 7 times In-depth Interview Reported that she’d had sex 7 times IDI probingWhen asked about the inconsistency with the CRF she explained that in the CRF interview she had been asked about sex acts and not rounds. Partner IDIHer partner reported having had sex 7 times in the same period CONCLUSION7 sex acts; CRF underreporting

Opposing theoretical paradigms PositivismConstructivism RealitySingle, objectiveMultiple, constructed KnowledgeNeutral & objectiveInfluenced by knower ValuesNo roleInfluence facts GeneralisationTime & context freeTime & context bound

Different methods QUANTQUAL Data typeNumericalNarrative Sample sizeLargeSmall InstrumentsClosedOpen AimExplainExplore, understand ProcessFixedInterative

Also: Continuation of the trial depends on informed consent Success of product depends on acceptability