Pragmatic trials at 50 – Back to the future? Jeremy Grimshaw Senior Scientist and professor Canada Research Chair in Health Knowledge Transfer and Uptake @GrimshawJeremy jgrimshaw@ohri.ca 9TH MAY 2017

Foundational paper 746 cites (SCOPUS)

Explanatory vs pragmatic trials Purpose Hypothesis testing Inform decision making Design Equalised Minimise/equalise non specific effects Optimal NSE seen as part of intervention Inclusion criteria Arbitrary Broad Intervention Emphasis on intervention fidelity and optimal implementation Prototypical elements of intervention defined but variations in intervention and uptake accepted Data collection Multiple end points Fewer end points - unobtrusive

Explanatory vs pragmatic trials

Large simple trials

Large simple trials ISIS-2 17,187 patients in 417 centres 2 x 2 factorial design I-v streptokinase (clot buster) or control 1 month aspirin or control Outcome – vascular mortality at 5 weeks and 15 month follow up, harms Highly significant reduction in vascular mortality at 5 weeks and longer follow up for both streptokinase and aspirin alone and in combination. Larger effects with earlier administration Practice changing (but required considerable further research to determine optimal implementation and delivery approaches)

Large simple trials Explanatory Pragmatic Purpose Hypothesis testing Inform decision making Design Equalised Minimise/equalise non specific effects Optimal NSE seen as part of intervention Inclusion criteria Arbitrary Broad Intervention Emphasis on intervention fidelity and optimal implementation Prototypical elements of intervention defined but variations in intervention and uptake accepted Data collection Multiple end points Fewer end points - unobtrusive

Large simple trials Drug trials Patient randomised trials Sample size - large (thousands) Commonly (but not exclusively) two arm trials Limited number of clinical endpoints Industry + public funding Address clinical uncertainty (clinical benefits/harms) Relatively silent on health service/implementation issues (QOL, patient experience, economic, implementation/delivery issues)

Health services research trials BMJ 1994;309:979-83

Health services research trials 204 women randomised to hysterectomy (control) vs endometrial ablation vs endometrial resection (2:1:1) Outcomes: operative complications, postoperative recovery, relief of menstrual symptoms, patient satisfaction at 6 and 12 months (psychological distress, longer term clinical outcomes, economic evaluation) Hysteroscopic surgery led to fewer post op complications, shorter post op recovery, similar relief of (non bleeding) menstrual symptoms, slightly less patient satisfaction, cost savings for patient and NHS over 12 months. Small number of women treated with hysteroscopic surgery required hysterectomy within 12 months.

Health services research trials Explanatory Pragmatic Purpose Hypothesis testing Inform decision making Design Equalised Minimise/equalise non specific effects Optimal NSE seen as part of intervention Inclusion criteria Arbitrary Broad Intervention Emphasis on intervention fidelity and optimal implementation Prototypical elements of intervention defined but variations in intervention and uptake accepted Data collection Multiple end points Fewer end points - unobtrusive

Health services research trials (Drugs), devices/technology, service delivery arrangements Patient and cluster randomised trials Sample size – moderate (hundreds) Diverse designs (multi-arm, factorial, patient preference etc); greater use of process evaluations etc Broad range of endpoints (clinical, QOL, implementation/delivery, economic) Public funding Address clinical and health services uncertainty

Implementation research trials Lancet 2001; 357:1406-9

Implementation research trials 244 UK general practices served by six radiology departments 2 x 2 factorial design Audit and feedback Educational messages Outcome – number of knee and lumbar spine x-rays, economic evaluation (focusing on NHS costs) Educational messages led to with 20% reduction in x-rays; audit and feedback had no effect. Cost saving to NHS.

Implementation research trials Explanatory Pragmatic Purpose Hypothesis testing Inform decision making Design Equalised Minimise/equalise non specific effects Optimal NSE seen as part of intervention Inclusion criteria Arbitrary Broad Intervention Emphasis on intervention fidelity and optimal implementation Prototypical elements of intervention defined but variations in intervention and uptake accepted Data collection Multiple end points Fewer end points - unobtrusive

Implementation research trials Implementation strategies (including service delivery arrangements Cluster randomised trials Sample size – small to moderate (tens of clusters) Diverse designs (multi-arm, factorial, patient preference etc); greater use of process evaluations etc Broad range of endpoints (implementation/delivery, economic) Public funding Address health services uncertainty

Recent developments Research policy interest Innovative designs Comparative effectiveness …. Innovative designs Stepped wedge designs Cohort and registry embedded trials Innovative partnerships Collaboration for Leadership in Applied Health Research and Care (CLARHC) program Learning healthcare organisations Implementation laboratories

summary Long tradition of pragmatic thinking and design in randomised trials Multiple approaches adopted to address pragmatic questions addressing different uncertainties and actors Pragmatic trials addressing clinical uncertainties may not address health service/implementation uncertainties Opportunities to consider how to maximise information (and research efficiency) from pragmatic trials to address wider range of uncertainties Public funding essential (and needs to be preserved!)

Heretical thought Systematic reviews of randomised trials should be the knowledge unit to inform clinical and health system decision making. Time to rethink Schwartz and Lellouch’s central argument that pragmatic trials aim to inform (local) decision making to consider how pragmatic trials efficiently contribute to global evidence production to inform decision making.

jgrimshaw@ohri.ca @GrimshawJeremy