YSGOL GWYDDORAU GOFAL IECHYD / SCHOOL OF HEALTHCARE SCIENCES PRIFYSGOL BANGOR / BANGOR UNIVERSITY Lynne Williams, Bangor University www.bangor.ac.uk/healthcaresciences @SHSBangor

 How do we assess how and why interventions achieve (or not) outcomes?  Are there differences between expected and observed outcomes?  How does context influence implementation and outcomes?  What’s the causal pathway for the intervention?  What are the mechanisms? Craig P, Dieppe P, MacIntyre S et al (2008). Developing and Evaluating Complex Interventions: New Guidance. London: MRC. Raine R, Fitzpatrick R, Barratt H et al (2016). Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Health Services and Delivery Research NIHR Vol 4 (16).

 To explain “for whom, how and why a complex intervention had a particular impact” (Liu et al, 2016)  To open the “black box” to examine what lies beneath (the “underlying processes”) (Grant et al, 2013)  Show how an intervention is implemented and what was delivered (as compared with what was intended to be delivered) http://evaluation.lshtm.ac.uk/process-evaluation/  Can provide a “snapshot” of receipt, implementation and context of trial intervention (Audrey et al, 2006) Audrey S, Holliday J, Parry-Langdon N et al (2006). Meeting the challenges of implementing process evaluation within randomized controlled trials: the example of ASSIST (A Stop Smoking in Schools Trial). Health Education Research 21 (3) 366-377 Grant A, Treweek S, Dreischulte T et al (2013). Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials 14:15 Liu H, Muhunthan J, Hayek A et al (2016). Examining the use of process evaluations of randomised controlled trials of complex interventions addressing chronic disease in primary health care—a systematic review protocol Systematic Reviews 5:138

 Process Evaluation runs parallel with the trial to understand the processes in relation to the intervention background, setting, and people involved  A logic model represents the intervention, and describes the resources required to ensure the implementation of the intervention, the components of the intervention, the mechanisms of impact, and the intended outcomes (UK Medical Research Council (MRC) Guidance)  Where implementation within process evaluation is described, this often refers to “the implementation, receipt and setting of an intervention and help in the interpretation of outcomes” (Oakley et al, 2006).

Understanding the context Engagement with policy makers, practitioners, patients Understanding the outcomes Acknowledging the social world we live in What are the causal mechanisms in the programme?

 Explanatory, contingent theory development  Context may be defined as space or place, whereby human interaction takes place under the “appropriate social and cultural conditions” (Pawson & Tilley, 1997: 57)  “change as the results of actions of social agents operating in a specific context whereby the action leads to outcomes by triggering mechanisms” http://evaluation.lshtm.ac.uk/process- evaluation/#realist

Context Mechanisms Outcomes Changes (intended & unintended), resulting from an intervention Context Outcomes Mechanisms What leads to response? How we respond in what ways and why? Features which are likely to affect how, and for whom, a programme is expected to work?

Sackley CM, Walker MF, Burton CR et al PRIFYSGOL BANGOR / BANGOR UNIVERSITY An Occupational Therapy intervention for residents with stroke-related disabilities in UK Care Homes (OTCH): cluster randomised controlled trial with economic evaluation Sackley CM, Walker MF, Burton CR et al The Health Technology Assessment 2016; Vol. 20: No. 15 DOI: 10.3310/hta20150 Pragmatic Phase III, parallel-group, cluster randomised controlled trial with an economic evaluation Primary outcome: The primary outcome measure was the Barthel Index (BI) score at 3 months after randomisation. The BI assesses dependency in 10 categories of self-care Aim: The predominant aim was to perform a definitive evaluation of OT for stroke and transient ischaemic attack (TIA) survivors in long-term institutional care Task related interventions on activities of daily living (feeding, dressing, toileting, bathing, transferring, mobilising) Supply of adaptive equipment Seating and positioning assessment Care home staff training

Example of the realist approach as used to advance learning about process evaluation function Process evaluation examining the fidelity of the occupational therapy intervention for residents a programme theory of fidelity to underpin the process evaluation multiple methods of data collection (e.g. in-depth interviews and critical incident reports) Four potential mechanisms through which fidelity within the trial could be investigated; the balancing of research and professional requirements building a positive rapport with care home staff working at re-engineering care homes’ environment learning about the intervention and its impacts over time

Process evaluations alongside trials of complex interventions can illuminate the underlying processes Provide a picture of the receipt, implementation and context of trial intervention Drawing on principles of the realist approach can guide process evaluation in order to identify what works, for whom and in what contexts