Applying theory to designing A&F interventions and evaluations in head to head trials Susan Michie Department of Psychology, UCL Ottawa December 2012

Summary Detailed description of intervention a starting point for identifying mechanisms of action i.e. theory May need to draw on more than one formal theory to generate hypotheses about mechanisms These hypotheses should guide intervention design, optimisation, evidence synthesis and trial design

What is theory? “A set of concepts and/or statements with specification of how phenomena relate to each other. Theory provides an organising description of a system that accounts for what is known, and explains and predicts phenomena.” Multidisciplinary consensus definition

Why theory? Some evidence that theory-based interventions more effective Provides a framework to facilitate –accumulation of evidence –communication across research groups Identifies mechanisms of action –evidence that can be used to improve interventions design head to head trials

MRC Guidance for developing and evaluating complex interventions Craig et al, 2009 BMJ

What theories? MRC guidance silent on this question NICE’s Behaviour Change evidence review (2008) –Identified evidence-based principles of behaviour change (see Abraham, Kelly, West & Michie, 2008) –No guidance on which theories to use Starting point for selecting theory –Understand intervention content i.e. active ingredients –Need a method for specifying content Taxonomies of behaviour change techniques (BCTs)

2006 Cochrane review Jamtvedt et al “Any summary of clinical performance of health care over a specified period of time” 118 trials A&F is typically effective –Effects vary 16% decrease to 70% increase in compliance –What explains variability? Types of A & F –“intensive”, “moderate” or “non-intensive ” –What do these terms mean?

Intensive A&F –(individual recipients) AND ((verbal format) OR (a supervisor or senior colleague as the source)) AND (moderate or prolonged feedback) Non-intensive –((group feedback) NOT (from a supervisor or senior colleague)) OR ((individual feedback) AND (written format) AND (containing information about costs or numbers of tests without personal incentives)) Moderate –(any other combination of characteristics than described in Intensive or Non-intensive group)

Problems of categorising by intensity Mixture of modes of delivery and content No theoretical rationale –Not surprising that no pattern of effect? Few recommendations for practice offered –‘A&F will continue to be an unreliable approach to quality improvement until we learn how and when it works best’ (Foy et al, 2005)

A theory-based approach Specify content as behaviour change techniques (BCTs) to allow theoretically based categorisation & analysis Generate theory-based hypotheses concerning effectiveness

Specifying content Two psychologists independently coded 13 papers from A&F review Identified 28 distinct, defined BCTs –Grouped into Goal/standard setting, Feedback & Action planning Inter-coder agreement 94% (84-100%)

Goal/Standard/Target As part of the intervention: 1.Were participants given a measurable target or goal of any kind? 2.Was the target for their individual behaviour (rather than a group target)? 3.Was the target for their group? 4.Were participants involved in setting their target? 5.Were participants involved in reviewing the target? 6.What was the frequency of review?

Feedback As part of the intervention, were participants: 7.given feedback about their own performance individually? 8.given feedback about their group’s performance individually? 9.given feedback about the group’s performance in a group? 10.required to collect their own feedback (self monitoring) 11.given feedback by someone else (externally generated feedback) 12.given feedback by people of high or more senior status 13.given feedback as a comparison with other people’s behaviours 14.given feedback as a comparison with targets 15.given feedback as a comparison with their own past behaviour 16.given written feedback 17.given face to face feedback 18.given a visual display of comparative data Timing of feedback: 19.What was the time period between behaviour monitored and feedback? 20.How frequently was feedback given? 21.How many times did participants get feedback in total? 22.Did feedback specifically address the behaviour to be changed?

Action Plan As part of the intervention were the participants: 23.given any action plan (advice suggestions on how to reach the target) 24.given a plan tailored to themselves 25.given a group action plan 26.involved in the development of their action plan 27.given the opportunity to review the action plan 28.what was the frequency of review?

A theory-based approach Specify content as behaviour change techniques (BCTs) to allow theoretically based categorisation & analysis Generate theory-based hypotheses concerning effectiveness

Self-regulation (control) Theory: Carver & Scheier, 82 GOAL Compare behaviour with standard Discrepancy noted Act to reduce discrepancy Environmental influences No discrepancy – goal reached Disengage from goal FEEDBACK GOAL/ STANDARD- SETTING ACTION-PLANNING

Theory-based hypotheses Feedback more effective when goal/target is set Most effective where goal/target and action plan 1.Feedback only Effective 2.Feedback + goal More effective 3.Feedback + goal + action plan Most effective 61 comparisons 8 comparisons 3 comparisons

Example: Cochrane review of Audit & Feedback Without explicit theory With theory

2012 theory-based Cochrane review Ivers et al 140 trials, effects small to moderate and variable –overall increase 4.3% in compliance (IQR 0.5% to 16%) Moderator analysis guided by theoretical predictions A&F is more effective when combined with –Explicit targets and an action plan Call for “better reporting” and “explicit use of theory” to develop hypotheses

Feedback more effective if … The source was a supervisor or colleague It was provided more than once It was delivered in both verbal and written formats –Understanding these effects theoretically would help to optimise intervention

Head to head trials: On what basis does one select intervention components? Need to have a theory about how A & F is working What functions are A & F playing? –Structure for noticing and reducing discrepancy Target, feedback, action plan –Cue to action –Reinforcement –Social support –?Others

Ensure all behaviour change techniques identified Within and beyond Audit & Feedback Both in intervention and control group –additional BCTs identified in 73 interventions & 34 control arms (Gardner et al, 2010)

Summary Detailed description of intervention a starting point for identifying mechanisms of action i.e. theory May need to draw on more than one formal theory to generate hypotheses about mechanisms These hypotheses should guide intervention design, optimisation, evidence synthesis and trial design

Additional slides

The COM-B system: Behaviour occurs as an interaction between three necessary conditions Psychological or physical ability to enact the behaviour Reflective and automatic mechanisms that activate or inhibit behaviour Physical and social environment that enables the behaviour Michie et al (2011) Implementation Science

What is a good theory? Criteria agreed across 4 disciplines 1.Clarity of constructs 2.Clarity of relationships between constructs 3.Measurability 4.Being explanatory 5.Describing causality 6.Achieving parsimony 7.Generalisability 8.Evidence base

28 Effective principles of individual behaviour change Maximise capability to regulate own behaviour –Develop relevant skills (e.g. goal setting, monitoring, feedback) –Develop specific plans to change Maximise opportunities to support self-regulation –Elicit social support –Avoid social and other cues for current behaviour –Change routines and environment Strengthen motivation to engage in the desired behaviour –Reward change –Develop appropriate beliefs E.g. benefits of changing, others’ approval, personal relevance, confidence to change –Develop positive feelings about changing Reduce motivation to continue with the undesired behaviour Abraham, Kelly, West & Michie, 2008, Psychology, Health and Medicine NICE Guidance for Behaviour change (2007)