Changing physician behavior: an exercise in futility? Jeremy Grimshaw Canada Research Chair in Health Knowledge Transfer and Uptake Director, Clinical Epidemiology Program Ottawa Health Research Institute

Personal perspective Focus has been on: professional and organizational behavior change. improving technical aspects of care ie how do we ensure patients get the right (evidence based) treatments at the right time. populations of physicians (and health care organizations).

My life as an atheoretical empiricist (and an unashamed positivist)

Background Research is consistently producing new findings that may contribute to effective and efficient patient care Watt observed that 50% reduction in mortality in ischemic heart disease over past 50 years has been due to advances in clinical care and suggested we were entering a post McKeown era where evidence based health services can improve population health The findings of such research will not change population outcomes unless health services and health care professionals adopt them in practice. Grimshaw, Ward, Eccles (2001). Oxford Handbook of Public Health. Watt (2002) Lancet

Background Consistent evidence of failure to translate research findings into clinical practice 30-40% patients do not get treatments of proven effectiveness 20–25% patients get care that is not needed or potentially harmful IOM round table identified failure to translate research findings as one of the major blocks to improved health outcomes ‘Evidence based medicine should be complemented by evidence based implementation’ Schuster, McGlynn, Brook (1998). Milbank Memorial Quarterly Grol (1997) BMJ Sung et al (2003). JAMA Grol R (2001). Med Care

Background Local health care organisations have relatively few resources for implementation activities and decision makers need to consider how best to use these to maximise benefits Dissemination and implementation strategies are not without costs. In some circumstances, the costs of development and introduction are likely to outweigh their potential benefits. In other circumstances, it may be more efficient to adopt less costly but less effective dissemination and implementation strategies.

Background If decision makers are to make decisions about implementation strategies informed by research evidence, they need information on: likely effectiveness of interventions (direction of effect, predicted effect size of intervention and relative effectiveness of different interventions); effect modifiers (practice environment, potential adopter, characteristics of behaviour); and, resources need to deliver interventions.

Background Rigorous evaluations (mainly randomised controlled trials) provide the best evidence of effectiveness of different interventions because: Effects of interventions are modest Limited understanding of likely confounders Substantial opportunity costs if ineffective or inefficient dissemination and implementation strategies used Eccles (2003) Quality and Safety in Health Care

Background Systematic reviews of rigorous evaluations of implementation interventions should inform decisions because they: identify and summarise evidence on the effectiveness of interventions allow the generalisability and consistency of research findings to be assessed allow exploration of data inconsistencies to be explored. Mulrow (1994) British Medical Journal Grimshaw (2003) Quality and Safety in Health Care

Overview of previous systematic reviews Generally ineffective Mixed effects Generally effective Dissemination of printed educational materials Audit and feedback Reminders Didactic educational session Local opinion leaders Educational outreach Multifaceted interventions Bero et al (1998). BMJ Grimshaw et al (2002). Medical Care

Problems with previous systematic reviews Use of vote counting techniques: Problems handling studies where statistical significance of comparison is uncertain (eg studies with unit of analysis errors) Don’t provide any estimate of effect size Inadequate consideration of quality of primary studies Conflation of multifaceted interventions into single intervention category eg educational outreach

Systematic review of guideline dissemination and implementation strategies Objective Systematic review of the effectiveness and efficiency of guideline dissemination and implementation strategies to promote improved professional practice.

Systematic review of guideline dissemination and implementation strategies Inclusion criteria Study designs – RCTs, CCTs, CBAs, ITS Participants – medically qualified health care professionals Interventions – guideline dissemination and implementation strategies Outcomes – objective measures of provider behaviour and/or patient outcome Search strategy Cochrane Effective Practice and Organisation of Care group’s search strategy

Results – included studies Search strategy identified approx 150,000 hits 5,000 hits identified as potentially relevant Full text 863 reports retrieved Included 285 reports of 235 studies, yielding 309 separate comparisons

Results – methodological quality Overall methodological quality poor (eg unit of analysis errors common) Poor description of interventions Only 27% of studies used theories and/or psychological constructs 29.4% comparisons reported any economic data

Results – single interventions Systematic review of guideline dissemination and implementation strategies Results – single interventions Intervention Number of CRCTs Median effect size Range Educational materials 5 +8.1% +3.6%, +17.0% Audit and feedback +7.0% +1.3%, +16.0% Reminders 14 +14.1% –1.0%, +34.0%

Systematic review of guideline dissemination and implementation strategies

Multifaceted interventions including educational outreach 13 RCT Systematic review of guideline dissemination and implementation strategies Multifaceted interventions including educational outreach 13 RCT Median effect +6.0% (range –4% to +17.4%)

Systematic review of guideline dissemination and implementation strategies Conclusions Imperfect evidence base for decision makers Many current rigorous evaluations have methodological weaknesses (eg unit of analysis errors) Poor reporting of study settings, barriers to change, content and rationale of intervention Generalisability of study findings is frequently uncertain

Systematic review of guideline dissemination and implementation strategies Conclusions Improvements in direction of effect in 86% of comparisons Reminders most consistently observed to be effective Educational outreach only led to modest effects Dissemination of educational materials may lead to modest but potentially important effects (similar effects to more intensive interventions) Multifaceted interventions not necessarily more effective than single interventions

UK MRC Framework for Evaluating Complex Interventions Continuum of increasing evidence

My life as an unashamed positivist

Ottawa Model Of Research Use Assess + Monitor + Evaluate barriers & supports interventions outcomes & degree of use Innovation Potential Adopters Interventions Adoption Outcomes Practice Environment Copyright, Logan, J. & Graham, ID 2003

Towards evidence based practice Ferlie and Shortell suggested four levels at which interventions to improve the quality of health care might operate: the individual health professional; health care groups or teams; organisations providing health care (e.g., NHS trusts); the larger health care system or environment in which individual organizations are embedded. Ferlie, Shortell (2001). Milbank Quarterly

Towards evidence based practice Levels of engagement Policy Macro (national/provincial) Meso (organisational) Managerial Provider Consumer

Towards evidence based practice Most clinical care occurs within the context of a provider – patient dyad. Interventions at other levels are mediated through the provider – patient dyad. Provider behaviour is most proximal determinants of evidence based practice. Changing provider behaviour is (one of) the right objective(s) to promote evidence based behaviour.

Implications for KU research Further research is required to develop a better theoretical understanding of professional behavior change by exploring determinants of provider and organisational behaviour to better identify modifiable and non modifiable factors Incremental predictive or action orientated models Emphasis should be on developing standard methods for operationalising theories and then testing their utility

Implications for KU research Further research is required to estimate the effectiveness and efficiency of dissemination and implementation strategies in the presence of different barriers and effect modifiers. Rigorous designs (pragmatic RCTs) Maximise informativeness by exploring causal mechanisms (based on theoretical constructs) and economic evaluations

Implications for KU research Further research is required to develop practical methods of identifying barriers and facilitators to change outside an academic context.

Implications for KU research Barriers to progress Lack of consensus around overarching frameworks and candidate theories Measurement issues Lack of interdisciplinary approach Lack of programmatic funding and approaches Lack of cohesion in KU research community

Summary KU research is about saving lives, improving health outcomes and enhancing the quality of health services. The good news – changing physician behavior is possible though current efforts only achieve modest effects. I am currently more optimistic than ever that with concerted research effort within increasingly cohesive research community, it should be possible to improve incrementally KU.

Implications for KU trainees (and researchers) Identify an research area that can engage you over a 3 – 5 year period Avoid dogma Avoid jargon Keep focused on the larger picture Challenge your supervisors (we don’t know what we’re doing either!)