1. Changing physician behavior: an exercise in futility?
Jeremy Grimshaw
Canada Research Chair in Health Knowledge Transfer and Uptake
Director, Clinical Epidemiology Program
Ottawa Health Research Institute

2. Personal perspective Focus has been on:
professional and organizational behavior change.
improving technical aspects of care ie how do we ensure patients get the right (evidence based) treatments at the right time.
populations of physicians (and health care organizations).

3. My life as an atheoretical empiricist (and an unashamed positivist)

4. Background
Research is consistently producing new findings that may contribute to effective and efficient patient care
Watt observed that 50% reduction in mortality in ischemic heart disease over past 50 years has been due to advances in clinical care and suggested we were entering a post McKeown era where evidence based health services can improve population health
The findings of such research will not change population outcomes unless health services and health care professionals adopt them in practice.
Grimshaw, Ward, Eccles (2001). Oxford Handbook of Public Health.
Watt (2002) Lancet

5. Background
Consistent evidence of failure to translate research findings into clinical practice
30-40% patients do not get treatments of proven effectiveness
20–25% patients get care that is not needed or potentially harmful
IOM round table identified failure to translate research findings as one of the major blocks to improved health outcomes
‘Evidence based medicine should be complemented by evidence based implementation’
Schuster, McGlynn, Brook (1998). Milbank Memorial Quarterly
Grol (1997) BMJ
Sung et al (2003). JAMA
Grol R (2001). Med Care

6. Background
Local health care organisations have relatively few resources for implementation activities and decision makers need to consider how best to use these to maximise benefits
Dissemination and implementation strategies are not without costs.
In some circumstances, the costs of development and introduction are likely to outweigh their potential benefits. In other circumstances, it may be more efficient to adopt less costly but less effective dissemination and implementation strategies.

7. Background If decision makers are to make decisions about
implementation strategies informed by research
evidence, they need information on:
likely effectiveness of interventions (direction of effect, predicted effect size of intervention and relative effectiveness of different interventions);
effect modifiers (practice environment, potential adopter, characteristics of behaviour);
and, resources need to deliver interventions.

8. Background
Rigorous evaluations (mainly randomised controlled trials) provide the best evidence of effectiveness of different interventions because:
Effects of interventions are modest
Limited understanding of likely confounders
Substantial opportunity costs if ineffective or inefficient dissemination and implementation strategies used
Eccles (2003) Quality and Safety in Health Care

9. Background
Systematic reviews of rigorous evaluations of implementation interventions should inform decisions because they:
identify and summarise evidence on the effectiveness of interventions
allow the generalisability and consistency of research findings to be assessed
allow exploration of data inconsistencies to be explored.
Mulrow (1994) British Medical Journal
Grimshaw (2003) Quality and Safety in Health Care

10. Overview of previous systematic reviews
Generally ineffective
Mixed effects
Generally effective
Dissemination of printed educational materials
Audit and feedback
Reminders
Didactic educational session
Local opinion leaders
Educational outreach
Multifaceted interventions
Bero et al (1998). BMJ
Grimshaw et al (2002). Medical Care

11. Problems with previous systematic reviews
Use of vote counting techniques:
Problems handling studies where statistical significance of comparison is uncertain (eg studies with unit of analysis errors)
Don’t provide any estimate of effect size
Inadequate consideration of quality of primary studies
Conflation of multifaceted interventions into single intervention category eg educational outreach

12. Systematic review of guideline dissemination and implementation strategies
Objective
Systematic review of the effectiveness and efficiency of guideline dissemination and implementation strategies to promote improved professional practice.

13. Systematic review of guideline dissemination and implementation strategies
Inclusion criteria
Study designs – RCTs, CCTs, CBAs, ITS
Participants – medically qualified health care professionals
Interventions – guideline dissemination and implementation strategies
Outcomes – objective measures of provider behaviour and/or patient outcome
Search strategy
Cochrane Effective Practice and Organisation of Care group’s search strategy

14. Results – included studies
Search strategy identified approx 150,000 hits
5,000 hits identified as potentially relevant
Full text 863 reports retrieved
Included 285 reports of 235 studies, yielding 309 separate comparisons

15. Results – methodological quality
Overall methodological quality poor (eg unit of analysis errors common)
Poor description of interventions
Only 27% of studies used theories and/or psychological constructs
29.4% comparisons reported any economic data

16. Results – single interventions
Systematic review of guideline dissemination and implementation strategies
Results – single interventions
Intervention
Number of CRCTs
Median effect size
Range
Educational materials 5
+8.1%
+3.6%, +17.0%
Audit and feedback
+7.0%
+1.3%, +16.0%
Reminders 14
+14.1%
–1.0%, +34.0%

17. Systematic review of guideline dissemination and implementation strategies

18. Multifaceted interventions including educational outreach 13 RCT
Systematic review of guideline dissemination and implementation strategies
Multifaceted interventions including educational outreach
13 RCT
Median effect +6.0% (range –4% to +17.4%)

19. Systematic review of guideline dissemination and implementation strategies
Conclusions
Imperfect evidence base for decision makers
Many current rigorous evaluations have methodological weaknesses (eg unit of analysis errors)
Poor reporting of study settings, barriers to change, content and rationale of intervention
Generalisability of study findings is frequently uncertain

20. Systematic review of guideline dissemination and implementation strategies
Conclusions
Improvements in direction of effect in 86% of comparisons
Reminders most consistently observed to be effective
Educational outreach only led to modest effects
Dissemination of educational materials may lead to modest but potentially important effects (similar effects to more intensive interventions)
Multifaceted interventions not necessarily more effective than single interventions

21. UK MRC Framework for Evaluating Complex Interventions
Continuum of increasing evidence

22. My life as an unashamed positivist

23. Ottawa Model Of Research Use Assess + Monitor + Evaluate
barriers & supports interventions outcomes
& degree of use
Innovation
Potential Adopters
Interventions
Adoption
Outcomes
Practice
Environment
Copyright, Logan, J. & Graham, ID 2003

24. Towards evidence based practice
Ferlie and Shortell suggested four levels at which interventions to improve the quality of health care might operate:
the individual health professional;
health care groups or teams;
organisations providing health care (e.g., NHS trusts);
the larger health care system or environment in which individual organizations are embedded.
Ferlie, Shortell (2001). Milbank Quarterly

25. Towards evidence based practice
Levels of engagement
Policy
Macro (national/provincial)
Meso (organisational)
Managerial
Provider
Consumer

26. Towards evidence based practice
Most clinical care occurs within the context of a provider – patient dyad.
Interventions at other levels are mediated through the provider – patient dyad.
Provider behaviour is most proximal determinants of evidence based practice.
Changing provider behaviour is (one of) the right objective(s) to promote evidence based behaviour.

27. Implications for KU research
Further research is required to develop a better theoretical understanding of professional behavior change by exploring determinants of provider and organisational behaviour to better identify modifiable and non modifiable factors
Incremental predictive or action orientated models
Emphasis should be on developing standard methods for operationalising theories and then testing their utility

28. Implications for KU research
Further research is required to estimate the effectiveness and efficiency of dissemination and implementation strategies in the presence of different barriers and effect modifiers.
Rigorous designs (pragmatic RCTs)
Maximise informativeness by exploring causal mechanisms (based on theoretical constructs) and economic evaluations

29. Implications for KU research
Further research is required to develop practical methods of identifying barriers and facilitators to change outside an academic context.

30. Implications for KU research
Barriers to progress
Lack of consensus around overarching frameworks and candidate theories
Measurement issues
Lack of interdisciplinary approach
Lack of programmatic funding and approaches
Lack of cohesion in KU research community

31. Summary
KU research is about saving lives, improving health outcomes and enhancing the quality of health services.
The good news – changing physician behavior is possible though current efforts only achieve modest effects.
I am currently more optimistic than ever that with concerted research effort within increasingly cohesive research community, it should be possible to improve incrementally KU.

32. Implications for KU trainees (and researchers)
Identify an research area that can engage you over a 3 – 5 year period
Avoid dogma
Avoid jargon
Keep focused on the larger picture
Challenge your supervisors (we don’t know what we’re doing either!)