Using Evidence in Your Work* From Evidence to Action A CIHR Funded Project *Based on a presentation for the National RAI Forum: "Making the Most of It“: Using Evidence in Your Work, by Sarah Bowen, Ph.D. (Winnipeg, MB, May 12, 2006)

WHAT IS EVIDENCE? Information that comes closest to the facts of a matter -the form it takes depends on the context or nature of the problem Findings of high-quality, methodologically appropriate research are most accurate evidence -but because research is often incomplete and sometimes contradictory or unavailable, other kinds of information are necessary supplements to or stand-ins for research The evidence base for a decision - multiple forms of evidence combined to balance rigour with expedience—while privileging the former (CHSRF) Information that comes closest to the facts of a matter -the form it takes depends on the context or nature of the problem Findings of high-quality, methodologically appropriate research are most accurate evidence -but because research is often incomplete and sometimes contradictory or unavailable, other kinds of information are necessary supplements to or stand-ins for research The evidence base for a decision - multiple forms of evidence combined to balance rigour with expedience—while privileging the former (CHSRF)

From: CHSRF Conceptualizing and Combining Evidence for Health System Guidance Scientific Evidence Professional Experiences & Expertise Habits & Tradition Resources Lobbyists & Pressure Groups Pragmatics & Contingencies Political Judgment Values

EVIDENCE-BASED OR EVIDENCE-INFORMED? Evidence-based: roots in quantitative research, particularly systematic reviews, clinical trials Evidence-informed: recognizes other factors affecting decision-making such as an incomplete evidence base Evidence-based: roots in quantitative research, particularly systematic reviews, clinical trials Evidence-informed: recognizes other factors affecting decision-making such as an incomplete evidence base

IS SOME EVIDENCE BETTER? A humorous look at hierarchies Class 0: Things I believe Class 0a: Things I believe despite the available data Class 1: Randomized controlled clinical trials that agree with what I believe Class 2: Other prospectively collected data Class 3: Expert opinion Class 4: Randomized controlled clinical trials that don’t agree with what I believe Class 5: What you believe that I don’t –Levels of Belief: From Shaughnessy & Slawson, 2004 Class 0: Things I believe Class 0a: Things I believe despite the available data Class 1: Randomized controlled clinical trials that agree with what I believe Class 2: Other prospectively collected data Class 3: Expert opinion Class 4: Randomized controlled clinical trials that don’t agree with what I believe Class 5: What you believe that I don’t –Levels of Belief: From Shaughnessy & Slawson, 2004

IS SOME EVIDENCE BETTER? Cont. Quantitative concept is dominant in health Emphasis is on experimental & clinical evidence (e.g. Cochrane Collaboration) Hierarchies of Evidence systematic reviews & meta-analyses well-designed Random Control Trials well-designed non randomized studies well-designed multi centre non-experimental studies expert opinion, descriptive studies Quantitative concept is dominant in health Emphasis is on experimental & clinical evidence (e.g. Cochrane Collaboration) Hierarchies of Evidence systematic reviews & meta-analyses well-designed Random Control Trials well-designed non randomized studies well-designed multi centre non-experimental studies expert opinion, descriptive studies

HIERARCHY CONCEPT LIMITS EVIDENCE-BASED HEALTH SERVICES Important perspectives of qualitative and mixed methods devalued Difficulties achieving successful evidence- base for planning and decision-making may overwhelm the importance of doing so Diminished recognition of importance of “context” for transferability of findings Important perspectives of qualitative and mixed methods devalued Difficulties achieving successful evidence- base for planning and decision-making may overwhelm the importance of doing so Diminished recognition of importance of “context” for transferability of findings

LEVELEXAMPLE of EVIDENCE USEDECISION-MAKERS CLINICAL Practice guidelines Direct care staff Managers PROGRAM Program designManagers POLICY Priorities for services Board & Senior Executive LEVELS OF EIDM

CHALLENGES DIFFER DEPENDING ON LEVEL OF DECISION Clinical –What is “best” practice? –Implementing guidelines & standards Program –Design (What does a program look like?) –Program Implementation Policy direction –Getting an issue “on the agenda” –Informing a policy trajectory Clinical –What is “best” practice? –Implementing guidelines & standards Program –Design (What does a program look like?) –Program Implementation Policy direction –Getting an issue “on the agenda” –Informing a policy trajectory

BARRIERS TO EVIDENCE USE Some barriers differ according to level: –Gap between research & practice –Differences between clinical & management: Culture Type of evidence Type of decisions, decision-making process BUT there are a number of common barriers.... Some barriers differ according to level: –Gap between research & practice –Differences between clinical & management: Culture Type of evidence Type of decisions, decision-making process BUT there are a number of common barriers....

COMMON BARRIERS: Absence of appropriate structures & processes Lack of confidence and authority to implement change Lack of education opportunities (research literacy) Difficulty accessing evidence Inadequate relevant, high quality information Lack of knowledge of the value of evidence Lack of senior level support (incl. knowledge of managers) Resource availability Skills Time Competing & conflicting demands Crisis Management culture

BARRIERS ARE NOT JUST EXTERNAL External (hard to modify) Political context Funding decisions –Resource allocation External requirements Crisis management culture Lack of information Perverse incentives External (hard to modify) Political context Funding decisions –Resource allocation External requirements Crisis management culture Lack of information Perverse incentives Internal (modifiable) Team/org culture or structure Leadership Competing priorities Research “literacy” Acceptance of inevitability of crisis management culture Resistance to change Insufficient time allocated Internal (modifiable) Team/org culture or structure Leadership Competing priorities Research “literacy” Acceptance of inevitability of crisis management culture Resistance to change Insufficient time allocated

What does it mean that there is “not enough time”? Not an organizational priority? (“there is always time for the important things”) –Resource allocation –Aligning resources and processes with stated values and goals –Viewing as an “add-on” Not an individual priority? (“how can you have enough time to do it over if you don’t have enough time to do it right?”) Not an organizational priority? (“there is always time for the important things”) –Resource allocation –Aligning resources and processes with stated values and goals –Viewing as an “add-on” Not an individual priority? (“how can you have enough time to do it over if you don’t have enough time to do it right?”)

ORGANIZATIONAL/UNIT/TEAM FACTORS Lack of mechanisms for consultation and input Centralized decision-making Leadership style Barriers to information flow Lack of transparency & accountability of decision-making processes Lack of processes & structures to allow “reflection” time Lack of mechanisms for consultation and input Centralized decision-making Leadership style Barriers to information flow Lack of transparency & accountability of decision-making processes Lack of processes & structures to allow “reflection” time

WHAT WORKS IN PROMOTING USE OF RESEARCH EVIDENCE? Collaboration Relationships “Personality factor” Leadership commitment to EI change Resource availability Willingness to change processes Sustainable interventions Decisions AND implementation Collaboration Relationships “Personality factor” Leadership commitment to EI change Resource availability Willingness to change processes Sustainable interventions Decisions AND implementation

USING EVIDENCE IN YOUR WORK Determine the level –Policy –Program –Clinical Clarify the challenge 1.Determining ‘best’ practice 2.Getting support for ‘best’ practice 3.Implementing ‘best’ practice Determine the level –Policy –Program –Clinical Clarify the challenge 1.Determining ‘best’ practice 2.Getting support for ‘best’ practice 3.Implementing ‘best’ practice

DETERMINING ‘BEST’ PRACTICE Assess the research –Literature analysis –Systematic reviews weighted –The trap of “decision-based evidence making” Evaluate for your setting –Transferability, context, resources, values Identify gaps Identify local evidence, experience of other programs (e.g. QI, evaluation activities) Assess the research –Literature analysis –Systematic reviews weighted –The trap of “decision-based evidence making” Evaluate for your setting –Transferability, context, resources, values Identify gaps Identify local evidence, experience of other programs (e.g. QI, evaluation activities)

GETTING SUPPORT FOR ‘BEST’ PRACTICE Identify champions Align with strategic priorities Use existing activities and structures Build collaborative relationships Integrate research with local evidence Speak to audience/stakeholder concerns Use effective communication strategies Identify champions Align with strategic priorities Use existing activities and structures Build collaborative relationships Integrate research with local evidence Speak to audience/stakeholder concerns Use effective communication strategies

IMPLEMENTING THE EVIDENCE Recognize implementation challenge –Plan, accountability, resources –“the personality factor” –“change management” Communicate evidence effectively –Link to personal, professional objectives Build collaborative relationships –Respect expertise of stakeholders Implementation & formative evaluation –Positive forum for identifying barriers Recognize implementation challenge –Plan, accountability, resources –“the personality factor” –“change management” Communicate evidence effectively –Link to personal, professional objectives Build collaborative relationships –Respect expertise of stakeholders Implementation & formative evaluation –Positive forum for identifying barriers

SUMMARY Evidence informed decision-making is important at all levels of health care provision: clinical, planning and policy Even with limited resources, we have opportunities to address internal barriers to evidence-informed decision-making Evidence should be used at all three stages of decision-making: determining a solution, getting support for a proposed solution, and implementing the decision. Evidence informed decision-making is important at all levels of health care provision: clinical, planning and policy Even with limited resources, we have opportunities to address internal barriers to evidence-informed decision-making Evidence should be used at all three stages of decision-making: determining a solution, getting support for a proposed solution, and implementing the decision.