Evidence-Based Management A New Approach to Teaching the Practice of Management ACPE Annual Meeting, April 26 30, 2013, New York Tony Kovner, Michiel Bosman, Eric Barends

EBMgt: My Experiences  EBMgt as a Process  Why Don’t Managers use it?  Learning from Medicine  Contributions of Organizational Behavior  Experience in Teaching  The Politics of EBMgt

EBMgmt: Definition  “…the systematic, evidence-informed practice of management, incorporating scientific knowledge in the content and process of making decisions.” (Rousseau 2012) (Rousseau 2012)

EBMgt as a process  Framing the question behind the decision  Finding the sources of information  Assessing the accuracy of information  Assessing the applicability of information  Assessing the actionability of information  Determining if the information is adequate (Hsu and others 2009 (Hsu and others 2009)

Why don’t Managers use EBMgt?  Never heard of it. They are using it.  Cost now more probable than benefits in the future  Managers have to be persuaded and trained  The process doesn’t clearly lead to one-best way to intervene  Politics impacts interests

Learning from EB-Medicine  Interventions that achieve positive predictable results  Hindrances that blocked implementation of EBM same now for EBMgt  Financial as well as political impacts

Contributions of Organizational Behavior  Hiring Talent-relying on structured interviews (Rousseau)  Challenging Performance goals impacting on performance (Latham and Locke)  Managers who set a vision outperform other managers (Kirkpattrick)

Teaching EBMgt  NYU / Wagner, Capstone course  Projects Hourly nurse rounding process Hourly nurse rounding process Redesigning the nurses’ clinical ladder Redesigning the nurses’ clinical ladder Improving the supply distribution process Improving the supply distribution process Identifying causes of emergency department waiting Identifying causes of emergency department waiting

The Politics of EBMgmt  Evidence is not sufficient to change people’s behavior  The quality of the argument and story-telling by persons presenting the evidence is what persuades stakeholders  How persuade the manager that this is in his own interest?

Michiel Bosman MD MMM FACPE MD, University of Amsterdam MMM, CMU Exec PhD (2015), OSU Spears School Collaborator, Center for EBMgmt Serial Entrepreneur

 “there is a large research-user gap”  “practitioners do not read academic journals”  “the findings of research into what is an effective intervention are not being translated into actual practice”  “the relevance, quality and applicability of research is questionable”  “practice is being driven more by fads and fashions than research”  “many practices are doing more harm than good” What field is this?

McMaster University Medical School, Canada Medicine: Founding fathers David Sackett Gordon Guyatt

How it all started

 More than 1 million articles in 40,000 medical journals per year (= 1995; now probably more than 2 million). For a specialist to keep up this means reading 25 articles every day (for a primary care physician more than 100!) Problem: too much ‘evidence’

Problem: too much evidence  HRM: 1,350 articles in 2010 (ABI/INFORM). For an HR manager to keep up this means reading 3 to 4 articles every day (for a ‘general’ manager more than 50!)

Problem: too many half truths BTW: most of the research is seriously flawed or irrelevant for practice. And some claim so is most management advice.

The 5 steps of Evidence-Based Practice 1. Formulate a focused question (Ask) 2. Search for the best available evidence (Acquire) 3. Critically appraise the evidence (Appraise) 4. Integrate the evidence with your professional expertise and apply (Apply) 5. Monitor the outcome (Assess)

1. Incompetent people benefit more from feedback than highly competent people. 2. Task conflict improves work group performance while relational conflict harms it. 3. Encouraging employees to participate in decision making is more effective for improving organizational performance than setting performance goals. True or false?

How evidence-based are managers?  959 (US) (Dutch) HR professionals  35 statements, based on an extensive body of evidence  true / false / uncertain On average: 35% - 57% correct HR Professionals' beliefs about effective human resource practices: correspondence between research and practice, (Rynes et al, 2002, Sanders et al 2008)

Evidence-Based Practice 1991 Medicine 1998 Education 1999 Social care, public policy 2000 Nursing 2005 Criminal justice 2010 Marketing 2013 Healthcare Management?

Evidence-Based Consulting  Close the research/practice gap  Professor + Consultant  EB Mgmt Workshops  Teach the 5 step EB Practice process  Answer business questions  CAT: Critically Appraised Topic

Closing the research/practice gap  Executive Doctorate in Management  University of Maryland College  Oklahoma State University  Development of tools to measure EB Mgmt Attitudes (EBMAS) and skills/competencies (Fresno)

Teaching Evidence Based Practice to managers

Evidence? intuition, expertise, personal experience, collective experience, organizational facts & data, best practices, benchmarking, outcome of scientific research

All managers base their decisions on ‘evidence’

BUT...

Managers give little or no consideration to the quality of the evidence they base their decisions on

SO...

Teach managers how to find and and critically appraise the evidence before they make their decision

The 5 steps of EBP 1.Formulate a focused question (Ask) 2. Search for the best available evidence (Acquire) 3. Critically appraise the evidence (Appraise) 4. Integrate the evidence with your professional expertise and apply (Apply) 5. Monitor the outcome (Assess)

0. Creating awareness Why do we need it?

Trust me, 20 years of management experience

Errors and Biases of Human Judgment

 Seeing order in randomness  Mental corner cutting  Misinterpretation of incomplete data  Halo effect  False consensus effect  Attribution error  Group think  Self serving bias  Sunk cost fallacy  Cognitive dissonance reduction  Confirmation bias  Outcome bias  Authority bias  Small numbers fallacy  Recall bias  Anchoring bias  Inaccurate covariation detection Errors and Biases of Human Judgment

1. Formulate a focused question

Asking the right question?  Does team-building work?  Does the introduction of self-steering teams work?  Does lean management improve the performance of our hospital?  Is 360 degree feedback for doctors effective?

 What is a ‘team’?  What kind of team?  In what contexts/ settings?  What counts as ‘team-building’?  What does ‘work’ mean? Focused question?  Does team-building work?

P =Population I = Intervention or success factor C = Comparison O = Outcome C = Context Answerable question: PICOC

P =Population I = Intervention or successfactor C = Comparison O = Outcome C = Context Focused question: PICOC  Employee productivity?  Patient satisfaction?  Return on investment?  Market share?  Organizational commitment?

2. Finding the best available evidence

Best available experiential evidence Best available internal evidence Organizational values and stakeholders’ concerns Best available external evidence Evidence-based decision scientific research

Scientific databases: management  ABI/INFORM  Business Source Elite  PubMed  PsycINFO  Web of Knowledge  ERIC

3. Critical appraisal of studies Making sense of evidence

Critical appraisal  Construct validity (lean six sigma = value stream mapping, root cause analysis, goal setting, participative decision making)  Internal validity (does it work?)  External validity (will it also work for my employees / organization?)

Levels of internal validity Levels of internal validity

Best research design? Best available Best available

Critical appraisal

Step 4: Turning evidence into practice

Applicable?  organizational characteristics  cultural & political aspects  financial aspects /cost-effectiveness / ROI  priorities  change readiness / resistance to change  implementation capacity  timing

TEST?

CAT: Critically Appraised Topic

CAT-walk

Evidence-based practice: If doctors can do it managers can do it!