1. Evidence-Based Practice AUPHA Annual Meeting, June 20, 2013, Monterey If doctors can do it managers can do it?

2. Evidence? outcome of scientific research, organizational facts & data, benchmarking, best practices, collective experience, personal experience, intuition

3. All managers base their decisions on ‘evidence’

4. However...

5. Many managers pay little or no attention to the quality of the evidence they base their decisions on

6. Trust me, 20 years of management experience

7. Teach managers how to critically evaluate the validity, and generalizability of the evidence and help them find ‘the best available’ evidence

8. Evidence-based decision

9. What is the added value of evidence- based practice for managers within the field of hospital care? Proof of concept

10. Teaching Hospital - 6 managers University Hospital - 4 managers Evidence based pilot

11. Phase 1: Training managers in the principles of EBP Phase 2: Examination of the current decision making processes that managers are using Phase 3: Evaluation of 4 completed projects from an EB perspective (retrospective) Phase 4: Making EB recommendations for 4 new projects (prospective) Phase 5: Evaluation Evidence based pilot

12. Some preliminary results

13. Decision making process

14.  Focus on procedures instead of evidence  Internal politics and power struggles  No critical appraisal of the evidence at hand  Relying on anecdotal evidence (workshops, best practices, popular management books, consultants)  One option (sometimes two)  Bias: Outcome, Halo, Confirmation, etc.

15. Post mortem analysis

16. Evidence-based perspective NOT: Did we made the right decision? BUT: Is there evidence from scientific research to support (or call into question) the approach taken? Best available evidence?

17. Post mortem: leadership training

18. leadership training: dm process  No problem definition  No organizational evidence consulted  Selection of training companies based on experience, recommendation or reputation  No explicit selection criteria / procedure  ‘Best’ presentation has won: one size fits all

19. leadership training: scientific evidence  15 meta analyses, 5 relevant  37 (‘systematic’) reviews, 2 relevant  Lots of relevant primary studies

20. leadership training: scientific evidence  Long history (30 yrs): moderate effect sizes  Senior & middle managers tend to benefit more than managers at the supervisory level  Effect on ‘poor’ leaders is limited.  Leadership trainings that focus on interpersonal / social skills show higher effect sizes than those based on a specific leadership ‘model’

21. Reactions Who knew? Denial Anger Bargaining Acceptance

22. Prospective / EB recommendations

23. Questions / projects  360 degree feedback  Financial incentives  Lean Six Sigma  Hand Hygiene  Goal setting  Value Based Health Care  Downsizing

24. Evidence-based perspective NOT: What works? BUT: What are, given the target group, the problem and the context involved, the main factors determining the success or failure of the project that need to be taken into account? Best available scientific evidence?

25. Prospective: Multi Source Feedback

26. Multi Source Feedback: background  IFMS: based on multi source feedback  Regulating bodies and insurance companies (KPI’s – prices/ revenue)  Based on CANMEDS, no standard method  New market: consulting firms

27. Process  Scoping session: inventory of the aspects relevant to the question  Session with leading academic  Search in relevant databases  Critical Appraisal  Summary / research synthesis  Recommendations / guidelines

28. Multi Source Feedback: scientific evidence  223 primary studies on MSF, 42 relevant  6 meta analyses or systematic reviews on MSF, 3 relevant  18 meta analyses or systematic reviews on ‘feedback’ or ‘performance appraisal’, 5 relevant

29. Content of the feedback (neg vs pos) Way of the delivery of the feedback Interpretation of the feedback Personality of the ratee Feedback orientation of the ratee Type and number of raters Selection of raters Rater reliablity (patients, nurses, colleagues) Type of response scale Development vs perfomance appraisal Organizational culture Perceived procedural justice Multi Source Feedback: main factors

30. Reactions Who knew? Wow! Great! Good stuff! Relevant!

32. Lessons learned I  New approach  Recalibrates the power dynamics (accountability!)  The profit is in the process  Different (better?) decisions were made  Doctors love it!

33. Lessons learned II  Hard for individual managers  It starts with the senior management team  It’s all about accountability  Support system  EBP > Planning & Control

34. One day, maybe … Chief Evidence Officer