1. Congres Evidence-Based Zorglogistiek, 13 oktober 2011 Evidence-Based Management What is it? Why do we need it? How does it look like in practice?

2. 1.Evidence based management: What is it? Postgraduate Course

3. Definition Evidence-based management means making decisions about the management of employees, teams or organizations through the conscientious, explicit and judicious use of four sources of information: 1. The best available scientific evidence 2. Organizational facts, metrics and characteristics 3. Stakeholders’ values and concerns 4. Practitioner expertise and judgment

4. Postgraduate Course Four sources

5. Evidence is not the same as ‘proof’ or ‘hard facts’ Evidence can be, or - so strong that no one doubts its correctness, or - - so weak that it is hardly convincing at all What is evidence? Postgraduate Course

6. McMaster University Medical School, Canada Medicine: Founding fathers David Sackett Gordon Guyatt Postgraduate Course

7. Management: Founding Mother Postgraduate Course

8. Jeffrey Pfeffer Robert Sutton Management: Founding Fathers Postgraduate Course

9. 2. Evidence-based management: Why do we need it? Postgraduate Course

10. EBMgt: some basic assumptions Postgraduate Course  Research produced by management scholars could be useful to organizations  Drawing on available evidence (including research produced by academics) is likely to improve decisions  Organizations do not appear to be strongly aware of nor use research findings  EBMgt is a potentially useful way of thinking about how we can incorporate research evidence into decision-making

11. Reason 1: Errors and Biases of Human Judgment Postgraduate Course

12.  Seeing order in randomness  Mental corner cutting  Misinterpretation of incomplete data  Halo effect  False consensus effect  Reinterpreting evidence  Group think Postgraduate Course Errors and Biases of Human Judgment  Confirmation bias  Authority bias  In-group bias  Recall bias  Anchoring bias  Inaccurate covariation detection  Distortions due to plausibility

13.  Seeing order in randomness  Mental corner cutting  Misinterpretation of incomplete data  Halo effect  False consensus effect  Reinterpreting evidence  Group think Postgraduate Course Errors and Biases of Human Judgment  Confirmation bias  Authority bias  In-group bias  Recall bias  Anchoring bias  Inaccurate covariation detection  Distortions due to plausibility

14. We are predisposed to see order, pattern and causal relations in the world. Patternicity: The tendency to find meaningful patterns in both meaningful and meaningless noise. Postgraduate Course Seeing order in randomness

15. We are patern seeking primates: association learning Postgraduate Course Seeing order in randomness

24. Postgraduate Course Points of impact of V-1 bombs in London

25. Postgraduate Course Points of impact of V-1 bombs in London

26.  A Type I error or a false positive, is believing a pattern is real when it is not (finding a non existent pattern)  A Type II error or a false negative, is not believing a pattern is real when it is (not recognizing a real pattern) Postgraduate Course Errors and Biases of Human Judgment Dr. Michael Shermer (Director of the Skeptics Society)

27.  A Type I error or a false positive: believe that the rustle in the grass is a dangerous predator when it is just the wind (low cost) Postgraduate Course Errors and Biases of Human Judgment

28.  A Type II error or a false negative: believe that the rustle in the grass is just the wind when it is a dangerous predator (high cost) Postgraduate Course Errors and Biases of Human Judgment

29. Pattern detection problem Assessing the difference between a Type I and Type II error is highly problematic (especially in split second ‘life and death’ situations), so the default position is to assume all patterns are real. Postgraduate Course Errors and Biases of Human Judgment

30. Postgraduate Course Errors and Biases of Human Judgment Jennifer Whitson, University of Texas Austin, corporate environments

31. Postgraduate Course Errors and Biases of Human Judgment Erroneous beliefs plaque both experienced professionals and less informed laypeople alike. stress peptic ulcer

32. Peptic ulcer – an infectious disease! This year's Nobel Prize in Physiology or Medicine goes to Barry Marshall and Robin Warren, who with tenacity and a prepared mind challenged prevailing dogmas. By using technologies generally available (fibre endoscopy, silver staining of histological sections and culture techniques for microaerophilic bacteria), they made an irrefutable case that the bacterium Helicobacter pylori is causing disease. By culturing the bacteria they made them amenable to scientific study. In 1982, when this bacterium was discovered by Marshall and Warren, stress and lifestyle were considered the major causes of peptic ulcer disease. It is now firmly established that Helicobacter pylori causes more then 90% of duodenal ulcers. The link between Helicobacter pylori infection and peptic ulcer disease has been established through studies of human volunteers, antibiotic treatment studies and epidemiological studies. Oct 2005

33. Postgraduate Course Errors and Biases of Human Judgment Doctors, teachers, lawyers and managers hold many erroneous beliefs, not because they are ignorant or stupid, but because they seem to be the most sensible conclusion consistent with the available evidence. They hold such beliefs because they seem to be the irresistible products of their own professional experience. They are the products, not of irrationality, but of flawed rationality They are the products, not of irrationality, but of flawed rationality

34.  Seeing order in randomness  Mental corner cutting  Misinterpretation of incomplete data  Halo effect  False consensus effect  Reinterpreting evidence  Group think Postgraduate Course Errors and Biases of Human Judgment  Confirmation bias  Authority bias  In-group bias  Recall bias  Anchoring bias  Inaccurate covariation detection  Distortions due to plausibility

35. Postgraduate Course Group think: management fads The nearly-forgotten fads:  Scientific Management/Taylorism  Business Process Reengineering  Management by results  Excellence  Total Quality Management  Learning Organizations  Knowledge Management

36. Postgraduate Course Group think: management fads The fads that haven’t been forgotten (yet):  Talent management  Management development  Executive coaching  Emotional intelligence  Employee engagement  Knowledge management  Myers Briggs Type Indicator  Belbin Team Roles

37. Postgraduate Course Group think: management fads “And there we see the power of any big managerial idea (or fad). It may be smart, like quality, or stupid, like conglomeration. Either way, if everybody's doing it, the pressure to do it too is immense. If it turns out to be smart, great. If it turns out to be stupid, well, you were in good company and most likely ended up no worse off than your competitors. Your company's board consists mostly of CEOs who were probably doing it at their companies. How mad can they get?

38. Postgraduate Course Group think: management fads The true value of conventional management wisdom is not that it's wise or dumb, but that it's conventional. It makes one of the hardest jobs in the world, managing an organization, a little easier. By following it, managers everywhere see a way to drag their sorry behinds through another quarter without getting fired. And isn't that, really, what it's all about?” (Colvin, 2004, Fortune)

39. Postgraduate Course So?

40.  Managers seem to be extremely good at generating ideas, theories, and explanations that have the ring of plausibility. They may be relatively deficient, however, in evaluating and testing those ideas once they are formed.  This requires that we think critically about experience, question our assumptions, and challenge what we think we know  (Show me the evidence!) Postgraduate Course Errors and Biases of Human Judgment

41. Reason 2: De ‘buitenwereld’ wordt steeds kritischer Postgraduate Course

42. Probleem 2: kritische geluiden “Managers maken Nederland ziek... Steeds meer vakmensen (zoals docenten, verpleegkundigen, artsen) hebben het gevoel dat ze worden aangestuurd door managers die van het vak geen verstand hebben maar wel de dienst uitmaken.” Ad Verbrugge

43. Postgraduate Course “Of het nu gaat om een ziekenhuis of een dropfabriek, te veel managers hebben de pretentie dat ze alles kunnen managen zonder ook maar te letten op de inhoud van het werk. Het zijn figuren die als een vlo van de ene "uitdaging" naar de andere springen, een spoor van verbittering en vernieling achter zich latend.” Geert Mak Probleem 2: kritische geluiden

44. Postgraduate Course “Nog meer managers, nog meer reorganisaties, nog meer power point-presentaties, nog meer holle retoriek over topprestaties en topkwaliteit. De groeiende korst van nepfuncties die onze bedrijven, scholen en andere organisaties nutteloos belasten wordt almaar dikker.” Dorien Pessers Probleem 2: kritische geluiden

45. Postgraduate Course Brede maatschappelijke ontwikkeling

46. Postgraduate Course “Waar de overheid, de dokter, de pedagoog en de manager vroeger een eenvoudig beroep op hun autoriteit konden doen, zullen zij nu met getallen en statistiek hun gelijk moeten aantonen.” Brede maatschappelijke ontwikkeling

47. Postgraduate Course Evidence based medicine Evidence based education Evidence based education Evidence based criminology Evidence based criminology Evidence based social welware Evidence based social welware Evidence based management? Evidence based management? Brede maatschappelijke ontwikkeling

48. Evidence based management: How does it look like in practice? Postgraduate Course

49. Four sources

50. Postgraduate Course JAMA, 1992

51. Postgraduate Course Push vs Pull Push: teaching (management) principles based upon a convergent body of research and telling students what to do. Pull: teaching (managers) how to find, appraise and apply the outcome of research (evidence) by themselves

52. Postgraduate Course The 5 steps of ‘pull’ EBP 1.Formulate an answerable question 2.Search for the best available evidence 3. Critically appraise the evidence 4.Integrate the evidence with your managerial expertise and organisational concerns and apply 5. Monitor the outcome

53. Postgraduate Course The 5 steps of ‘pull’ EBP 1. Formulate an answerable question 2. Search for the best available evidence 3. Critically appraise the evidence 4.Integrate the evidence with your managerial expertise and organisational concerns and apply 5. Monitor the outcome

54. Answerable question Postgraduate Course I am a consultant, my client a large health-care organization. The board of directors has plans for a merger with a smaller healthcare organization. However, it’s been said that the organizational culture differs widely between the two organizations. The board want’s to know if this can impede a successful outcome.

55. Postgraduate Course P =Population or problem I = Intervention or successfactor C = Comparison O = Outcome C = Context Answerable question: PICO(C)

56. Answerable question: PICOC Postgraduate Course P: What kind of Population are we talking about? Middle managers, back-office employees, medical staff, clerical staff? O: What kind of Outcome are we aiming for? Employee productivity, return on investment, profit margin, competitive position, innovation power, market share, customer satisfaction? P/C: And how is the assumed cultural difference assessed? Is it the personal view of some managers or is it measured by a validated instrument?

57. Postgraduate Course The 5 steps of ‘pull’ EBP 1.Formulate an answerable question 2. Search for the best available evidence 3. Critically appraise the evidence 4.Integrate the evidence with your managerial expertise and organisational concerns and apply 5. Monitor the outcome

58. Where do we search? Postgraduate Course

59. How do we search? Postgraduate Course

60. The 5 steps of ‘pull’ EBP 1.Formulate an answerable question 2.Search for the best available evidence 3. Critically appraise the evidence 4. Integrate the evidence with your managerial expertise and organisational concerns and apply 5. Monitor the outcome

61. Critical appraisal How to read a research article? Postgraduate Course

62. Critical appraisal Postgraduate Course 1.Study designs 2.Levels of evidence 3.Bias / confounding 4.Effect sizes 5.External validity

63. Postgraduate Course Which study for which question? Research designs The “best” evidence depends on the question type !

64. Postgraduate Course Levels of evidence

65. Postgraduate Course The 5 steps of ‘pull’ EBP 1.Formulate an answerable question 2.Search for the best available evidence 3. Critically appraise the evidence 4. Integrate the evidence with your managerial expertise and organisational concerns and apply 5. Monitor the outcome

66. Postgraduate Course 1. Is your organization / division / population so different from those in the study that its results cannot apply? 2. How relevant is the study to what you are seeking to understand or decide? 3.What are your organization’s potential benefits and harms from the intervention? 4. Is the intervention feasible in your setting? Organization concerns Always ask yourself to what extent the evidence is applicable in your situation:

67. Postgraduate Course The 5 steps of ‘pull’ EBP 1.Formulate an answerable question 2.Search for the best available evidence 3. Critically appraise the evidence 4.Integrate the evidence with your managerial expertise and organisational concerns and apply 5. Monitor the outcome

68. Postgraduate Course Monitor the outcome  Uitkomst gemeten?  Voormeting?  Controlegroep?

69. Postgraduate Course Do a trial!

70. Postgraduate Course Monitor the outcome  Business Process Redesign?  Six Sigma?  Lean management?  Lean Six Sigma?  TOC/ Theory of Constraints?  Performance Management?  Of …..

71. Postgraduate Course Vragen?