1. Evidence-Based Management Evidence-Based Practice Insights from Key Domains University of Edinburgh, May 2 nd 2014 Eric Barends Rob Briner

2.  A brief history  Current developments  Future developments  Challenges  Implications for practice & research 2 Evidence-Based Management

3. 3 Evidence based management: A brief history

4. 4 Manager > ‘Manage’ introduced mid 16th century: from Italian maneggiare, “to handle”, based on Latin manus, “hand”. Management? (we’re all managers)

5. History of management/business schools  Late 1800s - started as trade schools  1950s - Ford and Carnegie reports criticised business schools for not being sufficiently academic or scientific  1980s onwards – criticised for producing research irrelevant to practice and students without relevant skills or knowledge 5

6.  On the one hand they fear... the scorn of other, more traditional academic subjects. On the other hand, they often stand accused of being less than relevant to business. (Grey, 2001)  2000s onwards – criticized for producing ideas and students that lead to financial crisis and poor ethical behaviour (e.g., Enron, only one US president has MBA, MBA students cheat more than other graduate students) 6 History of management/business schools

7. Management schools are big business  Nat. Center for Education Statistics (USA, 2011-’12)  20% of all Batchelor’s Degrees  25% of all Master’s Degrees  Higher Education Statistics Agency (UK, 2012-’13)  27% of all Higher Degrees awarded  18% of all Higher Education qualifications obtained  UK Council for International Student Affairs (2011-’12)  36% of all business school students are international  30% of all international students studying business 7

8. EBP in Management  Many similar ideas in past (e.g., Mode 1-Mode 2, collaborative research, rigour and relevance)  Rousseau (2005) Presidential address  Peffer and Sutton (2006) Hard Facts, Dangerous Half- Truths, and Total Nonsense: Profiting from Evidence- based Management 8

9. EBP in Management  EBMgt Collaborative (2007-’08) Rousseau/Carnegie Mellon  Wiley Evidence-Based Management Insights (2007-’08) (attempt to create Cochrane database for management) Briner and Denyer  Presentation to AoM Board of Governors (2008) (attempt to persuade to develop systematic reviews in management)  Center for Evidence-Based Management – more later 9

10. Evidence based management: What is it? 10

11. 11

12. Evidence-based practice Central Premise: Decisions should be based on a combination of critical thinking and the ‘best available evidence‘. 12

13. Evidence? findings from scientific research, organizational facts & figures, benchmarking, best practices, professional experience 13

14. All managers base their decisions on ‘evidence’ 14

15. But…many managers pay little or no attention to the quality of the evidence they base their decisions on 15

16. Trust me, 20 years of management experience 16

17. SO... 17

18. Teach managers how to critically evaluate the validity, and generalizability of the evidence and help them find ‘the best available’ evidence 18

19. Evidence-based practice is about making decisions through the conscientious, explicit and judicious use of the best available evidence from multiple sources by 1. Asking:translating a practical issue or problem into an answerable question 2. Acquiring:systematically searching for and retrieving the evidence 3. Appraising:critically judging the trustworthiness and relevance of the evidence 4. Aggregating:weighing and pulling together the evidence 5. Applying:incorporating the evidence in the decision-making process 6. Assessing:evaluating the outcome of the decision taken to increase the likelihood of a favorable outcome. Definition 19

20. Evidence based decision Professional experience and judgment Organizational data, facts and figures Stakeholders’ values and concerns Scientific research findings Ask Acquire Appraise Aggregate Apply Assess 20

21. Evidence-based practice: Focus on the decision making process Think in terms of probability 21

22. Differences?  Research: low internal validity

23. Controlled vs uncontrolled studies

24. Differences?  Research: low internal validity  Lack of evidence summaries  Focus on multiple sources  Organizational (BIG) data  Stakeholders’ concerns

25. How evidence-based are we (managers)? “I’ve never thought I need more evidence before making a decision; I know what needs to be done, we get on with it and we get results.” 25

26. How evidence-based are we?  959 (US) + 626 (Dutch) HR professionals  35 statements, based on an extensive body of evidence  true / false / uncertain HR Professionals' beliefs about effective human resource practices: correspondence between research and practice, (Rynes et al, 2002, Sanders et al 2008) 26

27. 1. Incompetent people benefit more from feedback than highly competent people. 2. Task conflict improves work group performance while relational conflict harms it. 3. Encouraging employees to participate in decision making is more effective for improving organizational performance than setting performance goals. True (likely) or false (not likely)? 27

28. Outcome: not better than random chance 28

29. Evidence based management: Current developments 29

31. Postgraduate Course 31

33. Current developments  Education  Access to research databases  REAs  Building a community

34. Evidence based management: Future developments 34

35. Postgraduate Course  CEBMa Database of Evidence Summaries  Online learning modules  Accreditational bodies Future developments: practice

36. Postgraduate Course  Some move towards systematic review methods in management  Limited interest in teaching EBP in management schools  Management schools still a bit trapped: Cash cows for universities; focus on ‘top quality’ academic research; few incentives to be relevant Future developments: academia 36

37. Postgraduate Course  Though issues around accountability, ethics, corporate social responsibility could in part be resolved by research and teaching EBP  Need for a professional doctorate? Future developments: academia

38. Evidence based management: Challenges 38

39. In general incentives to do non-EBP and punishment (or no incentives) for doing EBP 39 Challenges (same for practitioners and academics)

40. Incentives for managers  Not rewarded for doing what ‘works’ – few evaluations  Speed and action valued more highly than accuracy and analysis  Managing and understanding power and politics to get things done more valued than understanding and using evidence to make decisions  It may be too late to change existing senior managers… 40

41.  Barriers from academic / buss school context In publishing and research In publishing and research In teaching content and teaching style In teaching content and teaching style  Barriers from organizational contexts Managers love fads and quick fixes Managers love fads and quick fixes Power and politics Power and politics 41 Other challenges

42. In general: academics don’t like EBMgt

43.  Ambivalence about the value and applicability of management research  Few incentives to get involved  Primary research (collecting new data) valued more highly than secondary research (reviewing existing data)  EBMgt not academics‘responsibility – this is about practice not research  Some concern that systematic reviews will expose the limited nature of management research  Some academics are like ‘gurus’ and feel that EBMgt might show their claims to be untrue Why don’t academics like EBMgt?

44. Espoused and more implicit goals of management school educators ESPOUSED GOALS  To develop critical thinking  To help students understand the full body of knowledge  To discuss gaps and limitations of our knowledge  To challenge thinking and assumptions  To educate  To teach students how to think for themselves  To maintain quality standards IMPLICIT GOALS  To help students feel successful  To select those bits of that are interesting or digestible or ‘cutting edge’ fads  To reassure students that what we’re teaching them is solid  To make sure students are satisfied  To entertain (edutainment)  Teach students what they need to pass assessments  To give ever-higher grades 44

45. ESPOUSED GOALS  To advance scientific understanding  Using the best research techniques  Publishing all results and replications – unbiased  Focus on what’s important  Being honest about existing evidence  To disseminate all our evidence and make publically available  Collaboration & cooperation IMPLICIT GOALS  To advance career  Use whatever techniques will get you published  Publishing (mostly) only positive results, no replications  Identifying ‘new’ or trendy topics – creating empires  Exaggerating how much we know  Locking up our evidence behind publishers’ pay walls  Competition for resources, slots in journals, between universities 45 Espoused and more implicit goals of management school researchers

46. Implications for research  Less focus on collecting new primary data  More focus on systematic reviews and understanding what we know and don’t know  Focusing research efforts on what is academically and/or practically important without compromising ‘academic freedom’  Broader-based research training – management researchers highly and narrowly specialized 46

47. WHAT’S STOPING US? In general: managers don’t like EBMgt

48.  Undermines formal authority  They feel it constrains freedom to make managerial decisions  Speed valued and rewarded more than accuracy  Feel they cannot use their own experience and judgment (not true)  Managers not necessarily rewarded for doing what works (organizations rarely evaluate)  THEY LOVE FADS & QUICK FIXES Why don’t managers like EBMgt?

49. Huge (peer) pressure to adopt fads & quick fixes Fads & quick fixes

51. Postgraduate Course 51

52. Postgraduate Course How are fads a problem?  “The main problem…is their lack of any solid intellectual foundation. Implicit in each fad is a cause effect statement that is rarely made explicit and never properly supported.” 52 (Donaldson & Hilmer, 1998)

53. 53

54. 54

55. 55 George Buckley James McNerney

56. QUICK FIXES Because quick fixes / fads  Can be career-enhancing for managers  Speed is often valued over accuracy  Do we crave quick and easy solutions? So who needs or wants academic research? So why do managers love quick fixes?

57. Implications for management practice  Focus on accountability  Focus on next generation  Management = profession  EBP > Professional standard >Accreditation 57

58. Think about this How can a profession / domain be evidence-based if it is not managed in an evidence-based way? 58 “No job is more vital to our society than that of a manager. It is the manager who determines whether our social institutions serve us well or whether they squander our talents and resources.” Henry Mintzberg