1. BRIDGING THE RESEARCH-PRACTICE GAP THOUGH EVIDENCE-BASED MANAGEMENT AND SYSTEMATIC REVIEW Friday, Aug 1 2014 12:30PM - 2:30PM at Pennsylvania Convention Center Room 202 B (Program Session #: 109 | Submission: 15526) Prof. Rob Briner and Prof. David Denyer @daviddenyer | @Rob_Briner | @CenterforEBMgt | #EBMgt

2. Welcome! 1.A brief introduction to evidence-based practice (EBM) and the role of systematic reviews (SR) in other fields 2.The nature, purposes and processes involved in evidence-based management (EBMgt) and SR –Plenary discussion: what questions do you have about EBP, EBMgt and SR? 3.The language of evidence-based management –Table discussions 4.Teaching EBMgt and developing SR skills –Table discussions 5.Academic and practitioner collaboration though EBMgt and SR –Table discussion 6.Resources available to develop EBMgt and SR

3. 1. A brief introduction to evidence- based practice (EBM) and the role of systematic reviews (SR) in other fields

4. Evidence-based practice and SRs not unusual in other fields All practitioners use some of types of evidence in their practice Main problem is not using enough good quality evidence nor critically appraising it for its trustworthiness Started in medicine with BMJ editorial (1991) stated that: ‘‘only about 15% of medical interventions are supported by solid scientific evidence’’ Spread to many other areas of practice

5. What is evidence-based practice? “Evidence-based management is about making decisions through the conscientious, explicit, and judicious use of four sources of information: practitioner expertise and judgment, evidence from the local context, a critical evaluation of the best available research evidence, and the perspectives of those people who might be affected by the decision.” (Briner et al., 2009, p. 19)

6. What is evidence-based practice?

17. Traditional literature reviews Do not usually focus on a specific or practice-relevant question Have diverse aims and purposes Adopt a wide variety of approaches and structure Do not use particular methods or do not explicitly state the methods used to conduct the review Are more prone to bias because, for example, authors can select studies that support their views and ignore those that do not Are less likely to identify the best available evidence Do not aim to be comprehensive by including all relevant and available research Are not replicable nor easy to update

18. Systematic literature reviews Always focus on specific and usually practice-relevant questions Have similar aims and purposes in that they all focus on answering specific questions Adopt similar approaches and a structure based on a set of broad principles Use particular and explicitly stated methods in order to: –Search for and identify relevant literature –Make decisions about what research to include or exclude –Judge the quality and relevance of the research –Integrate or synthesize findings Are less prone to bias, because, for example, decisions about what to include and exclude are made explicitly Are more likely to identify the best available evidence Aim to be comprehensive by including all relevant and available research within explicitly-stated boundaries or constraints Are replicable and easy to update

19. Systematic literature reviews: Medicine Cochrane Collaboration founded in 1993 it aims to help people make well- informed decisions by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of interventions in all areas of health care Cochrane Database of Systematic Reviews –1995 36 reviews –1999 500 reviews –2001 1000 reviews –2004 2000 reviews + 1400 published protocols (plans) –2012 5000+ reviews Reviews prepared by healthcare professionals who volunteer (10000 people worldwide) Cochrane Review Groups Application and debate about quality standards

20. Systematic literature reviews: Social Policy Campbell Collaboration founded in 2000 An independent, non-profit, and largely voluntary collaboration Mission is help people make well-informed decisions by preparing, maintaining and disseminating systematic reviews in education, crime and justice, and social welfare.

21. 2. The nature, purposes and processes involved in evidence-based management (EBMgt) and SR

22. The steps in Evidence-Based Practice 1.Asking: Translating a practical issue or problem into an answerable question 2.Acquiring: Systematically searching for and retrieving the evidence 3.Appraising: Critically judging the trustworthiness and relevance of the evidence 4.Aggregating: Weighing and and pulling together the evidence 5.Applying: Incorporating the evidence in the decision-making process 6.Assessing: Evaluating the outcome of the decision taken

23. The mains steps in conducting a Systematic Review 1.Identify and clearly define the question the review will address. 2.Determine the types of studies and data that will answer the question. 3.Search the literature to locate relevant studies. 4.Sift through all the retrieved studies in order to identify those that meet the inclusion criteria (and need to be examined further) and those that do not and should be excluded. 5.Extract the relevant data or information from the studies. 6.Critically appraise the studies by assessing the study quality determined in relation to the review question. 7.Synthesize the findings from the studies. 8.Consider potential effects of publications or other biases.

24. Plenary discussion What questions do you have about EBP, EMBgt and SR?

25. 6. Resources available to develop EBMgt and SR

26. Plenary discussion What questions do you have about EBP, EMBgt and SR?

27. 3. The language of evidence-based management

28. Table discussions What does 'evidence' mean? What different connotations does it have? What does 'evidence-based' mean?

29. Misinterpretations and barriers Misinterpretations: EBMgt is done by scholars not practitioners EBMgt is a single rigid formulaic method EBMgt is only concerned with effect size, randomised- controlled trials and meta-analysis etc. Barriers Preoccupation with ‘new knowledge’ and ‘new questions’ Evidence aware/informed or based? Integration with experience and other forms of evidence

30. 4. Teaching EBMgt and developing SR skills

31. Table discussions Why are SRs useful and important - academically and practically? How do (or could) you teach students and others to do this? Share examples of curricula, assignments, etc.

32. How EBMgt and SR skills can be taught to different groups Cranfield DBA Co-producing SRs Engagement Research Dissemination Exploitation Evaluation From Engagement Integration?

33. How EBMgt and SR skills can be taught to different groups Cranfield DBA, executive education Co-producing SRs Engagement To Dissemination Exploitation Research Evaluation Integration

34. 5. Academic and practitioner collaboration though EBMgt and SR

35. Table discussions How can EBMgt be done in practice? How common is it? What are the barriers to managers becoming more evidence-based? How can academics work with organizations and managers to produce systematic reviews that address practical questions?

36. HOW organisations ARE commissioning systematic reviews from academics AWA – Advanced Workplace Associates Network for Business Sustainability

37. 6. Resources available to develop EBMgt and SR

38. CEBMa Website

39. Other websites and publications

40. BRIDGING THE RESEARCH-PRACTICE GAP THOUGH EVIDENCE-BASED MANAGEMENT AND SYSTEMATIC REVIEW @daviddenyer | @Rob_Briner | @CenterforEBMgt | #EBMgt

41. Extra slides

42. Some people care about research impact –Hambrick - What if the Academy actually mattered? –Huff - Mode1/2 knowledge production –Van de Ven - Engaged scholarship –Bartunek - Collaborative research –Pearce - What do we know and how do we really know it? –Rousseau - Evidence-based Management 42

43. The situation The research conducted by most business school academics does not have a marked influence on the practice of management (Currie, Knights and Starkey, 2010)

44. The current evidence-base In general, managers prefer to update their knowledge and skills by reading popular management books, typically written by consultants, former executives or journalists (Pfeffer and Fong, 2002; Rynes, Bartunek and Daft, 2001)

45. Evidence-based management: integrate different forms of knowledge “Evidence-based management is about making decisions through the conscientious, explicit, and judicious use of four sources of information: practitioner expertise and judgment, evidence from the local context, a critical evaluation of the best available research evidence, and the perspectives of those people who might be affected by the decision.” (see Briner, Denyer, Rousseau, 2009)

46. Hierarchy of evidence?