Prevailing opinions about large scale, sustained change: congruence, contradictions and challenges Jenna P Breckenridge Madalina Toma Nicola M Gray Mary J Renfrew Bruce Guthrie Stephen MacGillivray With thanks to Ania Zubala for assistance with poster design The Scottish Improvement Science Collaborating Centre (SISCC), led by the University of Dundee with NHS Tayside, aims to improve health and care by developing the evidence base for sustainable, large-scale improvement in health and social care. The SISCC brings together researchers, NHS staff, policy makers, educators, and the third sector from across Scotland and internationally in a coordinated and coherent way, to add value to existing investment and deliver a ‘step change’ in improvement knowledge and practice, and maximise benefit for Scotland and beyond. Background Improvement work needs to be informed by robust evidence in order to make changes that are: effective, affordable, acceptable to stakeholders, avoid unintended negative consequences, and sustainable beyond the duration of a single project. The improvement science field is littered with various opinion pieces - where views on how to create large scale, sustained change are asserted with great confidence and authority - however it is unclear how many of these statements are supported by empirical evidence. Findings 13 opinion papers on large scale, sustained change 17 unsupported by evidence 43 propositions 15 supported by empirical evidence 13 supported by non-empirical evidence *One statement was supported by both empirical and non-empirical sources and was counted twice. Analysis of the propositions generated three thematic areas of discussion about large scale sustainable change within the quality improvement field: We wanted to find out if there was concordance between opinions about how to create large scale, sustained change and the empirical evidence base. What contextual factors are needed for large scale change? Need to align policy with local contextual need Theory is important but is insufficient on its own Financial resource to absorb change without overburdening the system Leadership that uses management strategies such as incentives No. of propositions and nature of supporting evidence: Empirical (3) Non-Empirical (3) Unsupported (9) Methods We hand searched the online archives of three journals over a 10 year period (1st January 2005 until 31st March 2015): Implementation Science, BMJ Quality and Safety, and Healthcare: The Journal of Delivery Science and Innovation. Using agreed inclusion criteria, two reviewers looked for think pieces, debates and opinion papers with a primary focus on sustainable, large scale change. We extracted the main propositions being made about how to achieve and sustain large scale improvement. For each proposition, we classified the type of evidence, if any, that was provided in support of the statement. This was done by two reviewers independently, and then cross-checked to agree a final list of propositions. The propositions were then grouped using an iterative documentary analysis process (Fitzgerald, 2007) to identify a cohesive set of themes that summed up the prevailing opinions and debates. How should we be achieving sustainability? Improvement should be embedded into organisational culture Ensure that improvement interventions are adapted to local context Testing, monitoring, review and evaluation help to support sustainability No. of propositions and nature of supporting evidence: Empirical (6) Non-Empirical (6) Unsupported (6) How should we be scaling up improvement work? Opinions are divided between: Starting small and scaling up progressively OR Starting big from the beginning No. of propositions and nature of supporting evidence: Empirical (5) Non-Empirical (4) Unsupported (2) Conclusions The majority of published opinions about how to achieve large scale sustained change do not draw on empirical evidence. There is disconnect between the certainty of opinion statements and the availability of evidence. This confirms that there are barriers to applying research knowledge in improvement work which need to be overcome. Moreover, the lack of supporting evidence does not necessarily mean that expert opinion is ‘incorrect’ and expert opinion could signpost future research priorities. SISCC is led by in partnership with Contact us: www.siscc.dundee.ac.uk siscc@dundee.ac.uk @EBImprovement