1. Workshop 4: Data flow: how can we achieve good data flow to achieve better care? Conference suite 4

2. DATA FLOW: HOW CAN WE ACHIEVE GOOD DATA FLOW TO ACHIEVE BETTER CARE Summary of findings – ENGAGING CLINICIANS IN QUALITY IMPROVEMENT THROUGH NATIONAL CLINICAL AUDIT Dominique Allwood Public Health Registrar and Improvement Science Fellow

3. Engaging Clinicians in Quality Improvement through National Clinical Audit Background Despite being well organised and reasonably well funded, large scale audits are undervalued as a potential resource to support quality improvement The Francis Inquiry report highlighted the need for trusts to focus more on quality HQIP commissioned ISL to undertake a project to understand more Aim To better understand the barriers and facilitators in engaging clinicians with national clinical audit to support local quality improvement Method Thirty participants contributed their views through interviews or focus groups conducted in 2013

4. Participants

5. Themes 1. Technical and Practical Issues Data collection Analysis and interpretation Presentation Resources Knowledge & skills Participation in learning 2. Communication Content and purpose Engagement 3. National and local alignment to quality improvement Motivations and incentives Structures, systems, staffing and culture

6. Data Collection Time Duplication IT systems, Interactivity Analysis Methodology Comparisons – case mix, adjustment, Data vs. information Presentation Timeliness Interpretation Customisation Knowledge and skills Education, training and ability Interpretation of data Presentational skills ChangeAccess to skills Participate in learning Events ConferenceWorkshops Peer groups/ networks Sharing best practice Guidance and tools HQIP Professional bodies and audit suppliers 1. Technical and practical factors

7. 2. Communication Content and Purpose Data collection vs. audit Audit vs. improvement & Improvement vs. judgement Evidence-based standards vs. aspirational targets RelevanceMechanism for improvement Communication and reporting Between clinical audit leads and clinical teams Between clinical audit leads and local audit team Between suppliers and trustsExternal reporting Engagement and Dialogue Between HQIP and local audit teamsBetween HQIP and clinical leadsBetween HQIP and suppliersInterpretation

8. 3. National and local alignment to quality improvement Incentives and motivations Recognition, competition, clinical excellence awards Regulatory – appraisal/ revalidation, Quality Accounts Financial – BPT, resource allocation Public disclosureAudienceOwnership Awareness National Clinical Audit HQIP Organisation Staffing of clinical audit teams Buy-in from Boards and Medical Directors Culture and organisationsLeadership

9. Key Findings (1) There is a need to clarify the content and purpose of some national clinical audits so that local healthcare providers understand the relevance and importance of audit and can use this data to make improvements to services Support with how to analyse and interpret data and present findings to others in user-friendly ways would help people to engage better with audit findings Incentives to drive improvement could act as both barriers and facilitators. Recognition of good practice was seen as a powerful motivator while financial incentives received a mixed response

10. Key Findings (2) The right resources, knowledge and skills are needed to encourage engagement with clinical audit. Face to face events were seen as one of the best tools for building understanding and sharing best practice Organisational structures, governance, staffing and culture all had a strong influence on the take up of clinical audit and on whether data was used to encourage improvement More could be done to engage clinical leads with HQIP. Participants would like to see greater involvement of clinicians and input from patients in the design and reporting of clinical audits

11. Stages Understanding why there is a need to change Engagement with content, relevance and relationship to recognised standards Understanding what to change Technical elements of collection, analysis, presentation and interpretation Understanding how to change Prioritising, action planning and learning from others, implementation

12. Discussion – variation Content, output, presentation, support, timeliness Whether factors were barriers or facilitators Processes and practices of individuals, organisations How much suppliers should be responsible for vs local clinician ownership Presentation, interpretation and summarising Highlighting areas for improvement

13. Recommendations Commissioners of audit (HQIP) Suppliers of audit Both commissioners and suppliers of audit Provider organisations Individual clinicians and audit teams

14. Recommendations Commissioners of national clinical audit: HQIP and NHS England Consider developing a strategy that places national clinical audits in the context of a vision for improvement Consider a balance of data on clinical effectiveness, patient experience and safety when commissioning audits Further work could be undertaken to evaluate the effectiveness of specific mechanisms and drivers for improvement Suppliers of national clinical audit Audits should include clear objectives about quality improvement, explanation of the purpose of data items Linking to evidence-based standards and differentiating non-evidence-based targets from descriptive data Consider ways to provide real-time feedback to support continuous improvement Customisable outputs that can be manipulated for local analysis. Involvement of clinicians and patients in design and reporting

15. Recommendations Both commissioners and suppliers of national clinical audit: Consider a dual approach to interpreting data focusing on: ensuring data is analysed and presented in a way that is meaningful with adjustment to allow valid comparisons supporting the user population to gain the relevant skill sets to interpret the data Explore streamlining data collection methods with consideration to broader informatics work Resources and opportunities for clinicians to discuss findings, share best practice, and compare performance Strengthen communication strategies to improve engagement and recognition dialogue with clinical leads, medical directors and clinical commissioners as well as the media Highlighting and celebrating improvements and providing support to manage local and national media.

16. Recommendations Provider organisations: Support clinical audit teams and clinicians to undertake national clinical audit and focus on ensuring adequate skills and resources (skills in collection, interpretation, analysis and presentation of data, quality improvement, change management, and clinical leadership) Trusts should ensure appropriate processes are in place use national clinical audit proactively in improvement and share best practice for management of national clinical audit processes and ensure this is integrated within the quality agenda of the organisation. Front-line staff: clinicians, audit staff and managers Greater ownership of and engagement with national clinical audits may be fostered by increasing understanding of local and national approaches to improving quality and the role of audit in the broader development of knowledge and skills for measurement and improvement of quality from undergraduate training onwards. Clinicians should explore available resources to support them in using national clinical audit for improvement and utilise forums to learn from and share good practice Strengthen relationships between the medical director, clinical audit teams, and clinical leads and managers

17. E: dominique.allwood1@nhs.net T: DrDominiqueAllw

18. Improving national audit data flow Kate Godfrey Director of Operations Quality Improvement and Development www.hqip.org.uk

19. Collection submission collation analysis reporting feedback interrogation action improvement collection submission collation analysis reporting feedback interrogation action improvement collection submission collation analysis reporting feedback interrogation action improvement collection submission collation analysis reporting feedback interrogation action improvement collection submission collation analysis reporting feedback interrogation action improvement collection submission collation analysis reporting feedback interrogation action improvement collection Data flow…..

20. Data Flow Collection, submission - Local Collation, analysis, reporting, feedback - National Interrogation, action, improvement - Local Assurance - local – Boards – Commissioners – Regulators

21. Recommendation 2 When commissioning new national clinical audits and reviewing contracts, consider a balance of data on clinical effectiveness, patient experience and safety. NCA methodology primarily drives improvement in clinical effectiveness by reporting how well structure and process measures are achieved against established standards and by comparing outcomes between providers

22. Recommendation 2 CORP looks at safety – rolling programmes We will be working with NHS England over the next 6 – 12 months to develop and agree the commissioning intent for patient-reported outcome and experience measures within the Programme.

23. Recommendation 4 Stimulate engagement through a communications strategy that focuses on creating broader awareness of HQIP’s remit and reach and the role of national clinical audit for quality improvement. in development - HQIP Communications Strategy Closer working with QI partners All new NCAs must have improvement driven aims and objectives to which all activities are aligned HQIP review reports, press releases to ensure improvement messages are clearly articulated

24. Recommendation 5 Further work could be done to evaluate the effectiveness of specific mechanisms and drivers (financial incentives, using data for appraisals and clinical excellence awards, workshops) for using national audit for improvement Best Practice Tariffs HQIP analyse feedback on the degree to which care improvement can be attributed to mechanisms and drivers such as those listed. This is undertaken at each commissioning decision point

25. Recommendation 5 Working with AoMRC to review and update the guidance on revalidation, focussing on supporting clinician’s participation in national audit and their use of national audit data to drive improvement in the quality of care delivery and patient outcomes. The extent to which NCAs report how their data is used as a mechanism/lever to drive improvement forms part of the ‘The Audit of Audits’ which has been completed by all National Clinical Audits and will publish in December 2014.

26. Recommendation 8 Consider ways to provide real-time feedback to support continuous improvement as well as providing datasets that can be manipulated for local analysis All new contracts require that data extracted is always accessible locally and this local availability should be accompanied by reporting mechanisms which have local utility to support quality improvement.

27. Recommendation 11 Explore how to streamline data collection methods and combine processes and systems with consideration to broader informatics work and the context in which national clinical audits sit. We will continue to streamline data collection and minimise local burden and share this responsibility with the HSC IC and the Confidentiality Advisory Committee who also scrutinise data set size and burden.

28. Group Work

29. Two Questions What can HQIP to do to support you? What can you do differently / Share good ideas

30. Recommendation 20 Clinicians, audit staff, and managers need to integrate more to take ownership of and engage with audit. Relationships between the medical director, clinical audit teams, and clinical leads are particularly important Locally… – NCA data flow within your organisation HQIP…. – Commissioning of NCA/ guidance / training / events