Copyright © Healthcare Quality Quest, 2013

Proposed standards for a national clinical audit — How we got involved and what we have learned

Method and problems encountered Background Proposed standards Applying the standards — lessons learned The way forward

Method and problems encountered Background Proposed standards Applying the standards — lessons learned The way forward

We aim to help healthcare organizations to improve the quality of patient care Teach clinical audit process — over 30,000 clinical staff and over 1000 clinical audit facilitators Carry out or supervise a number of large-scale clinical audits, usually technically challenging Teach clinical audit process to clinicians outside the UK — 8 countries Background — HQQ Sold over 2500 copies of book for clinicians

Background HQQ was asked to evaluate a national clinical audit HQQ needed an objective basis for the evaluation There were no published standards internationally for a national clinical audit or equivalent activity We needed to develop the standards in order to do the job of evaluating the national clinical audit

Method and problems encountered Background Proposed standards Applying the standards — lessons learned The way forward

Method Search strategy Literature databases Websites Journals Professional organizations Search engines Included Any document that described an expectation related to the design or conduct of a national clinical audit Sources screened and abstracted by one reviewer and validated by another Limitations Much literature about local clinical audits Some descriptions not available

Problems encountered Finding ‘evidence’ of good practice — Often deeply embedded in the description of a project; not easily accessible Language — Many different terms used in different countries Meaning — Clinical audit as large-scale data collection and collation versus quality improvement Little crossover with large-scale quality improvement studies — Much evidence was in the context of a national or large-scale quality improvement study Small benefits of knowing American and international language and practices

Recognizing terms used in other countries National clinical audits National outcomes studies National or international quality indicators measurement Activities included National performance indicator monitoring National quality improvement studies Registries

Method and problems encountered Background Proposed standards Applying the standards — lessons learned The way forward

Proposed standards for a national clinical audit — Structure Ethical basis The subject has an ethically based justification Governance There is continuous and effective governance Stakeholder involvement All key stakeholders are actively involved, including patients Resources Resources allow for appropriate, valid and reliable measurement and proactive facilitation of improvement Roles and responsibilities and project plan Roles and responsibilities at national and local levels are defined There is a published project plan for all aspects of the audit

Proposed standards for a national clinical audit — Process Recruitment The audit is continuously proactive in recruitment Improvement objectives The stated aims reflect an intention to drive improvement in quality Population or sample The population or sample is suitable for drawing valid conclusions and minimizing bias Data collection strategy The data collection strategy is suitable for the stated objectives Quality-of-care measures Measures used meet criteria established by the governance board and are capable of producing valid and reliable data

Proposed standards for a national clinical audit — Process Data elements and data sources Data permit judging compliance with quality-of-care measures but do not place a data collection burden Data collection protocol The protocol describes in detail all aspects of data collection and analysis Protection of patient identity The audit keeps patient identity protected in all stages Identification of ethics issues Any ethics issues are recognized and handled Training and support for data collectors Data collectors are trained and supported to provide complete, accurate, reliable and timely data

Proposed standards for a national clinical audit — Process Pilot testing The entire design, process and tools are pilot tested Reliability testing Reliability of data collected or acquired is tested Data linkages Links to any electronic record systems are secure Data quality management and assurance Data are quality assured and analysed appropriately Preliminary data and peer review Data are available on a timely basis for local peer review

Proposed standards for a national clinical audit — Process Identification of good practice and shortcomings Good practice and shortcomings in quality of patient care are explicitly identified Analysis of causes of shortcomings There is support for the analysis of causes of shortcomings in quality Facilitation of improvements There is proactive facilitation of improvements in the quality of patient care Reports Reports are complete, timely and useful Communication There is communication with all stakeholders throughout all stages

Proposed Standards for a national clinical audit — Outcomes Level of participation A target for participation is set and met Timeliness of preliminary reports Preliminary data are available for local validation and peer review within weeks Timeliness of complete reports Final reports are available within weeks from the validation stage deadline Evidence of improvements Improvements in patient care are sufficient to justify continuation Reliability of data Data are >90% reliable

Method and problems encountered Background Proposed standards Applying the standards — lessons learned The way forward

Applying the standards — lessons learned A national clinical audit is like a super tanker — it takes a long time to change direction With good operational definitions — assessment was straightforward Room for improvement in the national audit we evaluated — agreement on our judgements and warm reception from all concerned

Method and problems encountered Background Proposed standards Applying the standards — lessons learned The way forward

The way forward Publication — as a result of strong encouragement to share the work we did Application — The Standards are being pilot tested for a new national clinical audit project in Australia Interest from other countries — Representatives of Ministries of Health in 3 countries have indicated interest in national clinical audits and using the proposed standards as a basis for planning

Copyright © Healthcare Quality Quest,