Audit and SEA Made Easy GPST Teaching Dr Kate Lewin GPST Course Organiser NHS Education for Scotland

Definitely not rocket science

Aims of workshop Refresh your knowledge of, or introduce you to, audit and SEA Define criteria and standards Increase confidence in performing Audit/SEA in your own practice Appreciate the use of audit/SEA in quality assurance, also for revalidation, QOF etc. Signpost to resources for further guidance Discuss YOUR audits or audit plans in groups

What is audit? Audit-schmaudit A necessary evil The thing my supervisor keeps nagging me about

What is Audit? A range of definitions exist: Audit is about taking note of what we do, learning from it and changing if necessary Audit is the improvement in the quality of care through standard-setting, peer review, implementation of change and re-evaluation Quite simply, audit is a tool that enables you to monitor and then improve the quality of care you provide to your patients. Systematic critical analysis of the quality of health care

The Audit Cycle Choose the Topic Define Criteria & Standards or “What do you think you should be doing” Define Criteria & Standards or What do you think you should be doing Identify the Changes Required & Implement Them !! Collect the Data i.e. The Information on what you are doing Assess Performance against criteria & standards How are we doing in relation to what we should be doing ?

Eg. Clopidogrel Prescribing The Audit Cycle Choosing The Topic Eg. Clopidogrel Prescribing

Define Criteria & Standards The Audit Cycle Define Criteria & Standards Criteria - what you want to measure (yard-stick) eg. Patients should have clopidogrel prescribed only in accordance with NHSGGC guidelines (ie. aspirin contraindicated or postACS/TIA/Stroke/stent insertion) Standard - how well you should be doing 80% of patients should have clopidogrel prescribed in accordance with NHSGGC guidelines

The Audit Cycle Collect the Data Identify patients on clopidogrel 2. Pharmacist review of notes – identify when started, by whom, indication, whether ever on aspirin +/- PPI 3. Determine whether in accordance with guidelines

The Audit Cycle Assess Performance Compare our results with the standard previously set e.g. 21 of 116 patients on clopidogrel (18%) were prescribed according to NHSGGC guidelines – far below standard of 80%

The Audit Cycle Agree & Implement Changes Required Explore reasons for inappropriate use Feedback to colleagues, discuss changes and implement them Eg. Letters to patients/cardiologists, face-to-face medication review, raising awareness of prescribers

Repeat data collection once changes have had a chance to take hold The Audit Cycle Repeat the Audit!!! Data Collection 2 Repeat data collection once changes have had a chance to take hold

Re-assess Performance Has the standard now been met? The Audit Cycle Re-assess Performance Compare the results with the standards previously set and results of data collection 1 Has the standard now been met? e.g. Now find that 48 of 90 (53%) of patients on clopidogrel are prescribed within guidelines ie. Significant improvement but still below standard

Identify Further Changes Required The Audit Cycle Identify Further Changes Required Long term issues: Determine if further change is required to sustain performance, and decide when next to audit this topic (annually, every 2 years etc.)

Criteria – what you want to measure Simple logical statements, used to describe a measurable item of quality health care ie. What you want to measure e.g. Patients with asthma should have their inhaler technique assessed at least once every 12 months.

Standard – How well you should be doing! Describes the ideal level of care to be achieved for each criterion ie. How well you should be doing e.g. 80% of patients with asthma should have their inhaler technique assessed at least once every 12 months.

Examples of Criteria & Standards

Arriving at Standards Don’t get overly concerned - standard setting is flexible, can be revised upwards or down Those involved decide on the level of care they find desirable - it is a professional issue/decision. Guidance can be derived from the literature/textbooks, but ultimately you decide with your practice. Can be based on your own work and observations, varies between practices

Report format for audit Report Section Guidance Reason for Audit Explain why the topic was chosen Potential benefits to patients/staff/practice Criteria Set Try to limit to 1-3, relevant to topic Short simple logical statements Justify with reference to current evidence Standards Set Agree a measureable standard for each criterion Set a realistic timescale Preparation and Planning Who you discussed the audit with, and who assisted you How you collected and analysed the data

Report format cont. Report Section Guidance Data Collection 1 Present using simple descriptive statistics/tables/graphs Do not present irrelevant data Comment on comparison with standard set Description of Change Describe changes agreed and implemented Attach example as evidence if possible Data Collection 2 Compare with standard and with data collection 1 If standard not reached speculate as to why Conclusions What have you learned How do you intend to take forward in future ?peer review

What topics to audit? Areas relevant to your practice Linked to educational event attended Look at NES Audit Ideas Booklet – online link, lots of ideas, topic areas and suggested criteria!

Group Exercise 1 Criteria & Standards – understanding the difference Group Exercise 2 Implementing Change – sometimes the hardest part of doing an audit!! Time to discuss own audits?

Significant Event Analysis

What is a significant event? “ Any event thought by anyone in the team to be significant in the care of patients or the conduct of the practice “ (Pringle et al, 1995)

SEA What happened? Why did it happen? What have you learnt? What have you changed?

What happened? Record all of the facts relating to the identified significant event (including any relevant dates, times and people or organisations involved) Data source: those directly and indirectly involved Establish a clear and full picture of what happened Impact or potential impact

Why did it happen? Establish all of the main and underlying reasons why the event actually occurred. Eg. A written telephone message about an important meeting was not passed to the practice manager because it had been lost. But…. why was it lost? Because it was written on a post-it and left on top of a report, which was subsequently filed away by an unsuspecting member of staff. ie. Internal communication practices not up to scratch!

What have you learned? Highlight any learning issues you and/or the practice experience. For example it may be related to a training need or a lack of knowledge concerned with therapeutics, disease management or administrative procedures. It could also reflect a learning experience (good or not so good) in dealing with patients, colleagues, or other organisations Ensure that insight into the event has been established by the practice team or the individuals concerned

What have you changed? Often a change in some aspect of care is required to improve the provision of care and/or minimise the risk that a similar event will occur. If so, a description of the change actually implemented should be given rather than a “wish list” of thoughts

What have you changed? (cont.) Sometimes it is not possible to implement change, either because the likelihood of the event happening again is so rare or because change is outwith the control of the individual/practice. If this is the case, then reasons should be clearly documented. Regardless of the type of significant event, change should at least be considered, then either implemented or justifiably ruled out

Important points Doesn’t have to be an bad event Sharing Blame-free Could explore example of excellent practice Sharing Team activity Blame-free Constructive learning not finger-pointing Look beyond the superficial For underlying/systematic causes

Useful links for further information Guidance on Audit, RCGP Revalidation Toolkit (p28) http://www.rcgp.org.uk/PDF/Scot_Complete_Revalidation _Toolkit_(Read_Only).pdf Ideas for Audit, NES www.clinicalgovernance.scot.nhs.uk/.../ideasforaud itandSEA.rtf SEA – NPSA Guide 2008 http://www.npsa.nhs.uk/nrls/improvingpatientsafety/prima rycare/significant-event-audit/

Group Exercise 3 Analysis of SEAs Time to discuss own SEA's?