1. Audit and SEA Made Easy GPST Teaching, Beardmore, November 2010 Mairi Jamieson GPST Course Organiser NHS Education for Scotland

2. Definitely not rocket science

3. Aims of workshop  Refresh your knowledge of, or introduce you to, audit and SEA  Define criteria and standards  Increase confidence in performing Audit/SEA in your own practice  Appreciate the use of audit/SEA in quality assurance, also for revalidation, QOF etc.  Signpost to resources for further guidance

4. What is audit?

5. What is Audit?  Audit is about taking note of what we do, learning from it and changing if necessary  Audit is the improvement in the quality of care through standard-setting, peer review, implementation of change and re-evaluation  Quite simply, audit is a tool that enables you to monitor and then improve the quality of care you provide to your patients.  Systematic critical analysis of the quality of health care A range of definitions exist:

6. The Audit Cycle Define Criteria & Standards or “What do you think you should be doing” Collect the Data i.e. The Information on what you are doing Assess Performance against criteria & standards How are we doing in relation to what we should be doing ? Identify the Changes Required & Required & Implement Them !! Define Criteria & Standards or What do you think you should be doing Choose the Topic

7. The Audit Cycle Choosing The Topic Eg. Clopidogrel Prescribing

8. The Audit Cycle Define Criteria & Standards Criteria - what you want to measure (yard-stick) eg. Patients should have clopidogrel prescribed only in accordance with NHSGGC guidelines (ie. aspirin contraindicated or postACS/angiography) Standard - how well you should be doing 80% of patients should have clopidogrel prescribed in accordance with NHSGGC guidelines

9. The Audit Cycle Collect the Data 1.Identify patients on clopidogrel 2. Pharmacist review of notes – identify when started, by whom, indication, whether ever on aspirin +/- PPI 3. Determine whether in accordance with guidelines

10. The Audit Cycle Assess Performance Compare our results with the standard previously set e.g. 21 of 116 patients on clopidogrel (18%) were prescribed according to NHSGGC guidelines – far below standard of 80%

11. The Audit Cycle Agree & Implement Changes Required 1.Explore reasons for inappropriate use 2.Feedback to colleagues, discuss changes and implement them Eg. Letters to patients/cardiologists, face-to-face medication review, raising awareness of prescribers

12. The Audit Cycle Repeat the Audit!!! Data Collection 2 Repeat data collection once changes have had a chance to take hold

13. The Audit Cycle Re-assess Performance Compare the results with the standards previously set and results of data collection 1 Has the standard now been met? e.g. Now find that 48 of 90 (53%) of patients on clopidogrel are prescribed within guidelines ie. Significant improvement but still below standard

14. The Audit Cycle Identify Further Changes Required Long term issues: Determine if further change is required to sustain performance, and decide when next to audit this topic (annually, every 2 years etc.)

15. Examples of Criteria & Standards

16. Criteria Simple logical statements, used to describe a measurable item of quality health care ie. What you want to measure e.g. Patients with asthma should have their inhaler technique assessed at least once every 12 months.

17. Describes the ideal level of care to be achieved for each criterion ie. How well you should be doing e.g. 80% of patients with asthma should have their inhaler technique assessed at least once every 12 months. Standard

18. Arriving at Standards  Don’t get overly concerned - standard setting is flexible, can be revised upwards or down  Those involved decide on the level of care they find desirable - it is a professional issue.  Guidance can be derived from the literature/textbooks, but ultimately you decide.  Can be based on your own work and observations, varies between practices

19. Report format for audit Report SectionGuidance Reason for AuditExplain why the topic was chosen Potential benefits to patients/staff/practice Criteria SetTry to limit to 1-3, relevant to topic Short simple logical statements Justify with reference to current evidence Standards SetAgree a measureable standard for each criterion Set a realistic timescale Preparation and Planning Who you discussed the audit with, and who assisted you How you collected and analysed the data

20. Report format cont. Report SectionGuidance Data Collection 1Present using simple descriptive statistics/tables/graphs Do not present irrelevant data Comment on comparison with standard set Description of Change Describe changes agreed and implemented Attach example as evidence if possible Data Collection 2Compare with standard and with data collection 1 If standard not reached speculate as to why ConclusionsWhat have you learned How do you intend to take forward in future

21. Group Exercise 1 Criteria & Standards Group Exercise 2 Implementing Change

22. Significant Event Analysis

23. What is a significant event? “ Any event thought by anyone in the team to be significant in the care of patients or the conduct of the practice “ (Pringle et al, 1995)

24. What happened?  Record all of the facts relating to the identified significant event (including any relevant dates, times and people or organisations involved)  Data source: those directly and indirectly involved  Establish a clear and full picture of what happened  Impact or potential impact

25. Why did it happen?  Establish all of the main and underlying reasons why the event actually occurred.  Eg. A written telephone message about an important meeting was not passed to the practice manager because it had been lost. But…. why was it lost? Because it was written on a post-it and left on top of a report, which was subsequently filed away by an unsuspecting member of staff. ie. Internal communication practices not up to scratch!

26. What have you learned?  Highlight any learning issues you and/or the practice experience.  For example it may be related to a training need or a lack of knowledge concerned with therapeutics, disease management or administrative procedures.  It could also reflect a learning experience (good or not so good) in dealing with patients, colleagues, or other organisations  Ensure that insight into the event has been established by the practice team or the individuals concerned

27. What have you changed?  Often a change in some aspect of care is required to improve the provision of care and/or minimise the risk that a similar event will occur.  If so, a description of the change actually implemented should be given rather than a “wish list” of thoughts

28. What have you changed? (cont.)  Sometimes it is not possible to implement change, either because the likelihood of the event happening again is so rare or because change is outwith the control of the individual/practice. If this is the case, then reasons should be clearly documented.  Regardless of the type of significant event, change should at least be considered, then either implemented or justifiably ruled out

29. Important points  Doesn’t have to be an bad event  Could explore example of excellent practice  Sharing  Team activity  Blame-free  Constructive learning not finger-pointing  Look beyond the superficial  For underlying/systematic causes

30. “Nothing changes if nothing changes”

31. Group Exercise 3 Analysis of SEAs

32. Useful links for further information  Guidance on Audit, RCGP Revalidation Toolkit (p28)  http://www.rcgp.org.uk/PDF/Scot_Complete_Revalidation _Toolkit_(Read_Only).pdf  Ideas for Audit, NES  www.clinicalgovernance.scot.nhs.uk/.../ideasforau ditandSEA.rtf  SEA – NPSA Guide 2008  http://www.npsa.nhs.uk/nrls/improvingpatientsafety/prima rycare/significant-event-audit/