1. WHY USE THE RCGP OUT OF HOURS CLINICAL AUDIT TOOLKIT ? Dr. Agnelo Fernandes MBE FRCGP 6 th March 2008

2. Out of Hours Clinical Audit Toolkit Background:  QR 4 – OOHs Providers & PCTs - inconsistencies  DH commissioned RCGP to develop a Clinical Audit Toolkit (2006)  Information gathering during 2006  Development team - >60years OOHs experience  Audit Tool tested – autumn 2006  Toolkit approved by RCGP and DH early 2007  OOHs Clinical Audit Toolkit Published March 2007

3. Out of Hours Clinical Audit Toolkit  BACKGROUND QR4 (2004 / 2006): For OOHs Providers  Experienced OOH’s clinician to lead audit programme  Regular audit of a random patient contacts - any staff group  Sufficient sample to review clinical performance of individuals  Appropriate action taken on the results of these audits.  Regular reports of audits to the contracting PCT.  Where appropriate results shared with multidisciplinary team  “Providers must cooperate fully with PCTs to ensure audits include clinical consultations whose episode of care involved more than one provider organisation” - along “Patient journey”

4. Out of Hours Clinical Audit Toolkit  AIMS:  To support OOH providers for effective clinical audit.  To maximise the opportunities of audit  To be “Practical” – “Deliverable” – “Achievable”  To be a “Best Practice Framework” - locally tailored  For Continuous Quality Improvement (CQI)”

5. Out of Hours Clinical Audit Toolkit  Step 1: Role of clinical audit within the Organisation  Step 2: Define the Patient Pathway  Step 3: Define the Audit Criteria  Step 4: Define an Audit Tool  Step 5: Conduct the Audit  Step 6: Incorporate Wider Learning  Step 7: Repeat the Audit cycle

6. Out of Hours Clinical Audit Toolkit  Step 1- Organisation  Resources  Accountability  Roles & Responsibilities  Individual & Organisational Learning  Assessment, Feedback, Learning, Implementing change

7. Out of Hours Clinical Audit Toolkit  Step 2  Define the Patient Pathway  Local Variability  Generic Decision Making Processes

8. Out of Hours Clinical Audit Toolkit  Step 3 - Define the Audit criteria  Criteria against which performance can be audited  Established Principles – GMC, NMC, SBH, OOHs QR, RCGP- MRCGP Video consultation assessments & examples of current good practice  No evidence based Standards for Criteria in OOHs  The Standard set for each criterion is the mean for the individual organisation  Scoring system to benchmark mean score for each criterion as part of a formative approach

9. Out of Hours Clinical Audit Toolkit 1. REASON (elicited) 2. EMERGENCY identified 3. HISTORY (or Algorithm) 4. ASSESSMENT 5. CONCLUSIONS 6. EMPOWERING behaviour 7. MANAGEMENT decisions 8. PRESCRIBING 9. SAFETY- NETTING 10. RAPPORT developed 11. IT / Protocols / Algorithms 12. ACCESS criteria satisfied Audit CRITERIA

10. Out of Hours Clinical Audit Toolkit  Step 4: Practical Audit Tool  A generic audit tool  review of individual call handlers and clinicians (doctors, nurses, etc.)  face to face settings or on the telephone  Scoring – only as a benchmark to monitor progress within the individual organisation

11. Out of Hours Clinical Audit Toolkit  Step 5: Conducting the Audit – Information gathering  Information – IT systems, telephone calls, records  Telephone system records  Access times; Voice recordings  Computer (& paper) system records  Productivity Data; Outcome Data; Clinical Records  Feedback from patients & colleagues  Significant Events & Serious Untoward Incidents (SUI’s)

12. Out of Hours Clinical Audit Toolkit  Step 5: Conducting the Audit - Sampling  SAMPLING: random sample of patient contacts systematically reviewed for every working individual  Baseline minimum standard at least 1% or 4 examples (whichever is the larger) (cf. part-time staff)  If concerns - a further 4% calls/consultations reviewed  cf. Calls to reflect upon (CtR)  Early review – new staff 2% or 8 calls /consultations)  If complaint or concerns a greater number of reviews

13. Out of Hours Clinical Audit Toolkit  Step 5: Conducting the Audit – Feedback & Reports  Individual Audit Reports - (e.g. quarterly).  Individual Feedback – to facilitate “reflection”  Organisational Reports – monthly / quarterly  Organisational Feedback and Learning  Via Clinical Governance Group to the “Board”  facilitate learning and benchmark progress  targeted educational activity  Feedback and Learning involving multiple providers  segmented audits of patient pathway

14. Out of Hours Clinical Audit Toolkit  Step 5: Conducting the Audit - Acting on audit findings at individual and organisational levels  Individual action  Selected input and Learning Plan  Organisational actions  Resources for support & educational input  Persistent poor performer  Despite support & educational input  Reporting to PCT Performance unit

15. Out of Hours Clinical Audit Toolkit  Step 6: Incorporate learning from other aspects of the service  A multi-disciplinary and multi-agency Clinical Governance Group (includes patients and commissioners)  Organisational “Reflection” & Learning

16. Out of Hours Clinical Audit Toolkit  Step 7: Repeat the Audit Cycle  Routine clinical audit driving CPD  Performance Review triggered by Audit or Complaint /SUI  Audit to inform Appraisal & Performance Review

17. Why use the RCGP Out of Hours Clinical Audit Toolkit?  Support delivery of QR 4 - effective Clinical Audit  Practical generic audit “tool” (locally adaptable)  all Staff (clinicians + call handlers)  in different Settings ( telephone, F2F – home or Base )  in different Providers along the patient pathway  Promote “Cultural Change” – individual / organisational  service / contract  patient centredness - shared decision making  Audit – integral part Continuous Quality Improvement