Clinical Performance Development in NHS Direct Dr Andrew Lee NAGPC Annual Conference March 2004

Clinical performance indicators Standard national reports replaced individual site versions Reported at national, network, site, nursing team, and individual nurse level Separate 0845 and OOH Clinical performance targets –Sorting of symptomatic calls –Use of NHS CAS

Gold Standard Sorting Study

Aims Establish a benchmark of existing NHS practice in clinical risk tolerance for primary presentations (first point of contact) by telephone Identify and quantify any appropriate difference in the sorting profile for calls made to access GP OOH services and that for other NHS Direct calls Use the results to inform target setting for NHS Direct clinical performance indicators

Gold Standard Sorting Study First Stage - Benchmarking –113 GPs from England and Wales –Random sample of 600 NHS Direct calls and 300 GP OOH –Clinical details of each call reviewed independently by 10 reviewers

Gold Standard Sorting Study Second stage – Reviewing the Differences –Cases with significant disagreement between reviewers and NHSD identified –Consensus review of these by senior NHSD medical and nursing clinicians taking account of : NHS guidelines, clinical consistency and context of service model

Gold Standard Dispositions Dispositions used to calculate Gold Standard: –NHSD actual dispositions where consensus amongst reviewers same –NHSD actual dispositions where no consensus at all amongst reviewers –Dispositions chosen by senior NHSD clinicians in Consensus review stage

GP consensus? Unanimity amongst reviewers on level of care in only 7% of cases At least 2:1 consensus amongst reviewers on a level of care in 46% of cases At least 2:1 consensus amongst reviewers on whether referral to urgent care (999/A&E/GPOOH) needed or not in 74% of cases

When GP consensus re Urgent referral/Not High concordance with NHS Direct –NHSD actual disposition was same as reviewer consensus in 78% of cases –Gold standard is same as reviewer consensus in 96% of cases

Use of NHS CAS Compared the accuracy and risk of the three patterns of assessment: –Algo used and followed –Algo used but a different disposition chosen –No algorithm used Risk measured as potentially critical misses i.e. a Gold Standard disposition of Emergency or Urgent Care (999, A&E, GP 4 or 12) with an actual NHSD disposition below this level (in hours care or self care)

NHS CAS results Algorithm content safe Algorithm delivers gold standard sorting or nearer to it than altering in over 80% of use Not using an algorithm produced a rate of potentially critical misses nearly three times higher than when an algorithm was used whether the recommended disposition was followed or not

Comparison with other studies SWOOP study of nurse-led OOH triage (1997): 49.2% referred to GP OOH service (calls before midnight) NHSD Gold standard for GP OOH (combination of GP within 2 hours and GP within 6 hours) = 43% (+ some GP next day will go to OOH at weekends) Sheffield MCRU Appropriateness Study (2003): 13% overtriage and 1% undertriage (equivalent to a net reduction in urgency of 12%) NHSD Gold standard = reduction in emergency&urgent care 9.7% (0845) and 11.1% (OOH)

NHS CAS developments Disposition wording changed to make timeframes consistent with OOH access standards (1, 2 and 6 hours) Twice yearly major clinical content version releases ongoing Version 12 (April/May 2004) contains first Gold Standard related revisions

Summary GPs have difficulty agreeing on appropriate sorting NHS Directs sorting is not very different from what GPs can broadly agree on NHS Direct has a good safety record NHS Directs sorting can improve NHS Direct is taking actions to achieve that improvement NHS CAS will help nurses deliver that improvement