Monitoring clinical quality through indicators, metrics and dashboards National perspective Danny Keenan National Clinical Advisor Care Quality Commission

2 NHS Performance Framework; Implementation guidance April 2011 Technical Guidance for the 2011/12Operating Framework; Jan 2011 The NHS Outcomes Framework 2011/12 Quality metrics and indicators explained: what’s the difference between a metric and an indicator Discussion re Clinical Quality Augmenting national indicators with local measurements of quality How the Care Quality Commission will monitor improvements in quality Using clinical dashboards to monitor improvement in quality indicators and metrics

3 Metric or indicator? A metric is a measure Patients in primary care have their blood pressure measured An indicator is a way of using a metric for purposes of comparison: (Latin: indicare = to point out)

4 Metric versus indicator Unit A has a caesarean Section rate of 15% A metric An indicator

5 What is clinical quality?

6 Three domains of quality Safety Effectiveness Patient experience

7 Outcomes and indicators for the first framework Preventing people from dying prematurely Enhancing the quality of life for people with long-term conditions Helping people to recover from episodes of ill health or following injury Ensuring people have a positive experience of care Treating and caring for people in a safe environment and protecting them from avoidable harm EFFECTIVNESS PATIENT EXPERIENCE SAFETY Clinical quality The NHS Outcomes Framework 2011/12

8 Assessing quality Measuring clinical quality: Structure -Refers to the inherent characteristics which are associated with higher quality (e.g): Procedure volumes Advanced IT Nurse staffing ratios Stroke service Processes -Do you comply with evidence-based care guidance? Administration of aspirin and B blockers with acute coronary syndrome Compliance with desirable practices such as frequent determination of HbA1c in diabetics Outcomes -some general ones -some sector or condition specific

9 Example metrics for assessing quality Structure

10 Implementing the NHS Performance Framework

11 Augmenting national indicators with local measurements of quality Preventing people from dying prematurely Enhancing the quality of life for people with long-term conditions Helping people to recover from episodes of ill health or following injury Ensuring people have a positive experience of care Treating and caring for people in a safe environment and protecting them from avoidable harm EFFECTIVNESS PATIENT EXPERIENCE SAFETY Clinical quality The NHS Outcomes Framework 2011/12 There are a set of 50 indicators underneath these domains

12 Augmenting national indicators with local measurements of quality The Commissioning Outcomes Framework Derived from the NICE Quality Standards, among other things. There will always be room for local measures: North West : Advancing Quality ---AQuA (Advancing Quality Alliance): -Heart Attack -Hip or knee replacement -Heart failure -Pneumonia -Coronary artery bypass surgery -Next iteration:  Mental health, including dementia  COPD  Diabetes

13 NHS Performance Guidance- Implementation Guidance. April 2011 Measuring for Quality Improvement launched the development of a menu of Assured Quality Indicators to enable local clinical teams to identify indicators that support their improvement work and allow benchmarking with other clinical teams. Quality indicators from this Assured Menu will have a number of uses, including in commissioner contracts (particularly the Commissioning for Quality and Innovation (CQUIN) payment framework), publication of Quality Accounts, and information for the public through NHS Choices.

14 Indicators for Quality Improvement

15 How CQC will monitor improvements in quality The essential standards of quality and safety

16 CQC and registration

17 Quality and Risk Profile (QRP)

18 Quality Risk Profiles (QRP) are at the centre Bring together information about a care provider so as to estimate risk and prompt front line regulatory activity Act as a guide and aid CQC’s inspectors Not a judgement (or a rating, or a league table) in themselves Build over time Capable of using both numeric (quantitative) and textual (qualitative) data Essential tool to support monitoring of compliance Help inform subsequent judgements and add to the knowledge base about a care provider Rejects the use of a small number of numeric indicators to make judgements about service quality

19 …but builds on 10 years development 19 The philosophy draws on the work of Klein and Carter in the 1980s The mathematical approaches have been being developed by CQC and its predecessors since the early 2000s and have substantial academic input The qualitative analysis techniques allow us to treat textual information consistently draw on best practice in social research techniques The approach draws on and addresses Bevan and Hood’s reflections of the “measurement frenzy” in public service reform in the 2000s This approach is at the cutting edge of use of information, but we believe transferable between sectors

20 Updating and sharing of QRPs with providers 20 QRPs are updated regularly (monthly or more often depending upon data feeds) The QRPs will be shared with Providers as they are updated so that we are all working from the same data NHS trusts have had access to their QRP since September via a.pdf. This has been made available to commissioners, SHAs, Monitor and DH since October This summer we will release an online service which will give access to data as live and which will have vastly increased functionality

21

22 How the Care Quality Commission will monitor improvements in quality

23 How CQC will monitor improvements in quality Our main tool will be using Registration and………. Compliance with Registration Previously we had the Annual Health Check…proved to be a blunt tool Now that Registration has been in place, with Trusts, for a year compliance is becoming more important

24 Site visits The aim of site visits is to gather evidence of compliance They are short, focussed unannounced site visits, rather than set piece inspections that require the provider to spend a lot of time in preparation Site visits are direct checks of compliance rather than assessing compliance through the assurance systems the organisation has in place. Therefore site visits may include direct observation of care and spending time with people who use the service, their families and carers, unless not appropriate to do so. We may also talk to managers and staff. Experts by experience will join us on some site visits to help us engage with people who use services.

25 Judgement framework Regulatory judgement and response Stage 2: Does the evidence show compliance? Stage 3: What is the impact on people who use services and the likelihood of this happening? Is there: No concern Minor concern Moderate concern Major concern Stage 4: Validation Regulatory response Maintain registration - no further action Improvement actions: eg improvement letter Enforcement actions: Statutory warning notice Imposition or variation of conditions Fines Prosecution Suspension of registration Cancellation of registration Regulatory judgement Judgement of compliance or concerns Translates minor, moderate or major concerns into regulatory judgement Takes account of the provider’s capability to improve Action will be proportionate Many Trusts have some compliance problems With enforcement to ensure compliance we believe this will drive improvement

26 Using clinical dashboards to monitor improvement in quality indicators and metrics This will be determined locally, bearing in mind that there will be requirements laid down by the Department (including QIPP local agenda); CQC and Monitor. So a dashboard will have these through it: Measures to reassure the Board/Managers that the service they provide is effective, safe and patient responsive Measures negotiated with local commissioners Measures suggested by clinicians and for use in revalidation Measures suggested by patients, including patient outcomes and surveys

27 Using clinical dashboards to monitor improvement in quality indicators and metrics Pitfalls: Avoid that which is easy to measure but not useful The adverse effects of any measure which is that the measured item improves at the expense of other important areas The burden of measurement

28 Local dashboards Some overarching themes: Process: -Board Issues (Having a culture of quality and safety in an organisation and amongst staff) -Quality of Records -Information / HES quality/ NHS number -Commitment to safety -Issues relating to staff Sickness/stress, staff satisfaction, absence, training, vacancies -Commitment to Audit and Service Improvement -Commitment to Research

29 Local dashboards Some overarching themes: Outcomes: -Mortality -Healthcare associated infections -Patients: Satisfaction, complaints -Prescribing, administration and reconciliation of medicines -Patient Safety First outcomes -Never events -National audits; Cardiac Cancer -HES based outcomes

30 Thank you

31 Annex to the NHS Performance Framework; Implementation guidance April 2011 Technical Guidance for the 2011/12Operating Framework; Jan

32 Mental Health