Understanding NMC allegations data, and developing a coding frame to categorise future allegations data Rob Francis Matt Reynolds March 2017 Restricted external

Introduction: What’s this all about?

Context of the project: Project background In September 2015 we were asked to conduct research for the NMC into the types of allegations made against nurses and midwives. Context of the project: Internal and public interest in being able to provide greater insight about these issues Previous way of collecting and categorising this data (by case speciality) had some limitations in how robust and detailed it was. E.g. high use of generic ‘other’ options Appetite to be better equipped to understand how allegations change in the course of the fitness to practise (FtP) process

Three main objectives: Research objectives Three main objectives: To develop a dataset of cases that can be used to analyse allegations; To develop a codeframe that enables the NMC to accurately categorise allegations data, across all three stages of its fitness to practise process (Screening, Investigation, Adjudication); To answer the question ‘What kinds of allegations lead to nurses and midwives being brought before their regulator?’ So we were creating both a piece of research and a living system

So what types of allegations are there? The codes: So what types of allegations are there?

At the heart of our work is the new codeframe: WHAT HAVE WE CREATED? At the heart of our work is the new codeframe: Over 200 codes which describe the different types of allegations made against nurses and midwives Organised into 3 levels, enabling ease of navigation. Level 1: 20 groups of allegation types Level 2: 117 codes describing specific types of allegations, all sitting within the 20 Level 1 groups; Level 3: A further 99 codes which capture specific detail where this is given, e.g. who the victim of an allegation was, or how exactly the nurse or midwife was alleged to have done something wrong.

Methodology: how did we get here?

Methodology: overview Our work is based on review and analysis of case documents from the NMC’s FtP process SCREENING INVESTIGATION ADJUDICATION Guiding principles: Iterative: a multi-stage process of looking at cases, Coproduced: working closely with NMC colleagues, several points for input and feedback Robust: through internal and external checks, and scale Live: acknowledging that reporting needs and interests change Pragmatic: there is no single way to arrange codes

Key decisions we took with the NMC: Research design Research design Analysis of NMC database Scoping Main coding Developing the user guide Quantitative analysis Report Sampling Key decisions we took with the NMC: Look at cases from most recent one year period Review 900 cases, 300 from each stage Focus on the nature of allegations made, not the NMC’s decisions or processes

Creating new codes to capture types of allegations we were seeing Main coding Research design Analysis of NMC database Scoping Main coding Developing the user guide Quantitative analysis Report Sampling A qualitative process of reading and reviewing case documents, but done at quantitative scale (900) Reading relevant sections from each document to examine what allegations were being made Creating new codes to capture types of allegations we were seeing Internal and external discussions and sense-checking Chopping and changing, how codes are grouped, broken down, or merged – the codeframe was a living document! Some further cleaning of the data was required Final decisions about naming and hierarchy Key thing here – the data should inform the codeframe, not the other way around

Quantitative analysis Research design Analysis of NMC database Scoping Main coding Developing the user guide Quantitative analysis Report Sampling Once codeframe was finalised and we reviewed all cases, we analysed this data quantitatively Three main processes: Analysing how often different types of allegations are found in cases Breaking down results based on what stage a case reached; and whether it was a nurse or midwife Examining what types of allegations are often made as part of the same case What we didn’t look at: Reasons why different allegations were more or less common at different stages/for nurses or midwives How allegations changed for individual cases

so what allegations are being made? Findings: so what allegations are being made?

This shows the frequency of different groups of allegations Allegations groups This shows the frequency of different groups of allegations Patient-facing ones the most common Some groups involve more rare types of allegations

Comparison across stages Allegations relating to patient care are present in 27% of cases closed at Screening, 36% of cases closed at Investigation, and 48% of cases closed at Adjudication. The same pattern is evident for allegations around prescribing and medicines management; record keeping; and employment and contractual issues. These also represent many of the largest groups of allegations received. Cases with allegations about behaviour or violence invert this pattern. These allegations account for a fifth of cases closed at Screening stage (19%) but a smaller proportion of those closed at Investigation stage (15%) and an even smaller proportion of those reaching Adjudication (9%). Cases closed at Investigation most frequently involve criminal proceedings where a nurse or midwife was alleged to have been convicted, cautioned, or similar, than those closed at other stages (24%, compared to 19% of Adjudication and 15% of Screening cases). Cases in the Adjudication stage more often involve allegations of dishonesty (31%); Finally, cases in the Adjudication stage more often include allegations revolving around a nurse or midwife’s interactions with the NMC (12%, compared to 5% and 4% of cases closed at Screening and Investigation stages respectively.)

Comparison by role Cases involving midwives more often include allegations of record keeping issues (32% of cases with midwives, compared to 23% of cases with nurses). Cases involving nurses are more likely to involve prescribing and medicines management (26% of cases with nurses, compared to 22% of those with midwives). Cases involving nurses are more likely to involve references to criminal proceedings (20% of cases with nurses, compared to 9% of those with midwives). Cases involving midwives are more likely to include allegations of problems with the nurse or midwife’s communication (22%, compared to 16% of nurse cases) Cases involving nurses are more likely to include allegations about their behaviour or violence (15%, compared to 7% of midwife cases) For smaller groups of allegations, differences were not evident. However, this may be in part due to the very small number of midwife cases with such allegations.

Patient care appears in all top co-occurrences Top pairings of allegation groups Number of cases % of all cases Record keeping with Patient care 148 16% Prescribing and medicines management with Patient care 109 12% Record keeping with Prescribing and medicines management 92 10% Communication issues with Patient care 88 Other crimes and offences with Criminal proceedings 64 7% Motor vehicle related with Criminal proceedings 59 Dishonesty with Patient care 52 6% Dishonesty with Record keeping 49 5% Employment and contractual issues with Patient care 47 Communication issues with Record keeping 45 Remember: multiple allegations can be raised against nurses and midwives in the same case, and several codes can be applied. Patient care appears in all top co-occurrences Communication issues also common

What happened next?

EMBEDDING THE CODEFRAME IN PRACTICE: The user guide The codeframe is a living thing – so NMC staff need to know how to look after it We created a user guide to help ensure that consistency and clarity in how the codes are applied. This includes: types of cases for each code, and how codes relate to one another. do’s and don’t’s point out where allegations can fall under several codes and how to distinguish between them. The guide is targeted at: People coding new cases Anyone who wants to analyse particular codes or overall data on allegations

Putting the work into practice Implementing the codeframe Following a series of training workshops which included around 80 staff (OPM initially then handing over to NMC training team), the NMC are implementing the new codes, using our guide to support staff The NMC will be using the codeframe to collect this data itself on a much bigger scale: For all cases seen in a year (in the 1000s) Collecting data at each stage of the FtP process that a case passes through; enables tracking of allegations through the different stages

HOW is the work being used by the nmc? The new codeframe is improving the NMC’s data and the application of its data. This should enable the Regulatory Intelligence Unit (RIU) and other FtP teams to: Develop a more evidenced based approach to regulation by analysing the allegations data to inform the direction of resources Share data and analysis with partners to enhance public protection, enabling more detailed reporting on FtP case referrals Develop learning loops, feeding back to the professions about what types of allegations are arising – highlighting problem areas which in turn can inform awareness-raising work, training and development etc. Provide public & patients with greater insight into the nature of allegations to NMC.