Why standards or training do not fix all problems: How to identify and improve system problems Nigel Livesley MD, MPH Regional Director, South Asia USAID ASSIST Project URC Email: nlivesley@urc-chs.com Twitter: @NigelLivesley

% of women attending ANC clinic whose BP and Hb is measured

Standards and guidelines Knowledge and skills enhancement Improving health service delivery organisational performance in health systems. Pallas et al. International Health 4 (2012) 20–29 Standards and guidelines (accreditation, quality inspection) Knowledge and skills enhancement Leadership and management (supervision) Organisational structure Incentives Process improvement Organisational culture To highlight that there are many ways to improve quality of care. Talk about how we focus a lot on the first column, not so much on the 2nd but that the 2nd is also important.

The highest attainable standard of health Interaction between provider and patient

$$ Culture The highest attainable standard of health Interaction between provider and patient Culture Skills enhancement Healthcare process improvement Skills + process improvement Standards and guidelines Organizational design Leadership and management Incentives $$

% of women attending ANC clinic whose BP and Hb is measured

This could be due to poor individual performance or poor systems Primarily system problem There are barriers preventing nurses from doing their job Solutions Process improvement Management

This could be due to poor individual performance or poor systems Primarily individual problem Nurses aren’t doing their job Lack of knowledge Lack of skills Lack of motivation Solutions Standards Training Incentives Supervision Primarily system problem There are barriers preventing nurses from doing their job Solutions Process improvement Management

This could be due to poor individual performance or poor systems Primarily individual problem Nurses aren’t doing their job Lack of knowledge Lack of skills Lack of motivation Solutions Standards Training Incentives Supervision Primarily system problem There are barriers preventing nurses from doing their job Solutions Process improvement Management

% of women attending ANC clinic whose BP and Hb is measured

Huge variation from patient to patient and week to week

Improving processes of care…

…led to a nine fold increase in the number of women identified with risk factors 18 high-risk pregnancies identified (1.4%) 39 high-risk pregnancies identified (12.3%)

$$ Culture The highest attainable standard of health Interaction between provider and patient Culture Skills enhancement Healthcare process improvement Skills + process improvement Standards and guidelines Organizational design Leadership and management Incentives $$

To Err is Human: “the majority of medical errors do not result from individual recklessness … More commonly, errors are caused by faulty systems, processes, and conditions that lead people to make mistakes” “Health care organizations must develop a “culture of safety”... This will mean incorporating a variety of well-understood safety principles, such as designing jobs and working conditions for safety; standardizing and simplifying equipment, supplies, and processes; and enabling care providers to avoid reliance on memory.”

How well does the QI community do at making system changes?

System vs individual changes System change Leads to changed behavior without needing to be repeated AND Continues to work when new individuals join Individual change Needs continual effort to maintain new behavior OR Needs to be repeated when new individuals join

Examples Problem Individual changes System changes Patients not returning to clinic with lab results after testing Tell patients to return with results Put up posters and reminders Move lab test to start of clinic visit (before seeing clinician) Mothers’ are not breast feeding Train mothers on importance of breast feeding Train HW on importance of breast feeding Set target for early breast feeding Procure gowns that open in front for pregnant women so women are less exposed

Examples Problem Individual changes System changes Patients not returning to clinic with lab results after testing Tell patients to return with results Put up posters and reminders Move lab test to start of clinic visit (before seeing clinician) Mothers’ are not breast feeding Train mothers on importance of breast feeding Train HW on importance of breast feeding Set target for early breast feeding Procure gowns that open in front for pregnant women so women are less exposed

Examples Problem Individual changes System changes Patients not returning to clinic with lab results after testing Tell patients to return with results Put up posters and reminders Move lab test to start of clinic visit (before seeing clinician) Mothers are not breast feeding in hospital Train mothers on importance of breast feeding Train HW on importance of breast feeding Set target for early breast feeding Procure gowns that open in front for pregnant women so women are less exposed

Interventions to change systems remain underused Kellogg et al. BMJ Qal Saf 2017

Most changes focus on individuals 73% 27%

Skepticism about the ability of health care workers to identify and implement system changes

Skepticism should not stop us from trying to get better Skepticism about the ability of health care workers to identify and implement system changes Skepticism should not stop us from trying to get better

Number of kidney transplants per year 1933-1954

Number of SUCCESSFUL kidney transplants per year 1933-1954

Review meeting in 1963 216 transplants reported, most had died early: Cadaveric - 81% “Kidney transplantation is still highly experimental and not yet a therapeutic procedure

Increased use of new approaches 1 year cadaveric survival - 50%

Continued refinement 1 year cadaveric survival - 85%

This happened because of incremental improvements in: Surgical technique Immunosuppression Anti-infective therapy Reducing time to transplant Organ preservation Histocompatability typing

New things rarely work perfectly Making new things better takes testing and adaptation and science

New things rarely work perfectly Making new things better takes testing and adaptation and science The QI community tends to not practice what we preach when it comes to our own approaches

What are the barriers to making system changes?

Why do QI teams not make many system changes? Humans are predisposed to assign responsibility to people Identifying system problems and making system changes require a new set of skills

Fundamental attribution error “the tendency for people to place an undue emphasis on internal characteristics (personality) to explain someone else's behavior in a given situation rather than considering the situation's external factors” Wikipedia

Quiz show experiment Ross, Amabile and Steinmetz 1977 Eighteen pairs of students were randomly assigned to be: questioner told to come up with 10 questions they knew the answer to contestant 24 observers watched They new the questioner and participant roles were randomly assigned They new the questioners were coming up with their own questions    Observers rated the questioners and contestants on their ‘general knowledge’ Observers rated the questioners knowledge as superior  

Quiz show experiment Ross, Amabile and Steinmetz 1977 Eighteen pairs of students were randomly assigned to be: questioner told to come up with 10 questions they knew the answer to contestant 24 observers watched They new the questioner and participant roles were randomly assigned They new the questioners were coming up with their own questions    Observers rated the questioners and contestants on their ‘general knowledge’ Observers rated the questioners knowledge as superior  

Quiz show experiment Ross, Amabile and Steinmetz 1977 Eighteen pairs of students were randomly assigned to be: questioner told to come up with 10 questions they knew the answer to contestant 24 observers watched They new the questioner and participant roles were randomly assigned They new the questioners were coming up with their own questions    Observers rated the questioners and contestants on their ‘general knowledge’ Observers rated the questioners knowledge as superior  

Most public health interventions involve individual solutions Knowledge and skills enhancement Standards and guidelines Leadership and management (supervision) Organisational structure Incentives Process re-engineering Organisational culture Pallas et al. International Health 4 (2012) 20–29

QUESTIONS/COMMENTS Summary Despite talking about system change many QI initiatives still focus on changing individual performance Reasons for this include: Fundamental attribution error Traditional focus in health care education on individual performance Lack of emphasis on building skills to identify and solve system issues QUESTIONS/COMMENTS

What are some possible solutions?

Maternal mortality in a Delhi hospital in 2013 5854 deliveries Pt. Age of the patient Time of death after delivery Cause 1. 22 96 hrs PPH 2. 20 20 hrs 3. 21 6 hrs 4. 4 hrs PIH 5. 24 hrs

Women assessed only twice in six hours after delivery

Poll: What change did the team make? Train nurses in how to do assessment Re-emphasize the importance of following standard care Add new nurses Assign new patients specific beds and keep BP equipment in one place

First change: re-emphasize standards Solution: Letter from medical superintendent re-emphasizing that nurses should assess women 6 times in the first 6 hours.

Poll: What will happen? Fewer assessments will take place No change More assessments will take place

Sustained improvement!

0.16% of women identified with complications

Poll Why has the number of assessments gone up but the number of women identified with complications has not? The hospital does a great job of preventing complications The nurses do not know how to effectively assess women The nurses are not doing effective assessments because they don’t care The nurses are not doing effective assessments because something in the system is making it hard

The hospital does a great job of preventing complications Why has the number of assessments gone up but the number of women identified with complications has not? The hospital does a great job of preventing complications Not likely – the mortality rate in 2013 was very high The nurses do not know how to effectively assess women Not likely – measuring vitals and identifying complications are not that difficult The nurses are not doing effective assessments because they don’t care Not likely – no one likes mothers dying The nurses are not doing effective assessments because something in the system is making it hard This seems most likely

Poll Who do you think came up with the idea of reminding the nurses that they needed to do the assessment? The nurses who care for women after delivery The nurse manager The doctor in charge of the post-partum ward The hospital in-charge

Getting the right team is crucial The fundamental attribution error is weaker for our own performance – we see the effect of the system on our performance Teams with more involvement of the ‘workers’ develop more system changes Teams that have lots of changes related to education and directives are often not accessing useful information from the people doing the work

0.16% of women identified with complications Training and standards are not fixing this problem.

0.16% of women identified with complications How can we help them improve?

Poll What should the team do next? Medical superintendent should set a target of how many women with complications should be identified each week Weekly review of how many women with complications were identified Reward nurses who identify women with complications Ask the nurses what challenges they face in identifying women with complications

Improving ability to identify and management women with complications Talked to the nurses about why it was hard to do the assessment: Too much work Can’t reduce patient load Can’t recruit more nurses

Poll What should the team do? Give up Use incentives to motivate the nurses to work harder Look for activities that the nurses are spending time on that are either not useful or are harmful

Improving ability to identify and management women with complications Assessment took 20 minutes 15 m was spend looking for equipment and the specific woman 5 m was spend on patient care

Improving ability to identify and management women with complications There was no system for: keeping women who had just delivered in an easy to access location keeping the assessment equipment in the same place

Poll What should the team do next? Train nurses in how to do assessment Re-emphasize the importance of following standard care Add new nurses Assign new patients specific beds and keep BP equipment in one place

Women with complications are now 12 times more likely to be identified early New system of care Observation room Partnering with patients

Sustained improvement and NO MATERNAL DEATHS in post partum ward

Problem solving Teams that keep making individual level changes (education, directives, audit and feedback, incentives etc) may do so because of: Hierarchy issues Limited involvement of front line staff Skill issues Not analyzing the system well Not testing changes well

How to help teams develop more system changes Problem Try Hierarchy Explain the principle that anyone who will have to work differently should be part of the team deciding on changes Test any changes by asking the people who will have to work differently if they are feasible Not analyzing the system well Set constraints Do not consider individual performance

How to help teams develop more system changes Problem Try Hierarchy Explain the principle that anyone who will have to work differently should be part of the team deciding on changes Test any changes by asking the people who will have to work differently if they are feasible Not analyzing the system well Set constraints Do not consider individual performance

How to help teams develop more system changes Problem Try Hierarchy Explain the principle that anyone who will have to work differently should be part of the team deciding on changes Test any changes by asking the people who will have to work differently if they are feasible Not analyzing the system well Set constraints Do not consider individual performance

Constrain analysis to not consider individual factors

Constrain analysis to not consider individual factors

How to help teams develop more system changes Problem Try Hierarchy Explain the principle that anyone who will have to work differently should be part of the team deciding on changes Test any changes by asking the people who will have to work differently if they are feasible Not analyzing the system well Set constraints Do not consider individual performance Have only front line workers to the analysis Do not make any changes related to training or instruction

How to help teams develop more system changes Problem Try Hierarchy Explain the principle that anyone who will have to work differently should be part of the team deciding on changes Test any changes by asking the people who will have to work differently if they are feasible Not analyzing the system well Set constraints Give examples of system changes

How to help teams develop more system changes Problem Try Hierarchy Explain the principle that anyone who will have to work differently should be part of the team deciding on changes Test any changes by asking the people who will have to work differently if they are feasible Not analyzing the system well Set constraints Give examples of system changes Focus on building skills

Summary QI emphasizes making system changes Despite this, most changes, even within QI work focus on individual performance Cognitive (individual) and system factors contribute this Vicious circle There are a variety of strategies we can try to make more system level changes