Continuous Quality Improvement (CQI): Part 2 Bethany Geldmaker PNP, PhD Tracy Jebo MPH

Learning Objectives Review with Infant Mental Health Advisory Board: Part 1-- Foundational Continuous Quality Improvement Results of the Self-Assessment Survey results Vote on priority area to develop a CQI plan based on the self- assessment survey and Advisory Board feedback

Review Performance Management CQI PDSA Cycle

Performance Management and CQI Two primary components: Performance Management: Using data to improve performance CQI (Continuous Quality Improvement): Involving all stakeholders to examine the performance of a process and address identified gaps Performance Management and CQI are distinct processes. While some of the components may overlap, there are marked differences as well.

Quality Improvement Uses data on how you are performing to drive activities which lead to true improvement. Utilizes stakeholders to address identified gaps continually improve processes Addresses the question: How do we get better?

CQI Process: PDSA Cycle The use of a deliberate and defined improvement process Plan-Do-Study-Act Cycle will be used for this project

CQI Process: PDSA Cycle When applying PDSA there are 3 key questions and 4 QI principles that will guide efforts. The 3 key questions to keep front and center during QI work are: What are we trying to accomplish? How will we know that a change is an improvement? What changes can we make that will results in improvement?

CQI Process: PDSA Cycle The 4 CQI principles are: Develop a strong client focus Continually improve all processes Involve stakeholders Mobilize both data and team knowledge to improve decision-making

PDSA Cycle: Nine Steps The PDSA cycle is used to improve existing processes, not to plan or implement new programs Always relies on data to support its decisions

PDSA Cycle: Plan-Do-Study-Act Plan-Do-Study-Act Cycle (Steps 1-5) Step 1: Identifying a problem Step 2: Assembling a Q.I. team that can address the problem Step 3: Identify what the true cause(s) of the problem may be, map the process to understand the existing activities that lead to the problematic issue Step 4: Brainstorming and conducting background research to look for possible solutions Step 5: Working to develop a theory of improvement that can be tested

Plan-Do-Study-Act Plan-Do-Study-Act Cycle (Step 6) Step 6: Test your theory of improvement implement a small-scale test of the change to your process. Collect, chart, and display data to determine the effectiveness of the improvement. Document problems, unexpected observations, and unintended side effects, because these occurrences will aid in the learning process.

Plan-Do-Study-Act Plan-Do-Study-Act Cycle (Step 7) Step 7: Study the Results of your Test Use data to determine if the test of your theory was successful Compare the results of your test to the baseline data Some questions to consider include: Did your test work? How do you know? Did the results match your theory/prediction? What do the data show? Are there trends in your data? Did you have unintended side effects? Is there improvement? Do you need to test the improvement under other conditions?

Plan-Do-Study-Act Plan-Do-Study-Act Cycle (Steps 8 and 9) Step 8: Either standardize your improvement (if test was successful) or develop a new theory for improvement (if test was unsuccessful) Step 9: Establish Future Plans Sustaining change or beginning PDSA again

Tips for Beginning the Journey CQI can be confusing, daunting - lots of jargon, different tools, methods, approaches (Google = 2,760,000 results) Avoid starting with a large-scale project; think small and simple Choose a QI effort related to a familiar process, program or area Be patient – change takes time Unanticipated results are not failures. They are learning opportunities for your next trial. We will be going through the journey together! Feelings of uncertainty about how to begin or what steps to take next

FOCUS First Find an opportunity to improve. Organize a team who understands the process. Clarify the current knowledge of the process. In this stage, you are gathering the "who, what, when, and where“. Understand the cause of process variation. Here, you ask yourself the "why" question. Select the piece of the process you want to improve. This is not technically part of the “model for improvement” but is part of the PDSA cycle sometimes left off or forgotten. I thought it was actually important and relevant as a way to think about how to focus your improvement efforts. What we have already done in this process: -Found an opportunity to improve: Family Enrollment & Engagement -Organzied teams (you guys) – though you may want to bring in others who can add understanding and help as you: -Clarify the current knowledge of the process – which is what we’ll do today – And you’ll work to understand process variation and select your improvement project when you go home today.

Self-Assessment Survey Assessed Knowledge, Attitudes and Practices (KAP) about CQI and Professional Development: Role of CQI within Organization Use of CQI Strategies and Techniques Involvement in CQI Use of CQI Techniques to examine Enrollment/Engagement Desire to improve Enrollment/Engagement Institutional supports in place to support a focus on Enrollment/Engagement Understanding Capacity What was the basis for these categories?

Self-Assessment Survey Conducted in November 2013 Purpose: to capture what is known and unknown about the training process and the use of Continuous Quality Improvement (CQI) principles in five component areas: professional development training quality of professional development role of continuous quality improvement in professional development activities evaluation activities data collection and analysis 21 responses received

Summary of Survey Results

Summary of Survey Results Section 1: Professional Development Training Members of the ECMHAB are involved in early childhood mental health initiatives, supportive of offering cross-disciplinary training and knowledgeable about the populations they serve: 57% report having a full understanding of who needs PD 76% work with other programs to enroll individuals in PD All agreed that ECMH trainings should be offered across disciplines

Summary of Survey Results Section 1: Professional Development Training Less is known about trainings themselves 1/3 of ECMHAB members don’t know if the number of trainings meet the need, if trainings are well attended, if registration is easy to understand and complete, or if cost is a barrier 76% of respondents disagreed that current communication methods adequately reach all individuals needing PD

Summary of Survey Results Section 1: Professional Development Training There may be regional or specialty-specific deficits: >half of respondents reported that the number of trainings is not sufficient to meet the need in their region or specialty. Nearly half of respondents reported that participants do not know where to register and find training opportunities.

Summary of Survey Results Section 2: Quality of Professional Development Feedback from survey was generally positive regarding the quality of current PD efforts. Exceptions were: Nearly ¼ of respondents disagreed (1) trainers are able to accommodate differing literacy levels and language barriers (2) early childhood programs are implementing evidence-based programs and practices

Summary of Survey Results Section 2: Quality of Professional Development A significant number of individuals reported “Don’t Know” for several indicators of quality centering around two themes: The demographics and values of the training participants Their belief in evidence-based practices The quality of the trainers themselves Accommodating ethnic and cultural needs and differing literacy levels and language barriers

Summary of Survey Results Section 3: Role of CQI in PD Activities Members of the ECMHAB are knowledgeable about both the DSS and ECCS project, reporting familiarity (95%) and confidence (80%) in both logic models. 50/50 split between those with familiarity and experience in both CQI and PDSA and those without 62% report needing training on CQI

Summary of Survey Results Section 4: Evaluation Activities Most respondents are aware of the ongoing evaluation activities included in both the DSS and ECCS projects. There is a significant percentage who report not knowing about evaluation activities: Formally measuring participant knowledge and understanding after training (38%) Evaluation plans (19%) Collect quantitative and qualitative data to measure progress (24%) Interesting to note that while only 24% didn’t know that both quantitative and qualitative data are collected, on average at least 50% of respondents reported not knowing whether or not specific outcomes were measured following each training.

Survey of Summary Results Section 5: Data Collection and Analysis Members are aware of the opportunity they have to provide input into quality improvement measures. A lack of awareness regarding what outcome data is available and collected for PD.

Next Steps Provide ongoing training and education: CQI PDSA cycle Specific information and updates on the data collected under the evaluation plan PD qualifications for trainers 2. Vote on priority area to develop a CQI plan based on the self- assessment survey and Advisory Board feedback

Focus for 2014 CQI Plan Professional Development Training process Qualifications of trainers Demographics and values of training attendees (Who is attending?) Types of data collected for each training

Focus for 2014 CQI Plan What does the Board feel needs to be the focus Suggestions follow based on the survey

Focus for 2014 CQI Plan 2. Quality Cultural competence Evidence based trainings Stakeholder input

Focus for 2014 CQI Plan 3. Evaluation Review of process and outcome measures Monitoring of any factors that inhibit project implementation

Focus for 2014 CQI Plan 4. Performance Measure/ Outcomes - Review and monitoring of data collected following each training

Focus for 2014 CQI Plan Investigate areas potentially in need of improvement: Effectively reaching all individuals in need of PD Communication methods and networks Ensuring participants know where to register and how to find training opportunities Understanding if, where, and/or what type of gaps exist in training