1. Continuous Quality Improvement (CQI): Part 2
Bethany Geldmaker PNP, PhD
Tracy Jebo MPH

2. Learning Objectives
Review with Infant Mental Health Advisory Board: Part 1-- Foundational Continuous Quality Improvement
Results of the Self-Assessment Survey results
Vote on priority area to develop a CQI plan based on the self- assessment survey and Advisory Board feedback

3. Review
Performance Management
CQI
PDSA Cycle

4. Performance Management and CQI
Two primary components:
Performance Management: Using data to improve performance
CQI (Continuous Quality Improvement): Involving all stakeholders to examine the performance of a process and address identified gaps
Performance Management and CQI are distinct processes. While some of the components may overlap, there are marked differences as well.

5. Quality Improvement
Uses data on how you are performing to drive activities which lead to true improvement.
Utilizes stakeholders to address identified gaps continually improve processes
Addresses the question: How do we get better?

6. CQI Process: PDSA Cycle
The use of a deliberate and
defined improvement process
Plan-Do-Study-Act Cycle
will be used for this project

7. CQI Process: PDSA Cycle
When applying PDSA there are 3 key questions and 4 QI principles that will guide efforts.
The 3 key questions to keep front and center during QI work are:
What are we trying to accomplish?
How will we know that a change is an improvement?
What changes can we make that will results in improvement?

8. CQI Process: PDSA Cycle
The 4 CQI principles are:
Develop a strong client focus
Continually improve all processes
Involve stakeholders
Mobilize both data and team knowledge to improve decision-making

9. PDSA Cycle: Nine Steps
The PDSA cycle is used to improve existing processes, not to plan or implement new programs
Always relies on data to support its decisions

10. PDSA Cycle: Plan-Do-Study-Act
Plan-Do-Study-Act Cycle (Steps 1-5)
Step 1: Identifying a problem
Step 2: Assembling a Q.I. team that can address the problem
Step 3: Identify what the true cause(s) of the problem may be, map the process to understand the existing activities that lead to the problematic issue
Step 4: Brainstorming and conducting background research to look for possible solutions
Step 5: Working to develop a theory of improvement that can be tested

11. Plan-Do-Study-Act Plan-Do-Study-Act Cycle (Step 6)
Step 6: Test your theory of improvement
implement a small-scale test of the change to your process.
Collect, chart, and display data to determine the effectiveness of the improvement.
Document problems, unexpected observations, and unintended side effects, because these occurrences will aid in the learning process.

12. Plan-Do-Study-Act Plan-Do-Study-Act Cycle (Step 7)
Step 7: Study the Results of your Test
Use data to determine if the test of your theory was successful
Compare the results of your test to the baseline data
Some questions to consider include:
Did your test work? How do you know?
Did the results match your theory/prediction? What do the data show?
Are there trends in your data?
Did you have unintended side effects?
Is there improvement?
Do you need to test the improvement under other conditions?

13. Plan-Do-Study-Act Plan-Do-Study-Act Cycle (Steps 8 and 9)
Step 8: Either standardize your improvement (if test was successful) or develop a new theory for improvement (if test was unsuccessful)
Step 9: Establish Future Plans
Sustaining change or beginning PDSA again

14. Tips for Beginning the Journey
CQI can be confusing, daunting - lots of jargon, different tools, methods, approaches (Google = 2,760,000 results)
Avoid starting with a large-scale project; think small and simple
Choose a QI effort related to a familiar process, program or area
Be patient – change takes time
Unanticipated results are not failures. They are learning opportunities for your next trial.
We will be going through the journey together!
Feelings of uncertainty about how to begin or what steps to take next

15. FOCUS First Find an opportunity to improve.
Organize a team who understands the process.
Clarify the current knowledge of the process. In this stage, you are gathering the "who, what, when, and where“.
Understand the cause of process variation. Here, you ask yourself the "why" question.
Select the piece of the process you want to improve.
This is not technically part of the “model for improvement” but is part of the PDSA cycle sometimes left off or forgotten. I thought it was actually important and relevant as a way to think about how to focus your improvement efforts.
What we have already done in this process:
-Found an opportunity to improve: Family Enrollment & Engagement
-Organzied teams (you guys) – though you may want to bring in others who can add understanding and help as you:
-Clarify the current knowledge of the process – which is what we’ll do today –
And you’ll work to understand process variation and select your improvement project when you go home today.

16. Self-Assessment Survey
Assessed Knowledge, Attitudes and Practices (KAP) about CQI and Professional Development:
Role of CQI within Organization
Use of CQI Strategies and Techniques
Involvement in CQI
Use of CQI Techniques to examine Enrollment/Engagement
Desire to improve Enrollment/Engagement
Institutional supports in place to support a focus on Enrollment/Engagement
Understanding Capacity
What was the basis for these categories?

17. Self-Assessment Survey
Conducted in November 2013
Purpose: to capture what is known and unknown about the training process and the use of Continuous Quality Improvement (CQI) principles in five component areas:
professional development training
quality of professional development
role of continuous quality improvement in professional development activities
evaluation activities
data collection and analysis
21 responses received

18. Summary of Survey Results

19. Summary of Survey Results
Section 1: Professional Development Training
Members of the ECMHAB are involved in early childhood mental health initiatives, supportive of offering cross-disciplinary training and knowledgeable about the populations they serve:
57% report having a full understanding of who needs PD
76% work with other programs to enroll individuals in PD
All agreed that ECMH trainings should be offered across disciplines

20. Summary of Survey Results
Section 1: Professional Development Training
Less is known about trainings themselves
1/3 of ECMHAB members don’t know if the number of trainings meet the need, if trainings are well attended, if registration is easy to understand and complete, or if cost is a barrier
76% of respondents disagreed that current communication methods adequately reach all individuals needing PD

21. Summary of Survey Results
Section 1: Professional Development Training
There may be regional or specialty-specific deficits:
>half of respondents reported that the number of trainings is not sufficient to meet the need in their region or specialty.
Nearly half of respondents reported that participants do not know where to register and find training opportunities.

22. Summary of Survey Results
Section 2: Quality of Professional Development
Feedback from survey was generally positive regarding the quality of current PD efforts.
Exceptions were:
Nearly ¼ of respondents disagreed
(1) trainers are able to accommodate differing literacy levels and language barriers
(2) early childhood programs are implementing evidence-based programs and practices

23. Summary of Survey Results
Section 2: Quality of Professional Development
A significant number of individuals reported “Don’t Know” for several indicators of quality centering around two themes:
The demographics and values of the training participants
Their belief in evidence-based practices
The quality of the trainers themselves
Accommodating ethnic and cultural needs and differing literacy levels and language barriers

24. Summary of Survey Results
Section 3: Role of CQI in PD Activities
Members of the ECMHAB are knowledgeable about both the DSS and ECCS project, reporting familiarity (95%) and confidence (80%) in both logic models.
50/50 split between those with familiarity and experience in both CQI and PDSA and those without
62% report needing training on CQI

25. Summary of Survey Results
Section 4: Evaluation Activities
Most respondents are aware of the ongoing evaluation activities included in both the DSS and ECCS projects.
There is a significant percentage who report not knowing about evaluation activities:
Formally measuring participant knowledge and understanding after training (38%)
Evaluation plans (19%)
Collect quantitative and qualitative data to measure progress (24%)
Interesting to note that while only 24% didn’t know that both quantitative and qualitative data are collected, on average at least 50% of respondents reported not knowing whether or not specific outcomes were measured following each training.

26. Survey of Summary Results
Section 5: Data Collection and Analysis
Members are aware of the opportunity they have to provide input into quality improvement measures.
A lack of awareness regarding what outcome data is available and collected for PD.

27. Next Steps Provide ongoing training and education: CQI PDSA cycle
Specific information and updates on the data collected under the evaluation plan
PD qualifications for trainers
2. Vote on priority area to develop a CQI plan based on the self- assessment survey and Advisory Board feedback

28. Focus for 2014 CQI Plan Professional Development Training process
Qualifications of trainers
Demographics and values of training attendees (Who is attending?)
Types of data collected for each training

29. Focus for 2014 CQI Plan What does the Board feel needs to be the focus
Suggestions follow based on the survey

30. Focus for 2014 CQI Plan 2. Quality Cultural competence
Evidence based trainings
Stakeholder input

31. Focus for 2014 CQI Plan 3. Evaluation
Review of process and outcome measures
Monitoring of any factors that inhibit project implementation

32. Focus for 2014 CQI Plan
4. Performance Measure/ Outcomes - Review and monitoring of data collected following each training

33. Focus for 2014 CQI Plan
Investigate areas potentially in need of improvement:
Effectively reaching all individuals in need of PD
Communication methods and networks
Ensuring participants know where to register and how to find training opportunities
Understanding if, where, and/or what type of gaps exist in training