MOVING TO AN EVIDENCED- BASED PRACTICE Using Measurement to Support Continuous Quality Improvement

HOMEWORK FOR COCS / AGENCIES  Review the CQI Basic Training.  Assign staff, board, others to your CQI Committee.  Assign a CQI Coordinator.  Determine a meeting time.  Begin meeting and keep meeting. 2

THE CQI PROCESS OVERVIEW  What kind of program are you reviewing?  Identify your core processes.  What are you required outcomes?  Prioritize concerns/opportunities for re-design.  Engage in problem-solving.  Test to see if what you did worked.  Continue problem-solving as needed. 3

THE MEASUREMENT TOOL  To help Agencies think about measures, Michigan’s CTEH PIMIT Committee has provided a “Tool” to help you think through some measures related to both program type and key process.  Store the Tool for use with your Committee. 4

PROGRAM TYPES  Prevention (Housing Navigation)  Rapid re-housing (Housing Navigation)  Housing case management  Permanent supportive housing  Permanent housing (without supports)  Outreach  Overnight shelter/warming shelter  Ongoing shelter (up to 3 months)  Transitional housing  Youth outreach/shelter/transitional housing 5

6 WHAT ARE YOUR CORE PROCESSES? (Steps to delivering your services)  Referral  Outreach  Case management  Knowledge/linkages to community resources  Staff training/orientation  Facilitate Landlord/Tenant relationships  Facility management  Safety management (verbal, emotional, physical, environmental, or food safety)  Records management  Financial assistance process/transmittal process  Data entry and reports (HMIS)

PRIORITIZING PROCESSES FOR STUDY 1.Which of these is high risk, high volume, or problem prone? 2.Staff have identified an idea to improve a process. 3.Auditors have identified a concern. 4.Process is core to success and required routine measurement.

WHERE ARE YOU HEADED?  Learn about using the “Tools for Change” to support to CQI Process.  Determine required outcomes  Brainstorm ideas for other measures  Use the Tool to create measures for outcomes and key processes  Evaluate performance  Use Analytical Tools to support re-design 8

TOOLS FOR CHANGE  Structured minutes  Meeting Minutes Form  Staff Input Form  CQI Monitor Form  Understanding the Issue / Supporting the Search for Solutions:  Fishbone diagram  Process flow chart 9

TOOLS FOR CHANGE: Structured Minutes  Creates a living history of problem solving  Reduces the impact of crisis management  Improves time management for the meeting  Structured minutes have the following characteristics:  Use a consistent agenda each month, same core topic areas.  Track each process re-design using a CQI Monitor Form  Document the new suggestions that are presented, reviewed and prioritized at each meeting. Suggestions selected for active problem- solving are recorded with an initial schedule of activities.  As a group, the committee decides when to close any problem solving processes.  When the redesign is completed or dropped, the CQI form goes into a notebook that archives the history of problem solving. 10

OPEN SAMPLE MINUTES FORM OPEN STAFF INPUT FORM OPEN CQI MONITOR FORM Please Refer to Sample Forms

EXAMPLE OF STRUCTURED MEETING FLOW A.Announcements  Participant announcements B.New opportunities/suggestions  CQI Coordinator reviews any new opportunities to improve services (Staff Input, Audit Findings, Chart Review, etc.)  The Committee prioritizes suggestion/opportunities  The CQI Coordinator begins a CQI form for each process prioritized for review/redesign/test C.Ongoing re-design topics (Processes active review/ re-design/testing cycle)  CQI Coordinator briefly reviews those processes currently open  The CQI Coordinator documents discussion on the CQI form 12

EXAMPLE OF STRUCTURED MEETING FLOW (continued) D.Core performance measure review (according to a defined schedule).  CQI Coordinator brings data reports for scheduled review of selected outcomes. Usually not all outcomes are reviewed in a single meeting. If performance on an outcome is prioritized for problem-solving, the improvement process is documented on the CQI Form and presented under topic “C.” E.Assignments related to all of the above. 13

CQI MONITOR FORM ELEMENTS  Description of problem/opportunity  Review of schedule  Initial analysis findings, using a structure like:  Fishbone/Flowchart/Data Report  Summary of plan  Specifics of review, such as:  Dates, discussion, and findings  Date of closing  Status 14

TOOLS FOR CHANGE: FISHBONE ResourcesStaffConsumersProcedures Quarterly Case Management Bad Data Entry Managing Scheduling Large Case Loads HMIS Consumer Contracts Partnering AgenciesRe-certification Requirements Partnering Agencies Expectations No-Shows Locating the Consumer Example: HPRP fails to provide quarterly contact.

TOOLS FOR CHANGE: Process Flow Chart 16 THE GOODWILL INN Guest HSWCase Manager Inquiries Perform pre-intake Guest Eligible? Refer to other agency Perform Intake NO YES Guest Admitted Perform Assessment Complete Self- Sufficiency Matrix Achieved Goals & Objectives Update Case Plan Obtained Housing? Extension Requested? NO NO Exit Guest YES Complete Extension of Stay Staffing Assessment Review Case Activities Extension Approved? NO YES Close Case Plan Complete Self- Sufficiency Matrix

MONITORING PERFORMANCE & ASSESSING CHANGE ( BRAINSTORMING FROM PIMIT) Please See CQI Tool for Measurement Ideas

WHAT MEASURES?  Process measures  Risk management measures  Funder required outcomes  Other Outcome measures 18

SOME PLACES TO LOOK FOR PROCESS MEASURES  Policies and procedures/written procedures for critical processes  Currency of updates  Evidence of compliance  Procedures that do not add value or are redundant, difficult, complex, or inconsistently completed  Client records and other internal communication documents  Chart audits  Accuracy and completeness of medication administration documentation  Successful completion of the Privacy Questionnaire, orientation processes, etc. 19

SOME PLACES TO LOOK FOR PROCESS MEASURES (continued)  Internal and external audits. Audit findings.  Staff/Consumer identified ideas for improvement or identified concerns.  Suggestion box  STAFF INPUT FORM to the CQI Committee.  Patterns identified from staff/consumer/board satisfaction questionnaires or focus groups.  Findings from ongoing measurement of outcomes or processes. 20

EXAMPLES OF PROCESS MEASURES  Consumer satisfaction surveys/focus groups (clients, staff, board, partnering organizations). Questions like:  Rate the degree to which you feel safe in the facility (staff and clients)  Rate your relationship with your housing case manager.  Rate the probability that you will be able to pay your rent on time.  Rate how safe you feel in your housing/neighborhood. 21

EXAMPLES OF PROCESS MEASURES (continued)  HMIS data quality and other reports  Percentage of “Engagement”: Do staff have a relationship with the client that will support a controlled discharge process.  Data quality reports: Percentage of missing values or percentage of incongruent responses.  Case management: Percentage of completed referrals.  Counting reports: Total clients served, or total clients that participate in a particular service, or percentage of admissions who presented with a targeted characteristic. 22

EXAMPLES OF PROCESS MEASURES (continued)  Critical incidents (unexpected events that happen in your program)  Tracking L&I claims, staff injury  Tracking safety complaints by staff or consumers  Rent arrearage notices from landlords  Problems with transportation  No shows for home or office visits 23

EXAMPLES OF RISK MANAGEMENT MEASURES  Existence/Adequacy of program policies for critical processes  Fire evacuation  Response to illness and medical emergency  Physical safety (facility or interaction)  Home visit protocols  HQS inspections  Response to requests for information/subpoenas 24

EXAMPLES OF RISK MANAGEMENT MEASURES (continued)  Accuracy and completeness of client files including:  Homeless Certification  Income Verifications  “But For” Documentation (HPRP/ESG)  Medication administration  Identifying/Contact Information  Documentation of need, recommendation, and responses  Transportation incidents  Insurance coverage

FUNDER / GRANT REQUIRED OUTCOMES (Targeted changes made by the service consumer)  HMIS required performance measures:  Moving to stable housing at exit (Discharge Destination Report)  Retention in permanent housing (LOS Report)  Reduction of time in shelter (LOS Report)  Increased employment/income at exit (income/employment report)  Improved self-sufficiency (Self-Sufficiency Matrix Report)  Re-admissions to shelter 26

FUNDER / GRANT REQUIRED OUTCOMES (continued) (Targeted changes made by the service consumer)  Other outcomes identified by management  Consumer satisfaction such as the programs overall average satisfaction rating (coming soon on HMIS)  Critical incidents such as reduction in “no shows” (coming soon on HMIS)  Performance on skills/knowledge tests, such as demonstrated ability to prepare meals or passing a test on food preparation standards 27

EXAMPLES OF OUTCOME MEASURES  Outcome performance with benchmarking (HMIS Standardized Outcomes)  XX% of persons exiting to a positive destination (Discharge Destination Outcome Report)  XX% of persons in PSH who retain housing for at least 1 year (LOS Outcome Report)  The overall average length of stay in shelter with a decline to not more than XX days (LOS Outcome Report) for clients with a positive exit  XX% will be employed at exit (Income and Employment Outcome)  XX% will increase income (Income and Employment Outcome) 28

EXAMPLES OF OUTCOME MEASURES (continued)  Consumer satisfaction surveys/focus groups (clients, staff, board, partnering organizations)  XX% rating overall program experience as good or very good  XX% indicating their knowledge of community services increased  XX% rating satisfaction with their housing as good or very good  Critical incidents (unexpected events that happen in your program/community)  XX% will successfully complete referrals to needed resources  XX% decrease in “no shows” for housing case management 29

UNDERSTANDING PERFORMANCE RATES  Agencies will establish target percentages based on historical performance and by comparing performance to other “like” programs.  To establish “standards of practice” (define what is strong performance) and to share best practices, agencies will be asked to join benchmarking groups composed of comparable programs (program type, consumer characteristics, location/service environment).  MCAH will support this process for agencies not engaged in a CoC or regional evaluation process. 30

UNDERSTANDING PERFORMANCE RATES (continued) AN EXAMPLE Based on historical performance, consumers generally exit to a “positive housing destination” between 40% and 50% of the time. The overall percentage for family shelters in the region is 70%. The agency identifies a need to improve this rate and meets with other members of the comparison group to discuss common practices and ideas to improve performance. The shelter sets a target to reach 60% in 6 months and 70% by the end of the year. Using “Tools for Change” they carefully review existing procedures and develop a plan for change. They continue to measure to determine if their performance is improving and continue to refine their processes until performance is optimized. 31

REVIEW OF REQUIRED OUTCOMES Interpreting the Live Reports from the HMIS

QUESTIONS If you have any questions about the CQI process please contact: Barb Ritter (MCAH) or Chuck Steinberg (MCAH) Or Gerry Leslie (Flint – Metro Housing Partnership) 33