CARF Canada Performance Measurement Outcomes Quality Assurance, Quality Improvement, and Performance Measurement

CARF Canada Outcomes - Day One Planning Session Learning Objectives This session is intended to clarify the differences between Continuous Quality Improvement (CQI) Quality Assurance (QA) and place both in the context of Performance Measurement. The learning objectives are: Have a clear understanding of the differences in the concepts Be able to utilize the concepts to strengthen your efforts Review learning objectives. Review any housekeeping items, such as when breaks will occur, where bathrooms are, parking, etc.

What NOT to do…

Let’s start with definitions….

Continuous Quality Improvement A continual improvement process is an ongoing effort to improve products, services, or processes. These efforts primarily seek "incremental" improvement over time. Delivery (customer valued) services are constantly evaluated and improved through ongoing cycles of data gathering, review, and planning.

Quality Assurance A program for the systematic monitoring and evaluation of the various aspects of a project, service, or process to ensure that standards of quality are being met

So… any clearer? Let’s look deeper…

What is Quality Assurance? Quality Assurance (QA) is a process driven approach… it is targeted towards monitoring and improving existing processes. Includes the process of verifying whether the product or service meets or exceeds the customer expectations (i.e., how the service is delivered) It ensures that the product or services are developed or implemented based on agreed standards (e.g., accreditation). It prevents defects/errors.

Quality Assurance Auditing Compliance with Standards is a common approach to quality assurance in human services. This includes; Accreditation Standards Licensing Standards Funding Body Standards/Policies

Quality Assurance Internal Quality Assurance Systems include; File Audits Tracking, review and follow-up of incidents/events Client and Stakeholder Feedback Mechanisms that focus on the ‘how it was done’ aspect of services Others?

CQI is about applying a learning cycle… CARF Canada Outcomes - Day One Planning Session CQI is about applying a learning cycle… Deming is acknowledged as a pioneer in this area. Developed the Plan-Do-Check-Act cycle Many other versions have been developed and adapted since… Referenced from CARF’s “Managing Outcomes” monograph – Employment & Community Services version. Briefly review the five components.

Plan-Do-Study-Act

Plan-Do-Check-Reflect-Act CARF Canada Outcomes - Day One Planning Session Plan-Do-Check-Reflect-Act Review Data Collection, gather consumer input, design and Improve if needed Report findings to Stakeholders, Develop/Monitor QI Workplan, Improve Services Design Outcome Plan, Logic Models, Data Collection ACT DO PLAN REFLECT CHECK Consider Implications for Quality Improvement and Create Workplan Collect Data Review each of the steps in the Quality Improvement Cycle, starting with “Plan”. Explain that the “Planning” phase is the focus of today’s workshop. The cycle provides a context within which to place Logic Models. Creating and implementing Logic Models is one part of an overall evaluative process. Data collection can be done using electronic means, such as HOMES, or through other electronic or paper processes. Checking involves producing reports of the data you have collected. At this point, you are simply looking at accuracy and completeness. Reflection is the process of analyzing the data and considering implications for Quality Improvement. In this step, you look for trends and relate the data back to the client group served and type of service delivered. Is this what you expected to see? Does the data reflect the everyday experiences of delivering the program? Once the data has been analyzed, consider the implications for quality improvement. Acting includes completing a Performance Analysis, developing a performance plan (Section One, Standard C 5) and reporting findings to key stakeholders (Section One, Standard C 7). The performance plan should be goal oriented, practical, and should include specific actions to be taken and plans for monitoring and reviewing progress. These phases of the cycle will be covered in greater detail on Day Three of the workshop. The Quality Improvement implications during the first cycle often relate to fine tuning the data collection process (measurement plan or measurement grid) and ensuring accuracy and completeness (checking phase). Stakeholders should be involved in as many aspects of the is process as possible. Data Analysis Clean up data set-check for duplicates, fidelity, completeness

CQI Principles An approach emphasizing; The person doing the work as the expert “Us” –vs.- “Them” as outmoded and counterproductive thinking Leadership partnering with all employees to empower them to make decisions that can improve quality

Systems Thinking Seeing Cause and effect Interrelationships Implications

Systems Principles & Thinking A Contemporary Systems View

Summing it up… Quality Assurance focuses more on how services are delivered regardless of outcome or benefit. It can include consumer feedback on the client’s experience of services as well as checks and balances to ensure minimum standards are met (licensing, audits, accreditation).

Summing it up… CQI is specifically focused on the improvement piece… incremental learning based on measurement or system feedback (i.e., a focus on what the results are and how they can be improved) A macro systems focused (sometimes described as a meta-process) Utilizes logic models and process charts Tends to place highest value on outcomes performance (client benefit) A democratic/empowerment value base

So… where does Performance Measurement (and CARF) fit???

Performance Measurement Performance measurement is the process of collecting, analyzing and/or reporting information regarding the actual performance of an individual, group, organization, system or component. It can involve measuring processes (i.e., QA) within organizations, or looking at end results, to see whether outputs are in line with what was intended or should have been achieved.

Overview of Performance Measurement Define specific objectives/indicators/measures of performance across multiple domains for a program/service/project Set Specific Targets/Benchmarks Measure baseline conditions Measure and compare results to targets/benchmarks New performance measures or targets/benchmarks are generated to improve decision making (i.e., applying a continuous learning cycle)

Performance Measures Must Be Clearly defined Easily understood Directly related to stated objectives Relevant to decision makers and stakeholders

Requirements for Performance Measures Stakeholders must help develop performance measures The “role” of performance measures must be understood by participants Performance measures must facilitate decision making

The Bottom Line QA is about setting up systems to ensure things happened the way they were supposed to CQI is about using data about both processes and outcomes to learn and improve Performance Measurement is about how you gather data on actual performance for both… and this is CARF’s focus.

Wrap-up Questions?