1. CARF Canada Performance Measurement Outcomes
Quality Assurance,
Quality Improvement, and Performance Measurement

2. CARF Canada Outcomes - Day One Planning Session
Learning Objectives
This session is intended to clarify the differences between Continuous Quality Improvement (CQI) Quality Assurance (QA) and place both in the context of Performance Measurement. The learning objectives are:
Have a clear understanding of the differences in the concepts
Be able to utilize the concepts to strengthen your efforts
Review learning objectives.
Review any housekeeping items, such as when breaks will occur, where bathrooms are, parking, etc.

3. What NOT to do…

4. Let’s start with definitions….

5. Continuous Quality Improvement
A continual improvement process is an ongoing effort to improve products, services, or processes. These efforts primarily seek "incremental" improvement over time. Delivery (customer valued) services are constantly evaluated and improved through ongoing cycles of data gathering, review, and planning.

6. Quality Assurance
A program for the systematic monitoring and evaluation of the various aspects of a project, service, or process to ensure that standards of quality are being met

7. So… any clearer? Let’s look deeper…

8. What is Quality Assurance?
Quality Assurance (QA) is a process driven approach… it is targeted towards monitoring and improving existing processes.
Includes the process of verifying whether the product or service meets or exceeds the customer expectations (i.e., how the service is delivered)
It ensures that the product or services are developed or implemented based on agreed standards (e.g., accreditation).
It prevents defects/errors.

9. Quality Assurance
Auditing Compliance with Standards is a common approach to quality assurance in human services. This includes;
Accreditation Standards
Licensing Standards
Funding Body Standards/Policies

10. Quality Assurance Internal Quality Assurance Systems include;
File Audits
Tracking, review and follow-up of incidents/events
Client and Stakeholder Feedback Mechanisms that focus on the ‘how it was done’ aspect of services
Others?

11. CQI is about applying a learning cycle…
CARF Canada Outcomes - Day One Planning Session
CQI is about applying a learning cycle…
Deming is acknowledged as a pioneer in this area.
Developed the Plan-Do-Check-Act cycle
Many other versions have been developed and adapted since…
Referenced from CARF’s “Managing Outcomes” monograph – Employment & Community Services version.
Briefly review the five components.

12. Plan-Do-Study-Act

13. Plan-Do-Check-Reflect-Act
CARF Canada Outcomes - Day One Planning Session
Plan-Do-Check-Reflect-Act
Review Data Collection, gather consumer input, design and Improve if needed
Report findings to Stakeholders, Develop/Monitor QI Workplan, Improve Services
Design Outcome Plan, Logic Models, Data Collection
ACT DO
PLAN
REFLECT
CHECK
Consider Implications for Quality Improvement and Create Workplan
Collect Data
Review each of the steps in the Quality Improvement Cycle, starting with “Plan”.
Explain that the “Planning” phase is the focus of today’s workshop.
The cycle provides a context within which to place Logic Models. Creating and implementing Logic Models is one part of an overall evaluative process.
Data collection can be done using electronic means, such as HOMES, or through other electronic or paper processes.
Checking involves producing reports of the data you have collected. At this point, you are simply looking at accuracy and completeness.
Reflection is the process of analyzing the data and considering implications for Quality Improvement. In this step, you look for trends and relate the data back to the client group served and type of service delivered. Is this what you expected to see? Does the data reflect the everyday experiences of delivering the program? Once the data has been analyzed, consider the implications for quality improvement.
Acting includes completing a Performance Analysis, developing a performance plan (Section One, Standard C 5) and reporting findings to key stakeholders (Section One, Standard C 7). The performance plan should be goal oriented, practical, and should include specific actions to be taken and plans for monitoring and reviewing progress. These phases of the cycle will be covered in greater detail on Day Three of the workshop.
The Quality Improvement implications during the first cycle often relate to fine tuning the data collection process (measurement plan or measurement grid) and ensuring accuracy and completeness (checking phase).
Stakeholders should be involved in as many aspects of the is process as possible.
Data Analysis
Clean up data set-check for duplicates, fidelity, completeness

14. CQI Principles An approach emphasizing;
The person doing the work as the expert
“Us” –vs.- “Them” as outmoded and counterproductive thinking
Leadership partnering with all employees to empower them to make decisions that can improve quality

15. Systems Thinking Seeing Cause and effect Interrelationships
Implications

16. Systems Principles & Thinking
A Contemporary Systems View

17. Summing it up…
Quality Assurance focuses more on how services are delivered regardless of outcome or benefit.
It can include consumer feedback on the client’s experience of services as well as checks and balances to ensure minimum standards are met (licensing, audits, accreditation).

18. Summing it up…
CQI is specifically focused on the improvement piece… incremental learning based on measurement or system feedback (i.e., a focus on what the results are and how they can be improved)
A macro systems focused (sometimes described as a meta-process)
Utilizes logic models and process charts
Tends to place highest value on outcomes performance (client benefit)
A democratic/empowerment value base

19. So… where does Performance Measurement (and CARF) fit???

20. Performance Measurement
Performance measurement is the process of collecting, analyzing and/or reporting information regarding the actual performance of an individual, group, organization, system or component. It can involve measuring processes (i.e., QA) within organizations, or looking at end results, to see whether outputs are in line with what was intended or should have been achieved.

21. Overview of Performance Measurement
Define specific objectives/indicators/measures of performance across multiple domains for a program/service/project
Set Specific Targets/Benchmarks
Measure baseline conditions
Measure and compare results to targets/benchmarks
New performance measures or targets/benchmarks are generated to improve decision making (i.e., applying a continuous learning cycle)

22. Performance Measures Must Be
Clearly defined
Easily understood
Directly related to stated objectives
Relevant to decision makers and stakeholders

23. Requirements for Performance Measures
Stakeholders must help develop performance measures
The “role” of performance measures must be understood by participants
Performance measures must facilitate decision making

24. The Bottom Line
QA is about setting up systems to ensure things happened the way they were supposed to
CQI is about using data about both processes and outcomes to learn and improve
Performance Measurement is about how you gather data on actual performance for both… and this is CARF’s focus.

25. Wrap-up
Questions?