The Basics of Performance Measurement for Quality Improvement Nancy Showers, DSW 888-NQC-QI-TA NationalQualityCenter.org

Linking Performance Measurement and Quality Improvement Infrastructure

How to Go in Circles CQI Change CQI Measure

Trends in QM From monitoring (QA) to improvement projects (QM) From QA by administrators to QM by teams From core medical indicators to expanded scope of process indicators From 100% goals to goals by benchmarking From data by hand to data by computer From process to outcome indicators Accountability to/ inclusion of consumers From program to regional QM

Basics of Performance Measurement Why measure? What to measure? When to measure? How to measure? Strategic planning for measurement

Why Measure?

Reasons to Measure Separates what you think is happening from what really is happening Establishes a baseline: It’s ok to start out with low scores! Determines whether changes actually lead to improvements Avoids slippage

Reasons to Measure (cont.) Ongoing / periodic monitoring identifies problems as they emerge Measurement allows for comparison across sites, programs, EMAs, TGAs and states The Ryan White Treatment Modernization Act of 2006 mandates performance measurement The HIV/AIDS Bureau places strong emphasis on quality management

What to Measure

What is a Quality Indicator? An indicator is a surrogate for direct measurement of quality Quality cannot be measured directly An indicator is a measure thought to contribute to or reflect quality

Process Indicator Topic Areas Medical processes Case management processes Clinic / agency / EMA / state processes Patient utilization of care underutilization overutilization misutilization State, EMA,TGA common processes Coordination of care processes

Outcome Topics Patient Health Status Patient Satisfaction Intermediate outcomes like immune and virological status Survival Symptoms Disease progression Disability Subjective health status Hospital and ER visits Patient Satisfaction

What is a Good Indicator? Relevance-How Important is the Indicator? Does the indicator affect a lot of people or programs? Does the indicator have a great impact on the programs or patients/clients in your EMA, TGA or state? Measurability Can the indicator realistically and efficiently be measured given finite resources?

What is a Good Indicator? (Cont’d) Accuracy Is the indicator based on accepted guidelines or developed through formal group-decision making methods? Improvability Can the performance rate associated with the indicator realistically be improved given the limitations of services and population?

Specify criteria to define your measurement population Location: all sites, or only some? Gender: men, women, or both? Age: any limits? Client conditions: all HIV-infected clients, or only those with a specific diagnosis? Treatment status? To start, you need to define your eligibility criteria for the measurement population. The measurement population consists of those patients who are eligible for measurement based on pre-established criteria. Defining a population requires identifying both which records should be reviewed and which should not. The key point here is to select the focus of your data collection efforts. Consider the following criteria to define your measurement population: - Location: What facilities within the care system will be included? - Gender: Does the indicator apply exclusively to men or women, or to both? - Age: Are there particular age limits? - Patient condition: Is a confirmed diagnosis required, or simply symptoms or signs? Do certain conditions make the patient ineligible? - Active treatment status: How many visits are required for eligibility? Must the patient currently be in treatment? Must the treatment have occurred within a certain time frame? When you are finished addressing these questions, you will have a list of eligibility criteria. Sampling Records

Indicator Definition Tips Base the indicator on guidelines and standards of care when possible Be inclusive (of staff and consumers) when developing an indicator to create ownership Be clear in terms of patient / program characteristics (gender, age, patient condition, provider type, etc.) Set specific time-frames in indicator definitions

How to Measure

Create a Plan Decide on a sampling plan (sample size, eligible records, draw a random sample) Develop data collection tools and instructions Train data abstractors Run pilot test (adjust after a few records) Inform other staff of the measurement process Check for data accuracy Remain available for guidance Make a plan for display and distribution of data

Using a Random Sample Use a random sample if the entire population can’t easily be measured “Random selection” means that each record has an equal chance of being included in the sample. The easiest way to select records randomly is to find a random number table and pull each record in the random sequence. Every record needs an equal chance of being included in the sample. You can’t just pick out the records that you know are ”good.” This is not hard to do, but it takes a little while to explain. For detailed instructions, please use “Measuring Clinical Performance: A Guide for HIV Health Care Providers,” available at the web site given here.

Resources to Randomize the Random Sample “Measuring Clinical Performance: A Guide for HIV Health Care Providers” (includes random number tables) A useful website for the generation of random numbers is www.randomizer.org Common spreadsheet programs, such as MS Excel Random sampling is not hard to do. For detailed instructions, please use “Measuring Clinical Performance: A Guide for HIV Health Care Providers,” available at the website shown on the slide. A useful website for the generation of random numbers is also: www.randomizer.org Most spreadsheet programs, such as Microsoft Excel offer random number tables as well. Sampling Records

Collect “Just enough” Data The goal is to improve care, not prove a new theorem 100% is not needed Maximal power is not needed In most cases, a straightforward sample will do just fine The data you need for quality improvement is not the same as the data that drives a peer-reviewed study of a randomized clinical trial. You don’t need to count every chart. You don’t need a precisely defined sample. Simple sampling techniques work quite well enough.

Strategies Depend on Resources Data systems enhance capability More indicators can be measured Indicators can be measured more often Entire populations can be measured Outcome as well as process indicators can be measured Alerts, custom reports help manage care Personnel resources Person power for chart reviews, logs, other means of measurement is needed Expertise in electronic / manual measurement

Tips for the Electronic Era Strategically plan for the electronic era Decide on patient level vs. aggregate data Decide on common data system vs. electronic submission exported from varying data systems Design and program queries and reports before requiring data submission Don’t defer improvement projects while implementing electronic plans. Don’t expect an electronic system to entirely replace the need for manual systems

When to Measure

Frequency You don’t need to measure everything all of the time. You can sample a short period of time and extrapolate the results Balance the frequency of measurement against the cost in resources If limited resources, measure areas of concern more frequently, others less frequently Balance the frequency of measurement against usefulness in producing change Consider the audience. How will frequency best assist in setting priorities and generating change?

National HIVQUAL Data Reports Show national trends based on self-reported data by participating HIVQUAL grantees Provide an opportunity to compare program performance with national data to highlight areas of opportunity

The HIVQUAL Project 2005 Performance Data Title III and Title IV Programs

Questions for Data Follow-up What are the results for key indicators? What are the major findings based on the generated data reports and your data analysis? What is the frequency of patients / programs not getting care? What is the impact of not getting the care? How does the performance compare with benchmark data? What is the feasibility of improving the care?

Key Questions for Data Follow Up (Cont’d) How can you best share the data results with your key stakeholders (Part A/B QI committees, HIV providers, consumers, etc.)? How do you generate ownership among providers and consumers? How will you assist in initiating/implementing QI projects to address the data findings? Who will be responsible and what are the next steps?

On Our Way to… CQI Heaven Measure Change CQI