1. The Basics of Performance Measurement for Quality Improvement Nancy Showers, DSW
888-NQC-QI-TA
NationalQualityCenter.org

2. Linking Performance Measurement and Quality Improvement
Infrastructure

3. How to Go in Circles
CQI
Change
CQI
Measure

4. Trends in QM From monitoring (QA) to improvement projects (QM)
From QA by administrators to QM by teams
From core medical indicators to expanded scope of process indicators
From 100% goals to goals by benchmarking
From data by hand to data by computer
From process to outcome indicators
Accountability to/ inclusion of consumers
From program to regional QM

5. Basics of Performance Measurement
Why measure?
What to measure?
When to measure?
How to measure?
Strategic planning for measurement

6. Why Measure?

7. Reasons to Measure
Separates what you think is happening from what really is happening
Establishes a baseline: It’s ok to start out with low scores!
Determines whether changes actually lead to improvements
Avoids slippage

8. Reasons to Measure (cont.)
Ongoing / periodic monitoring identifies problems as they emerge
Measurement allows for comparison across sites, programs, EMAs, TGAs and states
The Ryan White Treatment Modernization Act of 2006 mandates performance measurement
The HIV/AIDS Bureau places strong emphasis on quality management

9. What to Measure

10. What is a Quality Indicator?
An indicator is a surrogate for direct measurement of quality
Quality cannot be measured directly
An indicator is a measure thought to contribute to or reflect quality

11. Process Indicator Topic Areas
Medical processes
Case management processes
Clinic / agency / EMA / state processes
Patient utilization of care underutilization overutilization
misutilization
State, EMA,TGA common processes
Coordination of care processes

12. Outcome Topics Patient Health Status Patient Satisfaction
Intermediate outcomes like immune and virological status
Survival
Symptoms
Disease progression
Disability
Subjective health status
Hospital and ER visits
Patient Satisfaction

13. What is a Good Indicator?
Relevance-How Important is the Indicator?
Does the indicator affect a lot of people or programs?
Does the indicator have a great impact on the programs or patients/clients in your EMA, TGA or state?
Measurability
Can the indicator realistically and efficiently be measured given finite resources?

14. What is a Good Indicator? (Cont’d)
Accuracy
Is the indicator based on accepted guidelines or developed through formal group-decision making methods?
Improvability
Can the performance rate associated with the indicator realistically be improved given the limitations of services and population?

15. Specify criteria to define your measurement population
Location: all sites, or only some?
Gender: men, women, or both?
Age: any limits?
Client conditions: all HIV-infected clients, or only those with a specific diagnosis?
Treatment status?
To start, you need to define your eligibility criteria for the measurement population.
The measurement population consists of those patients who are eligible for measurement based on pre-established criteria. Defining a population requires identifying both which records should be reviewed and which should not. The key point here is to select the focus of your data collection efforts.
Consider the following criteria to define your measurement population:
- Location: What facilities within the care system will be included?
- Gender: Does the indicator apply exclusively to men or women, or to both?
- Age: Are there particular age limits?
- Patient condition: Is a confirmed diagnosis required, or simply symptoms or signs? Do certain conditions make the patient ineligible?
- Active treatment status: How many visits are required for eligibility? Must the patient currently be in treatment? Must the treatment have occurred within a certain time frame?
When you are finished addressing these questions, you will have a list of eligibility criteria.
Sampling Records

16. Indicator Definition Tips
Base the indicator on guidelines and standards of care when possible
Be inclusive (of staff and consumers) when developing an indicator to create ownership
Be clear in terms of patient / program characteristics (gender, age, patient condition, provider type, etc.)
Set specific time-frames in indicator definitions

17. How to Measure

18. Create a Plan
Decide on a sampling plan (sample size, eligible records, draw a random sample)
Develop data collection tools and instructions
Train data abstractors
Run pilot test (adjust after a few records)
Inform other staff of the measurement process
Check for data accuracy
Remain available for guidance
Make a plan for display and distribution of data

19. Using a Random Sample
Use a random sample if the entire population can’t easily be measured
“Random selection” means that each record has an equal chance of being included in the sample.
The easiest way to select records randomly is to find a random number table and pull each record in the random sequence.
Every record needs an equal chance of being included in the sample. You can’t just pick out the records that you know are ”good.”
This is not hard to do, but it takes a little while to explain. For detailed instructions, please use “Measuring Clinical Performance: A Guide for HIV Health Care Providers,” available at the web site given here.

20. Resources to Randomize the Random Sample
“Measuring Clinical Performance: A Guide for HIV Health Care Providers” (includes random number tables)
A useful website for the generation of random numbers is
Common spreadsheet programs, such as MS Excel
Random sampling is not hard to do. For detailed instructions, please use “Measuring Clinical Performance: A Guide for HIV Health Care Providers,” available at the website shown on the slide.
A useful website for the generation of random numbers is also:
Most spreadsheet programs, such as Microsoft Excel offer random number tables as well.
Sampling Records

21. Collect “Just enough” Data
The goal is to improve care, not prove a new theorem
100% is not needed
Maximal power is not needed
In most cases, a straightforward sample will do just fine
The data you need for quality improvement is not the same as the data that drives a peer-reviewed study of a randomized clinical trial.
You don’t need to count every chart.
You don’t need a precisely defined sample. Simple sampling techniques work quite well enough.

22. Strategies Depend on Resources
Data systems enhance capability
More indicators can be measured
Indicators can be measured more often
Entire populations can be measured
Outcome as well as process indicators can be measured
Alerts, custom reports help manage care
Personnel resources
Person power for chart reviews, logs, other means of measurement is needed
Expertise in electronic / manual measurement

23. Tips for the Electronic Era
Strategically plan for the electronic era
Decide on patient level vs. aggregate data
Decide on common data system vs. electronic submission exported from varying data systems
Design and program queries and reports before requiring data submission
Don’t defer improvement projects while implementing electronic plans.
Don’t expect an electronic system to entirely replace the need for manual systems

24. When to Measure

25. Frequency
You don’t need to measure everything all of the time. You can sample a short period of time and extrapolate the results
Balance the frequency of measurement against the cost in resources
If limited resources, measure areas of concern more frequently, others less frequently
Balance the frequency of measurement against usefulness in producing change
Consider the audience. How will frequency best assist in setting priorities and generating change?

26. National HIVQUAL Data Reports
Show national trends based on self-reported data by participating HIVQUAL grantees
Provide an opportunity to compare program performance with national data to highlight areas of opportunity

27. The HIVQUAL Project 2005 Performance Data Title III and Title IV Programs

28. Questions for Data Follow-up
What are the results for key indicators?
What are the major findings based on the generated data reports and your data analysis?
What is the frequency of patients / programs not getting care?
What is the impact of not getting the care?
How does the performance compare with benchmark data?
What is the feasibility of improving the care?

29. Key Questions for Data Follow Up (Cont’d)
How can you best share the data results with your key stakeholders (Part A/B QI committees, HIV providers, consumers, etc.)?
How do you generate ownership among providers and consumers?
How will you assist in initiating/implementing QI projects to address the data findings? Who will be responsible and what are the next steps?

30. On Our Way to…
CQI Heaven
Measure
Change
CQI