Funded by HRSA HIV/AIDS Bureau The Basics of Performance Measurement for Quality Improvement Nancy Showers, DSW 888-NQC-QI-TA NationalQualityCenter.org

2 National Quality Center (NQC) Infrastructure Linking Performance Measurement and Quality Improvement

3 National Quality Center (NQC) How to Go in Circles Measure CQI Change CQI

4 National Quality Center (NQC) Trends in QM From monitoring (QA) to improvement projects (QM) From QA by administrators to QM by teams From core medical indicators to expanded scope of process indicators From 100% goals to goals by benchmarking From data by hand to data by computer From process to outcome indicators Accountability to/ inclusion of consumers From program to regional QM

5 National Quality Center (NQC) Basics of Performance Measurement Why measure? What to measure? When to measure? How to measure? Strategic planning for measurement

6 National Quality Center (NQC) Reasons to Measure Separates what you think is happening from what really is happening Establishes a baseline: It’s ok to start out with low scores! Determines whether changes actually lead to improvements Avoids slippage Ongoing / periodic monitoring identifies problems as they emerge

7 National Quality Center (NQC) Reasons to Measure (cont.) Measurement allows for comparison across sites, programs, and networks The Ryan White Treatment Modernization Act of 2006 mandates performance measurement The HIV/AIDS Bureau places strong emphasis on quality management  The lead agency is responsible for assuring that QI systems are established at subcontracting agencies and can require regular reporting.

8 National Quality Center (NQC) What is a Quality Indicator? An indicator is a surrogate for direct measurement of quality Quality cannot be measured directly An indicator is a measure thought to contribute to or reflect quality

9 National Quality Center (NQC) Process Indicator Topic Areas Medical processes Case management processes Clinic / cross clinic processes Patient utilization of care  Underutilization  Overutilization  Misutilization Coordination of care processes

10 National Quality Center (NQC) Outcome Topics Patient Health Status Intermediate outcomes like immune and virological status Survival Symptoms Disease progression Disability Subjective health status Hospital and ER visits Patient Satisfaction

11 National Quality Center (NQC) Network Indicator Priorities (Part D) Common medical indicators Population specific medical indicators Sub-contractor specific medical indicators Support of medical care indicators Network spanning indicators Data system indicators

12 National Quality Center (NQC) What is a Good Indicator? Importance-  Does the indicator affect a lot of people or programs?  Does the indicator have a great impact on the programs or patients/clients in your program Measurability  Numerator / Denominator=Performance  Can the indicator realistically and efficiently be measured given finite resources?

13 National Quality Center (NQC) What is a Good Indicator? (Cont’d) Accuracy  Is the indicator either based on accepted guideline or developed through formal group-decision making methods? Improvability  Is this indicator within our control?  Can the performance rate realistically be improved given the limitations of services and population?

14 National Quality Center (NQC) What is a Good Indicator? (Cont’d) Specificity  Does the indicator specify exactly which patients / locations / time frames are included? Location: all sites, or only some? Patient characteristics-age, gender, diagnosis, treatment status, etc. Hours, days, months, years Sampling Records

15 National Quality Center (NQC) Indicator Definition Tips 1.Base the indicator on guidelines and standards of care when possible 2.Be inclusive (of staff and consumers) when developing an indicator to create ownership 3.Be clear in terms of patient / program characteristics (gender, age, patient condition, provider type, etc.) 4.Set specific time-frames in indicator definitions

16 National Quality Center (NQC) Create a Plan Decide on a sampling plan (sample size, eligible records, draw a random sample) Develop data collection tools and instructions Train data abstractors Run pilot test (adjust after a few records) Inform other staff of the measurement process Check for data accuracy Remain available for guidance Make a plan for display and distribution of data

17 National Quality Center (NQC) Using a Random Sample Use a random sample if the entire population can’t easily be measured “Random selection” means that each record has an equal chance of being included in the sample. The easiest way to select records randomly is to find a random number table and pull each record in the random sequence.

18 National Quality Center (NQC) Resources to Randomize the Random Sample “Measuring Clinical Performance: A Guide for HIV Health Care Providers” (includes random number tables) A useful website for the generation of random numbers is Common spreadsheet programs, such as MS Excel Sampling Records

19 National Quality Center (NQC) Collect “Just enough” Data The goal is to improve care, not prove a new theorem 100% is not needed Maximal power is not needed In most cases, a straightforward sample will do just fine

20 National Quality Center (NQC) Strategies Depend on Resources Data systems enhance capability  More indicators can be measured  Indicators can be measured more often  Entire populations can be measured  Outcome as well as process indicators can be measured  Alerts, custom reports help manage care Personnel resources  Person power for chart reviews, logs, other means of measurement is needed  Expertise in electronic / manual measurement

21 National Quality Center (NQC) Tips for the Electronic Era Strategically plan for the electronic era  Decide on patient level vs. aggregate data  Decide on common data system vs. electronic submission exported from varying data systems  Design and program queries and reports before requiring data submission Don’t defer improvement projects while implementing electronic plans. Don’t expect an electronic system to entirely replace the need for manual systems

22 National Quality Center (NQC) Frequency You don’t need to measure everything all of the time. You can sample a short period of time and extrapolate the results Balance the frequency of measurement against the cost in resources If limited resources, measure areas of concern more frequently, others less frequently Balance the frequency of measurement against usefulness in producing change Consider the audience. How will frequency best assist in setting priorities and generating change?

23 National Quality Center (NQC) National HIVQUAL Data Reports Show national trends based on self-reported data by participating HIVQUAL grantees Provide an opportunity to compare program performance with national data to highlight areas of opportunity

24 National Quality Center (NQC) The HIVQUAL Project 2005 Performance Data Title III and Title IV Programs

25 National Quality Center (NQC) Questions for Data Follow-up What are the results for key indicators? What are the major findings based on the generated data reports and your data analysis?  What is the frequency of patients / programs not getting care?  What is the impact of not getting the care?  How does the performance compare with benchmark data?  What is the feasibility of improving the care?

26 National Quality Center (NQC) Key Questions for Data Follow Up (Cont’d) How can you best share the data results with your key stakeholders (QI committees, HIV staff, consumers, etc.)? How do you generate ownership among providers and consumers? How will you assist in initiating/implementing QI projects to address the data findings? Who will be responsible and what are the next steps?

27 National Quality Center (NQC) CQI Heaven Measure Change CQI On Our Way to …