1. Funded by HRSA HIV/AIDS Bureau Selecting an Indicator & Collecting Performance Data Barbara M Rosa, RN-C, MS

2. National Quality Center (NQC)2 Objectives  Understand the criteria in selecting an indicator for improvement  Develop an indicator with numerator & denominator for a network Part D program or a single site Part D program  Identify data sources and develop a data collection plan &/or manual data collection tool  Understand sampling methodology

3. National Quality Center (NQC)3 Agenda  15 min—Overview of indicator development and data display/analysis  15 min—Individual programs develop an appropriate indicator with numerator & denominator defined.  10 min—Share indicators with group  10 min—Breakout in 2 or 3 groups: networks, single sites, and/or programs with electronic data collection to develop a data collection plan.  10 min—Share plans with group

4. National Quality Center (NQC)4 What Is A Quality Indicator?  A quality indicator is tool to measure specific aspects of care and services that are optimally linked to better health outcomes while being consistent with current professional knowledge and meeting client needs.

5. National Quality Center (NQC)5 What Makes a Good Indicator?  Relevance  Does the indicator affect a lot of people or programs?  Does the indicator have a great impact on the programs or patients/clients in your network or clinic?  Measurability  Can the indicator realistically and efficiently be measured given finite resources?

6. National Quality Center (NQC)6 What Makes a Good Indicator?  Accuracy/Validity  Is the indicator based on accepted guidelines or developed through formal group-decision making methods?  Improvability  Can the performance rate associated with the indicator realistically be improved given the limitations of your services and population?

7. National Quality Center (NQC)7 Define Indicator  Who is eligible to be evaluated?  What part of this population should have received the care being measured? (Who should be counted in the denominator?)  What part of those who should have received the care did receive the recommended care? (Who should be counted in the numerator?)

8. National Quality Center (NQC)8 Example  Eligibility definition by the National HIVQUAL Project:  HIV+ ambulatory patients with at least 2 medical visits during the study period (i.e., 1/1/08 through 12/31/08) with at least one visit in each six-month period (January- June/July-December)

9. National Quality Center (NQC)9 Examples  Location: all sites, or only some?  Gender: men, women, or both?  Age: any limits?  Client conditions: all HIV-infected clients, or only those with a specific diagnosis?  Treatment status?

10. National Quality Center (NQC)10  Base the indicator on guidelines and standards of care when possible  Include staff and consumers when developing an indicator to create ownership  Be clear in terms of patient / program characteristics (gender, age, patient condition, provider type, etc.)  Set specific time-frames in indicator definitions Tips for Defining Indicators

11. National Quality Center (NQC)11 Types of Measures  Outcome measures are the voice of the customer  Outcome measures reflect the impact of our work on the care provided to our clients  Examples: CD4/VL, number of medical provider visits, incidence of disease rates, functional status indicators, mortality & morbidity rates, client/care team satisfaction rates, or cost of care delivered

12. National Quality Center (NQC)12 Types of Measures  Process measures are the voice of the system  Process measures tell us whether parts or steps in the system are working as planned  Examples: Waiting time for appointments, no show rates, percent of clients with self-management goals in place, the number of clients organized by acuity level in case management

13. National Quality Center (NQC)13 Types of Measures  Balance measures are used to monitor the effect one change might have on multiple parts of the system  Example: Mental Health screening & referral is improved. To prevent a potential negative outcome of the increase in referrals leading to lengthy waits for appointments, a balancing measure might be to monitor time between the referral and the mental health appointment

14. National Quality Center (NQC)14 Collecting & Managing Data  “Data In” characteristics:  Data that requires manual chart review should be completed as efficiently as possible (pull the chart only once)  Include the data collectors (e.g., HIV case managers) in the feedback process: this validates their work and improves data quality  Streamline potential duplication of data entry by systematic alignment

15. National Quality Center (NQC)15 Collecting & Managing Data  “Data out” characteristics:  Data should be displayed clearly; graphs and charts are effective means  Data should be shared throughout the system along with program goals and improvement activities  Data should be evaluated and acted upon as outlined in the QM Plan  Share data, as appropriate, with consumers, partners, funders, and other stakeholders  Use quality improvement data to inform operational and strategic planning efforts; alignment within the organization and system is smart business

16. National Quality Center (NQC)16 Constructing A Sample-- Definitions  Sampling: allows teams to make inferences about a total population based on observations of a smaller subset of that group  Over sampling: may be required if measurement pertains exclusively to one group within a population

17. National Quality Center (NQC)17 Constructing A Sample-- Methodology  Define the selection criteria  Gender  Age  Patient condition  Treatment status  Identify eligible cases  Randomly select cases

18. National Quality Center (NQC)18 Constructing A Sample-- Sampling Chart Total Eligible Population Minimum Total Records Charts to Pull* 140-1596483 160-1796787 180-1997091 200-2497598 250-29979103 300-34982107 350-39985111

19. National Quality Center (NQC)19 Sampling Methodology  Create a numbered list of all eligible patients  Go to web site: www.randomizer.orgwww.randomizer.org  Click on link to Randomize  Put in total number of eligible patients, e.g. 200  Put in number of sample size, e.g., 41  Click randomize now, and you will get a list of 41 random numbers between 1 and 200  You may have to do women and men separately because of over sampling for women

20. National Quality Center (NQC)20 www.randomizer.org

21. National Quality Center (NQC)21 How many sets of number would you like? How many numbers per set? Number range? From To Do you wish each number in a set to remain unique? Do you wish to sort your outputted numbers? How do you wish to view the output? Randomize Now www.randomizer.org 41 1 200 1 YES NO Place Markers Off

22. National Quality Center (NQC)22 Contact Information National Quality Center (NQC) 212-417-4730 NationalQualityCenter.org 212-417-4730 NationalQualityCenter.org Info@NationalQualityCenter.org HIVQUAL-US 212-417-4620 HIVQUAL.org 212-417-4620 HIVQUAL.org JMM30@health.state.ny.us