How Can I Use My Completeness Report to Improve Data Quality? January 2016 How Can I Use My Completeness Report to Improve Data Quality? Ryan White Services Report HIV/AIDS Bureau, Health Resources and Services Administration (1,2) Welcome to the Ryan White Service Report video series. Each video provides an overview of a topic related to the Ryan White Services Report – otherwise known as the RSR, which is reported to the HIV/AIDS Bureau within the Health Resources and Services Administration. This video walks through the Completeness Report to help you identify gaps in your data and improve your RSR reporting processes and data quality.

Why Completeness Reports? This video will cover… 2 Report Structure 3 Tips for Analysis 1 Why Completeness Reports? First, we’ll briefly describe the purpose of the Completeness Report and how you can access it. Then, we’ll go over the report structure, including key definitions. Finally, we’ll review simple tips for analyzing your report.

Reveal gaps in your data Why Completeness Reports? Encourage you to fill gaps for future reports Why Completeness Reports? You might be asking why you’re receiving a completeness report. The Completeness Report is a tool HAB created to help you evaluate the completeness of your RSR data. (1) It can show you where there are gaps in your data and data collection processes (2) so you can fill those gaps for future submissions. 3

Who gets a report? Who gets a report? Grantees can create reports with data from one or more providers Each provider that submitted a client-level data file Who gets a report? Who gets a report? Multiple formats (e.g., Excel, PDF) A single report for one file or for multiple files (1) Every provider that submitted an RSR client-level data XML file gets a Completeness Report. Reports are available through the RSR Web System after file upload. (2) Grantees can create a report for one or multiple providers. (3) Reports are also available in multiple formats. The Excel and .CSV versions facilitate data analysis, while PDF and Word versions might be better for emailing and printing. (4) Finally, if you upload more than one RSR XML file, you can create a report for a single file or create a report that consolidates your multiple files.

Measures of completeness clients with 1. A reported value Measures of completeness 2. Nothing reported (missing) What exactly does complete data mean? Completeness is the extent to which you’re reported required data. We measure completeness through two main indicators. (1) The percent of required clients with: (2) A value reported, (2) And, the percent of clients with nothing reported at all. We call this last category of data “missing data.” These two categories are a completeness rate and a missing rate. The sum of these rates is 100%. 5

Measures of completeness What about “unknown” values? Prior to 2014, “unknown” was a valid response for many data elements Only gender, transgender sub-group, and a few clinical data elements still allow for “unknown” Those “unknown” responses are included in the completeness rate Measures of completeness Note that “unknown” was a valid response for many RSR data elements before 2014. Therefore, the completeness report used to present an unknown rate. That’s now gone. Gender, transgender sub-group, and a handful of clinical data elements still allow for the “unknown” response option. These “unknown” responses are considered reported values and therefore, fall into the completeness rate. 6

Fantasia Department of Health First Page Now, let’s talk about the structure of the Completeness Report. The first page of the Completeness Report describes the purpose of the report and the measure’s main definitions. The page also contains the grantee or provider name.

Fantasia University Health Center Fantasia Department of Health Fantasia University Health Center First Columns This particular Completeness Report is for a grantee with multiple providers. It is structured by provider so a given provider’s data elements are together. Let’s look at the main column headings. The first two columns contain type and name of each RSR client-level data items. Note that all services, both core medical and support, are lumped together in one line of the report. Column three shows the number of clients served during the reporting period with a service. This number is the same for all data elements. The fourth column contains the number of clients required for reporting for that particular data element. The numbers are different for each data element b/c reporting on some data elements depends on the services a client has received and other client characteristics. For example, only transgender clients need a transgender subgroup. Only HIV-positive clients with an outpatient/ambulatory care visit need clinical data. The number reported in this fourth column is the denominator in the completeness calculations corresponding to the data element in the same row.

Fantasia Department of Health Last Pages of Report To identify the requirement conditions for each data element, go to the end of the report.

Fantasia Department of Health Fantasia University Health Center Measure Calculations Fantasia University Health Center The next columns are the numerators in the rates and the rates themselves. To illustrate the calculations, let’s look at some clinical data elements as examples. This provider reported 1,880 clients. Of these, 1,375 were HIV positive and had an outpatient/ambulatory care visit, meaning they are required to have clinical data. So, 1,375 is the denominator in the clinical data calculations. This provider is doing great with CD4 count and viral load! Viral load is especially important because HAB uses these data to demonstrate the health outcomes of the Ryan White program. However, this provider is struggling with reporting prescribed ART. The provider only reported these data for 170 clients, leading to a 12% completeness rate.

Client Services 56 Let’s review the client services calculation, because it can be a bit tricky. Remember that Total Clients Reported is the number of clients reported with a service and the next column is the number of required clients for a given data element. For the client services data element, if the number in that second column is bigger than the first column, you have reported some clients without a service. Let’s look at this example. This provider reported 54 with a service and 56 clients in total. This means the provider reported 2 clients without a services, leading to a missing rate of 4% for the client services item. A client must have a service to be included in the report

Analysis tips Who gets a report? Look for completeness rates of less than 90% Download your report in Excel and sort from worst to best Analysis tips Who gets a report? Compare your rates to national averages Share report with staff/ providers to trouble-shoot issues and develop improvement plan Now, that you have a good sense of what your completeness report means, here are some analysis tips. (1) Download your report in Excel and sort by worst to best to identify struggling providers and tricky data elements. (3) Check out a document we have on TARGET that presents the completeness rate national averages and distributions. See where you fall. (4) Once you’ve identified your problem areas, meet with your staff and/or providers to understand what is causing missing data and develop improvement strategies. HAB cares! We will follow up with grantees and providers that do not meet certain completeness thresholds.

National Averages Finally, let’s take a look at those national averages. This document on TARGET gets updated every year after new RSR data are uploaded. You can see the national averages across all providers and the percent of providers that fall into each completeness bucket. When we compare screened for Hep C with viral load, we see they have the same missing average – 9%. This means, on average, providers are missing screened for Hep C and viral load data for about 9% of their required clients. However, when we look at the distributions, we see, that for screened for Hep C, most providers fall at the tails – in the best and worst categories, whereas for viral load, providers are more evenly distributed throughout the categories. https://careacttarget.org/library/rsr-data-completeness-rates-2010-2014

Questions? Learn more about data quality: https://careacttarget.org/library/data-quality- why-it-matters Learn how to access the Completeness Report: https://careacttarget.org/library/reviewing-your- data-upload-rsr-tools-2016 Contact us with questions: Data.TA@caiglobal.org Now we have come to the end of our video. To learn more about how to improve data quality and analyzing your Completeness Report: Review a presentation on data quality, posted on the TARGET Center Website. Review a webinar on how to access the Completeness Report and other tools in the RSR Web System. (2) Contact us with questions or concerns at this email address. Thanks for joining us!