17F4-final-presentation Improving the Data Audit Process at the Michigan Arthroplasty Registry Collaborative Quality Initiative From IOE 481 Project Team #4: Pranita Balusu Brandon Brathwaite Klaudia Mastela Final Report Submitted to: Rochelle Igrisan, MBA, MSN, RN, Senior Project Manager April Richmond, BSN, RN, Senior Clinical Information Analyst Thomas Zheng, PhD, MS, Expert Statistician Richard Hughes, PhD, Co-Director Mark P. Van Oyen, PhD, IOE 481 Faculty Instructor Mary Duck, UMHS IOE 481 Liaison Pranita Hi Everyone, we're team 4, I'm Pranita, and this is Klaudia and Brandon. Today we’ll present our final report for our project, which is improving the data audit process at the Michigan Arthroplasty Registry Collaborative Quality Initiative, or MARCQI. We’ll start off with a quick overview of MARCQI and the audit process, and then go into our methods, findings, conclusions, and lastly our final recommendations. 17F4-final-presentation December 5th, 2017

Introduction & Background $$ $$ Quality Improvement Data Pranita So MARCQI is a statewide collaborative between 60 hospitals across Michigan.

Introduction & Background MARCQI Objectives Improve the quality and safety of care Identify the poorly performing new technologies Show both patients and purchasers that the hospital sites under MARCQI are improving the value of arthroplasty services MARCQI Audit Goals Ensure data is complete Ensure data is accurate Ensure data is interpreted consistently across all hospital sites Pranita So MARCQI is a statewide collaborative between

Audit Process Audit Process Billing Audit On-Site Audit P4P Points Pranita

Audit Process Billing Audit 4 60 12 Pranita Give scale information: There are 4 full-time nurse consultants located at the MARCQI coordinating center here in Ann Arbor. These consultants are responsible for looking at billing data for each of the 60 hospitals/sites within MARCQI, which each perform at least 200 annual hip & knee arthroplasties. These billing audits are performed on an annual, mid-year basis. 4 60 12

Project Scope In Scope Out of Scope MARCQI’s billing audit process Excel template for billing audits Training session and user manual for template On-site audit process Site selection and frequency P4P point system Pranita

Billing Department Data Billing Audit Process Billing Department Data MARCQI Data Last Name First Name Date of Birth Surgery Date Procedure Code Admit Date Discharge Date Complete Dataset Pranita

Goals and Objectives Primary Goal Tasks Recommend process improvements that increase the efficiency of MARCQI’s billing audit process, the team will complete the following tasks: Interview MARCQI nurse coordinators and the audit process project manager Obtain data samples from MARCQI nurse coordinators and the project manager Observe billing audits Research Excel formulas to identify more efficient ways of comparing two sheets Analyze data collected from observations Brandon: The primary goal of our project was to recommend process improvements which increase the efficiency of the MARCQI billing audit process from its current state. In order to achieve this, the following tasks were necessary. First, interviews were to be conducted with members of the MARCQI staff including the nurse coordinators at the MARCQI office and our project manager for the audit process. We also seeked to obtain data sample from MARCQI and our project managers, including MARCQI literature and and historical data. We needed to obtain an observation of the billing audit process in its current state by organizing with the MARCQI nurse coordinators. It was also essential to research Excel formulas and methods for comparing data from two sheets. Finally, we would have to analyze and interpret the data we obtained throughout our observations.

Methodology Method Interviews and Surveys Weekly meetings Qualtrics Surveys with 4 auditors Data Collection and Observations Historical MARCQI Datasets for 2015-2016 for 1 site (~4700 rows of data) Mock Billing Audit Alternate Methods of Data Comparison Excel compared to other methods Literature Review MARCQI Literature Technical Articles Brandon Sample Sizes: (how many people did you interview? How many people did the team distribute the questionnaire to?) Interviews primarily consisted of our weekly meetings at the MARCQI office, at which the leading nurse coordinator, the project manager, MARCQI Co-Director, and an expert statistician would be present. We were able to come in to the meetings with a Questionnaire distributed to the 4 nurse consultants located at the MARCQI office. How much sample data from previous data audits looked at? One historical set of site data and MARCQI database data for a single hospital (site data with ~4700 rows, database data with ~3900)

Findings & Conclusions Surveys Brandon Survey Results: Length of manual process? - Dependent on hospital, but questionnaire results from coordinators indicates they take approximately 11 hours per site

Findings & Conclusions Interviews Surveys Auditors don’t have enough time to conduct enough audits Audits take on average ~11 hours per site Over half the issues are from missing cases The template should identify inconsistencies and missing cases to reduce time Brandon

Findings & Conclusions Observing Billing Audit Very manual and repetitive Audits have on average ~170 errors per site Brandon

Findings & Conclusions Observing Billing Audit Brandon: Current paperwork designed by MARCQI intended to establish a procedure for the billing audits. As you can see in the first screenshot, the paperwork asks questions as potential solutions, such as if the dates are in the same format. However, it does not provide a specific recommendation or change that should be made to ensure standardization of the data. The screenshot highlighted #N/A indicates an entry that has a non-matching value which must be manually checked by the auditors, even after following the described process.

Findings & Conclusions Observing Billing Audit Very manual and repetitive Audits have on average ~170 errors per site Standardize the process Automate comparison of fields Automate identification of missing cases Brandon

Findings & Conclusions Alternate Methods of Data Comparison Excel Other Programs Audit time reduction Add-ons Time to design User experience Limited flexibility Python R SAS Greater computing speed Flexible Klaudia

Findings & Conclusions Alternate Methods of Data Comparison Excel Other Programs Audit time reduction Add-ons Time to design User experience Limited flexibility Python R SAS Greater computing speed Flexible Klaudia

Findings & Conclusions Literature Review Methods of comparing data using Excel: The template should: Comparing data on 2 sheets Consolidate data from 2 sheets into 1 in order to compare rows Highlighting differences in data Use conditional formatting to highlight difference Automatically eliminating duplicate data Sort and filter cases with multiple copies from different sources Klaudia

User Manual & Training Session Recommendations Excel Template User Manual & Training Session Use a template Standardize the process Combine the data Automatically count duplicates Automatically identify inconsistencies Automatically identify possible missing cases Address possible issues Describe additional tips for Excel Klaudia

Recommended Billing Audit Klaudia

Recommended Billing Audit Klaudia

Recommended Billing Audit Klaudia

Recommended Billing Audit Klaudia

Recommended Billing Audit Klaudia

Recommended Billing Audit Klaudia

Expected Impact Complete and Accurate MARCQI Dataset Standardized Process Shorter Billing Audit Times Complete and Accurate MARCQI Dataset Klaudia

Thank you Questions? Klaudia