1. 17F4-final-presentation
Improving the Data Audit Process at the Michigan Arthroplasty Registry Collaborative Quality Initiative
From IOE 481 Project Team #4: Pranita Balusu Brandon Brathwaite Klaudia Mastela
Final Report Submitted to:
Rochelle Igrisan, MBA, MSN, RN, Senior Project Manager April Richmond, BSN, RN, Senior Clinical Information Analyst Thomas Zheng, PhD, MS, Expert Statistician Richard Hughes, PhD, Co-Director Mark P. Van Oyen, PhD, IOE 481 Faculty Instructor Mary Duck, UMHS IOE 481 Liaison
Pranita
Hi Everyone, we're team 4, I'm Pranita, and this is Klaudia and Brandon. Today we’ll present our final report for our project, which is improving the data audit process at the Michigan Arthroplasty Registry Collaborative Quality Initiative, or MARCQI. We’ll start off with a quick overview of MARCQI and the audit process, and then go into our methods, findings, conclusions, and lastly our final recommendations.
17F4-final-presentation
December 5th, 2017

2. Introduction & Background$$ $$
Quality Improvement
Data
Pranita
So MARCQI is a statewide collaborative between 60 hospitals across Michigan.

3. Introduction & Background
MARCQI Objectives
Improve the quality and safety of care
Identify the poorly performing new technologies
Show both patients and purchasers that the hospital sites under MARCQI are improving the value of arthroplasty services
MARCQI Audit Goals
Ensure data is complete
Ensure data is accurate
Ensure data is interpreted consistently across all hospital sites
Pranita
So MARCQI is a statewide collaborative between

4. Audit Process Audit Process Billing Audit On-Site Audit P4P Points
Pranita

5. Audit Process Billing Audit 4 60 12
Pranita
Give scale information: There are 4 full-time nurse consultants located at the MARCQI coordinating center here in Ann Arbor. These consultants are responsible for looking at billing data for each of the 60 hospitals/sites within MARCQI, which each perform at least 200 annual hip & knee arthroplasties. These billing audits are performed on an annual, mid-year basis. 4 60 12

6. Project Scope In Scope Out of Scope MARCQI’s billing audit process
Excel template for billing audits
Training session and user manual for template
On-site audit process
Site selection and frequency
P4P point system
Pranita

7. Billing Department Data
Billing Audit Process
Billing Department Data
MARCQI Data
Last Name
First Name
Date of Birth
Surgery Date
Procedure Code
Admit Date
Discharge Date
Complete Dataset
Pranita

8. Goals and Objectives Primary Goal Tasks
Recommend process improvements that increase the efficiency of MARCQI’s billing audit process, the team will complete the following tasks:
Interview MARCQI nurse coordinators and the audit process project manager
Obtain data samples from MARCQI nurse coordinators and the project manager
Observe billing audits
Research Excel formulas to identify more efficient ways of comparing two sheets
Analyze data collected from observations
Brandon: The primary goal of our project was to recommend process improvements which increase the efficiency of the MARCQI billing audit process from its current state. In order to achieve this, the following tasks were necessary. First, interviews were to be conducted with members of the MARCQI staff including the nurse coordinators at the MARCQI office and our project manager for the audit process. We also seeked to obtain data sample from MARCQI and our project managers, including MARCQI literature and and historical data. We needed to obtain an observation of the billing audit process in its current state by organizing with the MARCQI nurse coordinators. It was also essential to research Excel formulas and methods for comparing data from two sheets. Finally, we would have to analyze and interpret the data we obtained throughout our observations.

9. Methodology Method Interviews and Surveys
Weekly meetings
Qualtrics Surveys with 4 auditors
Data Collection and Observations
Historical MARCQI Datasets for for 1 site (~4700 rows of data)
Mock Billing Audit
Alternate Methods of Data Comparison
Excel compared to other methods
Literature Review
MARCQI Literature
Technical Articles
Brandon
Sample Sizes: (how many people did you interview? How many people did the team distribute the questionnaire to?) Interviews primarily consisted of our weekly meetings at the MARCQI office, at which the leading nurse coordinator, the project manager, MARCQI Co-Director, and an expert statistician would be present. We were able to come in to the meetings with a Questionnaire distributed to the 4 nurse consultants located at the MARCQI office.
How much sample data from previous data audits looked at? One historical set of site data and MARCQI database data for a single hospital (site data with ~4700 rows, database data with ~3900)

10. Findings & Conclusions
Surveys
Brandon
Survey Results: Length of manual process? - Dependent on hospital, but questionnaire results from coordinators indicates they take approximately 11 hours per site

11. Findings & Conclusions
Interviews
Surveys
Auditors don’t have enough time to conduct enough audits
Audits take on average ~11 hours per site
Over half the issues are from missing cases
The template should identify inconsistencies and missing cases to reduce time
Brandon

12. Findings & Conclusions
Observing Billing Audit
Very manual and repetitive
Audits have on average ~170 errors per site
Brandon

13. Findings & Conclusions
Observing Billing Audit
Brandon: Current paperwork designed by MARCQI intended to establish a procedure for the billing audits. As you can see in the first screenshot, the paperwork asks questions as potential solutions, such as if the dates are in the same format. However, it does not provide a specific recommendation or change that should be made to ensure standardization of the data. The screenshot highlighted #N/A indicates an entry that has a non-matching value which must be manually checked by the auditors, even after following the described process.

14. Findings & Conclusions
Observing Billing Audit
Very manual and repetitive
Audits have on average ~170 errors per site
Standardize the process
Automate comparison of fields
Automate identification of missing cases
Brandon

15. Findings & Conclusions
Alternate Methods of Data Comparison
Excel
Other Programs
Audit time reduction
Add-ons
Time to design
User experience
Limited flexibility
Python R
SAS
Greater computing speed
Flexible
Klaudia

16. Findings & Conclusions
Alternate Methods of Data Comparison
Excel
Other Programs
Audit time reduction
Add-ons
Time to design
User experience
Limited flexibility
Python R
SAS
Greater computing speed
Flexible
Klaudia

17. Findings & Conclusions
Literature Review
Methods of comparing data using Excel:
The template should:
Comparing data on 2 sheets
Consolidate data from 2 sheets into 1 in order to compare rows
Highlighting differences in data
Use conditional formatting to highlight difference
Automatically eliminating duplicate data
Sort and filter cases with multiple copies from different sources
Klaudia

18. User Manual & Training Session
Recommendations
Excel Template
User Manual & Training Session
Use a template
Standardize the process
Combine the data
Automatically count duplicates
Automatically identify inconsistencies
Automatically identify possible missing cases
Address possible issues
Describe additional tips for Excel
Klaudia

19. Recommended Billing Audit
Klaudia

20. Recommended Billing Audit
Klaudia

21. Recommended Billing Audit
Klaudia

22. Recommended Billing Audit
Klaudia

23. Recommended Billing Audit
Klaudia

24. Recommended Billing Audit
Klaudia

25. Expected Impact Complete and Accurate MARCQI Dataset
Standardized Process
Shorter Billing Audit Times
Complete and Accurate MARCQI Dataset
Klaudia

26. Thank you
Questions?
Klaudia