Performance Comparison Among Major EHR Systems Tiankai Wang, PhD and David Gibbs, PhD Health Information Management Department

Background and Significance Over 4,000 Certified Health IT products available (ONC, 2018) “Health IT selection is challenging and the impact of making a wrong decision is costly and time consuming. While the certified health IT comparison tool marketplace is robust and diverse, there are still significant gaps in not only the marketplace itself, but also in the ability of providers to use the tools to make informed decisions.” (p. 7-8) “Comparison tools can ease the decision-making burden throughout a very complex process.” (p. 23) ONC, 2016 Report to Congress: Feasibility of Mechanisms to Assist Providers in Comparing and Selecting Certified EHR Technology Products Graphic licensed under CC BY-SA

Motivations Financial Quality Prior Literature Meaningful use incentives are temporary Long-term financial benefits from EHR remain unclear (Collum, Menachemi, & Sen, 2016) Quality Patient outcomes Increasingly dependent on EHR systems Prior Literature EHR as a concept Little evidence to differentiate products and vendors

Objective and Hypothesis Demonstrate how EHR systems can be objectively compared using available data detailing hospital financial performance and standard quality scores Hypothesis H1: EHR systems demonstrate different financial and quality performance

Measurements Measurements in available dataset Return on Assets (ROA) (Gunny, 2010) Bed Utilization Rate (BUR) (Belciug & Gorunescu, 2015) Length of Stay (LOS) (Hoyer et al., 2016) Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) Survey of patients perspectives on hospital care (Carter & Silverman, 2016) Total Performance (TP) Score Clinical care, patient experience, patient safety, efficiency/cost reduction HCAHPS plus additional quality measures (Haley, et al., 2016) Overall Rating HCAHPS plus additional quality measures from Hospital Compare (Hu, Jordan, Rubinfeld, Schreiber, Nerenz, & Waterman, 2017)

Sample Selection Definitive Healthcare, a subscribed healthcare data provider The dataset is comprehensive The dataset is updated The original dataset contains 8,825 unique hospital observations from 2011 to 2016 The final sample contains 3,266 observations

Empirical Model Performance =β0 + EHR vendor indicates + ∑Controls + Year fixed effects + ε Performance: Return on Assets (ROA) Bed Utilization Rate (BUR) HCAHPS summary star rating (HCAHPS) Length of stay (LOS) Hospital Compare Overall Rating (Overall_rating) Value-based Purchasing Total Performance Score (TP_score) Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS)

Empirical Model (cont’d) Independent Variable: EHR vendor indicators Control Variables: Medicare mix, Medicaid mix, uncompensated care mix, market concentration index, size, teaching status, ownership, location, Year fixed effects

Regression Results VARIABLES ROA BUR HCAHPS LOS Overall_Rating TP_Score Intercept 0.1113*** 0.3066*** 3.5212*** 5.7166*** 3.3112*** 49.5168*** (0.000) Vendor A -0.0031 0.0142** 0.1060*** -0.1837*** 0.0768** 0.2219** (0.335) (0.025) (0.028) (0.042) Vendor B 0.0162* 0.0057 0.0644*** -0.0646*** 0.1482** 0.7628* (0.099) (0.523) (0.001) (0.010) (0.013) (0.067) Vendor C 0.0295*** 0.0145* 0.2803*** -0.2677*** 0.2811*** 2.6372*** (0.002) (0.056) Vendor D -0.0223** -0.0216*** 0.2128*** -0.4685*** 0.2315** 0.3146 (0.024) (0.005) (0.258) Vendor E -0.0104 0.0344*** -0.0403 -0.0461 0.1118*** -0.3723 (0.582) (0.470) (0.216) (0.009) (0.547) Controls Yes Year FE Observations 3,265 2,891 3,262 2,923 2,468 Adjusted R-squared 0.059 0.384 0.145 0.117 0.143

Limitations No details about EHR customizations or configurations Selection bias Framework for analysis remains an option

Conclusion This study demonstrates how a variety of financial and quality factors associated with candidate EHR systems may be analyzed to measure how well the systems meet organizational priorities. Combining this approach with others that focus on patient outcomes will provide EHR acquisition teams with additional information during the decision-making process.

Contact Information Tiankai Wang, PhD tw26@txstate.edu David Gibbs, PhD dgibbs@txstate.edu

References Belciug S. & Gorunescu, F. (2015). Improving hospital bed occupancy and resource utilization through queuing modeling and evolutionary computation. Journal of Biomedical Informatics, 53:261-69. Carter J.C. & Silverman, F.N. (2016). Using HCAHPS data to improve hospital care quality. TQM Journal, 28(6):974-90. Collum, T.H., Menachemi, N, & Sen, B. (2016). Does electronic health record use improve hospital financial performance? Evidence from panel data. Health Care Manage Rev., 41(3):267-74. Gunny, K.A. (2010). The Relation Between Earnings Management Using Real Activities Manipulation and Future Performance: Evidence from Meeting Earnings Benchmarks. Contemporary Accounting Research,27(3):855-88. Haley D.R., Mei, Z., Spaulding, A., Hamadi, H., Jing, X., & Yeomans, K. (2016). The Influence of Hospital Market Competition on Patient Mortality and Total Performance Score. Health Care Manager, 35(3):220-30. Hoyer E.H., Friedman, M., Atanelov L, Needham, D.M., Brotman, D.J., Lavezza, A., …Colantuoni, E. (2016). Promoting mobility and reducing length of stay in hospitalized general medicine patients: A quality-improvement project. Journal of Hospital Medicine, 11(5):341-47. Hu, J., Jordan, J., Rubinfeld, I., Schreiber, M., Nerenz, D., & Waterman, B. (2017). Correlations Among Hospital Quality Measures: What “Hospital Compare” Data Tell Us. American Journal of Medical Quality, 32(6):605-10. ONC. 2016. Report to Congress Feasibility of Mechanisms to Assist Providers in Comparing and Selecting Certified EHR Technology Products. Retrieved from https://www.healthit.gov/sites/default/files/macraehrpct_final_4-2016.pdf ONC. 2018. Certified Health IT Products List. Retrieved from https://chpl.healthit.gov