1. Performance Comparison Among Major EHR Systems
Tiankai Wang, PhD and David Gibbs, PhD
Health Information Management Department

2. Background and Significance
Over 4,000 Certified Health IT products available
(ONC, 2018)
“Health IT selection is challenging and the impact of making a wrong decision is costly and time consuming. While the certified health IT comparison tool marketplace is robust and diverse, there are still significant gaps in not only the marketplace itself, but also in the ability of providers to use the tools to make informed decisions.” (p. 7-8)
“Comparison tools can ease the decision-making burden throughout a very complex process.” (p. 23)
ONC, 2016 Report to Congress: Feasibility of Mechanisms to Assist Providers in Comparing and Selecting Certified EHR Technology Products
Graphic licensed under CC BY-SA

3. Motivations Financial Quality Prior Literature
Meaningful use incentives are temporary
Long-term financial benefits from EHR remain unclear
(Collum, Menachemi, & Sen, 2016)
Quality
Patient outcomes
Increasingly dependent on EHR systems
Prior Literature
EHR as a concept
Little evidence to differentiate products and vendors

4. Objective and Hypothesis
Demonstrate how EHR systems can be objectively compared using available data detailing hospital financial performance and standard quality scores
Hypothesis
H1: EHR systems demonstrate different financial and quality performance

5. Measurements Measurements in available dataset Return on Assets (ROA)
(Gunny, 2010)
Bed Utilization Rate (BUR)
(Belciug & Gorunescu, 2015)
Length of Stay (LOS)
(Hoyer et al., 2016)
Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS)
Survey of patients perspectives on hospital care
(Carter & Silverman, 2016)
Total Performance (TP) Score
Clinical care, patient experience, patient safety, efficiency/cost reduction
HCAHPS plus additional quality measures
(Haley, et al., 2016)
Overall Rating
HCAHPS plus additional quality measures from Hospital Compare
(Hu, Jordan, Rubinfeld, Schreiber, Nerenz, & Waterman, 2017)

6. Sample Selection
Definitive Healthcare, a subscribed healthcare data provider
The dataset is comprehensive
The dataset is updated
The original dataset contains 8,825 unique hospital observations from 2011 to 2016
The final sample contains 3,266 observations

7. Empirical Model
Performance =β0 + EHR vendor indicates + ∑Controls + Year fixed effects + ε
Performance: Return on Assets (ROA)
Bed Utilization Rate (BUR)
HCAHPS summary star rating (HCAHPS)
Length of stay (LOS)
Hospital Compare Overall Rating (Overall_rating)
Value-based Purchasing Total Performance Score (TP_score)
Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS)

8. Empirical Model (cont’d)
Independent Variable: EHR vendor indicators
Control Variables: Medicare mix,
Medicaid mix,
uncompensated care mix,
market concentration index,
size, teaching status, ownership,
location,
Year fixed effects

9. Regression Results VARIABLES ROA BUR HCAHPS LOS Overall_Rating
TP_Score
Intercept
0.1113***
0.3066***
3.5212***
5.7166***
3.3112***
***
(0.000)
Vendor A
0.0142**
0.1060***
***
0.0768**
0.2219**
(0.335)
(0.025)
(0.028)
(0.042)
Vendor B
0.0162*
0.0057
0.0644***
***
0.1482**
0.7628*
(0.099)
(0.523)
(0.001)
(0.010)
(0.013)
(0.067)
Vendor C
0.0295***
0.0145*
0.2803***
***
0.2811***
2.6372***
(0.002)
(0.056)
Vendor D **
***
0.2128***
***
0.2315**
0.3146
(0.024)
(0.005)
(0.258)
Vendor E
0.0344***
0.1118***
(0.582)
(0.470)
(0.216)
(0.009)
(0.547)
Controls
Yes
Year FE
Observations
3,265
2,891
3,262
2,923
2,468
Adjusted R-squared
0.059
0.384
0.145
0.117
0.143

10. Limitations No details about EHR customizations or configurations
Selection bias
Framework for analysis remains an option

11. Conclusion
This study demonstrates how a variety of financial and quality factors associated with candidate EHR systems may be analyzed to measure how well the systems meet organizational priorities.
Combining this approach with others that focus on patient outcomes will provide EHR acquisition teams with additional information during the decision-making process.

12. Contact Information
Tiankai Wang, PhD
David Gibbs, PhD

13. References
Belciug S. & Gorunescu, F. (2015). Improving hospital bed occupancy and resource utilization through queuing modeling and evolutionary computation. Journal of Biomedical Informatics, 53:
Carter J.C. & Silverman, F.N. (2016). Using HCAHPS data to improve hospital care quality. TQM Journal, 28(6):
Collum, T.H., Menachemi, N, & Sen, B. (2016). Does electronic health record use improve hospital financial performance? Evidence from panel data. Health Care Manage Rev., 41(3):
Gunny, K.A. (2010). The Relation Between Earnings Management Using Real Activities Manipulation and Future Performance: Evidence from Meeting Earnings Benchmarks. Contemporary Accounting Research,27(3):
Haley D.R., Mei, Z., Spaulding, A., Hamadi, H., Jing, X., & Yeomans, K. (2016). The Influence of Hospital Market Competition on Patient Mortality and Total Performance Score. Health Care Manager, 35(3):
Hoyer E.H., Friedman, M., Atanelov L, Needham, D.M., Brotman, D.J., Lavezza, A., …Colantuoni, E. (2016). Promoting mobility and reducing length of stay in hospitalized general medicine patients: A quality-improvement project. Journal of Hospital Medicine, 11(5):
Hu, J., Jordan, J., Rubinfeld, I., Schreiber, M., Nerenz, D., & Waterman, B. (2017). Correlations Among Hospital Quality Measures: What “Hospital Compare” Data Tell Us. American Journal of Medical Quality, 32(6):
ONC Report to Congress Feasibility of Mechanisms to Assist Providers in Comparing and Selecting Certified EHR Technology Products. Retrieved from
ONC Certified Health IT Products List. Retrieved from