Staff Perception Survey before and after EHR/CPOE Implementation Jean Loes Marcia Ward, Douglas Wakefield, John O’Brien

Introduction Among the most notable challenges to implementing clinical information systems are the varying levels of acceptance and use by healthcare providers and employees: –Research has shown that experiences shape the degree to which users will accept the technology initially (Dixon, 1999; Herbert & Benbasat, 1994). –Research has shown that users’ attitudes regarding risks to service quality and disruptions in workflow hinder implementation (Hu, et al., 2002; Zheng, et al., 2005). All of the significant models of information technology use suggest that perceptions of the impact on work and outcomes are significant determinants of technology use and adoption (Kufafka, et al., 2003). Copyright © 2006 Trinity Health 2

Copyright © 2009 Trinity Health 3 Background on Measures Measures have been created to explore attitudes toward technology including perceived usefulness and ease of use (Davis, 1989). Recently we developed a measure of healthcare worker perceptions of the effect of clinical information systems on workflow processes and outcomes (Wakefield et al., 2007). This Information Systems Expectations and Experiences (I- SEE) survey was administered before and after “Go-Live” of a comprehensive system change in several hospitals. Analysis of the survey factor structure identified scales that assessed respondents’ perceptions related to communication changes, changes in selected work behaviors, perceptions of the implementation strategy, and the impact on quality of patient care (Wakefield et al., 2007).

Objective The purpose of this study is to examine changes in perceptions about quality before and after implementation of a comprehensive clinical information system. This study is the first to explore these changes in multiple small hospitals and across various employee categories. Copyright © 2009 Trinity Health 4

Survey Methods The I-SEE survey instrument was modified. Respondents were asked to indicate their current perceptions at each wave. The survey was administered at three times: –Wave 1 – March 2007 – before any changes –Wave 2 – March 2008 – during training and preparation –Wave 3 – March 2009 – after implementation Implementation “Go-Live” occurred: –July 2008 for 3 hospitals –September 2008 for 4 hospitals Copyright © 2006 Trinity Health 5

Survey Responses Number and (Response Rate) Copyright © 2009 Trinity Health 6 Hospital WAVE WAVE Wave Ellsworth Municipal Hospital73 (28%)59 (23%)41 (16%) Franklin General County Hospital93 (51%)56 (31%)43 (24%) Hancock County Memorial Hospital79 (65%)63 (52%)34 (28%) Kossuth Regional Health Center54 (28%)47 (24%)35 (18%) Mercy Medical Center - New Hampton58 (50%)62 (54%)48 (42%) Mitchell County Regional Medical Center45 (21%)49 (23%)44 (21%) Palo Alto County Health Systems90 (50%)42 (23%)49 (27%) Total Returned Surveys511 (40%)385 (30%)305 (24%)

Survey Items with an Average Response of “Strongly Agree” Survey Items Overall Mean Response 31. Patient care is consistently given according to the “9 Rights” I enjoy my job Overall patient care is safe in the areas I work Access to information to make good patient care decisions is available I get a great deal of professional satisfaction from my job Communications ensuring high quality and safe patient care routinely occur when patients are transferred to other facilities 5.05 Copyright © 2006 Trinity Health 7

Items that Changed across Waves Copyright © 2009 Trinity Health 8

Number of Respondents in Each Staff Group Copyright © 2009 Trinity Health 9 Professional/Employee GroupsNumber Providers (physicians and midlevel providers) 49 Registered Nurses343 Other-Clinical (professional, technical, patient support) 458 Other-NonClinical (clerical, senior management, marketing) 293

Comparison across Staff Groups To explore whether the staff groups differed, we compared their responses across the three waves. –The Other – NonClinical group was excluded because they had minimal if any contact with the clinical information system and were not directly involved in patient care. Significant interactions were found for three survey items shown on the next slides. The pattern from before to after implementation was: –Other – Clinical respondents showed no change or increases in survey responses after implementation –Registered nurses showed no change in survey responses after implementation –Providers showed sizable decreases in survey responses after implementation. Copyright © 2006 Trinity Health 10

Significant Differences among Staff Groups Copyright © 2006 Trinity Health 11

Significant Differences among Staff Groups Copyright © 2006 Trinity Health 12

Significant Differences among Staff Groups Copyright © 2006 Trinity Health 13

Comparison between Physicians and Mid-level Providers To further explore whether subsets of the provider group differed, we compared their responses across the three waves. –Physicians included 11 supervisory and 13 nonsupervisory physicians. –Mid-level Providers included 22 nurse practitioners, physicians assistants, CRNAs, etc. Significant interactions were found for four survey items. As shown on the next slide, the pattern from before to after implementation was: –Mid-level Providers showed increases in survey responses after implementation to two survey items –Mid-level Providers showed no change in survey responses after implementation to two survey items –Physicians showed increases in survey responses after implementation to two survey items –Physicians showed decreases in survey responses after implementation to two survey items. Copyright © 2006 Trinity Health 14

Significant Differences between Physicians and Mid-Level Providers Copyright © 2006 Trinity Health 15

CPOE Rates after Implementation Copyright © 2009 Trinity Health 16

Relationship between Survey Items at Wave 3 and CPOE Use Rates Correlations between survey items at Wave 3 and CPOE use rates indicate that staff at hospitals with higher CPOE use rates also tended to respond more in agreement to three survey items: –“Patients are rarely asked the same questions by the staff” (r=.93) –“Access to information to make good patient care decisions is available” (r=.79) –“I get a great deal of professional satisfaction from my job” (r=.73) The strongest relationship indicated that staff at hospitals with higher CPOE use rates tended to strongly disagree with the survey item “Too many verbal orders are made on my unit” (r= -.96) Copyright © 2006 Trinity Health 17

Relationship between Survey Items at Wave 2 and CPOE Use Rates Correlations between survey items at Wave 2 and CPOE use rates indicate that staff at hospitals with higher CPOE use rates tended to respond more in agreement to three survey items before Go-Live: –“I spend about the right amount of time documenting patient care” (r=.69) –“Patients are rarely asked the same questions by the staff” (r=.63) –“Overall patient care is safe in the areas I work” (r=.59) Thus, higher agreement with these items may predict CPOE use rates after implementation. Copyright © 2006 Trinity Health 18

References Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and use acceptance of information technology. MIS Quarterly, 13 (3), Dixon, D. R. (1999). The behavioral side of information technology. International Journal of Medical Informatics, 56, Halbesleben JRB, Wakefield DS, Ward MM, Brokel J, Crandall D. (2009). The relationship between super users’ attitudes and employee experiences with clinical information systems. Medical Care Research & Review, 66: Hebert, M., & Benbasat, I. (1994). Adopting information technology in hospitals: the relationship between attitudes/expectations and behavior. Hosp Health Serv Adm, 39 (3), Hu, P. J., Chau, P. Y. K., & Sheng, O. R. L. (2002). Adoption of telemedicine technology by health care organizations: An exploratory study. Journal of Organizational Computing and Electronic Commerce, 12 (3), Kukafka, R., Johnson, S. B., Linfante, A., & Allegrante, J. P. (2003). Grounding a new information technology implementation framework in behavioral science: A systematic analysis of the literature on IT use. Journal of Biomedical Informatics, 36, Wakefield DS, Halbesleben JRB, Ward MM, Qiu Q, Brokel J, Crandall D. (2007) Development of a measure of clinical information systems expectations and experiences. Medical Care, 45: Zheng, K., Padman, R., Johnson, M. P., & Diamond, M. S. (2005). Understanding technology adoption in clinical care: clinician adoption behavior of a point-of-care reminder system. International Journal of Medical Informatics. 74(7-8), Copyright © 2009 Trinity Health 19

Acknowledgments This work was supported by funding from AHRQ – 5UC1HS – “EHR Implementation for the Continuum of Care in Rural Iowa” Mercy Medical Center – North Iowa Mercy North Iowa Network The University of Iowa – Center for Health Policy and Research The University of Missouri – Center for Health Care Quality Copyright © 2009 Trinity Health 20