1 Evaluating Return-on-Investment for a Hospital Clinical Information System Computers in Nursing Snyder-Halpern, Rita PhD, RN, C, CNAA HSCI 740 June 8, 2004

2 Topics Technology Being Evaluated Method of Measuring Cost and Outcomes Sources of Data Results Viability of Approach at Work Site Q&A

3 Technology Being Evaluated  Rural hospital clinical information system – part of larger regional healthcare enterprise CIS solution  Capture and retrieval of clinical data  Supports ability to access clinical documentation for IP, OP, ER, Ambulatory patient admit types  Physician order-entry capabilities  Patient record management

4 Method of Measuring Costs Measured costs based on initial IT investment, with planning for maintenance and support (15% -> 23% of total development cost within first 4 years), ongoing training (training costs rise in year 2 then drop off in year 2->7) — Training costs based on past data related to annual staff turnover (did not account however for how system might impact staff turnover rates – for better or worse) — Assumed negative correlation between IT training costs and IT use over time (e.g., assumed even though less money invested in training would still result in same/improved use of technology/ system). — Did not look at correlation of use of CIS and hospital’s increased revenue – translated savings based on paper, anticipated reduction of FTEs, and hospital staff time savings for retrieval of information as dollars that could go to the bottom line. — Calculated maintenance based on present value. With vendors – can designate cap on maintenance each year thru contract vehicle, thus using better data point for planning purposes.

5 Method of Measuring Outcomes Outcomes (designated four CIS benefits) — Linked to common IT metrics, not organizational strategic initiatives (e.g., decrease in paper vs. increase in number of patients/doctors attracted to hospital) — FTEs -numbers could actually increase with increased attraction of number of physicians/patients — Data research/retrieval times – look at benchmarks and do compare (easier to do for processes like registration/admission vs. general medical record retrieval) — Medical records savings – based on reduction of FTEs in medical records department – look at how to measure better leverage of staff vs. reduction of staff

6 Sources of Data Considered only cost of IT (investment) for organization in conducting cost/benefit analysis — Did not look at variations in IT investment related to revenue (e.g., consider increase in number of patients that could be seen as a result of time savings thereby increasing revenues – would contribute to additional FTEs vs. reduction in FTEs) Did not look at use patterns from old CIS and examine impact of not having system capabilities in place to support full physician documentation processes Only referenced brief experience of one other organization as it related to its quality monitoring capabilities – never reviewed IT investment/use/revenue correlations of other hospital  Case examples for use of different sources of data Use of IT Cost of IT for the org. Org. Revenue Cost of IT development Operations & Maintenance Revenue- generating hospital? CIS Three sources of data No review of old system use patterns

7 Results Increase in FTE hours and impact on productivity — Duel entry of effort due to lack of training for new processes – because physicians training/acceptance of system was lacking, nurses tried to minimize impact to physicians (data already in CIS as flowsheet was also being hand-written by nurses to summarize assessments and scanned in – impacted initial cost estimates for reduction of FTEs/hourly rates. Physicians could not enter in orders but they needed to use system to pull out other clinical data (e.g., lab/rad results, etc.) so still had to use paper to support documentation processes No reduction in printing costs Faulty assumption that paper usage would decrease – did not look at business impact (physicians were accustomed to printing out clinical information to support their documentation processes). New system did not consider business processes affected by IT. Need for additional investment in training Need for additional investment in development due to vaporware

8 Viability of approach at work site Based on experience with healthcare providers, have seen this as a viable approach only under following circumstances: — When making minor upgrades to existing system (not for major system implementation efforts) — When money is no object — System aligned with desired business practices of physicians and medical staff (not IT staff) — Physicians and nurses were engaged early on in JAD discussions & system demonstrations and were already championing effort — Staff were more technically advanced (not first new system in the door within last year or two) and system savvy – needed less training — Existing and Desired business processes were being defined as system was being developed (system did not drive redefinition of business processes)

9 Questions Two challenges impacted the anticipated benefits: — Product delivery delays — Source data capture problems In regard to these problems, what additional costs should they have anticipated? How could HIPAA rules/regulations been considered in terms of cost/benefit implications (hint: in old system physicians had extensive printing capabilities -more paper floating around – less security). What data sources could they have used to build their case for this? What revenue streams could they have examined to examine correlation between IT investment and revenue? Why is measuring costs based on reduction in FTEs a death spiral?