Thepul Ginige Lecture-7 Evaluation Approaches of Information System Thepul Ginige

Evaluation Approaches Several evaluation approaches can be used: – Cost-benefit analysis (CBA) – Key Performance indicators (KPI) – Revised decision evaluation approach – User involvement evaluations – The information system utility approach 2

What is Cost-benefit analysis? Cost-benefit analysis is a tool for assessing whether or not the costs of an activity can be justified by the outcomes and impacts. Cost-benefit analysis measures both inputs and outputs in monetary terms. 3

Steps Identifying Costs – The first step is to identify and quantify all costs associated with a proposed action. Identifying Benefits – The next step is to identify and quantify all benefits anticipated as a result of successful implementation of the proposed action. Evaluate Costs and Benefits – The final step when creating a cost benefit analysis is to weigh the costs and benefits to determine if the proposed action is worthwhile 4

What is Key Performance indicators? Performance indicators are essential tools which will tell you if your business is on target or veering off course. Using the right indicators will help you deliver the right results. Key Performance Indicators cuts straight to the 75 + KPIs that matter. It explains what key performance indicators are, gives you short overviews of each metric and describes how to use the measure effectively 5

Examples of KPIs Net profit Revenue growth rate Return on investment (ROI) Customer satisfaction index Customer online engagement level Six Sigma level Process or machine downtime level 6 Reference : Key Performance Indicators (KPI):The 75 measures every manager needs to know, Bernard Marr, FT Prentice Hall, ISBN

Why Evaluate? Three purposes of the evaluation process: 1.To compare the results with the goals and expected effects of the system 2.To direct work towards the expected result with the help of evaluation during the development of the system 3.To use the findings and outcomes as an experience base for the next project (Peterson & Jelger, 1988)

Two Approaches to IS Evaluation Objectivist Approach Quantitative Subjectivist Approach Qualitative Important system attributes that can be measured and interpreted Observation results are dependent on context and observer; different individuals or groups may hold a different opinion about a systems value

Two Approaches to CIS Evaluation Has rarely produced positive evaluation results when applied to complex information systems – Consumed large amounts of resources – Expected and measured objective parameters, such as time saving, did not exceed those of the control Very often, the influence of such a system will not visible in direct time savings, but in: – Improved cooperation between departments – Increased quality of documentation – Better patient care Subjective Approach Objective Approach

Objectives of the CIS Evaluation Framework Internal To provide a method for assessing IS systems in order to quantify value and identify opportunities for improvement External To contribute to the broader healthcare community by designing a generalizable methodology for value- rating IS system technologies in order to facilitate decisions on technology implementations

Information System Evaluation Framework

The Goals of Evaluation The generic goal of most evaluations is to provide "useful feedback" to a variety of audiences including sponsors, donors, client- groups, administrators, staff, and other relevant constituencies. Most often, feedback is perceived as "useful" if it aids in decision-making. 12