1 Why evaluate information technology investments? Pertemuan Matakuliah: A Evaluasi Efektivitas Sistem Informasi Tahun: 2006

2 Introduction Highest ongoing capital expenditure but economically justifiable Need to prioritize different IT projects Formal evaluation process is a learning experience. Ex : for computer (requirement, budget, expertise) Judgments only based on perceptions

3 Different approaches to evaluation 1.Ex-ante and ex-post evaluation Ex-ante : Evaluations to forecast and evaluate the impact of future situations. Example : financial estimate Purpose : to support IT investment justification Ex-post : Assess the value of existing situations Purpose : to assess the value of IT investment and support operational decisions

4 Different approaches to evaluation 2. Formative and summative evaluation Formative : improving program performance, take place while program is in operation, rely to large extent of qualitative data and responsive to the needs of program owner and operators. Summative : monitoring the process and products of system, gathering user feedback for further development, assessing the effectiveness of the system and overall performance

5 Establish the costs It can be difficult to allocate costs to specific projects when a product will be utilized throughout organization Investment cost sometimes escalates during the development/implementation phase Ongoing cost can be difficult to estimate due to the fact that lifespan of IT investment is unknown

6 Problems of benefit evaluation Types of benefit that a firm can expect from its application (range, forms, task, understanding) Hard to link the intangible benefit to profitability Perform IT benefit evaluation as part of business process. Ex : JIT

7 Estimating Benefits Estimating Benefits presents several challenges. –In a Displacement Model, the costs being replaced, are known. –They are real and can easily be quantified. If the system will replace 10 workers in the Paint booth at $18.00, then you will save $180 a day in direct costs. Indirect costs would include, Fringe, G&A, benefits, etc. These costs become savings or benefits to the corporation.

8 Estimating Benefits –In a Cost Avoidance Model, the costs being avoided, can usually be estimated with a high degree of certainty. –If a system is being deployed to avoid hiring additional staff, then the cost of the staff can also be estimated with a good confidence level. –There is additional risk with this model in that the costs are probably accurate, but there is no quantifiable data on which to base the estimate. As with the displacement model, these costs also become savings or benefits to the corporation.

9 Estimating Benefits The most difficult benefit to accurately estimate is those resulting from an IT Investment that is either a new solution or product unto itself. –Examples of this are a “.com”, a B2B or B2C portal type of solution. The problems, except for those that have done exactly the same thing previously, are as follows: –If a similar solution has been deployed by another company, they won’t tell you how they made it work and how many benefits that are receiving, much less quantify it. You don’t really know what will happen.

10 Estimating Benefits Use data from similar sources or projects. –It may be appropriate to retain some consultants who have done this type of project before. –Consultants can often provide guidance that would not be obtainable elsewhere. –Be cautious in the selection process. They may not be as good as they say that they are. Quantify as much as possible. –Support it with extensive market research. –Ask for justification of each estimate. –Use a reasonableness test. Does it make sense.

11 Estimating Benefits Backup estimates. –The company that was going to sell $65,000 worth of Hot Wheels a month was just plain wrong. It isn’t going to happen. –Question all the estimates that you receive. Use benefit numbers that are very conservative. –If you are wrong and things are better, great. If they are worse, you’ll know sooner. Make sure that the benefits projected identify any associated costs that would offset the benefits.

12 Estimating Benefits Each solution and the firms that is deploying it at any specific time will receive different benefits. Be as complete as you can in your analysis. Apply financial principles to the benefits estimates as you would to the cost estimates. –Both change over time. –Look outside the solution that is being estimated. Risk may be external to the solution. Regulations, conflicts and changing demographics all impact costs and benefits. Consider everything that affects the company in your analysis.

13 Estimating Benefits Estimating intangible benefits, uses one of two approaches. –Approach One: Negotiation with the operating units to assess the value of the intangible. This is usually done by asking what is the value to the benefit to stakeholders. Answers will vary but ask enough and a common value can be obtained. –Approach Two is Imputation. This is the practice of 'filling in' missing data with plausible values and applying a test of the value.

14 Estimating Benefits In estimating intangible benefits, consider that the very nature of this type of benefit. –Through real, it is difficult to apply quantifiable values to it. –The values applied can all ways be questioned and challenged. Realizing these difficulties, many firms as part of their estimating methodology, consciously choose to exclude intangible benefit valuations from their calculations.