Measuring IT Innovation Benefits

Evaluation of IT innovation Why measure IT benefits? A new IS/IT system is an investment; it must be strategically/financially justified. To evaluate an already existing system: Were the targets achieved? To enable comparison of different proposals To learn and create tools for managing IT projects Are we being conned by IT salespeople?

Investments in IT are difficult to evaluate Both costs and benefits are difficult to quantify; intangible factors are likely to be significant. It is difficult to follow (and plan for) the extremely rapid developments in IT. Measuring IT benefits

An IT assessment should answer three main questions: 1.How does IT improve business performance? 2.How to decide the IT projects in which to invest? 3.How to assess the performance of systems after implementation?

IT project costs (examples)  Hardware  Software  Programming  Problem analysis  Project management  Training  Operation  ‘Hidden’ costs, such as users helping other users in the period after implementation, or troubleshooting

IT project costs “Companies spend ten times more money making a software work than on the software itself” Joe Levy President Quadragon Group Comdex 2002

Tangible benefits Tangible benefits can be measured directly. Tangible benefits of IT/IS are improved business processes (as we have discussed throughout the course). These translate directly into reduced cost, - and indirectly into more business.

Intangible benefits Better information and decision-making Better coordination and supervision Improved morale Ability to evaluate more alternative Organisational learning Intangible benefits are those that cannot be measured in monetary terms:

To consider...  IT is very often only one part of a larger BPR which may include changes in organisation, IT/IS, logistics, etc.  With falling prices of IT hardware and software, and similarity between products, IT can often be considered a commodity; - in some cases staff training may cost more than the implemented system

Traditional measurement methods Return on investment (ROI) Net present value (NPV) Cost-benefit ‘Informal’ methods  Traditional project appraisal methods evaluate financial ratios only (monetary value). Intangible benefits are ignored or converted to a monetary value.

There are at least 30 current available evaluation methods for IT benefits (Andreasen 1999). They can be split into: Objective methods, that quantify inputs and output Subjective methods, relying on attitudes and opinions of users and system builders Alternative measurement methods

Information Economics (IE) Return on Management (ROM) SESAME... etc.  These methods are rarely used in construction, due to little awareness of these methods and complexity of the methods

In construction... Research show that there is some awareness of alternative IT evaluation methods, but very few companies use them. A general lack of evaluation methodologies with focus on IT, many rely on ‘gut feel’. IT evaluation techniques are considered more costly than the value they generate.

Pittard 1995 Awareness of IT appraisal techniques used in sample of UK construction companies

Andresen 1999 Procedures used in sample of Danish construction firms

Measuring IT benefits There is a need for construction-specific tool/framework that enables all parties to communicate and exchange information on possible IT investments in a form that is readily recognisable within the culture of the construction sector.

IT benefits measurement One approach measures value in distinct areas in which IT innovation can benefit a business: Efficiency: Effectiveness: Output / planned output Performance: Level of new outputs enabled, i.e. ~ the strategic impact of IT

IT benefits measurement Efficiency  money Effectiveness  a score/index of improved operations Performance  qualitative evaluation of improved business performance (long term) The framework will measure value, but not only in quantifiable terms:

IT benefits measurement process

IT benefits measurement Construction business process Efficiency benefits Effectiveness benefits Performance benefits Business planning Marketing Information management Procurement Finance Client management Design Construction Operation & Maintenance Human resources

Example of typical benefits ProcessEfficiencyEffectiveness PerformancePerformance ConstructionReduced construction times Improved productivity Reduced waste Improved quality of output Reduced technology risks Ability to exchange data Improved idea- sharing among project teams Improved integration Improved project relationships with strategic partners

IT benefits measurement The method makes use of sets of tables that must be filled out before and after an IT project. Full paper available at:

‘Conclusion’ 1.Decide key targets; monetary as well as non- monetary performance indicators 2.Assign a manager responsible for evaluating benefits (a champion) 3.Evaluate results against targets 4.Evaluate the measurement method and feedback A simplified procedure for any IT innovation benefit measurement may be concluded as: