By: Satchanawit Phongkriangkrai Chapter 9: Evaluating the Outcomes of Information Systems Plans Managing information technology evaluation – techniques and processes* By: Satchanawit Phongkriangkrai * An earlier version of this chapter appeared in the European Management Journal, Vol. 10, No.2. June, pp. 220–229.

Overview: Information systems planning

Agenda Evaluation: emerging problems Strategy and information systems Evaluating feasibility: findings Linking strategy and feasibility techniques CODA: From development to routine operations

Introduction The size and continuing growth in IT investments from the early 1990 served to place IT issue above the ranks in most organization . Many organizations have encountered IT investment problems, e.g., high risks, and hidden cost processes. At least 20% of IT investment is wasted and between 30% and 40% of IT projects realize no net benefits. Major problems occur in how the IT investment is evaluated and controlled.

Evaluation: emerging problems Evaluation brings into question the notion of cost, benefit, risk, value, and process. There are many problems that impact on IT evaluation: inappropriate measures understating human and organizational costs overstating cost neglecting intangible benefits, etc. The fundamental and common failure relates to strategy alignment.

Strategy and information systems Organizational investment climate is affected by: The financial health and market position of the organization Industry sector pressures The organizational business strategy and direction The management and decision-making culture Decision-making culture Conservative Innovative Decision-making style Directive Consensus-driven

Alignment Alignment creates strategic climate in which IT investment can be related to business/organizational direction. Lack of alignment is a common problem in public sector information. When the organization lacks alignment of IT evaluation practice, they tend to become separated from business needs and plans, and from organizational realities. Another critical alignment is that between: what is done with IT how it fits with the information needs of the organization

IT Strategic grid Developed by McFarlan and McKenney in 1983 Classifies systems where IT investment has been made and where it should be applied. Demonstrates whether the IT vestments are being made: into core system, business growth, competitiveness. Identifies four classes of firms.

Figure Strategic grid analysis

Value chain Established by Porter and Millar in 1991 Consists of the primary and support activities. Primary activities of a typical manufacturing company: inbound and outbound logistics operations marketing and sales support service Support activities: firm infrastructure human resource management technology development procurement

IT investment mapping Developed by Peters in 1993 The basic dimensions of map were arrived at after reviewing the main investment concerns. Two dimension over 50 IT projects investment orientation Infrastructure – software/hardware environment Business Process – finance and accounts Market Influence – increasing repeat sales and benefit: Business Expansion Risk Minimization Enhance Productivity

Figure IT investment mapping

Figure Investment map comparing business and IT plans

Multiple methodology Developed by Earl in 1989 A multiple methodology inquires IT investment more closely with the strategic aims and direction of the organization, and its key needs. Three strategy formulas are: A top-down approach – use critical factor success (CFC) to establish objectives; A bottom-up evaluation – evaluate current systems; Inside-out innovation – create new strategic options. The purpose is to identify through internal and external analysis of business needs and opportunities to relate the development of IS applications to business strategy.

Evaluating feasibility: findings Evaluating IT projects at its feasibility stage are based on the right strategic climate. Feasibility evaluation provides results of interest and to sponsor of research. Value of IT/IS are justified by understating cost and using notational figure for benefit realization. Willcocks and Lester looked at 50 organizations from both private and public section of manufacturing and service. There is little evidence of a concern for assessing risk in any formal manner.

Figure IT evaluation: feasibility findings

Linking strategy and feasibility techniques A method uses evaluation techniques to measure the type of IT project, develop techniques relating the IT investment to business/organization value. The method of evaluation needs to be reliable, consistent in its measurement over time. The three evaluation techniques are: Return on management (ROM), Matching objectives, projects and techniques, From cost-benefit to value.

Linking strategy and feasibility techniques (Return on management (ROM)) Developed by Strassman in 1990 ROM is modern IT investment evaluation. ROM measures the performance based on the added value – the difference between net revenues and payment external supplier – to an organization provided by management. However, ROM is not widely used in UK. The problem is whether it really represents what IT has contributed to business performance.

Linking strategy and feasibility techniques (Matching objectives, projects and techniques) Butler Cox (1990) suggests five main purposes: surviving and functioning as a business improving business performance by cost reduction/increasing sales achieving a competitive leap enabling the benefits of other IT investments to be realized being prepared to compete effectively in the future The matching of IT investment can now be categorized as follows: Mandatory investments Investments to improve performance Competitive edge investments Infrastructure investments Research investments

Figure Classifying IT investments

Figure Matching projects to techniques

Linking strategy and feasibility techniques (From cost-benefit to value) Known as the information economics approach Developed by Parker et al. 1988 This technique copes with many problems of IT evaluation, both at the level of methodology and of process. Information economics looks beyond benefit to value. Value is the sum of: Enhanced return on investment, Business domain assessment, Technology domain assessment.

Figure The information economic approach

Linking strategy and feasibility techniques (From cost-benefit to value (Cont’d)) Enhanced ROI Value Linking – assesses indirect benefits added to other departments. Value Acceleration – reduced time-scales for operations. Value Restructuring – measures the benefits of restructuring a department, or job. Innovation valuation – considers the value of competitive advantage and calculate risks, pioneer cost, or failing cost. Business Domain Assessment Strategic Match: matches the business goals to the proposed project/IT investment. Competitive Advantage – assesses the degree of an advantage in the market. Management Information – assesses the contribution toward the management need for information. Competitive Response – assesses the degree of corporate risk associated with not undertaking the project. Organization Risk – implements the project in terms of personnel, skills, and experience.

Linking strategy and feasibility techniques (From cost-benefit to value (Cont’d)) Technology Domain Assessment Strategic IS Architecture – measures the degree to which the project fits into the overall information system plan. Definitional Uncertainty – assesses the complexity of the area and probability of non-routine change. IS Infrastructure Risk – evaluates a project’s dependence on new or untried technologies.

CODA: From development to routine operations

CODA: From development to routine operations (Cont’d) The evaluation cycle is useful for controlling a specific project, building organizational know-how on IT and management. However, some of limitations in evaluation techniques and processes are: Weak linkage between evaluations carried out different stages, Projects shelved, Stakeholders not included in feasibility stage, Continual problems from feasibility stage.

CODA: From development to routine operations (Cont’d) Suggestion to improve evaluation: Linking evaluation across stages and time Determining stakeholders who are participants in evaluation at all stages Avoiding the fall-of interest in evaluation at later stage Providing adequate evaluation techniques for the long term process.

Conclusions The profile of IT evaluation has increased high expenditure, disappointed expectations, underdeveloped and undermanaged area. Need more effective evaluation practice: traditional techniques, tailor modern techniques. In the past of the evaluation practice, value is price. The future challenge is to continue the measurement of value and build techniques and processes.

Thank you Q&A