Chapter 13 Measuring, Reporting, and Controlling Managing the Information Technology Resource Jerry N. Luftman © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 1

Chapter Outline Importance of measurement What and how to measure Quality and efficacy of measures Reporting to different audiences Controlling IT through effective SLAs Negotiating a service level agreement Making SLAs work – metrics Introducing internal SLA to firm SLA myths Who owns measurements and what/when to do them Can enterprise succeed with or without measurements? © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 2

Why Is There IT? © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 3

Productivity Study of IT and Business Executives Source: InformationWeek research productivity study of 300 IT and business executives © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 4

Measuring IT IT expenditure as percent of revenue IT expenditure per employee Revenue per IT dollar Total IT expense budget vs. actual Total IT capital budget vs. actual Employees supported per IT employee © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 5

Importance of Measurement Business wants to know if IT is being run efficiently and effectively Identify opportunities for improving effectiveness and efficiency Effectiveness Doing things right Efficiency Doing things the right way © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 6

Measuring IT – Samples Overall – qualitative measures Overall – quantitative measures Network Data center © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 7

What and How to Measure Need to measure Business impact Customer relationships Internal organization impact Investment impact Value chain impact Know how processes are changing between any measurements, and explain any changes in results No ideal set of metrics © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 8

10-Step Approach to Developing Measurements Commitment of resources, time, and cooperation from senior business and IT management Have and review formal strategic plan, mission statements, and goals Committees need to be established for each of 38 IT processes Each committee meets and defines goals of each IT process as first objective Identify critical success factors © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 9

10-Step Approach to Developing Measurements Establish guidelines to measure if, and to what degree, critical success factors have been met Complete pilot should be done and members jointly participate in the pilot Recommendation to install these measurements should be raised to senior management for endorsement Distribution and implementation of new measures Analyze results and review with all partners © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 10

5 Stages of Benchmarking Planning Analysis Integration Action Maturity Source: Robert C. Camp and http://www.apqc.org © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 11

Benchmarks Planning Analysis Identify what is to be benchmarked Identify comparative companies Determine data collection method and collect data Analysis Determine current performance “gap” Project future performance levels © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 12

Benchmarks Integration Action Maturity Communicate benchmark findings and gain acceptance Establish functional goals Action Develop action plans Implement specific actions and monitor progress Recalibrate benchmarks Maturity Leadership position attained and practices fully integrated into process © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 13

Iterative Process for Benchmarking Source: Dataquest, Gartner Group. © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 14

Quality of Measures Often best when forecasting trends Can lead to misleading results Gain correct insight as to what measurement indicates Good quality may be significant indicator of good performance © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 15

Dimensions of Service Quality Reliability Consistency of performance and dependability Responsiveness Willingness/readiness of employees to provide service in timely manner Competence Possession of required skills to perform service Access Approachability and ease of contact Courtesy Politeness, respect, consideration, friendliness © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 16

Dimensions of Service Quality Communications Keeping customers informed in language they understand Credibility Trustworthiness, believability, honesty Security Freedom from danger, risk, doubt Understanding/Knowing the Customer Making effort to understand customer’s needs Tangibles Physical evidence of service © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 17

Dimensions of Successful IT Functions Service quality System quality Information quality Use User satisfaction Individual impact Work group impact Organizational impact © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 18

Source: Anthony, R.N. Planning and Control Systems: A Framework for Analysis, Cambridge, MA: Harvard University Press (1965). © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 19

Variety in Reporting Historical Data Statistical Data Assist managers in determining repairs Each has different characteristics © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 20

Characteristics of Effective SLAs Based on current “user” expectation Defined by location, function, and service type Uses benchmarks and baselines for measuring performance Specifies minimums, penalties, and incentives Specifies reports and tracking tools to be monitored Agreement not too few, not too many Gets adjusted periodically to reflect changes © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 21

Negotiating a SLA Get a Baseline Define Responsibility Allow for Modifications Ensure Reports Are in Terms Anyone Can Understand Don’t Forget the End-User Experience © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 22

Examples of Service Level Metrics Application availability Average application response time Number of application crashes per unit of time Average throughout Network availability and bandwidth Server availability © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 23

Examples of Service Level Metrics Elapse time to repair hardware failure Mean time between server failure Number of operating system failure per unit of time Number of middleware failure per unit of time Number of production jobs not completed during batch night shift © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 24

Questions Concerning Who Owns Measurement Who should own measurement? Is measure a process that should be owned by the people doing the measurement? Is it a corporate asset or cost? Is this question similar to who should own payroll? Should the owners of internal processes that are being measured simply own it? Does it make a difference? © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 25

What to do with Measurements Establish report card for performance Create effective and understandable benchmarks Senior management needs to know current status and effectiveness of IT © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 26

Performance Measures for Efficient Order Management Short order lead times In-stock availability Order accuracy Access to order status information Response time to customer inquiries © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 27

Performance Measures for Efficient Customer Management Lower inventory carrying cost Reduced human intervention Greater order accuracy Improved production planning/forecasting Lower order administration Lower number of complaints Improved customer decision making based on preferences Increase number of automatic restocking customers Lower stock-out occurrences © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 28

When Measurements Should be Done Continuously Daily Weekly Monthly Semiannually Annually © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 29

Can Enterprise Succeed Without Measurement? Only 42% of companies prepare business case for CRM project Only 45% have centralized CRM responsibility 57% can’t justify investments because they don’t know how to measure customer profitability Only 10% measure ROI © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 30

How Does Enterprise Benefit from Measurement? Performance is related to effectiveness, efficiency, and quality To understand process it must be measured and compared Part of monitoring project performance Determine if vendors are performing to service level agreements © 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 31