1. Chapter 13 Measuring, Reporting, and Controlling
Managing the Information Technology Resource
Paula Goulding
ICT622

2. Chapter Outline Importance of measurement What and how to measure
Quality and efficacy of measures
Reporting to different audiences
Controlling IT through effective SLAs
Negotiating a service level agreement
Making SLAs work – metrics
Introducing internal SLA to firm
SLA myths
Who owns measurements and what/when to do them
Can enterprise succeed with or without measurements?
ICT622

3. Why Is There IT?
ICT622

4. Productivity Study of IT and Business Executives
Source: InformationWeek research productivity study of 300 IT and business executives
ICT622

5. Measuring IT IT expenditure as percent of revenue
IT expenditure per employee
Revenue per IT dollar
Total IT expense budget vs. actual
Total IT capital budget vs. actual
Employees supported per IT employee
ICT622

6. Importance of Measurement
Business wants to know if IT is being run efficiently and effectively
Identify opportunities for improving effectiveness and efficiency
Effectiveness
Doing things right
Efficiency
Doing things the right way
ICT622

7. Measuring IT – Samples Overall – qualitative measures
Overall – quantitative measures
Network
Data center
ICT622

8. What and How to Measure Need to measure
Business impact
Customer relationships
Internal organization impact
Investment impact
Value chain impact
Know how processes are changing between any measurements, and explain any changes in results
No ideal set of metrics
ICT622

9. 10-Step Approach to Developing Measurements
Commitment of resources, time, and cooperation from senior business and IT management
Have and review formal strategic plan, mission statements, and goals
Committees need to be established for each of 38 IT processes
Each committee meets and defines goals of each IT process as first objective
Identify critical success factors
ICT622

10. 10-Step Approach to Developing Measurements
Establish guidelines to measure if, and to what degree, critical success factors have been met
Complete pilot should be done and members jointly participate in the pilot
Recommendation to install these measurements should be raised to senior management for endorsement
Distribution and implementation of new measures
Analyze results and review with all partners
ICT622

11. 5 Stages of Benchmarking
Planning
Analysis
Integration
Action
Maturity
Source: Robert C. Camp and
ICT622

12. Benchmarks Planning Analysis Identify what is to be benchmarked
Identify comparative companies
Determine data collection method and collect data
Analysis
Determine current performance “gap”
Project future performance levels
ICT622

13. Benchmarks Integration Action Maturity
Communicate benchmark findings and gain acceptance
Establish functional goals
Action
Develop action plans
Implement specific actions and monitor progress
Recalibrate benchmarks
Maturity
Leadership position attained and practices fully integrated into process
ICT622

14. Iterative Process for Benchmarking
Source: Dataquest, Gartner Group.
ICT622

15. Quality of Measures Often best when forecasting trends
Can lead to misleading results
Gain correct insight as to what measurement indicates
Good quality may be significant indicator of good performance
ICT622

16. Dimensions of Service Quality
Reliability
Consistency of performance and dependability
Responsiveness
Willingness/readiness of employees to provide service in timely manner
Competence
Possession of required skills to perform service
Access
Approachability and ease of contact
Courtesy
Politeness, respect, consideration, friendliness
ICT622

17. Dimensions of Service Quality
Communications
Keeping customers informed in language they understand
Credibility
Trustworthiness, believability, honesty
Security
Freedom from danger, risk, doubt
Understanding/Knowing the Customer
Making effort to understand customer’s needs
Tangibles
Physical evidence of service
ICT622

18. Dimensions of Successful IT Functions
Service quality
System quality
Information quality
Use
User satisfaction
Individual impact
Work group impact
Organizational impact
ICT622

19. Source: Anthony, R.N. Planning and Control Systems: A Framework for Analysis, Cambridge, MA: Harvard University Press (1965).
ICT622

20. Variety in Reporting Historical Data Statistical Data
Assist managers in determining repairs
Each has different characteristics
ICT622

21. Characteristics of Effective SLAs
Based on current “user” expectation
Defined by location, function, and service type
Uses benchmarks and baselines for measuring performance
Specifies minimums, penalties, and incentives
Specifies reports and tracking tools to be monitored
Agreement not too few, not too many
Gets adjusted periodically to reflect changes
ICT622

22. Negotiating a SLA Get a Baseline Define Responsibility
Allow for Modifications
Ensure Reports Are in Terms Anyone Can Understand
Don’t Forget the End-User Experience
ICT622

23. Examples of Service Level Metrics
Application availability
Average application response time
Number of application crashes per unit of time
Average throughout
Network availability and bandwidth
Server availability
ICT622

24. Examples of Service Level Metrics
Elapse time to repair hardware failure
Mean time between server failure
Number of operating system failure per unit of time
Number of middleware failure per unit of time
Number of production jobs not completed during batch night shift
ICT622

25. Questions Concerning Who Owns Measurement
Who should own measurement?
Is measure a process that should be owned by the people doing the measurement?
Is it a corporate asset or cost?
Is this question similar to who should own payroll?
Should the owners of internal processes that are being measured simply own it?
Does it make a difference?
ICT622

26. What to do with Measurements
Establish report card for performance
Create effective and understandable benchmarks
Senior management needs to know current status and effectiveness of IT
ICT622

27. Performance Measures for Efficient Order Management
Short order lead times
In-stock availability
Order accuracy
Access to order status information
Response time to customer inquiries
ICT622

28. Performance Measures for Efficient Customer Management
Lower inventory carrying cost
Reduced human intervention
Greater order accuracy
Improved production planning/forecasting
Lower order administration
Lower number of complaints
Improved customer decision making based on preferences
Increase number of automatic restocking customers
Lower stock-out occurrences
ICT622

29. When Measurements Should be Done
Continuously
Daily
Weekly
Monthly
Semiannually
Annually
ICT622

30. Can Enterprise Succeed Without Measurement?
Only 42% of companies prepare business case for CRM project
Only 45% have centralized CRM responsibility
57% can’t justify investments because they don’t know how to measure customer profitability
Only 10% measure ROI
ICT622

31. How Does Enterprise Benefit from Measurement?
Performance is related to effectiveness, efficiency, and quality
To understand process it must be measured and compared
Part of monitoring project performance
Determine if vendors are performing to service level agreements
ICT622