2. Notes accompany this presentation. Please select Notes Page view. These materials can be reproduced only with written approval from Gartner. Such approvals must be requested via e-mail: vendor.relations@gartner.com. Gartner is a registered trademark of Gartner, Inc. or its affiliates. Benchmarking IT: The Cost/Quality Balance Chris Smith

3. Comparing IT Performance While high level metrics are helpful and can be used to communicate the value of IT - remember They are database averages from surveys and Do not necessarily reflect your environment’s intricacies A Benchmark provides a more accurate view of your spending & staffing levels as it compares your organization to a set of peers with similar Service Levels Environment complexity Workload Number of locations supported Geographic location Level of outsourcing

4. £11,250,000 Specific Measures and Definitions Gartner collects data in line with its industry standard “Consensus Models” to ensure comparability “Workload” represents a service provided and is balanced with cost.  Devices  MIPS  Minutes  Volume  Users Workload  Software  Hardware  Personnel  Occupancy  Transmission Cost To compare with actual spending, an organisation’s workload is multiplied by their peers’ average unit cost: 12,500 x £900 The organisation’s PC Count Peers’ cost for supporting the organisation’s PCs Peers’ average cost per PC Based on operational expense Labour not depreciated

5. Cost per Desktop A B Complexity / Workload LowHigh The diagram below serves to illustrate the fundamental flaw in the “database average” approach to benchmarking: Company B appears to spend more than Company A when viewed at the simplistic “cost per desktop” level. However, when a more contextual analysis is performed and the degree of complexity is factored in (the red line represents the extrapolated trend line from a Gartner benchmark database), then it becomes apparent that Company B is actually the more cost efficient, and Company A spends more than other organisations with the same degree of complexity. “Cost difference” Key Principles of Benchmarking

6. Government Trends in IT Spend to Run, Grow and Transform the Business Data Source: Gartner IT Key Metrics Data 2010 IT Spending to Run, Grow, and Transform the Business The distribution of IT spending between run, grow and transform activities has been similar for the last 4 years The proportion of spend across these three categories is very similar for National and Local government

7. What if You Are Outsourced? Determination of whether an overall deal is good requires not just a comparison to some high level spending numbers but also some measures across other dimensions: -Customer Satisfaction…measures here can provide indications of how the provider is doing and whether services meet the needs and priorities of the business. -Relationship Assessment…provides a view on the levels of trust and control in managing the deal. -Cost Assessment…in an open relationship provide insight into whether and how efficiencies in service delivery can be gained; across roles played by both service provider and service recipient -Business Alignment… focussing on innovation and developments is the deal delivering the business improvements expected. Is the deal generating the business outcomes expected? Is the service provider delivering the service outcomes expected, and is pricing still competitive? Is the working relationship healthy? Is the service provider meeting the contractual requirements? Are both operational and management staff happy with the day-to-day and medium-term "outcomes" being generated?

8. Changing the Conversation from IT Cost to IT Value All CIOs must manage a diverse set of stakeholders The toughest stakeholders are those who hold or control the IT budget directly and indirectly Obtaining clarity over which stakeholders maintain this IT budget control is vital IT is expected to play one of 4 specific and different roles in any organization The CIO does not always/often/ ever choose the role IT is expected to play Controlling the conversation about the value of IT, within its chosen role, is critical to the CIO’s survival and the strategic impact IT can bring to the business through innovation

9. The overall performance of the IT organization at XYZ is rated better than the average organization. CIO Scorecard Results at a Glance Better than average performance is found in the areas of Cost Containment and Infrastructure & Operations. Average performance is found in the areas of applications organization and enterprise architecture. An improvement area is found in business process management. The effectiveness and innovation ratings for XYZ are slightly above average. INNOVATION EFFECTIVENESS EFFICIENCY TRANSFORMGROWRUN Business IT Overall (relative to peer) = + -

10. How to Compare Peer groups are determined based upon the role that IT plays in the enterprise The database demonstrates clear differentiation in the average performance in the underlying maturity models for “Team Players” and “Grinders.” XYZ is compared to other “Grinder” organizations based upon current role and “Team Player” based upon aspiration for each CIO Scorecard evaluation component. Evaluation Component XYZGrinder Team Player Infrastructure & Operations 2.832.393.11 Cost Containment 3.442.042.59 Applications Organization 2.682.693.88 Business Process Management 1.401.281.60 Enterprise Architecture 2.202.383.12 Open Innovation Readiness 1.252.853.28 Effectiveness/Innovation Enterprise Viewpoint 3.272.583.84 Effectiveness/Innovation IT Organization Viewpoint 3.562.653.89 *Rating based upon 1 to 5 scoring with 5 as most mature

11. The Major Dimensions of IT Performance There are a number of dimensions in which IS performance can be managed. These dimensions are inter- related and over-focus on one will adversely affect the others -Process -Value for Money -Alignment -Service Availability -Service Quality

12. Service Level Attainment – Sample Output Wintel Availability 98.0%99.0%100%98.5% 99.5% Network Availability 98.0%99.0%100%98.5% 99.5% Voicemail Availability 98.0%99.0%100%98.5% 99.5% MAC Timeframe 4 days2 days0 days3 days 1 day Mainframe Response 4 sec2 sec0 sec3 sec 1 sec Wintel Response 4 sec2 sec0 sec3 sec1 sec Apps. % of Time Plan 250%150%50%200% 100% UNIX Response 4 sec2 sec0 sec3 sec 1 sec Mainframe Availability 98.0% 99.0%100%98.5% 99.5% App. % of Budget Plan 250%150%50%200%100% UNIX Availability Apps. S2 Restore 4 days2 days0 days3 days 1 day Apps. S1 Restore 4 days2 days0 days3 days1 day  XYZ availability and response service level are competitive with peer organisations  End-user-facing service levels, such as help desk first call resolution, tend to fall below average peer performance as well as XYZ targets Telecom. Availability 98.0%99.0%100%98.5% 99.5% Desktop S1 Restore 16 hrs8 hrs0 hrs12 hrs 4 hrs First Call Resolution 0%50%100%25% 75% Desktop S2 Restore 16 hrs8 hrs0 hrs12 hrs 4 hrs Mean Time to Answer 120 sec60 sec0 sec90 sec 30 sec XYZ Actual: XYZ Target: Peer Average: 98.0% 99.0%100%98.5% 99.5%

13. Process Maturity – Sample Output  XYZ exceeds database average maturity levels in two thirds of the assessed disciplines.  IT governance and relationship management are areas of particular strength, suggesting a strong focus on business value.  In contrast, low maturity for asset management and business continuity may indicate risk for higher cost and greater downtime Asset Management Bus. Continuity and DR Capacity Planning Change Management Configuration Management Database Administration Facilities Management Incident Management Inventory Management IT Financial Management Monitoring Performance Management Problem Management Release Management Security Management Service Desk Service Level Management Storage Management Task Management Relationship Management IT Governance 053241 Peer AverageClient XYZ

14. 1531225344 Business Alignment – Sample Output Management assigns the greatest satisfaction to reducing operating cost—a strong XYZ focus in years past An increase in industry competition, however, has shifted focus from cost savings to competitive advantage, including improving time to market and enabling revenue growth Faster Time to Market IMPORTANCE SATISFACTION Competitive Advantage Enable Revenue Growth Reduce Operating Cost Improve Customer Support Reduce Bus. Disruptions Enhance Business Agility Reduce Direct IT Cost

15. IT Service Satisfaction – Sample Output  Consistent with Process maturity findings, line of business managers are concerned with XYZ’s preparedness in terms of disaster recovery  Managers gave Voice and Data communications their highest satisfaction rating—4.44

16. Help Desk Quality

17. Help Desk Productivity

18. Desktop Productivity

19. Notes accompany this presentation. Please select Notes Page view. These materials can be reproduced only with written approval from Gartner. Such approvals must be requested via e-mail: vendor.relations@gartner.com. Gartner is a registered trademark of Gartner, Inc. or its affiliates. Benchmarking IT: The Cost/Quality Balance Chris Smith