Compuware Corporation E2E Performance Monitoring to the Mth Tier (Mainframe Integrated) The New Industry Standard via the Apdex Alliance SCCMG November 2, 2007 Thomas A. Halinski Yuri Grinshteyn

Compuware Corporation End-to-End Performance Monitoring to the Mth Tier (Mainframe Integrated) – The New Industry Standard via the Apdex Alliance How many reports do you use to determine if your end to end systems (including the Mainframe) needs tuning? Would it help if you could convert many measurements into one number? Are you sure you are looking at application performance from the same perspective as your End-User (i.e. user satisfaction with enterprise applications)? Is your enterprise interested in an integrated approach to monitoring, identifying and tuning the end user E2E transactions? Here’s what the Apdex Alliance and the Industry are now suggesting and a step beyond with “dashboards” that are both for IT and Business Management.

Compuware Corporation Page 3 Defining Service Management and the Apdex Standard Service Management Problem Apdex Approach Performance Measurement Problem Industry Guideline/A Better Mousetrap The Cost of Poor Application Quality © NetForecast, Inc., All rights reserved.

Compuware Corporation Page 4 The IT Value Chain Half of enterprises are – providing poor performance or do not know how well they are serving their users (business success) NetForecast/BCR survey – Half of enterprises are postponing launching new applications due to performance concerns (curtailing business) Network World survey Which Half are you? Performance measured by success of the business, i.e. The User Experience Users Business Infrastructure Business Users Infrastructure

Compuware Corporation Page 5 The 80/20 Rules Have Flipped Old 80/20 Rules – 80% of your users are in your primary offices – 80% of your traffic is inside your network – Therefore, if you deliver good service to the 80% you know, then you are well ahead of the game New 80/20 Rules – 80% of the users are outside your primary offices – 73% of application service problems are reported by end users, not by the IT department Forrester Research – 82% of enterprises say that poor performance is impairing employee productivity Network World survey The 80/20 Rules Have Flipped

Compuware Corporation Page 6 * The Apdex Alliance is a group of companies collaborating to promote an application performance metric called Apdex. Apdex is a numerical measure of user satisfaction with the performance of enterprise applications, and reflects the effectiveness of IT investments in contributing to business objectives. See for details. An “end-to-end performance monitoring” view of an enterprise, is based on the industry standard from the “End User Perspective”, as proposed by the Apdex Alliance*. This includes the “application” perspective. It means that “Performance Is The User Experience” - deterministic of “Business Value” and bottom line ($). E2E Performance Monitoring End User Perspective - Definition

Compuware Corporation Page 7 Today’s Problem: Many Numbers, Little Insight Measured Response Time (seconds) App AApp BApp CApp DApp E Day Average Best Hour Worst Hour th Percentile Which application is in trouble?

Compuware Corporation Page 8 Example: 100 Numbers Start with what you have – Your measurement tool produced 100 samples The samples are – Single application – User-level response time measurements – One hour period of observation Is the application operating well?

Compuware Corporation Page 9 Numbers Beget Numbers Incremental Time Period (sec) Number of Samples in Time Period sec Average 7.6 Median 3.9 Mode 4.8 Standard Deviation th Percentile22.5 Minimum 1.3 Maximum 59.6 Now you have 137 numbers. Can you answer the question, “Is the application operating well?” 2

Compuware Corporation Page 10 Defining Performance and the Apdex Standard Service Management Problem Apdex Approach Performance Measurement Problem Industry Guideline/A Better Mousetrap The Cost of Poor Application Quality © NetForecast, Inc., All rights reserved.

Compuware Corporation Page 11 Apdex Defined Apdex is a numerical measure of user satisfaction with the performance of enterprise applications It defines a method that converts many measurements into one number – Uniform 0-1 scale, 0 = no users satisfied, 1 = all users satisfied Standardized method – It is a comparable metric across all applications, and – Across enterprises

Compuware Corporation Page 12 Deconstructing Application Transactions Session = Period of time that a user is “connected” to an application Start the application End or suspend the application Task = Each interaction with the application during the session Type or choose Enter or click System responds User waits User reads or thinks Type Wait Read Enter or click System responds Process = A group of user interactions that accomplish a goal Get new , add an employee, check on inventory status, etc. Idle

Compuware Corporation Page 13 The Task Defined Task response time is the elapsed time required for an application system to respond to a human user input such that the user can effectively proceed with the process they are trying to accomplish – Time when the user is waiting in order to proceed – User feels the responsiveness of the application – Long Task time makes the user less productive The Task is what a user can time with a stopwatch

Compuware Corporation Page 14 How Users View Application Task Performance Satisfied – User maintains concentration – Performance is not a factor in the user experience – Time limit threshold is unknowingly set by users and is consistent Tolerating – Concentration is impaired – Performance is now a factor in the user experience – User will notice how long it is taking Frustrated – Performance is typically called unacceptable – Casual user may abandon the process – Production user is very likely to stop working

Compuware Corporation Page 15 How Apdex Works Start with a sufficient number of Task measurement samples Target response time “T” defines the satisfied zone (0-T sec) – T is shown as a subscript of all Apdex values (for example 0.80 T ) Count the number of samples within three performance zones – Satisfied, Tolerating, Frustrated Tolerating count Apdex T = Total samples Satisfied count 2 + Given Target response time T and Sufficient response time measurement samples Then Note Frustrated samples are not in numerator but are counted in total samples Index 0 = Failure; 1 = Perfection (all users satisfied)

Compuware Corporation Page 16 Putting it All Together Tolerating Apdex T = Total samples Satisfied 2 + Frustrated Satisfied Tolerating Good Fair Poor Unacceptable 0.00 T 0.50 T 1.00 T 0.85 T 0.94 T 0.70 T Excellent Report Group: Application User Group Time Period Existing Task Response Time Measurement Samples T F 1. Define T for the application T = the application target time (threshold between satisfied and tolerating users). F = threshold between tolerating and frustrated users is calculated (F = 4T). 2. Define a Report Group (details available are tool dependent). 3. Extract data set from existing measurements for Report Group. 4. Count the number of samples in three performance zones. 5. Calculate the Apdex formula. 6. Display Apdex result (T is always shown as part of the result).

Compuware Corporation Page 17 The Apdex View of the 100 Numbers User productivity is impaired if the application responds in more than 8 seconds –T = 8 sec Apdex for the 100 measurements = –The application barely providing “Good” performance 100 numbers = Good Fair Poor Unacceptable 0.00 T 0.50 T 1.00 T 0.85 T 0.94 T 0.70 T Excellent

Compuware Corporation Page 18 Defining Performance and the Apdex Standard Service Management Problem Apdex Approach Performance Measurement Problem Industry Guideline/A Better Mousetrap The Cost of Poor Application Quality © NetForecast, Inc., All rights reserved.

Compuware Corporation Proper performance management is delivered in two steps: 1.) Ongoing end-user performance measurement, reporting and tracking. 2.) Flow-based performance measurements and diagnostics to identify which silo the issue resides in.

Compuware Corporation Page 20 Service Monitoring Web Servers or Fat Client Application Servers Mainframe / Database Servers Response Times Via End User Perspective (Clients) Mth Tier - APM Mainframe monitoring can be triggered at the application code level by watching SLAs set on the Java or.NET network methods which are used to communicate to the Mainframe Only available with “Integrated” E2E Performance Monitoring Response Times From the application layer (Server)

Compuware Corporation Page 21 APPLICATION SERVICE MANAGEMENT Monitoring Individual Tiers ©2006, NetForecast, Inc. and Apdex Alliance. All rights reserved.

Compuware Corporation Page 22 Desktop Windows WAN UNIX Mainframe DBMS/ Mainframe SAN Report Aggregation Report Report Aggregation Typical Silo Management ©2006, NetForecast, Inc. and Apdex Alliance. All rights reserved.

Compuware Corporation Page 23 Report First Reason Why Silo Performance Does Not Equal User Performance * Performance between “silos” is missing when Monitoring is Non-Integrated ****** Desktop Windows WAN UNIX Mainframe DBMS/ Mainframe SAN

Compuware Corporation Page 24 Assumed Path of a User’s Task Actual Path of a User’s Task Second Reason Why Silo Performance Does Not Equal User Performance Some Tier Performance Tools miss/aggregate Actual Data ©2006, NetForecast, Inc. and Apdex Alliance. All rights reserved. Desktop Windows WAN UNIX Mainframe DBMS/ Mainframe SAN

Compuware Corporation Page 25 Defining Performance and the Apdex Standard Service Management Problem Apdex Approach Performance Measurement Problem Industry Guideline/A Better Mousetrap The Cost of Poor Application Quality © NetForecast, Inc., All rights reserved.

Compuware Corporation Page 26 Client Clients Applications Transaction Types Transaction Response Data Correlation Business Model Applications Locations Business Transactions Users Network Session Response Traffic\Utilization Latency IP End Points WAN Links Server Servers Counter Data WMI Methods SQL Calls Mainframe Faults Measurement Data Device WMI Counter Data SNMP Vantage E2E Integrated Performance Analysis Data Correlation E2E Integrated Solution

Compuware Corporation Page 27 End User Monitoring Approaches Agentless – Network attached appliance – Monitor all transactions from all users Active – Dedicated workstation that executes synthetic transactions – Flexible scripting Passive – End user workstation agent End User experience monitoring

Compuware Corporation Page 28 Integrated with end user monitoring Focus on the application – Web server tier – Application server tier J2EE &.NET – Heap, CPU, connection pools – Long running and CPU intensive methods, transactions, and database calls – Pinpoint source of memory leaks – Database server tier Hundreds of Oracle, SQL Server, DB2 metrics – Performance of individual SQL calls Agent-based and agentless implementation approaches Server Monitoring Server monitoring

Compuware Corporation Page 29 APPLICATION CHARACTERIZAITON Quickly identify transactions that are having problems and what time the problem occurred

Compuware Corporation Page 30 END TO END VISIBILITY Real time status and usage of applications, systems and network application performance

Compuware Corporation Page 31 Integrated E2E for the Mth Tier - Mainframe Monitoring Software Monitor application performance Identify excessive resource consumption (CPU & Wait) Resolve excessive resource consumption (CPU & Wait) Improve “End User” transactions and reduce costs

Compuware Corporation Page 32 Network monitoring Transaction profiling tools allow you to see application calls from a Windows or UNIX server to the mainframe. If the call exceeds the threshold, it triggers a mainframe measurement for that alert.

Compuware Corporation Page 33 CPU10.53 sec Mth tier monitoring

Compuware Corporation Page 34 Mth tier monitoring – identify excessive CPU usage

Compuware Corporation Page 35 Mth tier monitoring – resolve excessive CPU usage “…using an INSPECT verb in conjunction with a reserved word, such as SPACES or LOW-VALUES, causes an exit from the user application program. Changing the word SPACES to [an actual space] eliminates the CPU time…”

Compuware Corporation Page 36 Defining Performance and the Apdex Standard Service Management Problem Apdex Approach Performance Measurement Problem Industry Guideline/A Better Mousetrap The Cost of Poor Application Quality © NetForecast, Inc., All rights reserved.

Compuware Corporation Page 37 Service Management Overview Maps business services to IT infrastructure; determines root-cause and business impact in real time Communicates service compliance status via dashboards and reports ITIL, Six Sigma best practice support Acquires data from any technical and business sources Monetize the value of IT components

Compuware Corporation Page 38 Real-Time Service Visualization

Compuware Corporation Page 39

Compuware Corporation Page 40

Compuware Corporation Page 41 Questions