1. Performance Engineering Methodology Chapter 4

2. Performance Engineering Performance engineering analyzes the expected performance characteristics of a system during the different phases of its lifecycle. Performance engineering – 1) develops practical strategies that help predict the level of performance a system can achieve and – 2) provides recommendations to realize the optimal performance level.

3. Typical PE Questions Can the insurance claim system meet its performance requirements of sub-second response time when a natural disaster occurs (e.g., a hurricane). Response Time Is the infrastructure of a government agency scalable and can it cope with the computing demands of the new required online security mechanisms? Scalability Is the reservation system for cruise lines able to respond to anticipated peak of customer inquiries after a TV ad campaign? Reliability

4. PE Larger Questions How can one plan, design, develop, deploy, and operate IT services that meet ever increasing demands for performance, availability, reliability, and security? Is a given IT system properly designed and sized for a given load condition?

5. PE Activities Understand the key factors that affect a system’s performance. Measure the system and understand its workload. Develop and validate a workload model that captures the key characteristics of the actual workload. Develop and validate an analytic model that accurately predicts the system’s performance. Use the models to predict and optimize the system’s performance.

6. Modeling Process

7. Motivating Example: a Call Center

8. Call Center Goals: – Foster better relationships with customers, creating customer loyalty and ensuring quality service. – Improve efficiency and service performance. – Identify and explore new sales opportunities. Main Functions: – Order status inquiry – Shipment tracking – Problem resolution status inquiry Requirements: sub-second response time and 24x7 operation.

9. QoS Questions Is the system design able to meet the subsecond response time for all functions? Response Time What will be the impact of doubling the number of system representatives in the next year? Scalability Can acceptable performance levels be maintained after integrating the system with the mainframe-based inventory application? Scalability Is the system capacity adequate to handle up to 1,000 calls in the busiest hour and yet preserve the subsecond response time goal? How do failures in the database server affect the 24x7 availability goal? What is the impact of starting to offer Web-based self-service to customers?

10. At the Requirements Analysis Phase Workload definition: – Call center’s view: Arrival rate of phone calls – IT system’s view: Functions received from the representatives. – DB server view: SQL requests from the application server. – LAN view: packet size distribution and interpacket arrival time.

11. At the System Design Phase What should the system throughput be to meet sub-second response times? – 200 customer service representatives and 80% are working during the peak hour. – Average think time of 30 sec. Model of the call center system

12. Call Center Model Using the interactive response time law: Z: average think time, 30sec N: number of active representatives in the system, 200X80% = 160 X0: system throughput R: average response time < 1sec

13. At the System Development Phase What should be the capacity of the DB server so that the performance goals are met? – Each submitted functions requires 2.2 SQL calls on average. – From the Forced Flow Law:

14. At the Operation Phase Assume DB server is a problem. Response times exceed sub-second goal. Measurements during peak hour: – 57600 queries/hour – Each query needs 50 msec of CPU, performs 4 I/Os on disk 1 and 2 I/Os on disk 2. Each I/O takes 8 msec on average. – X0 = 57600 / 3600 = 16 queries/sec – Service demands: Dcpu = 0.05 sec; Ddisk1 = 4 x 0.008 = 0.032 sec; Ddisk2 = 2 x 0.008 = 0.016 sec.

15. At the Operation Phase (cont’d)

16. The residence times at the CPU and disks for open QN model Response time of the DB server: R DB =

17. At the Evolution Phase The company is considering to develop Web applications to allow customers access to the information they need without assistance from a customer representative. Web self-services reduce transaction costs and enhance the customer experience. Security requirements mandate that new applications be developed for Web access (authentication, auditing, DB access control mechanisms).

18. At the Evolution Phase Local queries and web queries:

20. Results for Evolution Scenario Open Multiclass Queuing Networks - Utilizations This wokbook comes with the books "Performance by Design," "Capacity Planning for Web Services," and "Scaling for E-Business" by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 2004, 2002 and 2000. Classes ® Queues ¯12Total cpu10.800000.150000.95000 d120.512000.200000.71200 d230.256000.100000.35600

21. Results for Evolution Scenario Open Multiclass Queuing Networks - Queue Lengths This wokbook comes with the books "Performance by Design," "Capacity Planning for Web Services," and "Scaling for E-Business" by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 2004, 2002 and 2000. Classes ® Queues ¯12Total cpu116.000003.0000019.00000 d121.777780.694442.47222 d230.397520.155280.55280

23. Performance Engineering Methodogy