1. Ó 1998 Menascé & Almeida. All Rights Reserved.1 Part V Workload Characterization for the Web (Book, chap. 6)

2. Ó 1998 Menascé & Almeida. All Rights Reserved.2 Configuration Plan Investment Plan Personnel Plan Understanding the Environment Workload Characterization Workload Model Validation and Calibration Workload Forecasting Performance Prediction Cost Prediction Valid Model Cost Model Developing a Cost Model Performance Model Cost/Performance Analysis

3. Ó 1998 Menascé & Almeida. All Rights Reserved.3 Learning Objectives (1) Introduce the workload characterization problem. Discuss a simple example of characterizing the workload for an intranet. Present a workload characterization methodology.

4. Ó 1998 Menascé & Almeida. All Rights Reserved.4 Learning Objectives (2) Discuss the following steps: – analysis standpoint – identification of the basic component – choice of the characterizing parameters – data collection – partitioning the workload Characteristics of Web workloads: – burstiness

5. Ó 1998 Menascé & Almeida. All Rights Reserved.5 What is Workload Characterization?

6. Ó 1998 Menascé & Almeida. All Rights Reserved.6 Workload The workload of a system can be defined as the set of all inputs that the system receives from its environment during any given period of time. HTTP requests Web Server

7. Ó 1998 Menascé & Almeida. All Rights Reserved.7 Workload Characterization Depends on the purpose of the study –cost x benefit of a proxy caching server –impact of a faster CPU on the response time Common steps –specification of a point of view from which the workload will be analyzed –choice of set of relevant parameters –monitoring the system -> raw performance data –analysis and reduction of performance data –construction of a workload model.

8. Ó 1998 Menascé & Almeida. All Rights Reserved.8 A Simple Example A construction and engineering company is planning to roll out new applications and to increase the number of employees that have access to the corporate intranet. The main applications are health human resources, insurance payments, on-demand interactive training, etc. Main problem: response time of the human resource system

9. Ó 1998 Menascé & Almeida. All Rights Reserved.9 A Simple Example (2) A D E C B Clients Servers Network...

10. Ó 1998 Menascé & Almeida. All Rights Reserved.10 A Simple Example: basic questions What is the purpose of the study? What workload we want to characterize? –Client’s point-of-view: user commands, server responses –server p.o.v: HTTP requests –network p.o.v.: traffic (packet size distribution), inter-packet arrival time What is the level of the workload description? –High-level characterization in terms of Web applications ; –Low-level characterization in terms of resource usage. How could this workload be precisely described?

11. Ó 1998 Menascé & Almeida. All Rights Reserved.11 Workload Characterization: concepts and ideas Basic component of a workload: refers to a generic unit of work that arrives at the system from external sources. –DB transaction, –interactive command, –process, –HTTP request Depends on the nature of service provided

12. Ó 1998 Menascé & Almeida. All Rights Reserved.12 Workload Characterization: concepts and ideas Workload characterization process –analyzes workload and identifies the basic components/features having impact on system’s performance –yields parameters that retain characteristics capable of driving performance models –workload model is a representation that mimics the workload under study Workload models can be used for: –selection of systems –performance tuning –capacity planning

13. Ó 1998 Menascé & Almeida. All Rights Reserved.13 Workload Description Hardware Software User Resource-oriented Description Functional Description Business Description

14. Ó 1998 Menascé & Almeida. All Rights Reserved.14 Workload Description Business characterization: a user-oriented description that describes the load in terms such as number of employees, invoices per customer, etc. Functional characterization: describes programs, commands and requests that make up the workload Resource-oriented characterization: describes the consumption of system resources by the workload, such as processor time, disk operations, memory, etc.

15. Ó 1998 Menascé & Almeida. All Rights Reserved.15 A Web Server Example The pair (CPU time, I/O time) characterizes the execution of a request at the server. Our basic workload: 10 HTTP requests First case: only one document size (15KB) 10 executions ---> (0.013 sec, 0.09 sec) More realistic workload: documents have different sizes.

16. Ó 1998 Menascé & Almeida. All Rights Reserved.16 Execution of HTTP Requests (sec)

17. Ó 1998 Menascé & Almeida. All Rights Reserved.17 Representativeness of a Workload Model System Performance Measures P real System Performance Measures P model Workload Model Real Workload

18. Ó 1998 Menascé & Almeida. All Rights Reserved.18 A Refinement in the Workload Model The average response time of 0.55 sec does not reflect the behavior of the actual server. Due to the heterogeneity of the its components, it is difficult to view the workload as a single collection of requests. Three classes: –small documents (0 < CPU <= 0.01, 0 < I/O <= 0.05) –medium doc.s (0.01 < CPU <= 0.03, 0.05 < I/O <= 0.14) –large doc.s (CPU > 0.3, I/O > 0.14)

19. Ó 1998 Menascé & Almeida. All Rights Reserved.19 Execution of HTTP Requests (sec)

20. Ó 1998 Menascé & Almeida. All Rights Reserved.20 Three-Class Characterization

21. Ó 1998 Menascé & Almeida. All Rights Reserved.21 Class Characterization A class comprises components that are similar to each other concerning resource usage Clustering the workload into classes increases the predictive power of a model

22. Ó 1998 Menascé & Almeida. All Rights Reserved.22 Completing the characterization Two major points: –Class characterization: statistical description of each class (e.g. CPU, I/O) number of components of each class –Requests arrival rate: depends on number of users generating requests think time (how often a user interacts with the server)

23. Ó 1998 Menascé & Almeida. All Rights Reserved.23 Workload Models A model should be representative and compact. Natural models are constructed either using basic components of the real workload or using traces of the execution of real workload. Artificial models do not use any basic component of the real workload. –Executable models (e.g.: synthetic programs, artificial benchmarks, etc) - not adequate for performance models –Non-executable models, that are described by a set of parameter values that reproduce the same resource usage of the real workload.

24. Ó 1998 Menascé & Almeida. All Rights Reserved.24 Workload Models The basic inputs to analytical models are parameters that describe the service centers (i.e., hardware and software resources) and the customers (e.g. requests and transactions) Typical parameters: –component (e.g., transactions) inter-arrival times; –service demands –component sizes –execution mix (e.g., levels of multiprogramming)

25. Ó 1998 Menascé & Almeida. All Rights Reserved.25 A Workload Characterization Methodology Choice of an analysis standpoint Identification of the basic component Choice of the characterizing parameters Data collection Partitioning the workload Calculating the class parameters

26. Ó 1998 Menascé & Almeida. All Rights Reserved.26 Selection of characterizing parameters Each workload component is characterized by two groups of information: Workload intensity –arrival rate –number of clients and think time –number of processes or threads in execution simultaneously Service demands (D i1, D i2, … D iK ), where D ij is the service demand of component i at resource j.

27. Ó 1998 Menascé & Almeida. All Rights Reserved.27 Data Collection This step assigns values to each component of the model. –Identify the time windows that define the measurement sessions. –Monitor and measure the system activities during the defined time windows. –From the collected data, assign values to each characterizing parameters of every component of the workload.

28. Ó 1998 Menascé & Almeida. All Rights Reserved.28 Partitioning the workload Motivation: real workloads can be viewed as a collection of heterogeneous components. Partitioning techniques divide the workload into a series of classes such that their populations are composed of quite homogeneous components. What attributes can be used for partitioning a workload into classes of similar components?

29. Ó 1998 Menascé & Almeida. All Rights Reserved.29 Partitioning the Workload Resource usage Applications Objects Geographical orientation Functional Organizational units Mode: interactive, transaction, batch

30. Ó 1998 Menascé & Almeida. All Rights Reserved.30 Workload Partitioning: Resource Usage

31. Ó 1998 Menascé & Almeida. All Rights Reserved.31 Workload Partitioning: Internet Applications

32. Ó 1998 Menascé & Almeida. All Rights Reserved.32 Workload Partitioning: Document Types

33. Ó 1998 Menascé & Almeida. All Rights Reserved.33 Workload Partitioning: Geographical Orientation

34. Ó 1998 Menascé & Almeida. All Rights Reserved.34 Modes Transaction: –workload characterized by arrival rate, –when a client has completed the service, it leaves the system –the number of clients in the system (population) varies extensively with time Interactive: –workload characterized by population (i.e. number of active client workstations or terminals) and by think time

35. Ó 1998 Menascé & Almeida. All Rights Reserved.35 Modes Batch: –workload characterized by population, which is fixed –when a client has completed the service, it leaves the system, but it is replaced by a new client (the output is short-circuited with input)

36. Ó 1998 Menascé & Almeida. All Rights Reserved.36 Calculating the class parameters How should one calculate the parameter values that represent a class of components? –Averaging: when a class consists of homogeneous components concerning service demands, an average of the parameter values of all components may be used. –Clustering of workloads is a process in which a large number of components are grouped into clusters of similar components.

37. Ó 1998 Menascé & Almeida. All Rights Reserved.37 Clustering Analysis

38. Ó 1998 Menascé & Almeida. All Rights Reserved.38 Clustering Analysis Software packages: SAS, SPSS, etc. A clustering algorithm attempts to find natural groups of components based on similar resource requirements Output of clustering algorithm: for each cluster, statistical description of centroid, + number of components -- this is the characterization of the associated class

39. Ó 1998 Menascé & Almeida. All Rights Reserved.39 Clustering Analysis: required steps Data analysis –sampling from real data –choice of logarithmic or other scale –discarding outliers (with care!) –trimming data to 95. or 98. percentile: D i t = (meas. D i - min.{D i }) / (max.{D i } - min{D i }) Distance measures –euclidean or other metric

40. Ó 1998 Menascé & Almeida. All Rights Reserved.40 Clustering Analysis: required steps Scaling –apply z score transform to use only adimensional values: z score = (meas. val - mean val) / standard dev Clustering algorithms –minimal spanning tree (hierarchical) –k-means algorithm (non-hierarchical) N.B. - the number of clusters must be kept small

41. Ó 1998 Menascé & Almeida. All Rights Reserved.41 New Phenomena in the Internet and WWW Self-similarity - a self-similar process looks bursty across several time scales. Heavy-tailed distributions in workload characteristics, that means a very large variability in the values of the workload parameters.

42. Ó 1998 Menascé & Almeida. All Rights Reserved.42 WWW Traffic Burst 10 6 10 7 Bytes Chronological time (slots of 1000 sec)

43. Ó 1998 Menascé & Almeida. All Rights Reserved.43 Incorporating New Phenomena in the Workload Characterization Burstiness Modeling burstiness in a given period can be represented by a pair of parameters (a,b) –a is the ratio between the maximum observed request rate and the average request rate during the period. –b is the fraction of time during which the instantaneous arrival rate exceeds the average arrival rate. (a = 6, b = 5%) => Web server throughput degraded by 12 to 20%

44. Ó 1998 Menascé & Almeida. All Rights Reserved.44 Part V: Summary Workload Characterization what is it? basic concepts workload description and modeling representativeness of a workload model Methodology (1) Choice of an analysis standpoint Identification of the basic component Choice of the characterizing parameters Data collection

45. Ó 1998 Menascé & Almeida. All Rights Reserved.45 Part V: Summary Methodology (2) Partitioning the workload Calculating the class parameters Averaging Clustering techniques and algorithms Taking burstiness into account