1. 1 Performance Modeling and System Management for Multi-Component Online Services Christopher Stewart and Kai Shen University of Rochester

2. 2 “Their [very-large-scale systems] complexity approaches that of an urban community or the economy, where technicians may understand components, but can neither predict nor control the whole system.” CRA Conference on Grand Research Challenges in Information Systems September 2003 Motivational Research Challenge

3. 3 Multi-Component Online Services Typically, deployed in heterogeneous distributed environments. Software architecture composed of many distinct components. These systems are difficult to model and predict. Often, system management is intuitive and heuristic. WS BidUserItem DB Partial Deployment of RUBiS Auction Service Trans 100Mb 1 Gb Datacenter

4. 4 Talk Outline 1. Overview. Contributions. Approach and Sub Goals. 2. Key application characteristics. 3. Models to predict whole-system performance. 4. Model-driven system management. 5. Related Work and Conclusion.

5. 5 Modeling Multi-Component Systems Modeling single component systems  Application characteristics, workload (req./sec), system management policies, and available resources. Modeling multi-component systems  Per-component resource demands,  Complex inter-component interactions,  Component placement and replication. We identify key component-level and inter- component application characteristics. We use these characteristics to model system performance and guide system management.

6. 6 Our Approach An “application profile” describes component-level and inter- component behaviors. Sub Goals: Create the application profile offline. Transparent to application and component middleware. Predicted Throughput and Response Time Application Application Profile Performance Modeling Component Placement Available Resources & Input Workload

7. 7 Talk Outline 1. Introduction 2. Key application characteristics. Per-component resource needs. Inter-component communication. o Overhead of remote invocations. 3. Models to predict whole-system performance. 4. Model-driven system management. 5. Related Work and Conclusion.

8. 8 Per-component Resource Usage For Each Component Predict the amount of resource required.  θ CPU, θ DiskIO, θ NetIO. Parameters: Workload and Workload Traits.  λ Workload = Rate (Requests per Second).  δ Workload =. Component resource usage profile is f():  f (λ Workload,δ Workload ) →.

9. 9 We extrapolate average resource needs from offline profiling results. We use functional fitting. Application: RUBiS Setting:  2.66 GHZ P4, 512 MB.  Linux, JBOSS 3.23. Per-component Usage Results 0 2 4 6 8 10 12 external request rate Bid Bid Fitting Web Server Web Server Fit CPU Required 0% 2% 4% 6% 8% 10%

10. 10 Definition: Roundtrip  A single synchronous communication between two components. Roundtrips increase response time.  Delay = Roundtrips * Latency. Roundtrips are difficult to identify without application knowledge. WS ABC DB Sample Application Inter-Component Communication

11. 11 Identifying Inter-component Roundtrips Algorithm 1.Transparently capture communications at the system call level. 2.Inter-component write/read pairs comprise a roundtrip if The read occurs after a write. Difference between timestamps exceeds network delay time(150μ). 2 Roundtrip Communications

12. 12 Inter-component Roundtrips Per Request 1.84 0.91 Categ. Web Server Database 7.74 0.72 1.39 7.43 5.32 0.76 6.54 4.00 0.23 8.90 0.10 0.43 0 0 0.31 0.23 1.21 0.46 3.34 Region Buy Bid Trans. User Comment Query Item RUBiS Read-only Request Mix

13. 13 Talk Outline 1. Introduction 2. Key application characteristics. 3. Models to predict whole-system performance. Throughput Response Time 4. Model-driven system management. 5. Related Work and Conclusion.

14. 14 Performance Models Predicted Throughput and Response Time Application Component Profiles Remote Invocation Profile Communication Profile Performance Modeling Component Placement Available Resources & Input Workload  We identify key component-level and inter- component characteristics.  We use these characteristics to model system performance and guide system management.

15. 15 Throughput Prediction The system throughput is the lowest workload at which an available resource at any distributed node is exhausted. Predicting saturation workload of a single node. 1. Acquire expected resource consumption per request.  Sum the expected resource usage of the individual components. 2. Add remote invocation overhead. 3. Find the saturation workload by solving constrained equations:  For each resource X: θ x (λ Workload ) < x available

16. 16 10 RT Simple Prediction Example workload = 4 req./sec. A cpu=20% B cpu=10% Throughput is 5 req./sec. Node 1 Node 2 node1 resp =.2 +.2 * 4/5 * 5 node1 resp = 1 sec. node2 resp =.166 sec. Network Delay = 10 X 150μ Response Time = 1.168 sec. Assumptions  Poisson request inter- arrival time. No automatically triggered requests.  Any request execution time distribution. Obtained during profiling. Response Time Prediction Queuing model: M/G/1 Network Delay  Roundtrips affect the average response time.  Data volume delay is also considered. Response Time is: Queuing time + Network delay.

17. 17 Validation Setup Application: RUBiS  Composed of Web Server, 9 EJB Components, and a back-end database.  Component Container: JBOSS 3.2.3  Database servers run MYSQL 4.0 Setting:  12 Node Cluster 8 2GHz Xeon processors, 4 2.66 P4 processors 2GB, 512 MB memory  1 Gigabit Ethernet

18. 18 Throughput Model Validation Base model RI model Full model Measurements Input workload (requests/second) Throughput (requests/second) 0 50 100 150 200 250 50 100 150 200 250  Base model overestimates performance by 15%.  The full model is accurate within 1%.  The effect of implementation artifacts (i.e. Remote Invocation) may be profound.

19. 19 Response Time Model Validation RI model Full model Measurements Input workload (in proportion to the saturation throughput) Average response time (millis) Base model 20 60 100 140 180 50% 60% 70% 80% 90%  Full model improves accuracy by 18% compared to RI model.  Full model predicts response time within 9% at all workload levels.

20. 20 Talk Outline 1. Introduction 2. Key application characteristics. 3. Models to predict whole-system performance. 4. Model-driven system management. Performance optimization techniques based upon accurate performance models. 5. Related Work and Conclusion.

21. 21 System Management Current, system management techniques are imprecise.  Inadequate system models.  Administrators may provide generic heurists/tips. We use accurate model-based predictions to guide system management.  High-performance component placement.  Capacity planning / Cost-effectiveness analysis.

22. 22 High Performance Component Placement Goal: Discover a good component placement strategy. 2 Intuitive approaches.  All replication, Low replication. 2 Model-based approaches.  Simulated annealing, Sampling. Model based system management outperforms intuitive approaches. Simulated annealing Random sampling All replication Low replication Input workload (requests/second) Average response time 100 140 180 220 260 0 100 200 300 400

23. 23 Related Work Application profiling.  Amza et al., 2002.  Barham et al., 2005 (Magpie). Performance modeling.  Doyle et al., 2003.  Urgaonkar et al., 2005. Distributed component placement.  Hunt and Scott, 1999 (Coign).  Amiri et al., 2000 (ABACUS).  Ivan et al., 2002.

24. 24 Take Away Points We transparently identify and profile key application characteristics. Using application profiles, we accurately model system performance. We demonstrate the effectiveness of model- based system management. www.cs.rochester.edu/~stewart/component.html

25. 25

26. 26 Extra Slides Someone asked a good question.

27. 27 Detailed Operating System Level Profiles We modified the LTT (Yaghmour). LTT instruments the Linux Kernel with trace points and forwards them to a logging daemon. We added trace points at the CPU context switch, the socket level interface, and disk IO. Our approach is still applicable with a different OS and less accurate measurement techniques.

28. 28 Additional Profiling Issues Nonlinear functional mappings  Occurs if requests have strong inter-dependencies.  e.g. CPU contention on a spin lock  The model is still applicable. May require more data points to acquire a decent approximation. Worst case scenario: Table based mappings [E. Anderson]

29. 29 Profiling Cost Given an N-component service. Non-concurrent profiling (i.e. one component at a time)  O(N 2 )  N = 11, Approx. 10 hours. Concurrent Profiling  Requires larger cluster.  O(N)  N=11, Approx. 1 ½ hours.