1. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science An Analytical Model for Multi-tier Internet Services and its Applications Bhuvan Urgaonkar, Giovanni Pacifici, Prashant Shenoy, Mike Spreitzer, Asser Tantawi University of Massachusetts and IBM TJ Watson

2. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 2 Internet Applications  Proliferation of Internet applications auction siteonline gameonline store  Growing significance in personal, business affairs  Focus: Modeling Internet applications

3. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 3 Why Model Internet Applications?  Capacity provisioning  How many servers does the application need?  Performance prediction  E.g., predict response time  Application configuration  Tune various application parameters  Request policing  Turn away excess requests during overloads

4. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 4 Internet Application Architecture  Multi-tier architecture  Each tier uses services provided by its successor  Session-based workloads  Caching, replication HTTPJ2EEDatabase request processing in an online bookstore search “moby” queries response Melville’s ‘Moby Dick’ Music CDs by Moby

5. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 5 Existing Application Models  Models for Web servers [Chandra03, Doyle03]  Do not model Java server, database etc.  Black-box models [Kamra04, Ranjan02]  Unaware of bottleneck tier  Extensions of single-tier models [Welsh03]  Fail to capture interactions between tiers  Existing models inadequate for multi-tier Internet applications

6. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 6 Talk Outline Motivation Application Model  Evaluation of the Model  Dynamic Capacity Provisioning  Summary and Future Research

7. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 7 Baseline Application Model  Model consists of two components  Sub-system to capture behavior of clients  Sub-system to capture request processing inside the application clientsapplication

8. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 8 Modeling Clients  Clients think between successive requests  Infinite server system to capture think time Z  Captures independence of Z from processing in application Client 1 Client 2 Client N Z Z Z Q0Q0 applicationclients

9. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 9 Modeling Request Processing Q1Q1 Q2Q2 QMQM tier 1tier 2tier M p M =1p3p3 p1p1 p2p2 S1S1 S2S2 SMSM  Transitions defined to capture circulation of requests  Request may move to next queue or previous queue  Multiple requests are processed concurrently at tiers  Processor sharing scheduling discipline  Caching effects get captured implicitly! N

10. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 10 Putting It All Together Q0Q0 Q1Q1 Q2Q2 QMQM p M =1p3p3 p1p1 p2p2 Z Z S1S1 S2S2 SMSM N  A closed-queuing model that captures a given number of simultaneous sessions being served tier 1tier 2tier M client

11. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 11 Model Solution and Parameter Estimation  Mean Value Analysis (MVA) Algorithm  Computes mean response time  Visit ratios  Equivalent to trans. probs. for MVA  V i ≈ λ i / λ req ; λ req at policer, λ i from logs  Service times  Use residence time X i logged at tier i  For last tier, S M ≈ X M  S i = X i – ( V i+1 / V i ) · X i+1  Think time  Measured at the entry point of application

12. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 12 Talk Outline Motivation Application Model Evaluation of the Model  Dynamic Capacity Provisioning  Summary and Future Research

13. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 13 Evaluation of Baseline Model  Auction site RUBiS  One server per tier ApacheJBOSS Mysql  Concurrency limits not captured 150 75

14. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 14 Q0Q0 Q1Q1 Q2Q2 QMQM Z Z S1S1 S2S2 SMSM N  Requests may be dropped due to concurrency limits  Need to model the finiteness of queues! Handling Concurrency Limits dropped requests

15. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 15 QMQM p1p1 pMpM S1S1 SMSM Q0Q0 Q1Q1 Q2Q2 QMQM Z Z S1S1 S2S2 SMSM N  Approach: Subsystems to capture dropped requests  Distinguish the processing of dropped requests Handling Concurrency Limits drop Q1Q1

16. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 16  Enhanced model can capture concurrency limits Response Time Prediction

17. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 17 Query Caching at the Database  Caching effects  Captured by tuning V i and/or S i  Bulletin-board site RUBBoS  50 sessions  SELECT SQL_NO_CACHE causes Mysql to not cache the response to a query  More model enhancements  Replication at tiers  Multiple session classes

18. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 18 Prototype Data Center  40+ Linux servers  Gigabit switches  Multi-tier applications  Auction (RUBiS)  Bulletin-board (RUBBoS)  Apache, JBOSS (replicable)  Mysql database

19. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 19 Dynamic Capacity Provisioning WorkloadResponse time Server allocations  Auction application RUBiS  Factor of 4 increase in 30 min  Server allocations increased to match increased workload  Response time kept below 2 seconds

20. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 20 Talk Outline Motivation Baseline Application Model Evaluation of the Model Dynamic Capacity Provisioning Summary and Future Research

21. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 21 Summary and Future Work  Analytical model for Multi-tier Internet Applications  Mean-value analysis  Concurrency limits, replication, caching, multiple classes  Model validation using 3-tier applications  Dynamic provisioning, request policing  Future work  Handling load imbalances at replicated tiers  Handling more diverse workloads  Handling other kinds of scheduling disciplines at servers

22. U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 22 Thank you! More information at: http://www.cs.umass.edu/~bhuvan