1. AUTONOMOUS RESOURCE PROVISIONING FOR MULTI-SERVICE WEB APPLICATIONS Jiang Dejun,Guillaume Pierre,Chi-Hung Chi WWW '10 Proceedings of the 19th international conference on World wide web 1

2. Agenda  Introduction  Related Work  Autonomous Provisioning  Evaluation  Conclusion  Comments 2

3. Introduction(1/5) 3  Major web sites such as Amazon.com and eBay  Are not designed as monolithic 3-tier applications but as a complex group of independent services querying each other  A service is a self-contained application providing elementary functionality  database holding customer information  an application serving search requests

4. Introduction(2/5)  To provide acceptable performance to their customers  providers often impose themselves a Service Level Agreement (SLA) apply dynamic resource provisioning to respect the SLA target by adding resources when violating the SLA removing resources when possible without violating the SLA 4

5. Introduction(3/5)  An essential question in resource provisioning of multiservice web applications  select which service(s) should be (de-)provisioned such that the whole application maintains acceptable performance at minimal cost.  This is a challenge because multi-service applications involve large number of components that have complex relationships with each other 5

6. Introduction(4/5)  Two possible approaches  models the entire application as a single queuing network Too complex to capture all services relationships  assigns a fixed SLA to each service separately May waste resources Only the front-end service should be given an SLA  each service should be autonomously responsible for its own provisioning  by collaboratively negotiating its performance objectives with other services to maintain the front-end’s response time  “What-if analysis” 6

7. Introduction(5/5)  The authors show the system which can effectively provision resources to both  traditional multi-tier web application  complex multi-service applications 7

8. Related Work  Resource provisioning for single-tier [1,4] or multi- tier Web applications [7,10,12,14,15,17]  In [18], model work flow patterns within multiservice applications to predict future workloads for each services component  In [15,16], the works focus on when resources should be provisioned  But allocating new resources becomes much faster now 8

9. Autonomous Provisioning – System Model(1/5)  A service  a single-tier functional service with an HTTP or SOAP interface hosted in an application server  a single-tier data service with an SQL interface hosted in a database server  Within a multi-service application  Services are commonly organized as a directed acyclic graph  assume that the services of one application are not used simultaneously by other applications 9

10. Autonomous Provisioning – System Model(2/5) 10

11. Autonomous Provisioning – System Model(3/5)  assume that some machines are always available to be added to an application  Each resource can be assigned to only one service at a time  Such resource may be a physical machine or a virtualized instance with performance isolation 11

12. Autonomous Provisioning – System Model(4/5) Predict Negotiation Global Decision 12

13. Autonomous Provisioning – System Model(5/5)  Step 1:  each service carries out “what-if analysis” to predict its future performance If the service was assigned an extra machine or removed one  Periodically send result to parent node  Step 2:  Intermediate node selects the maximum performance gain and minimum loss among the children nodes and itself  Finally:  Root node select which service(s) to provision when the SLA is (about to be) violated 13

14. Autonomous Provisioning – Performance Model(1/3)  use an M/M/n/PS queue to capture the performance of an n-core machine  Expected Response Time:  R server : the average response time of the service  n : the number of CPU cores assigned to service  λ : the average request rate  S server : the mean service time 14

15. Autonomous Provisioning – Performance Model(2/3)  A service may also use caches to offload some of the incoming requests from the service itself.  Adding caches potentially improves response time for two reasons  First, cache hits are processed faster than cache misses.  Second, the service itself and all children nodes receive less requests, and can thus process them faster 15

16. Autonomous Provisioning – Performance Model(3/3)  The performance model calculates the caching impact on the response time as follows  R(m) : the response time of the backend server across m CPU cores  S cache : the cache service time  ρ n : the expected cache hit ratio with n nodes 16

17. Autonomous Provisioning – Model parameterization(1/5)  Most of the model parameters can be measured offline or monitored at runtime.  the request rate can be monitored by the administrative tools of application servers and database servers  The cache service time can be obtained by measuring cache response time offline  But expected cache hit ratio( ρ n ) and mean service time(S server ) are harder to measure 17

18. Autonomous Provisioning – Model parameterization(2/5)  Expected cache hit ratio( ρ n )  Using virtual caches  Stores only metadata such as the list of object in caches and their sizes  It receives all requests directed to the service and applies the same operations as a real cache with the same configuration would 18

19. Autonomous Provisioning – Model parameterization(3/5)  Mean service time(S server )  Previous research works measure the service time via profiling under low workload But authors found that while workload increases, the prediction error rate become higher  To achieve acceptable prediction results, authors apply a classical feedback control loop to adjust the service time at runtime.  The system continuously estimates the service’s response time under the current conditions and compares the error between the predicted response time and the measured one. 19

20. Autonomous Provisioning – Model parameterization(4/5) 20

21. Autonomous Provisioning – Model parameterization(5/5)  Define a threshold as a configuration parameter  If the error rate exceeds the threshold, recomputed the service time  S’ server : the corrected service time  R server : the latest measured response time  n : the number of current CPU cores  λ : the current request rate 21

22. Autonomous Provisioning – Resource Provisioning of service instances (1/3)  Each service reports performance promises to its parent on behalf of its children and itself:  it reports the best performance gain (loss) possible by adding (removing) a server to (from) a service of the subtree consisting of its children nodes and itself.  Assuming a service i has k immediate children services V i,J : the average number of service executions on service J caused by one request from service i 22

23. Autonomous Provisioning – Resource Provisioning of service instances (2/3) 23

24. Autonomous Provisioning – Resource Provisioning of service instances (3/3) 24

25. Autonomous Provisioning – Resource Provisioning of cache instances (1/3)  Provisioning cache instances is harder  it not only changes the performance of the concerned service, but also changes the traffic to its children, which in turn affects their performance  Each service periodically informs its children of the relative workload decrease (increase) it would address to them if it was given one more (one less) cache instance 25

26. Autonomous Provisioning – Resource Provisioning of cache instances (2/3)  To calculate expected traffic  EIR : expected invocation ratio = expected cache miss rate  V i,j : the average number of service executions on service j caused by one request from service i  W i : the request rate of node i  K : the number of predecessors in the graph 26

27. Autonomous Provisioning – Resource Provisioning of cache instances (3/3) 27

28. Autonomous Provisioning – Shifting Resource Among Services  In many cases, it can be more efficient to simply reorganize resource assignments within the application without retrieving machines from the resource pool  V i,j may change due to an update in the application code or a change in user behavior  Shifting resource may be an oscillating behavior  To prevent it, one should define a performance threshold as the criterion for deciding whether to shift 28

29. Evaluation – Experimental Setup(1/3)  All experiments are performed on the DAS3 cluster at VU University Amsterdam.  The cluster consists of 85 nodes a dual-CPU/dual-core 2.4GHz AMD Operon DP 280 4GB RAM a 250 GB IDE hard drive  Nodes are connected with a 10Gbps LAN the network latency between nodes is negligible  set the prediction error threshold for dynamically adjusting the service time to 3%. 29

30. Evaluation – Experimental Setup(2/3) 30  Author implement the local performance monitor on application server using the MBean servlet from JBoss.  The database server monitoring is based on performance data collected by the admin tool of MySQL.  Author developed the negotiation agent in Java using plain sockets.

31. Evaluation – Experimental Setup(3/3) 31

32. Evaluation – Model Validation(1/2)  Compare predicted values with the measured response times  using the “XSLT” and “Product” services from Figure 7(c) separately  Set the SLA of each service to a maximum response time of 400ms  Initially assign one server to each 32

33. Evaluation – Model Validation(2/2) 33

34. Evaluation – Comparison(1/2) 34  Comparison with “Analytic”  Figure 7(a)  A well known model  Set SLA of 500ms for the whole application  Analytic does not address multi-service application

35. Evaluation – Comparison(2/2) 35  Comparison with per-service SLA  Figure 7(b)  Set SLA to 500ms 12 req 16 req 25 req 23 req

36. Evaluation – Provisioning under varying load intensity(1/2) 36  Using figure 7(c) and 7(d)  Set SLA of 500ms  Workload first increases from 2 req/s to 22 req/s, then decrease back to 2 req/s

37. Evaluation – Provisioning under varying load intensity(2/2) 37 10 req18 req16 req8 req

38. Evaluation – Provisioning under varying load distribution(1/2) 38  Add workload to Service 2 and 3 at the same rate  At time 35, the workload of service 3 decrease while workload of service 2 increase as the same

39. Evaluation – Provisioning under varying load distribution (2/2) 39

40. Evaluation – Provisioning under varying load locality (1/2) 40  Define locality as the hit rate for a cache holding 10,000 objects  Increase workload until time 25 when the SLA was violated, and then changing the locality of service3

41. Evaluation – Provisioning under varying load locality (2/2) 41

42. Conclusion 42  The paper takes a different stand and demonstrates that provisioning resources for multi- service applications.  Which can be achieved in a decentralized way where each service is autonomously responsible for its own provisioning  Propose to give an SLA only to the front-end service  To author’s best knowledge, no other published resource provisioning algorithm can match or outperform their approach

43. Comments 43  The paper is using physical machines as resources  In virtual machine, we can dynamic set each VM’s CPU upper bound It means that maybe we can make the cost less than the paper  The paper is more complex than what we want to do now, but there are something we can refer to  Something like the approach of adjusting the profiling

44. 44  The End  Thanks