1. Computer Science 1 Resource Overbooking and Application Profiling in Shared Hosting Platforms Bhuvan Urgaonkar Prashant Shenoy Timothy Roscoe † UMASS Amherst and Intel Research †

2. Computer Science 2 Motivation r Proliferation of Internet applications m Electronic commerce, streaming media, online games, online trading,… r Commonly hosted on clusters of servers m Cheaper alternative to large multiprocessors Clients Internet Streaming Games E-commerce cluster

3. Computer Science 3 Hosting Platforms r Hosting platform: server cluster that runs third-party applications r Application providers pay for server resources m CPU, disk, network bandwidth, memory r Platform provider guarantees resource availability m Performance guarantees provided to applications r Central challenge: Maximize revenue while providing resource guarantees

4. Computer Science 4 Design Challenges r How to determine an application’s resource needs? r How to provision resources to meet these needs? r How to map applications to nodes in the platform? r How to handle dynamic variations in the load?

5. Computer Science 5 Talk Outline þ Introduction r Inferring Resource Requirements r Provisioning Resources r Handling Dynamic Load Variations r Experimental Evaluation r Related Work

6. Computer Science 6 Hosting Platform Model r Hosting Platforms: Dedicated vs Shared m Dedicated: Applications get integral # nodes m Shared: Applications may get fractional # nodes r Our focus: Shared Hosting Platforms m Nodes may have competing applications r Capsule: component of an application running on a node m Example: e-commerce application: HTTP server, app server, database server

7. Computer Science 7 Provisioning By Overbooking r How should the platform allocate resources? m Provision resources based on worst-case needs r Worst-case provisioning is wasteful m Low platform utilization r Applications may be tolerant to occasional violations m E.g., CPU guarantees should be met 99% of the time r Possible to provide useful guarantees even after provisioning less than worst-case needs ð Idea: Improve utilization by overbooking resources

8. Computer Science 8 Application Profiling r Use the Linux trace toolkit time Begin CPU quantumEnd CPU quantum ON OFF r Profiling: process of determining resource usage m Run the application on an isolated set of nodes m Subject the application to a real workload m Model CPU and network usage as ON-OFF processes

9. Computer Science 9 Resource Usage Distribution time Measurement Interval Cumulative Probability Fractional usage 01 1 r(100) 0.99 r(99) Probability Fractional usage 01

10. Computer Science 10 Capturing Burstiness: Token Bucket r Token Bucket (σ, ρ) m Resource usage over t ≤ σ.t + ρ Algorithm by Tang et al r Additional parameter T m Satisfy token bucket guarantees only for t ≥ T ρ1ρ1 ρ2ρ2 time usage σ 1.t + ρ 1 σ 2.t + ρ 2

11. Computer Science 11 Profiles of Server Applications r Applications exhibit different degrees of burstiness m May have a long tail r Insight: Choose (σ, ρ) based on a high percentile 0 0.05 0.1 0.15 0.2 0.25 0.3 00.10.20.30.40.50.60.7 Apache Web Server, 50% cgi-bin Probability Fraction of CPU 0 0.05 0.1 0.15 0.2 0.25 0.3 00.1.20.30.40.50.60.70.8 Streaming Media Server, 20 clients Probability Fraction of NW bandwidth

12. Computer Science 12 Resource Overbooking r Applications specify overbooking tolerance O m Probability with which capsule needs may be violated r Controlled overbooking via admission control: Σ K (σ k ·T min + ρ k )·(1 - O k ) ≤ C·T min Pr (Σ K U k > C) ≤ min (O 1,…,O k ) r A node that has sufficient resources for a capsule is feasible for it

13. Computer Science 13 Mapping Capsules to Nodes r A bipartite graphs of capsules and feasible nodes m Greedy mapping: consider capsules in non-decreasing order of degrees: O( c. Log c ) m Guaranteed to find a placement if one exists! m Multiple feasible nodes => best fit, worst fit, random… 1 2 3 1 2 3 4 capsules nodes capsules nodes 1 3 3 1 2 4 Final Mapping

14. Computer Science 14 Handling Flash Crowds r Detect overloads by online profiling r Reacting to overloads (ongoing work) m Compute new allocations m Change allocations, move capsules, add servers

15. Computer Science 15 Talk Outline þ Introduction þ Inferring Resource Requirements þ Provisioning Resources þ Handling Dynamic Load Variations r Experimental Evaluation r Related Work

16. Computer Science 16 The SHARC Prototype r A Linux-based Shared Hosting Platform m 6 Dell Poweredge 1550 servers m Gigabit Ethernet link r Software Components m Profiling  Vanilla Linux + Linux trace toolkit m Control plane  Overbooking, placement m QoS-enhanced Linux kernel  HSFQ schedulers

17. Computer Science 17 Experimental Setup r Prototype running on a 5 node cluster m Each server: 1 GHz PIII with 512MB RAM and Gigabit ethernet m Control plane runs on a dedicated node m Applications run on the other four nodes r Workload: mix of server applications m PostgreSQL database server with pgbench (TPC-B) benchmark m Apache web server with SPECWeb99 (static & dynamic HTTP) m MPEG streaming server with 1.5 Mb/s VBR MPEG-1 clients m Quake I game server with “terminator” bots

18. Computer Science 18 Resource Overbooking Benefits r Small amounts of overbooking can yield large gains m Bursty applications yields larger benefits Placement of Apache Web Servers

19. Computer Science 19 Capsule Placement Algorithms r Diverse requirements: worst-fit outperforms others r Similar requirements: all perform similarly

20. Computer Science 20 Performance with Overbooking r Performance degradation is within specified overbooking tolerance ApplicationMetricIsolated100 th 99 th 95 th Avg ApacheTput (req/s) 67.967.5166.9164.8139.8 PostgreSQL Tput (trans/s) 22.822.4622.2121.789.04 Streaming Viol (sec) 000.310.595.23

21. Computer Science 21 Related Work r Single node resource management m Proportional share schedulers: WFQ, SFQ, BVT, … m Reservation based schedulers: Nemesis, Rialto, … r Cluster-based resource management m Cluster Reserves [Aron00], Aron thesis [Aron00] m MUSE [Chase01]: economic approach m Oceano [IBM], Planetary computing [HP] m Clusters for high availability: Porcupine [Saito99] m Grid computing

22. Computer Science 22 Concluding Remarks r Resource management in shared hosting platforms m Application profiling to determine resource usage m Revenue maximization using controlled overbooking m Ability to handle dynamic workloads (ongoing work) r URL: http://lass.cs.umass.edu