Computer Science 1 Resource Overbooking and Application Profiling in Shared Hosting Platforms Bhuvan Urgaonkar Prashant Shenoy Timothy Roscoe † UMASS Amherst and Intel Research †

Computer Science 2 Motivation r Proliferation of Internet applications m Electronic commerce, streaming media, online games, online trading,… r Commonly hosted on clusters of servers m Cheaper alternative to large multiprocessors Clients Internet Streaming Games E-commerce cluster

Computer Science 3 Hosting Platforms r Hosting platform: server cluster that runs third-party applications r Application providers pay for server resources m CPU, disk, network bandwidth, memory r Platform provider guarantees resource availability m Performance guarantees provided to applications r Central challenge: Maximize revenue while providing resource guarantees

Computer Science 4 Design Challenges r How to determine an application’s resource needs? r How to provision resources to meet these needs? r How to map applications to nodes in the platform? r How to handle dynamic variations in the load?

Computer Science 5 Talk Outline þ Introduction r Inferring Resource Requirements r Provisioning Resources r Handling Dynamic Load Variations r Experimental Evaluation r Related Work

Computer Science 6 Hosting Platform Model r Hosting Platforms: Dedicated vs Shared m Dedicated: Applications get integral # nodes m Shared: Applications may get fractional # nodes r Our focus: Shared Hosting Platforms m Nodes may have competing applications r Capsule: component of an application running on a node m Example: e-commerce application: HTTP server, app server, database server

Computer Science 7 Provisioning By Overbooking r How should the platform allocate resources? m Provision resources based on worst-case needs r Worst-case provisioning is wasteful m Low platform utilization r Applications may be tolerant to occasional violations m E.g., CPU guarantees should be met 99% of the time r Possible to provide useful guarantees even after provisioning less than worst-case needs ð Idea: Improve utilization by overbooking resources

Computer Science 8 Application Profiling r Use the Linux trace toolkit time Begin CPU quantumEnd CPU quantum ON OFF r Profiling: process of determining resource usage m Run the application on an isolated set of nodes m Subject the application to a real workload m Model CPU and network usage as ON-OFF processes

Computer Science 9 Resource Usage Distribution time Measurement Interval Cumulative Probability Fractional usage 01 1 r(100) 0.99 r(99) Probability Fractional usage 01

Computer Science 10 Capturing Burstiness: Token Bucket r Token Bucket (σ, ρ) m Resource usage over t ≤ σ.t + ρ Algorithm by Tang et al r Additional parameter T m Satisfy token bucket guarantees only for t ≥ T ρ1ρ1 ρ2ρ2 time usage σ 1.t + ρ 1 σ 2.t + ρ 2

Computer Science 11 Profiles of Server Applications r Applications exhibit different degrees of burstiness m May have a long tail r Insight: Choose (σ, ρ) based on a high percentile Apache Web Server, 50% cgi-bin Probability Fraction of CPU Streaming Media Server, 20 clients Probability Fraction of NW bandwidth

Computer Science 12 Resource Overbooking r Applications specify overbooking tolerance O m Probability with which capsule needs may be violated r Controlled overbooking via admission control: Σ K (σ k ·T min + ρ k )·(1 - O k ) ≤ C·T min Pr (Σ K U k > C) ≤ min (O 1,…,O k ) r A node that has sufficient resources for a capsule is feasible for it

Computer Science 13 Mapping Capsules to Nodes r A bipartite graphs of capsules and feasible nodes m Greedy mapping: consider capsules in non-decreasing order of degrees: O( c. Log c ) m Guaranteed to find a placement if one exists! m Multiple feasible nodes => best fit, worst fit, random… capsules nodes capsules nodes Final Mapping

Computer Science 14 Handling Flash Crowds r Detect overloads by online profiling r Reacting to overloads (ongoing work) m Compute new allocations m Change allocations, move capsules, add servers

Computer Science 15 Talk Outline þ Introduction þ Inferring Resource Requirements þ Provisioning Resources þ Handling Dynamic Load Variations r Experimental Evaluation r Related Work

Computer Science 16 The SHARC Prototype r A Linux-based Shared Hosting Platform m 6 Dell Poweredge 1550 servers m Gigabit Ethernet link r Software Components m Profiling  Vanilla Linux + Linux trace toolkit m Control plane  Overbooking, placement m QoS-enhanced Linux kernel  HSFQ schedulers

Computer Science 17 Experimental Setup r Prototype running on a 5 node cluster m Each server: 1 GHz PIII with 512MB RAM and Gigabit ethernet m Control plane runs on a dedicated node m Applications run on the other four nodes r Workload: mix of server applications m PostgreSQL database server with pgbench (TPC-B) benchmark m Apache web server with SPECWeb99 (static & dynamic HTTP) m MPEG streaming server with 1.5 Mb/s VBR MPEG-1 clients m Quake I game server with “terminator” bots

Computer Science 18 Resource Overbooking Benefits r Small amounts of overbooking can yield large gains m Bursty applications yields larger benefits Placement of Apache Web Servers

Computer Science 19 Capsule Placement Algorithms r Diverse requirements: worst-fit outperforms others r Similar requirements: all perform similarly

Computer Science 20 Performance with Overbooking r Performance degradation is within specified overbooking tolerance ApplicationMetricIsolated100 th 99 th 95 th Avg ApacheTput (req/s) PostgreSQL Tput (trans/s) Streaming Viol (sec)

Computer Science 21 Related Work r Single node resource management m Proportional share schedulers: WFQ, SFQ, BVT, … m Reservation based schedulers: Nemesis, Rialto, … r Cluster-based resource management m Cluster Reserves [Aron00], Aron thesis [Aron00] m MUSE [Chase01]: economic approach m Oceano [IBM], Planetary computing [HP] m Clusters for high availability: Porcupine [Saito99] m Grid computing

Computer Science 22 Concluding Remarks r Resource management in shared hosting platforms m Application profiling to determine resource usage m Revenue maximization using controlled overbooking m Ability to handle dynamic workloads (ongoing work) r URL: