Computer Science 1 Resource Overbooking and Application Profiling in Shared Hosting Platforms Bhuvan Urgaonkar Prashant Shenoy Timothy Roscoe † UMASS Amherst and Intel Research † Fifth USENIX OSDI, Boston, Dec 2002

Computer Science 2 Introduction r Proliferation of Internet applications m E-commerce, streaming media, online games, … r Commonly hosted on clusters of servers m Cheaper alternative to large multiprocessors Clients Internet Streaming Games E-commerce cluster

Computer Science 3 Hosting Platforms r Hosting platform: server cluster that runs third-party applications r Applications pay for server resources m CPU, network bandwidth, memory, disk r Platform provider guarantees resource availability r Challenge: Maximize # hosted applications while providing resource guarantees

Computer Science 4 Design Challenges r How to determine an application’s resource needs? r How to provision resources to meet these needs? r How to map applications to servers in the platform? r How to handle dynamic variations in load?

Computer Science 5 Talk Outline þ Introduction r Inferring Resource Requirements r Provisioning Resources r Mapping Applications to Servers r Experimental Evaluation r Related Work

Computer Science 6 Terminology r Hosting platform models m Dedicated: Applications get integral # nodes m Shared: Applications may get fractional # nodes Applications Platform nodes http App serv DB serv r Capsule: component of an application running on a node

Computer Science 7 Provisioning By Overbooking r Worst-case provisioning is wasteful m Low utilization of resources r Applications may be tolerant to occasional violations m E.g., CPU guarantees should be met 99% of the time r Possible to provide useful guarantees even after provisioning less than worst-case needs ð Overbook resources to improve utilization m E.g., Airline reservations

Computer Science 8 Application Profiling r Use the Linux Trace Toolkit [Yaghmour00] time Begin CPU quantumEnd CPU quantum ON OFF r Profiling: process of determining resource usage m Run the application on an isolated set of nodes m Subject the application to a real workload m Model CPU and network usage as ON-OFF processes

Computer Science 9 Resource Usage Distribution time Measurement Interval Cumulative Probability Fractional usage 01 1 A 0.99 B Fractional usage ON-OFF PROCESS Probability 01 1 PDF CDF

Computer Science 10 Profiles of Server Applications r Applications exhibit different degrees of burstiness r Need to capture variability in resource usage Postgres Server, 10 clients Probability Fraction of CPU Apache Web Server, 50% cgi-bin Probability Fraction of CPU Streaming Media Server, 20 clients Probability Fraction of NW bandwidth

Computer Science 11 Capturing Burstiness: Token Bucket r Token Bucket (σ, ρ) m Resource usage over t ≤ σ.t + ρ ρ1ρ1 ρ2ρ2 time usage σ 1.t + ρ 1 σ 2.t + ρ 2 time ON-OFF PROCESS r Choose (σ, ρ) based on a high percentile

Computer Science 12 Resource Overbooking Mechanism r Applications specify overbooking tolerance O i m Probability with which capsule needs may be violated r Controlled overbooking via admission control: m Resource requirements of all capsules are met Σ K (σ k ·T min + ρ k )·(1 - O k ) ≤ C·T min m Overbooking tolerances of all capsules are met Pr (Σ K U k > C) ≤ min (O 1,…,O k ) r A node that has sufficient resources for a capsule is feasible for it

Computer Science 13 Mapping Capsules to Nodes r A bipartite graphs of capsules and feasible nodes r Greedy mapping: consider capsules in non-decreasing order of degrees r Multiple feasible nodes => random, best fit, worst fit… capsules nodes capsules nodes Final Mapping

Computer Science 14 Talk Outline þ Introduction þ Inferring Resource Requirements þ Provisioning Resources þ Mapping Applications to Servers r Experimental Evaluation r Related Work

Computer Science 15 The SHARC Prototype r A Linux-based Shared Hosting Platform m 6 Dell Poweredge 1550 servers m Gigabit Ethernet link r Software Components m Profiling  Vanilla Linux + Linux Trace Toolkit m Control plane  Overbooking, placement m QoS-enhanced Linux kernel  HSFQ schedulers

Computer Science 16 Experimental Setup r Prototype running on a 5 node cluster m Each server: 1 GHz PIII with 512MB RAM and Gigabit ethernet m Control plane runs on a dedicated node m Applications run on the other four nodes r Workload: mix of server applications m Apache web server with SPECWeb99 (static & dynamic HTTP) m PostgreSQL database server with pgbench (TPC-B) benchmark m MPEG streaming server with 1.5 Mb/s VBR MPEG-1 clients m Quake I game server with “terminator” bots

Computer Science 17 Resource Overbooking Benefits r Small amounts of overbooking can yield large gains r Bursty applications yield larger benefits Placement of Apache Web Servers No Ovb Ovb=1% Ovb=5% Web Servers Placed Number of Nodes Placement of Streaming Media Servers No Ovb Ovb=1% Ovb=5% Media Servers Placed Number of Nodes

Computer Science 18 Performance with Overbooking Isolated100th99th95thAverage Performance of Postgres Throughput (trans/s) CPU Provisioning Isolated100th99th95thAverage Performance of Apache Throughput (req/s) CPU Provisioning r Performance degradation is within specified overbooking tolerance

Computer Science 19 Handling Flash Crowds r Detect overloads by online profiling r Reacting to overloads (ongoing work) m Compute new allocations m Change allocations, move capsules, add servers

Computer Science 20 Related Work r Single node resource management m Proportional share schedulers: WFQ, SFQ, BVT, … m Reservation based schedulers: Nemesis, Rialto, … r Cluster-based resource management m Cluster Reserves [Aron00] m MUSE [Chase01]: economic approach m Oceano [IBM], Planetary computing [HP] m Clusters for high availability: Porcupine [Saito99] m Grid computing [Globus]

Computer Science 21 Concluding Remarks r Resource management in shared hosting platforms m Application profiling to determine resource usage m Controlled overbooking to improve utilization m Mapping applications to servers r Future work m Handling dynamic workloads m Managing memory and disk bandwidth r URL: