Intelligent Workload Factoring for A Hybrid Cloud Computing Model Hui Zhang Guofei Jiang Haifeng Chen Kenji Yoshihira Akhilesh Saxena NEC Laboratories America Princeton, NJ July 10 th,

2 IT trends: Internet-based services and Cloud Computing  Trend on IT applications – Adoption of service oriented architectures & Web 2.0 applications, e.g. Software as a Service (SaaS) Mobile commerce Open collaboration Social networking Mashups  Trend on IT infrastructure – Adoption of cloud computing architecture. Computations return to the data centers. – Promise of management simplification, energy saving, space reduction, … Blue Cloud

3 What is Cloud Computing? 4+ billion phones by 2010 [Source: Nokia] Web 2.0- enabled PCs, TVs, etc. Businesses, from startups to enterprises  An emerging computing paradigm –Data & services : Reside in massively scalable data centers Can be ubiquitously accessed from any connected devices over the internet. The unique points to cloud computing users are the Elastic infrastructure and the Utility model: provision on demand, charge back on use. [IBM]

4 Cloud Computing is not a reality yet for the majority  “Little Investment In Cloud & Grid Computing for 2009.”  “CIOs are looking primarily to tested, well-understood technologies that can result in savings or increased business efficiencies whose support can be argued from a financial point of view” –a survey by Goldman Sachs & Co., July Private cloud? Public cloud? Choose one, please! Let me think about it. What about current application platform? What about data privacy? What about the performance? Why the full package? ….

5 Local data center (small, dedicated) A hybrid cloud computing infrastructure model Remote cloud (large, pay per use) Dynamic Workload  IT customers can have the best Total Cost of Ownership (TCO) strategy with their applications running on a hybrid infrastructure –Local data center, small and fully utilized for best application performance. –Remote cloud, infinite scaling, use on demand and pay per use. User requests Workload factoring

6 The economic advantage of hybrid cloud computing model: a case study To host Yahoo! Video website workload A local data center hosting 100% workload Hosting solution Annual Cost ($$) Cost on running a 790-servers data center A local data center: workload of 95% time Amazon EC2: peak workload of 5% time + Amazon EC2 hosting 100% workload Workload Factoring US $ 1.384M † † †: assume over-provisioning over the peak load Cost on running a 99-servers data center + US $ 7.43K ‡ ‡ ‡: only consider server cost. Amazon EC2 pricing: $0.10 per machine hour – Small Instance (Default).

Hybrid Cloud Computing architecture Design goals 1.smoothing the workload dynamics in the base zone application platform and avoiding overloading scenarios through load redirection; 2.making trespassing zone application platform agile through load decomposition not only on the volume but also on the application data popularity. (1) (2) (3)

Intelligent workload factoring: problem formulation Problem statement: Input: –requests (r 1, r 2, …, r M ). –data objects (d 1,d 2, …,d N ). –request-data relationship types (t 1 =(d i,d j,…), t 2 =(d x,d y,…),…, t R ) each request belongs to one of the R types Output: –Request partition schemes (R 1, R 2,…, R K ) and data partition schemes (D 1,D 2,…,D K ) for K locations. Problem: a fast online mechanism to make the optimal decision on request and data partition for minimal cross-location data communication overhead. Solution: –fast data frequency estimation Graph model generation –greedy bi-section partition Hypergraph partition [Karypis99] Loc. 1 Loc. 2 d1d1 d3d3 d2d2 d5d5 d4d4 d6d6 A hypergraph partition problem model (NP-hard) Where: Subject to request type i;# of requests for type-i; sum of the vertex weights in Location-k Loc-i capacity of res. type t (1: storage, 2: computing)

The fast top-k data item detection algorithm 9 Timet0t0 Data popularity P old Data popularity P new  Design goal  Starting at t 0, reach an estimation accuracy on the top-k data items in P new within the minimal time.  The key ideas leading to the detection speedup  filtering out old popular data items in a new distribution  filtering out unpopular data items in this distribution.

Speedup analysis of the fast top-k algorithm  Problem model –Formally, for a data item T, we define its actual request rate p(T) = total requests to T/total requests. –FastTopK will determine an estimate p’(T) such that with probability greater than α. We use Z α denote the percentile for the unit normal distribution. For example, if α = 99.75%, then Z α = 3.  Main speedup result –Define an amplification factor X for the rate change of a data item before and after the historical topk-K filtering as –Theorem 1: Let N C before be the number of samples required for basic fastTopK, and N C fafter be the number of samples required for filtering fastTopK –Notation: X 2 speedup of the detection process even with a X-factor on rate amplification due to historical information filtering.

11 Fast and memory-efficient workload factoring scheme “Base zone” Arriving request n n y “Trespassing zone” Fast top-k data item detection scheme end “Base zone” end y Panic mode? Does it belong to the top-k list?

12 A complete request dispatching process in hybrid cloud computing Round-robin dispatching Arriving request Trespassing zone n end LWL Base zone Workload factoring Workload shaping Available server? Drop the request Admit the request drop admit end Drop the request end y

Testbed setup 13 EC2 S3 load controller a http request request forwarding Dispatching decision http reply rtsp://streamServer_x//… IWF

14 Workload factoring evaluation: incoming requests t0t0

15 Workload factoring evaluation: results (I)

Workload factoring evaluation: results (II) 16 Base zone server capacity Trespassing zone server capacity

17 Conclusions  We present the design of intelligent workload factoring, an enabling technology for hybrid cloud computing. –Targeting enterprise IT systems to adopt a hybrid cloud computing model where a dedicated resource platform runs for hosting application base loads, and a separate and shared resource platform serves trespassing peak load of multiple applications.  The key points in our research work –Matching infrastructure elasticity with application agility is a new cloud computing research topic. –Workload factoring is one general technology in boosting application agility. CDN load redirection is a special case.

18 Backup slides

19 Multi-application workload management Multi-application workload management architecture