Internet Service Migration and Placement Part 1 Instructor: Xiaodong Zhang Xiaoning Ding 11/08/2004

Outline  Background  OPUS: An Overlay Peer Utility Service Overview Architecture Research issues  Model-based resource provisioning Overview Web service model Model-based resource allocator

Outsourcing Services & Utility-based services Outsourcing services  Customer-owned or leased system.  The service provider takes responsibility for managing the customer ’ s IT and network system – the computing infrastructure – based on customer-defined service level agreements (SLA).  Billed on a monthly or fixed-fee basis. Utility-based services  The service provider owns the infrastructure  leases the infrastructure to the customers  pay for what you use  Example: Internet data center enabling ASPs to deliver ASP services

Utility & SLA  Utilities deliver IT resources (CPU, storage, and bandwidth) to hosted application and, ultimately, end users  much as the electric utility transparently delivers power on demand to customers.  Applications agree to Service Level Agreements (SLAs) with the utility

 Dedicate fixed resources per application  Reprovision manually as needed  Overprovision for surges High variable cost of capacity Static Provisioning

Load Is Dynamic World Cup soccer site May-June 1998 Seasonal fluctuations Event surges (11x) ita.ee.lbl.gov M T W Th F S S M T W Th F S S Week Week ibm.com external site February 2001 Daily fluctuations Workday cycle Weekends off

Adaptive Provisioning offer economies of scale  Network access  Power and cooling  Administration and security  Surge capacity

Overlay network and Mobile code  Increasing number of important network services are deploying overlays CDN, Replicated services, Storage systems... Dynamically map data and functions onto network resources  Programs and data will adaptively migrate and replicate in response to changing network conditions, client access characteristics,... Programs dynamically run at optimal network locations Data dynamically flow to where it is required.

Outline  Background  OPUS: An Overlay Peer Utility Service Overview Architecture Research issues  Model-based resource provisioning Overview Web service model Model-based resource allocator

OPUS: An Overlay Utility Service App demand (per network region) Overlay node Peering Allocate nodes to services based on current demand

OPUS: Overview  targeting utilities consisting of a distributed set of thousands of server sites, each with potentially 1000's of individual machines, cooperating together to fulfill aggregate SLAs  Simultaneously hosts multiple distributed applications replicated web services application-layer multicast content distribution networks....

Opus tasks  Resource allocation Allocate resources among competing applications Maximize aggregate performance Based on changing application and network characteristics, SLAs  Replica placement Closely related to resource allocation Where to place individual application replicas Consider dynamically changing client access patterns, network failures, etc.

Opus tasks  Overlay topology construction create overlays that meet application requirements of performance, delay, and reliability minimize consumed network resources  Request routing discover the service replica capable of delivering the highest quality of service

OPUS: Architecture

The service overlay  Each Opus site runs an instance of site manager coordinating resource usage at that site and exchange status summaries with other opus sites.  Interconnects all active nodes and provides overlay services  “ Backbone ” for coordinated, decentralized resource allocation and resource control

The service overlay  Assist the construction and maintenance of application overlay  Dynamic and self-healing  Scalability issue Hierarchical data dissemination in dicast Think globally but act locally

Adaptive per-application overlay  Each application uses its application overlay to Route internal application traffic Disseminate content Synchronize state information …  The topology and site allotments are subject to change by resource allocator

Security and isolation  Allocating resources to applications at the granularity of individual nodes  Future plan: using virtual machine  Using VLAN to isolate traffic on the wire

Research Issues  Overlay topology construction  Resource allocation  Scalable tracking of system characteristics  Reliability QoS guarantees

Overlay topology construction  Emphasize scalability Quantify the benefits of competing structures Develop scalable distributed constructing algorithms  Initial work A general overlay topology that enables dynamic tradeoffs between network performance/reliability and cost Focus on network cost and relative delay penalty (RDP) to characterize overlay topology Two candidate overlay topologies: K-spanner and LAST.

Overlay topology construction

 Distributed algorithms for building and maintaining the topology Selectively probing using probabilistic techniques and hierarchy Using partial, approximate and probabilistic knowledge of network infomation Having each node gradually migrate to its “ proper ” location in the overlay.

Resource allocation classical economic model  Customers are associated with utility functions specifying the value of the services result from a allotment. (concave functions)  Opus maximizes global value across all applications.  Optimal solution: the marginal value of an additional resource unit is in equilibrium across all customers.

Resource allocation Allocated Resources Throughput (Value) App2 App1 Gradient 2 Gradient 1

Resource allocation  Scalability consideration Adapt from economic resource allocation  Decentralized federation of autonomous local “ markets ” exchanging information to converge toward a global equilibrium Celluar structure  Cell: an entire Opus site or a portion of large site  Cells plan their internal allocation locally  Cells operate to trade load or resources

Tracking system characteristics  Nodes are partitioned into clusters of size d.  Each cluster elects an agent responsible for disseminating local cluster information  Agents from d adjacent clusters form second-level clusters  All nodes are organized into a tree called dicast tree. Height=log d N

Tracking system characteristics Hierarchical data dissemination in dicast

Tracking system characteristics  Data travels up the tree, and may be aggregated with data from the nodes  At each level of the tree, an overlay propagates the data among all participating cluster members  Updates are buffered awaiting the arrival of further updates until a threshold is reached, and updates are aggregated  Each node may has exact information of “ nearby ” nodes in the same cluster Aggregate information of remote cluster

Reliability QoS Guarantees Address network level failures  Restricted flooding Redundantly transmit the same data over multiple logical path  Minimizing the overhead Intermediate nodes re-evaluate the reliability of the remainder of the path, and choose between forwarding redundant data and suppressing duplicate data

Reliability QoS Guarantees 0.96 A B JD S S  A  D: 0.96*0.98*0.99=0.931 S  A  D: 0.97*0.97*0.99=0.931 S  A and B  D: (1-(1-0.96*0.98)*(1-0.97*0.97)) *0.99=0.987

Reliability QoS Guarantees  To match the overlay topology with the failure characteristics of underlying network Construct overlays with disjoint paths to lower the failure correlation among logical overlay links Collect statistical information about loss correlation Use network topology information

Outline  Background  OPUS: An Overlay Peer Utility Service Overview Architecture Research issues  Model-based resource provisioning Overview Web service model Model-based resource allocator

Overview  Addresses the provisioning problem Multiple competing services hosted by a shared server cluster (utility) How much resource does a service need to meet SLA targets  Applications Static web content Heavily resource-intensive Predictable in average per-request resource demands

System architecture

The use of the model

Web service model

α Zipf locality parameter λ Offered load in requests/s S Average object size T Total number of objects M Memory size for object cache RpRp CPU response time H Object cache hit ratio λsλs Storage request load in IOPS RsRs Average storage response time

Web service model μs,φμs,φ Peak storage throughput in IOPS RpRp CPU response time H Object cache hit ratio λsλs Storage request load in IOPS RsRs Average storage response time R Average total response time

Model-based resource allocator  Periodically invoked by the utility OS executive to adjust the allotments  Focus on memory and storage resources, ignore CPU constraints  Output an allotment vector for each service CPU share,Memory and storage allotment [M, φ ]

Model-based resource allocator  Resource provisioning primitives Candidate plans initial candidate allotment vectors LocalAdjust modifies a candidate vector to adapt to local resource constraint or surplus GroupAdjust modifies a set of candidate vectors to adapt to a resource constrait or surplus

Model-based resource allocator Generating Initial Candidates Ρ target  R p Φ=μ s Rp, Φ, ρ target  R s (4) λ,H  λ s (2) Φ=λs / ρ target |Φ-Φ desired |<ε H  M (1)

References  Utility Computing White Paper:  Service Utilities:  D. G. Andersen, H. Balakrishnan, M. F. Kaashoek, and R. Morris, "Resilient Overlay Networks," in 18th ACM Symposium on Operating Systems Principles (SOSP), October 2001, pp  "OPUS: Overlay Utility Service", Rebecca Braynard, Dejan Kostic, Adolfo Rodriguez, Jeff Chase and Amin Vahdat, poster at 18th ACM Symposium on Operating System Principles (SOSP), Banff, Canada, October ( poster)  R. Braynard, D. Kostic, A. Rodriguez, J. Chase, and A. Vahdat. Opus: an Overlay Peer Utility Service. IEEE OPENARCH  Ronald P. Doyle, et. al., ``Model-based resource provisioning in a Web service utility", Proceedings of the 4th USENIX Symposium on Internet Technology, 2003.