Systems Issues for Scalable, Fault Tolerant Internet Services Yatin Chawathe Eric Brewer To appear in Middleware ’98 http://www.cs.berkeley.edu/~yatin/papers/sns-crc.ps

Motivation Proliferation of network-based services Two critical issues must be addressed by Internet services: System scalability Incremental and linear scalability Availability and fault tolerance 24x7 operation

A Reusable SNS Framework Clusters of workstations are ideal for Internet services [FGC+97] But, clusters are difficult to manage To ensure linear scalability, service must distribute load across the cluster Service must grow the cluster with increasing load Partial failures within a cluster complicate fault management Isolate common requirements of cluster-based Internet apps into a reusable substrate -- the Scalable Network Services (SNS) framework

Architecture ... ... Outside World Worker Worker Driver Worker Internal Network SNS Manager Worker Driver Worker Worker Driver Worker ...

Workers Workers are grouped into classes. Within a class, workers are identical Workers can receive tasks from the outside world, or from other workers Workers have a simple serial interface for tasks The originator sends a task to the consumer by specifying the class and inputs for the task Tasks are atomic and restartable Worker Drivers present a narrow interface between the SNS substrate and the worker application

Centralized SNS Manager SNS Manager is intentionally centralized makes it easier to reason about and implement the various policies “all” we need to do is ensure the fault tolerance of the manager, and make sure it is not a performance bottleneck Three key functions Resource location Load balancing and scalability Fault tolerance

Resource Location Persistent Connection Find Register Found Worker Worker Driver Worker Worker Driver Persistent Connection Find Register Found Multicast Beacons Multicast Beacons Multicast Beacons SNS Manager

Load Balancing Load measurement and reporting Each worker examines incoming requests and estimates the “load” that would be generated Simplest load metric: queue length at workers Workers periodically report their current load to the SNS Manager SNS Manager maintains load history and aggregates load reports from all workers Load reports are piggybacked on manager beacons to rest of the system

Load Balancing Each worker performs local load balancing decisions Use lottery scheduling -- # of tickets are inversely proportional to worker load Stale load reports can cause oscillations Use a correction factor based on the number of requests that were sent since last load report

Auto-launch for Scalability Worker replication to handle short traffic bursts Multiple workers handle requests in parallel If load on a class of workers gets too high, the SNS Manager launches a new one Overflow pool for long bursts non-dedicated set of machines (e.g. users’ desktop machines) when all dedicated nodes are exhausted, harness an overflow node; release it after burst subsides useful for incremental scalability

Fault Tolerance Starfish Fault tolerance Two mechanisms: “Peer” monitoring as opposed to primary/secondary fault tolerance Two mechanisms: Timeouts and retries Preemptive detection and component restart Reliance on soft state simplifies crash recovery

Fault Tolerance Worker Worker Driver Worker Worker Driver Worker AmRestarting ReRegister SNS Manager SNS Manager SNS Manager SNS Manager SNS Manager

Example Applications TranSend Wingman TopGun Mediaboard MARS Web proxy for on-the-fly content distillation Wingman The world’s only graphical web browser for the 3COM PalmPilot TopGun Mediaboard PDA groupware: shared electronic whiteboard for the 3COM PalmPilot MARS MBone archive server

Evaluation

Evaluation

Evaluation Workers 4 & 5started Worker 3 started Worker 2 started

Summary Reusable architecture substrate for building Internet service applications Application developers program their services to a well-defined narrow interface SNS takes care of resource location, spawning, load balancing, fault tolerance Number of interesting applications on top of the SNS substrate Next step: SNSv2 NINJA