Towards Autonomic Computing: Service Discovery and Web Hotspot Rescue Weibin Zhao Department of Computer Science Columbia University April 5, 2006 With Prof. Henning Schulzrinne

04/05/06 Service Discovery & Web Hotspot Rescue2 Autonomic Computing The growing complexity of computing systems Interconnected heterogeneous systems Roaming users in changing networking environments A myriad of network-enabled devices A grand vision Build self-managing systems so as to reduce management complexity and cost A grand challenge Fully autonomic systems vs. systems that have substantial autonomic components

04/05/06 Service Discovery & Web Hotspot Rescue3 Thesis Research Autonomic networking and distributed systems Self-configuration of networking applications Dynamic scalability of Internet servers Service discovery A building block Enable end systems to discover desired services on networks automatically Web hotspot rescue A prototype system Allow web sites to scale dynamically to handle short-term load spikes effectively

04/05/06 Service Discovery & Web Hotspot Rescue4 Thesis Outline Service discovery Enhancements to Service Location Protocol (SLP) Selective anti-entropy for high availability partial replication Web hotspot rescue DotSlash – an automated web hotspot rescue system Traffic prediction for overload prevention Summary

04/05/06 Service Discovery & Web Hotspot Rescue5 Service Discovery Service description  service access points Service providers advertise Service users query Discovery mechanisms Multicast Service registries (directory services) Existing service discovery systems Jini, UPnP, UDDI, SLP, … Challenges Scalability: small scale, local domain, one service type New discovery scenarios: best match, multi-access-point services match discovery

04/05/06 Service Discovery & Web Hotspot Rescue6 Our Approach Enhance Service Location Protocol (SLP)  leverage existing efforts IETF standard for service discovery in IP networks Flexible and powerful: multicast, directories, scopes, search filters, security features Enhancements to SLP Mesh enhancement (mSLP) Remote service discovery Preference filters Global attributes

04/05/06 Service Discovery & Web Hotspot Rescue7 mSLP Overview SLP Entities UA: user agent SA: service agent DA: directory agent DA 1 (S 1, S 2 ) DA 3 (S 2, S 3 ) DA 2 (S 1, S 2 ) DA 4 (S 3 ) (S 1, S 2 ) (S 2 ) (S 3 ) mSLP Example mSLP Scope-based fully- meshed peer DA architecture Simplify SA registration Improve DA consistency

04/05/06 Service Discovery & Web Hotspot Rescue8 mSLP Design Peer relationship management Learn about new peers Set up, maintain, tear down a peer relationship Exchange information about known peers Registration propagation control In shared scopes (partial replication) New updates only Version control Propagation methods Anti-entropy (for initial data and after failures) Direct forwarding (for newly accepted updates)

04/05/06 Service Discovery & Web Hotspot Rescue9 Anti-Entropy For high availability full replication [PODC 87 ] Eventual consistency by exchanging new updates only Subset : all updates accepted by R i Summary vector: largest TS for each subset Exchange updates in all subsets during a session Replica 2 (Scope 1 ) Replica 3 (Scope 1, Scope 2 ) Replica 1 (Scope 1,Scope 2 ) (3) {Update 2 :Scope 2 :TS 2 } ? (2) {Update 1 :Scope 1 :TS 1, Update 3 :Scope 1 :TS 3 } (1) {Update 1 :Scope 1 :TS 1, Update 3 :Scope 1 :TS 3 } {Update 1 :Scope 1 :TS 1, Update 2 :Scope 2,TS 2, Update 3 :Scope 1 :TS 3 } Problems for partial replication

04/05/06 Service Discovery & Web Hotspot Rescue10 Selective Anti-Entropy For high availability partial replication [PODC02] Exchange updates in any number of subsets during a session Use safe sessions only (no summary problem) (R 6, { }, R 1 ) is safe (R 6, { }, R 2 ) and (R 6, { }, R 4 ) are not safe R 2 (S 1 ) R 3 (S 1,S 2 ) R 4 (S 2,S 3 ) R 5 (S 3 ) R 1 (S 1,S 2,S 3 ) R 6 (S 1,S 2 )

04/05/06 Service Discovery & Web Hotspot Rescue11 Thesis Outline Service discovery Enhancements to Service Location Protocol (SLP) Selective anti-entropy for high availability partial replication Web hotspot rescue DotSlash – an automated web hotspot rescue system Traffic prediction for overload prevention Summary

04/05/06 Service Discovery & Web Hotspot Rescue12 Web Hotspots Flash crowds, the Slashdot effect Short-term dramatic load spikes Dynamic content sites: more vulnerable, different bottlenecks Challenges Capacity planning (clusters, mirrors, CDNs): not cost- effective Web Server Internet

04/05/06 Service Discovery & Web Hotspot Rescue13 Our Approach DotSlash An automated web hotspot rescue system Address different bottlenecks Usage model Mutual-aid community for different web sites Three types of communities Open Closed: authentication Flood-insurance closed: authentication + tokens

04/05/06 Service Discovery & Web Hotspot Rescue14 DotSlash Overview Origin Server Rescue Server Client1 (1) (2) (3)(4) HTTP redirect vh1. (8) Reverse proxy (5) vh1. (6) (9) (10) (7) Dynamic DNS Origin DNS Rescue DNS Cache Client2 (2) (1) (4) (3) DNS RR HTTP redirect & cache miss DNS RR & cache hit

04/05/06 Service Discovery & Web Hotspot Rescue15 DotSlash Components Basic system (static & dynamic content) Dynamic virtual hosting Request direction Workload monitoring Rescue control Rescue server discovery Extensions for dynamic content Dynamic replication of application programs On-demand query result caching

04/05/06 Service Discovery & Web Hotspot Rescue16 Workload Migration Request redirection at origin server DNS-RR: first-level crude load distribution Add rescue server IP address to local DNS HTTP redirect: second-level fine-grained load balancing Policies: weighted round robin based on rescue capacity Dynamic virtual hosting at rescue server Assign virtual host name to origin server Used in origin server’s HTTP redirect Map client requests Its own name:  its own contentwww.rescue.com An alias: vh1. (HTTP redirect)  origin servervh1. An origin server: (DNS-RR)  origin serverwww.origin.com

04/05/06 Service Discovery & Web Hotspot Rescue17 Rescue Management Workload monitoring Network and CPU utilization Load regions Trigger different rescue actions Rescue protocol Rescue server discovery DotSlash registries Replicated based on mSLP Registry discovery via DNS SRV dot-slash.net Desired load region Heavy load region Light load region Rescue Server Origin Server SOS 200 OK TOKEN RATE SHUTDOWN KEEPALIVE

04/05/06 Service Discovery & Web Hotspot Rescue18 Server States & Rescue Actions Normal Rescue SOS Get rescueRelease rescue Provide rescueShutdown last rescue Increase P r Decrease P r Get more rescue Increase R r Decrease R r Provide more rescue Shutdown some rescue P r : Redirect probability R r : Allowed redirect rate

04/05/06 Service Discovery & Web Hotspot Rescue19 Dynamic Script Replication Dynamic content web sites LAMP configuration: Linux, Apache, MySQL, PHP Remove web/application server bottleneck Origin ServerDatabase Rescue Server MySQLApache (5) PHP(6) (1) (2) (3) (4) Client (7) (8)

04/05/06 Service Discovery & Web Hotspot Rescue20 On-demand Query Result Caching Reduce workload at read-mostly databases Query Result Cache Data Driver Web/Application Server Query Result Cache Data Driver Web/Application Server Database Server Client Origin Server Rescue Server Database Server

04/05/06 Service Discovery & Web Hotspot Rescue21 Caching & Data Driver Control SOS StateRescue State Heavy Load Desired Load Light Load Cache On Cache Off Cache On Upper Threshold Lower Threshold Normal State Cache On Bypass Cache Rescue Request Database Write Database Read no--Normal yesno-Turn offCache+DB yes noNormalDB+cache yes Redirect Data Driver Control Caching Control

04/05/06 Service Discovery & Web Hotspot Rescue22 Implementation Three configurations in using DotSlash Dots_Apache Dots_Apache + Dots_PHP Dots_Apache + Dots_PHP + Dots_MySQL BINDmSLP DA HTTP Shared Memory SLP DNS DotSlash Rescue Protocol DotSlash Daemon Client Apache DotSlash Daemon DotSlash Module

04/05/06 Service Discovery & Web Hotspot Rescue23 Evaluation Experimental Setup LAMP: Redhat 9.0, Apache , PHP 4.3.6, MySQL Dynamic DNS: BIND 9.2.2, dot-slash.net Service discovery: enhanced SLP LAN and PlanetLab Workload Static content: httperf (HP Labs) Dynamic content: RUBBoS (Rice University) Metrics M ax data rate delivered M ax request rate supported

04/05/06 Service Discovery & Web Hotspot Rescue24 Relieving Network Bottleneck Max Request Rate (reqs/s)Max Data Rate (kB/s) Without Rescue 9 54 With Rescue Improvement978%1007% Origin Server A PlanetLab node behind DSL Rescue Server A local machine connected to Internet2

04/05/06 Service Discovery & Web Hotspot Rescue25 Workload Control and Migration Request/redirect rate at origin server Rescue rate at rescue servers Network Workload ControlWorkload Migration Data rate at origin server and rescue servers Total data rate delivered to clients

04/05/06 Service Discovery & Web Hotspot Rescue26 Removing Web/Application Server Bottleneck Origin ServerHCLC Max Rate (reqs/s) 118   245 Improvement208%500% Number of Rescue Servers 910 Different Configurations for Origin Server HC: 2 GHz CPU, 1GB LC: 1GHz CPU, 512 MB Origin Server: HC / LC Rescue Server: LC Database Server: HC CPU Utilization Origin Server: 50-60% Rescue Servers: ~50% Database Server: 95%

04/05/06 Service Discovery & Web Hotspot Rescue27 Reducing Database Workload (Read-only) Test Case Max Rate (reqs/s) Number of Rescue Servers Compared to READ Compared to READ r READ %- READ c %- READ r %100% READ r,c %462% READ r,sc %333% c: cache r: rescue sc: shared cache

04/05/06 Service Discovery & Web Hotspot Rescue28 Reducing Database Workload (Submission) Test Case Max Rate (reqs/s) Number of Rescue Servers Compared to SUB Compared To SUB r SUB %- SUB c %- SUB r %100% SUB r,c %150% c: cache r: rescue

04/05/06 Service Discovery & Web Hotspot Rescue29 Thesis Outline Service discovery Enhancements to Service Location Protocol (SLP) Selective anti-entropy for high availability partial replication Web hotspot rescue DotSlash – an automated web hotspot rescue system Traffic prediction for overload prevention Summary

04/05/06 Service Discovery & Web Hotspot Rescue30 Traffic Prediction Traditional approach Based on a number of history intervals At a single time scale Use curve fitting Our approach Predict upper bound of future web traffic volume For overload prevention Use a multiple-time-scale approach Only based on current interval At different time scales: self-similarity Use statistical properties of web traffic

04/05/06 Service Discovery & Web Hotspot Rescue31 Design Prediction algorithm [WWW03] Given a time scale T D(T): difference of traffic volume between adjacent intervals  (D(T)): mean of D(T)  (D(T)): standard deviation of D(T) Divide T into n sub-intervals: T’=T/n  (D(T))=n H  (D(T’)),  (D(T))=n H  (D(T’)), H: Hurst parameter Parameter selection T: prediction interval, <=100 second n: scaling factor, [10, 100] H: Hurst parameter, [0.8, 0.9]

04/05/06 Service Discovery & Web Hotspot Rescue32 Experimental Results Three servers on Day74 n=10, H=0.85 Server41 on Day65 n=10, H=0.85, T=100 seconds 1998 World Cup data set, 1.35 billion requests, 30 servers, 92 days

04/05/06 Service Discovery & Web Hotspot Rescue33 Thesis Summary Major thesis contributions Enhancements to Service Location Protocol Selective anti-entropy for high availability partial replication DotSlash – an automated web hotspot rescue system Web traffic prediction for overload prevention Open-source software releases Future work Apply DotSlash to other Internet servers, P2P systems, web services Address security issues in DotSlash Location-based service discovery

04/05/06 Service Discovery & Web Hotspot Rescue34 Major Publications Request for Comments (RFCs) RFC 3528: mSLP RFC 3421: preference filters RFC 3832: remote service discovery Conference and journal papers DotSlash: PODC04, WCW04, GI05, ICAC06 Traffic prediction: WWW03 Selective anti-entropy: PODC02 Service discovery: ICCCN00, ICCCN02, JSS05