M. S. Thesis Defense, 09/24/20011 Migratory TCP (MTCP) Transport Layer Support for Highly- Available Network Services Kiran Srinivasan DisCoLab Division of Computer and Information Sciences Rutgers University Advisor: Liviu Iftode

M. S. Thesis Defense, 09/24/20012 Services Rather Than Servers Client oblivious to a given server’s location Client is interested only in the content provided and QoS A Service -> set of equivalent servers Internet services Most services are based on TCP Clients demand both high-availability and performance

M. S. Thesis Defense, 09/24/20013 TCP-based Internet Services Adverse conditions Core network congestion, overloaded servers, failed servers or under DoS attack TCP response Network delays => packet loss => retransmission TCP limitations Creates an implicit binding of service to a server ! Clients cannot change to another server for sustained service

M. S. Thesis Defense, 09/24/20014 Migratory TCP Transport layer protocol that supports dynamic connection migration Extension to TCP A client connection can transparently migrate to different servers during its lifetime Modified server applications cooperate to support the handoff Client applications do not change Servers can be geographically distributed Does not depend on application-specific info

M. S. Thesis Defense, 09/24/20015 Contributions Designed a transport layer protocol that enables dynamic connection migration Implemented a prototype on FreeBSD Demonstrated the utilization of our protocol in building highly-available services Conducted preliminary performance evaluation F. Sultan, K. Srinivasan, L. Iftode - “Transport Layer Support for Highly-Available Network Services ”, HotOS 2001

M. S. Thesis Defense, 09/24/20016 Outline Motivation Protocol Design Prototype Implementation Preliminary Performance Evaluation Protocol Utilization Related Work Conclusions and Future Work

M. S. Thesis Defense, 09/24/20017 The Migration Model Client Server 1 - overloaded Server 2 congestion

M. S. Thesis Defense, 09/24/20018 Triggers and Initiators Client Server 1 Server 2 MIGRATE_TRIGGER MIGRATE_INITIATE

M. S. Thesis Defense, 09/24/20019 Design Issues How to resume the service at the new server? Without the involvement of the client application Without full migration of the server application With low overhead What is the state that needs to be transferred? Application state Protocol state

M. S. Thesis Defense, 09/24/ Contract Between Server Application and MTCP Application Define fine-grained per-connection application state Export per-connection application state snapshot periodically to the kernel After migration, import per-connection state to resume service MTCP Transfer the per-connection state Synchronize the application state with the protocol state

M. S. Thesis Defense, 09/24/ Server Application Interface export_state Export the state snapshot at the origin server Many times periodically import_state Import the state snapshot at the destination server Enables the destination server to resume from the last snapshot Only once at the destination server Light-weight connection migration

M. S. Thesis Defense, 09/24/ Server Application Example while ((s = accept(ssock)) != -1) { if (import_state(s, &state)) num = state else num = 0 recv(s, &buf) state = ++ num export_state(s, &state) recv(s, &buf) state = ++ num export_state(s, &state)... }

M. S. Thesis Defense, 09/24/ Contract Between Server Application and MTCP Application Define fine-grained per-connection application state Export per-connection application state snapshot periodically After migration, import per-connection state to resume service MTCP Transfer the per-connection state Synchronize the application state with the protocol state

M. S. Thesis Defense, 09/24/ MTCP Protocol Server 1Server 2 Application Protocol Connections

M. S. Thesis Defense, 09/24/ State Synchronization Example while ((s = accept(ssock)) != -1) { if (import_state(s, &state)) num = state else num = 0 recv(s, &buf) state = ++ num export_state(s, &state) recv(s, &buf) state = ++ num export_state(s, &state)... }

M. S. Thesis Defense, 09/24/ State Synchronization Example while ((s=accept(ssock)) != -1) { if (import_state(s, &state)) num = state else num = 0 recv(s, &buf) state = ++num export_state(s, &state) recv(s, &buf) state = ++num export_state(s, &state)... }

M. S. Thesis Defense, 09/24/ State Synchronization Example

M. S. Thesis Defense, 09/24/ Log-Based State Synchronization Logs are maintained in the protocol At export time, protocol discards logs (sync-ed) Logs form part of the state to be transferred during migration Resumption of service starts from the last state snapshot

M. S. Thesis Defense, 09/24/ Implementation Modified the TCP/IP stack in FreeBSD kernel Lazy connection migration mechanism Evaluation on synthetic benchmarks Target applications HTTP server PostgreSQL front-end

M. S. Thesis Defense, 09/24/ Lazy Connection Migration C (0) C’C’ (2) (3) Client Server 1 Server 2 (1) (4)

M. S. Thesis Defense, 09/24/ Implementation: Per-Connection State snap not ack’ed Write Buffer Application Write Application Protocol

M. S. Thesis Defense, 09/24/ Implementation: Per-Connection State X From network Read pointer Read since snap Read before snap ack’ed Application Protocol Read Buffer snap

M. S. Thesis Defense, 09/24/ Preliminary Performance Evaluation Experimental setup Two servers and a client: P III 233MHz, 256 MB RAM Servers are connected by a dedicated network link Benchmarks Microbenchmark Simple web server like application

M. S. Thesis Defense, 09/24/ Microbenchmark One migration averaged over 200 runs for each state size All times are in microseconds The state size is in bytes

M. S. Thesis Defense, 09/24/ Microbenchmark The time for migration varies linearly with state size

M. S. Thesis Defense, 09/24/ Stream Server Experiment Server application sends a stream of 256K bytes in chunks of 1K each Server performance degrades after a threshold Degradation is emulated by means of delays in the server Migration policy module inserted into kernel at the client Smoothed rate estimator Migration occurs when the estimated rate drops to 70% of max. observed rate

M. S. Thesis Defense, 09/24/ Stream Server Experiment Results Effective rate is almost equal to the best rate from the server

M. S. Thesis Defense, 09/24/ Implementation Summary Addition of ~5000 lines of kernel code Changes to TCP finite state machine Addition of two logical TCP states: Migrating and Blackhole Time spent on the design and implementation was 9 months

M. S. Thesis Defense, 09/24/ Protocol Utilization For high availability For performance through load balancing schemes Servers trigger migrations based on a load balancing policy Fault tolerance Eager transfer of connection state information

M. S. Thesis Defense, 09/24/ Related Work Classes of related work TCP Fault tolerance Process migration

M. S. Thesis Defense, 09/24/ TCP Related Works HTTP server fail-over by connection migration [Snoeren ‘00] soft TCP and HTTP state maintained at back-up server Fault-tolerant TCP [Alvisi ‘00] failure masking persistent connections across server crashes Stream Control Transmission Protocol (SCTP) - RFC 2960 multi-homing (many IP addresses / endpoint) ensures connectivity in the face of network failure

M. S. Thesis Defense, 09/24/ TCP Related Works (cont’d) MSOCKS [Bhagwat `98] Proxy based “TCP splicing” for mobile clients with multiple interfaces Indirect TCP [Bakre ‘95] Connection handoffs between MSRs allow mobile hosts to maintain open connections with a fixed host

M. S. Thesis Defense, 09/24/ Fault Tolerance Related Works Log-based rollback recovery The NonStop kernel [Bartlett ’81]

M. S. Thesis Defense, 09/24/ Process Migration Related Works Condor [Litzkow ‘88] Locus [Popek ‘81] MOSIX [Barak ‘85] Sprite [Ousterhout ‘88]

M. S. Thesis Defense, 09/24/ Conclusions We provide a transport layer protocol that enables building of highly-available services The transport layer protocol allows dynamic server endpoint connection migration We implemented a prototype in a FreeBSD kernel We designed an API for the server applications Preliminary performance evaluation on a web server- like application achieved constant rate in the presence of server degradation

M. S. Thesis Defense, 09/24/ Future Work Investigate migration policies Alternate state transfer implementations Recursive connection migration mechanism Future applications

M. S. Thesis Defense, 09/24/ Acknowledgment Florin – for contributing to the design and implementation of the project Other members of DisCo Lab

M. S. Thesis Defense, 09/24/ Thank You Demo in CoRE 336 !