1. Reliable Sockets: A Foundation for Mobile Communications
Victor C. Zandy
Computer Sciences Department
University of Wisconsin-Madison
Paradyn/Condor Week (March 2001, Madison WI)

2. Motivation Network communication is unreliable
Modems disconnect spontaneously
Computers run on batteries
Many IP addresses are not static
Assignment by DHCP
Mobile computers move across networks
Applications do not respond well to these failures

3. Reliable Sockets (Rocks)
Sockets that tolerate
IP address changes
Link failures
Extended periods of disconnection
Automatically detect failures and recover
No loss of in-flight data
Applications are oblivious to failures

4. Rocks are General Purpose
Rocks can be used for
UDP and TCP (and everything over them)
Connected sockets and listening sockets
Interoperate with plain sockets
Transparent, user-level, and portable

5. Applications Remote shells Remote GUI-based applications
Mail, editor
Long-running builds
Remote GUI-based applications
Office apps
Mobile and reliable UDP
Streaming video and audio

6. Applications Process migration
Checkpoint Condor jobs with open sockets
Migrate desktop applications

7. Related Work Emphasize mobility, not reliability
No extended periods of disconnection
Lack mechanisms for failure detection and automatic reconnection
Based on kernel modifications
Must be root to install
Unportable
Protocol internals
Mobile IP, TCP Migrate, MSOCKS

8. TCP Sockets Host A Application Sockets API TCP Socket Send Recv Kernel
Port
10000
IP:
Network

9. TCP Data Flow Host A Host B Sockets API Sockets API Send Send Recv2 1 3 4 5
write
Sockets API
Sockets API
Send 2 1 3
Send
Recv
Recv
Port
10000
Port 22
IP:
IP:

10. TCP Data Flow Host A Host B Sockets API Sockets API Send Send Recv2 1 3 4 5
write
Sockets API
Sockets API
Send 4 5
Send
Recv
Recv 2 1 3
Port
10000
Port 22
IP:
IP:

11. TCP Data Flow Host A Host B Sockets API Sockets API In-flight data2 1 3 4 5
write
Sockets API
Sockets API
In-flight data
Send 4 5
Send
Recv
Recv 2 1 3
Port
10000
Port 22
IP:
IP:

12. TCP Data Flow Host A Host B Sockets API Sockets API Send Send Recv2 1 3 4 5 2 1 3
write
read
Sockets API
Sockets API
Send 4 5
Send
Recv
Recv 2 1 3
Port
10000
Port 22
IP:
IP:

13. Socket Failures Host A Host B New IP Address Host movement
Lease expiry
Process migration
Sockets API
Sockets API
Disconnection
Host suspension
Link failure
Send
Send
Recv
Recv
Port
10000
Port 22
IP:
IP:

14. Effect on Applications
Host A
Host B
write
read
Sockets API calls fail
Sockets API
Sockets API
In-flight data is lost
Send
Send
Recv
Recv
Port
10000
Port 22
IP:
IP:

15. What Rocks Do Detect socket failure Automatically reconnect
Hide failure from the application
Automatically reconnect
Recover in-flight data

16. Reliable Sockets Host A Application Sockets API Rock In-Flight
Rocks Library
Sockets API
TCP Socket
Send
Recv
Kernel
Port
10000
IP:
Network

17. Rock Data Flow Host A Host B Sockets API Sockets API In-Flight
write
read
Sockets API
Sockets API
In-Flight
In-Flight
Count bytes read.
Copy data.
Count bytes sent.
Sockets API
Sockets API
Send
Send
Recv
Recv
Port
10000
Port 22
IP:
IP:

18. Response to Failure Host A Host B Sockets API Sockets API In-Flight
write
Sockets API
Sockets API
In-Flight
In-Flight
Sockets API
Sockets API
Send
Send
Recv
Recv
Port
10000
Port 22
IP:
IP:

19. Response to Failure Host A Host B Sockets API Sockets API In-Flight
Send
Send
Recv
Recv
Port
10000
Port 22
IP:
IP:

20. Each rock detects the failure within seconds.
Response to Failure
Host A
Host B
Sockets API
Sockets API
Each rock detects the failure within seconds.
In-Flight
In-Flight ! !
Sockets API
Sockets API
Send
Send
Recv
Recv
Port
10000
Port 22
IP:
IP:

21. Response to Failure Host A Host B Sockets API Sockets API
Each rock suspends:
Close TCP socket
Block application
Attempt to reconnect
In-Flight
In-Flight
Sockets API
Sockets API
IP:
IP:

22. Response to Failure Host A Host B Sockets API Sockets API
Each rock suspends:
Close TCP socket
Block application
Attempt to reconnect
In-Flight
In-Flight
Sockets API
Sockets API
New IP Address
IP:
IP:

23. Recovery Host A Host B Sockets API Sockets API In-Flight In-Flight
Send
Send
New TCP Connection
Recv
Recv
Port
30001
Port 22
IP:
IP:

24. Recovery Host A Host B Sockets API Sockets API Authenticate. In-Flight
Send
Send
Recv
Recv
Port
30001
Port 22
IP:
IP:

25. Retransmit in-flight data not received by remote application.
Recovery
Host A
Host B
Sockets API
Sockets API
Authenticate.
Retransmit in-flight data not received by remote application.
In-Flight
In-Flight
Sockets API
Sockets API
Send
Send
Recv
Recv
Port
30001
Port 22
IP:
IP:

26. Retransmit in-flight data not received by remote application.
Recovery
Host A
Host B
read
Sockets API
Sockets API
Authenticate.
Retransmit in-flight data not received by remote application.
Then resume the rock.
In-Flight
In-Flight
Sockets API
Sockets API
Send
Send
Recv
Recv
Port
30001
Port 22
IP:
IP:

27. Reconnection
Host A
Host B

28. Connection end moves to new IP address
Reconnection
Connection end moves to new IP address
Host B
Host A
Change IP Address

29. Reconnection
Each end attempts to reconnect to its peer at its last known address.
Host B
Host A
Connection does not complete

30. As long as one end does not move, they eventually reconnect.
Reconnection
As long as one end does not move, they eventually reconnect.
Host B
Host A

31. Reconnection They cannot reconnect if both ends move. Host B Host A
Connection does not complete
Connection does not complete

32. Reconnection Network Proxy Host B Host A 101.8.7.1 207.10.0.1
Where is A?
Where is B?

33. Expanded Rocks API
API allows rocks-aware applications to control rocks behavior
Fine control of reconnection
Notification when rock is suspended
Manual control of reconnection addresses
Notification when rock is resumed

34. Expanded Rocks API New socket options Policies Parameters
Extended getsockopt and setsockopt
Policies
Which ports are excluded?
Parameters
Reconnection timeout
Sensitivity to connection failures

35. Performance Reconnection latency
1-2 seconds to reconnect
Usually less than time to acquire DHCP lease
Suspended rocks have negligible overhead

36. Conclusion Rocks make sockets completely reliable
Protect from link failures and IP address changes
Use with any application
Our release is ready for download
Ready for remote shells and remote GUIs
See the demo on Wednesday!

37. Detecting Failures Users expect quick response to failures. Heartbeat:
Periodically send heartbeat to peer
Watch for too many missed heartbeats
Sockets API Errors:
Too slow to rely upon
Not reported for idle connections

38. Detecting Failures The TCP keep-alive probe is inadequate
It waits two hours to send its first probe
User cannot change its period