J.H.Saltzer, D.P.Reed, C.C.Clark End-to-End Arguments in System Design Reading Group 19/11/03 Torsten Ackemann

Introduction In a modular system, functions can often be put into different sub-systems –By network, client, joint venture, redundantly Many functions require knowledge only the applications have –Providing the function in the network is impossible This is known as the “end-to-end argument” –Reasoning against low-level function implementation

End-to-end Caretaking Considering “Careful file transfer” –Host A linked via multi hop network to host B –Goal is to move file without damage –There is an file transfer application and the data communication system –Specific steps of transaction: File transfer application on host A reads file Applications asks comm system for transmission Comm network transmits file from A to B Comm system on host B reads packets and delivers them to file transfer application on host B File transfer application on host B writes file

End-to-end Caretaking: Threats Threats to the transaction: –Reading incorrect data from the disk –The software on both hosts might make a mistake in buffering and copying the data –Processor or memory on hosts A and B might be faulty –Comm system might drop or change bits in a packet, lose packets or deliver packets more than once –Either host may crash part way through the transaction How to cope with these threats?

End-to-end Caretaking: Solutions Reinforce each of the steps along the way? –using duplicate copies, timeout and retry, redundancy, crash recovery etc. –Uneconomical if all threats are low in probability Alternative is “end-to-end check and retry” –Perform all steps, then check on receiving host Additional step to read file back into memory on host B, then calculate and send checksum to host A –Retry from the beginning if checksums don’t match –Two or more failures on the same transfer indicate that the system is in need of repair

End-to-end Caretaking: Reliability Proposal: Comm system guarantees reliable data transmission –For example by redundancy with packet checksums, sequence numbers, retry mechanism –Reliability of transmission can be increased to any level –The application still has to counter treats outside the comm system Read and write failures, software failures etc. –The extra effort in the comm system has no effect on the correctness of the outcome The application has to provide an end-end reliability guarantee anyway

Performance Aspects: Reliability Application is not aware of packets, and retransmits whole file in case of errors Comm system can retransmit on packet level Effort on lower levels to increase reliability can increase performance significantly But: No “perfect” reliability not necessary on lower levels Tradeoff based on performance rather than a requirement based on correctness Reliability measures in comm system lower performance, too!

Performance Aspects Performing the function at the lower level may cost more! –Job can’t be done as efficiently since information is more limited –May be more efficient if it can be performed with minimal perturbartion Applications that don’t need the function have to pay for it

Example: Delivery Guarantees Acknowledgement of Delivery of a message Knowing that a message got delivered is not important for an application Knowing that the other host acted on the message is important This can only be done by the application on the other host, not the comm system

Example: Secure Transmission of Data Encryption in the network requires trust to manage encryption keys Data is in the clear as it passes into the target node Authenticity still has to be checked by application Network encryption ensures that a node cannot deliberately transmit information that should not be exposed

Example: Duplicate Message Transmission Some network designs allow(ed) a message to be delivered twice The application might do that, too Thus, suppression must be accomplished by the application anyway Function can be omitted from lower levels

Example: Transaction Management Distributed data storage system –Accessing data via identifier, version, type of access: read/write (, value to be written) Network is simplified and does not suppress duplicate messages –Identifier plus version suffices to detect duplicate writes –Duplicate read only leads to duplicate response Acknowledgement needed is that data is stored safely –Can only be provided by the higher levels By eliminating ACKs, the number of messages is halved!

Identifying the Ends Analysis of application requirements is essential For voice traffic, unusually strong version of the argument applies –Bit-perfect communication leads to delays in packet delivery –It is better to accept slightly damaged but not delayed packets However, this changes if the conversation is not real-time (voice mail) –Accuracy is more important than performance suddenly The end-to-end argument is not an absolute rule, but a guideline –Carefully identify the end points!

Conclusions End-to-end arguments are “Occam’s Razor” for networking Comm systems are often specified before the applications –Designer may be tempted to “help” users by taking on more functions than necessary

Statement/Questions End-to-end allows rapid deployment of new services by anybody –Reason for the success of the Internet? Will history repeat with Wireless vs. UMTS? –Application level nets and overlay nets are end-to-end applications Political Dimension –It is about control: “Walled gardens” of providers –Users want to give up control for comfort? –End-to-end is less secure? –More than two stakeholders? Governments etc. End-to-end transparency is the key –Smart nodes in between just have to appear dumb