NetPerL Seminar An Analysis of TCP Processing Overhead David D. Clark, Van Jacobson, John Romkey, Howard Salwen IEEE Communications Magazine, June 1989 Presented By: Jim Harris July 23, 2002 6/27/2019 Cal Poly Network Performance Research Group

Cal Poly Network Performance Research Group Background “Classic paper” – notable authors, citations Issue: performance of TCP protocol Scientific approach to address issue Seminal conclusions on TCP/IP performance that have withstood the passage of time 6/27/2019 Cal Poly Network Performance Research Group

Cal Poly Network Performance Research Group The Analysis Started with TCP in BSD UNIX, and modified to remove dependencies on UNIX Determined common path for TCP, and implemented a “fast path” See figure 6/27/2019 Cal Poly Network Performance Research Group

Cal Poly Network Performance Research Group Analysis Terminology 6/27/2019 Cal Poly Network Performance Research Group

A First Case Study – Input Processing Common belief that receiving is more complex [paper refuted belief] Three stages of receive processing Search for TCB TCP checksum Packet header processed First, study packet header processing Counted instructions 6/27/2019 Cal Poly Network Performance Research Group

TCP Output Processing / IP Sending (output) side less detailed analysis Used instruction count again IP processing Instruction estimates for both receiving and sending: common path Header prediction example IP block copy of header template 6/27/2019 Cal Poly Network Performance Research Group

Support Functions / Checksums and TCBs The buffer layer Buffer layers for protocols have same goal Specifications of protocol suite intentionally broad in this area Timers and Schedulers Timer code can be cost demanding Checksums and TCBs Issue of CPU or outboard chip for checksum Temporal coherency for TCB blocks observed 6/27/2019 Cal Poly Network Performance Research Group

Some Speed Predictions Receiving a packet estimated at 300 instructions for TCP/IP Used factor of 4/3 for RISC to get 400 Used 100Mbp/s FDDI and 10 MIPS processor with one ack for every 2 packets yields a packet sized of 500 bytes before protocol is the limiting factor, i.e., bottleneck 6/27/2019 Cal Poly Network Performance Research Group

Cal Poly Network Performance Research Group Why Are Protocols Slow? Real sources of overhead Operating system: interrupt processing, buffer management, context switches (scheduling), timer operation Byte touching operations: moving data in memory, checksumming Data copy biggest factor: CPU and DMA 6/27/2019 Cal Poly Network Performance Research Group

A Direct Measure of Protocol Overhead Used Sun-3/60 to measure time with logic analyzer [our HP16700 has this capabilty] 6/27/2019 Cal Poly Network Performance Research Group

A Direct Measure of Protocol Overhead (cont) Times yield estimated instruction counts that are consistent with above analysis 70% of time in TCP, and 80% of that in output (send) side - counter to common belief Times support OS impact Per byte operations swamp protocol processing costs: 771 us versus 100 us Protocol processing small compared to OS processing: 240 us versus 100 us 6/27/2019 Cal Poly Network Performance Research Group

Some Dangerous Speculations If OS and memory overhead the real limits, then move the processing to a special outboard processor Problems identified with buffer layer between host and outboard processor Alternative: design of simple network interface hardware – minimize interrupts, minimize data copies, compute checksum with data copy 6/27/2019 Cal Poly Network Performance Research Group

Cal Poly Network Performance Research Group Protocols in Silicon Not necessary to abandon TCP/IP protocol to improve performance Recommend moving to: Efficient network interface card, or Simple network controller on cpu bus Desirable to have a programmable element using general protocols TCP has 15 years of improvements which will continue [as of 1989, now TCP/IP are at least 28 years old] Putting protocols in hardware is not required 6/27/2019 Cal Poly Network Performance Research Group