The TCP-ESTATS-MIB Matt Mathis John Heffner Raghu Reddy Pittsburgh Supercomputing Center Rajiv Raghunarayan Cisco Systems J. Saperia JDS Consulting, Inc IETF 62, March 2005

The TCP Extended Statistics MIB Use TCP’s ideal diagnostic vantage point –Observe what the path is doing to segments –Observe what the application is doing to TCP TCP already measures many path properties –RTT, RTT variance, MTU, window size Easily instrumented to measure other properties –Reordering, loss rate, congestion signals Instrument why tcp_output() stops sending –Receiver window, congestion window or the sender? Per-connection controls to support workarounds

Example: a hard diagnostic problem Most symptoms scale with RTT –TCP Buffer Space, Network loss and reordering, etc –On a short path TCP compensates for the flaw Local Client to Server: all applications work –Including all standard diagnostics Remote Client to Server: all applications fail –Leading to faulty implication of other components This is the essence of the “End-to-end Problem”

How extended TCP statistics can help Without TCP instrumentation –Symptoms are reduced on short sections of long paths –Nearly all diagnostics yield a false pass on short paths With TCP ESTATS –Measure key properties of a short section of the path –Extrapolate to the full path to pass judgment –Tools get more sensitive as you test shorter sections –Example uses Web100.org instrumented TCP Target is a simple TCP discard service

Example diagnostic tool output End-to-end goal: 4 Mb/s over a 200 ms path including this section Tester at IP address: xxx.xxx Target at IP address: xxx.xxx Warning: TCP connection is not using SACK Fail: Received window scale is 0, it should be 2. Diagnosis: TCP on the test target is not properly configured for this path. > See TCP tuning instructions at Pass data rate check: maximum data rate was Mb/s Fail: loss event rate: % (3960 pkts between loss events) Diagnosis: there is too much background (non-congested) packet loss. The events averaged losses each, for a total loss rate of % FYI: To get 4 Mb/s with a 1448 byte MSS on a 200 ms path the total end-to-end loss budget is % (9733 pkts between losses). Warning: could not measure queue length due to previously reported bottlenecks Diagnosis: there is a bottleneck in the tester itself or test target (e.g insufficient buffer space or too much CPU load) > Correct previously identified TCP configuration problems > Localize all path problems by testing progressively smaller sections of the full path. FYI: This path may pass with a less strenuous application: Try rate=4 Mb/s, rtt=106 ms Or if you can raise the MTU: Try rate=4 Mb/s, rtt=662 ms, mtu=9000 Some events in this run were not completely diagnosed.

Changes with -06 draft Overhauled listen table –Designed to instrument generic SYN flood defenses Restructured per connection tables –Required “perf” table Expose TCP state variables (no memory footprint) Basic performance instrumentation First tier diagnostic instrumentation –3 optional tables for more detailed diagnosis Path (loss, reordering, duplication, etc) Stack (impact and state of control algorithms) Application (Is the data motion timely?) –1 table of writeable controls for workarounds LimCwnd, LinRwnd, and LimSsthresh Cleanup descriptions, references

Open Technical Issues Too much required stuff? –There may be further juggling between tables The proper SMI type for “Duration”, we want: –microsecond resolution for short flows –days (or months?) scale for exit stats –meaningful deltas at all scales What have we forgotten?

Next steps Last call for input from implementers, researchers MIB doctor review WG last call sometime this summer