draft-ietf-ippm-tcp-throughput-tm-04.txt 1 TCP Throughput Testing Methodology IETF 78 Maastricht Reinhard Schrage Barry Constantine

draft-ietf-ippm-tcp-throughput-tm-04.txt 2 Key Comments from Anaheim  As the prerequisite to TCP layer testing, RFC2544 should not be listed as the only means to conduct pre-qualification testing –Revised the draft accordingly, generalizing the network pre-qualification step to Layer 2 / 3 stress testing (with RFC2544 listed as an example)  Need to specify the performance requirements for the TCP client and server testers –Provided “rule of thumb” guideline for hardware performance; testing above 100 Mb/s may require high performance servers or dedicated test equipment  Diagnostic information would be useful –Added result interpretations for each test step and some examples  Experimentation would be required to validate the methodology –Conducted lab and field testing, the results of which have flowed through draft versions 01 – 04 and are highlighted in the following slides

draft-ietf-ippm-tcp-throughput-tm-04.txt 3 Lab Test Environment Cisco 3750E = 10G Switches CoS (Layer 2) QoS (Layer 3) Policing Cisco 3925 = Router (1G Line Rate) QoS (Layer 3) Traffic Shaping Queue Management (RED, WRED) CPE Network Simulator Cisco3925 Cisco3750E Network Provider TCP Test Client TCP Test Server  Out of the lab testing effort, two (2) basic metrics were a useful means of comparing various test configurations (basic throughput, QoS schemes, etc.)

draft-ietf-ippm-tcp-throughput-tm-04.txt 4 TCP Efficiency Metric Transmitted Bytes - Retransmitted Bytes x 100 Transmitted Bytes x 100 = 98% 1000  The TCP Efficiency metric is the percentage of bytes that were not retransmitted and is defined as:  As an example, if 1000 TCP segments were sent and 20 had to be retransmitted, the TCP Efficiency would be calculated as:

draft-ietf-ippm-tcp-throughput-tm-04.txt 5 TCP Transfer Time Metric  The TCP Transfer Time metric is the time it takes to transfer a block of data across simultaneous TCP connections –Can simplify the study of aggregate throughput across multiple connections  An example would be the bulk transfer of 100 MB upon 5 simultaneous TCP connections (each connection uploading 100 MB) –500 Mbit/s Ethernet service – “Ideal” TCP Transfer Time would be ~8 seconds  Simple metric, but easy for network field technicians to comprehend and compare test results

draft-ietf-ippm-tcp-throughput-tm-04.txt 6 Traffic Policing Test Results Cisco M 100M Link, Policed to 10M TCP ClientTCP Server TCP Throughput Mbits/sec Time, sec The graph clearly shows that the four (4) TCP connections (hosts) did not evenly share the 10M link since traffic was policed and TCP was not “smoothed”

draft-ietf-ippm-tcp-throughput-tm-04.txt 7 TCP Client Traffic Shaping Test Results Cisco M 100M Link, Shaped to 10M TCP Throughput Mbits/sec Time, sec The graph clearly shows that the four (4) TCP connections (hosts) did evenly share the 10M link since traffic was shaped and TCP was “smoothed” by the shaping function TCP Server

draft-ietf-ippm-tcp-throughput-tm-04.txt 8 Field Trial 300 Mbps Ethernet Service - 75 msec RTT  Layer 2 traffic testing was conducted (test results = “Pass”), but Enterprise user was not achieving file transfer performance  TCP testing (40 TCP sessions with 64 KB window) was conducted, results charted per the draft recommendation –The TCP Efficiency metric was used to contrast West->East vs. East->West performance (West -> East direction had severe performance issues) TCP Tester TCP Tester

draft-ietf-ippm-tcp-throughput-tm-04.txt 9 Next Steps  Additional review / comments from working group members would be most welcome!  Plan to conduct 1-2 more field trials with the methodology during the month of August  Incorporate comments and field trial findings into draft-05