1. © 2006 Andreas Haeberlen, MPI-SWS 1 Monarch: A Tool to Emulate Transport Protocol Flows over the Internet at Large Andreas Haeberlen MPI-SWS / Rice University Marcel Dischinger MPI-SWS Krishna Gummadi MPI-SWS Stefan Saroiu University of Toronto

2. 2 © 2006 Andreas Haeberlen, MPI-SWS Motivation New protocol design is an active research area PCP [NSDI 2006], TCP BIC [Infocom 2004], TCP Vegas, TCP Nice [OSDI 2002], TFRC [Sigcomm 2000] State of the art for protocol evaluation: Analysis Simulation/emulation Testbeds But: We still do not understand protocol performance over the real, heterogeneous Internet Fiber, cable, DSL, cellular, wireless,...

3. 3 © 2006 Andreas Haeberlen, MPI-SWS The Monarch tool Scale: Can use millions of live Internet paths Realism: Captures the Internet's heterogeneity Accuracy: Emulation is accurate Monarch emulates transport protocols over the Internet at large

4. 4 © 2006 Andreas Haeberlen, MPI-SWS Outline Overview How Monarch works Evaluation Applications Summary

5. 5 © 2006 Andreas Haeberlen, MPI-SWS Internet How Monarch works Standard TCP Data packet (1500 bytes) ACK packet (40 bytes) TCP Receiver TCP Sender Internet TCP Sender TCP Receiver Host under our control

6. 6 © 2006 Andreas Haeberlen, MPI-SWS Internet How Monarch works TCP Sender TCP Receiver Standard TCP Monarch Probe packet, 1500 bytes (e.g. TCP ACK) Response packet, 40 bytes (e.g. TCP RST) Data packet, 1500 bytes TCP Sender TCP Receiver Internet host (not under our control) Host under our control

7. 7 © 2006 Andreas Haeberlen, MPI-SWS Internet How Monarch works Standard TCP Monarch Probe packets Response packets Data packets TCP Sender TCP Receiver ? TCP Sender Big packets Small packets Big packets Small packets Path is stressed in the same way  Similar throughput, delay, loss, reordering

8. 8 © 2006 Andreas Haeberlen, MPI-SWS What are the limitations? How many hosts will respond? Which protocols can it emulate? What are sources of inaccuracy?

9. 9 © 2006 Andreas Haeberlen, MPI-SWS How many hosts will respond? At least 18% of the hosts respond Probe packet Host type BroadbandAcademicRouter TCP ACK7.2 %13.4 %69.6 % ICMP TsReq18.0 %4.9 %63.0 % ICMP EchoReq25.0 %8.9 %89.3 % UDP Packet7.4 %4.1 %7.3 % Any probe28.4 %18.1 %90.3 % Easy to find 10,000 hosts or more if necessary

10. 10 © 2006 Andreas Haeberlen, MPI-SWS Which protocols can it emulate? Monarch works with many protocols TCP BIC, TCP Nice, TCP Westwood, TCP Vegas, Highspeed TCP, Scalable TCP, Fast TCP SACK, FACK, Window scaling PCP, RAP, TFRC Some protocols cannot be used ECN, XCP Congestion experienced !

11. 11 © 2006 Andreas Haeberlen, MPI-SWS What are sources of inaccuracy? Monarch interprets upstream packet loss as downstream packet loss Upstream losses cause additional retransmissions Paper discusses other challenges Delayed ACKs, middleboxes, upstream reordering

12. 12 © 2006 Andreas Haeberlen, MPI-SWS Monarch knows when it made a mistake Monarch uses IPIDs to find potential inaccuracies When a flow has completed, it is classified as: Confirmed: No upstream loss or upstream reordering Inaccurate: Upstream loss or upstream reordering Indeterminate: Not enough information to self-diagnose ID 17 ID 18 ID 17 ID 19

13. 13 © 2006 Andreas Haeberlen, MPI-SWS Outline Overview How Monarch works Evaluation Applications Summary

14. 14 © 2006 Andreas Haeberlen, MPI-SWS Methodology Run real and emulated TCP transfers over the same links, compare the results Endpoints: 4 hosts under our control: Seattle, Houston, Boston, Saarbrücken (Germany) 356 PlanetLab nodes Each flow transfers 500kB 12,166 flow pairs total

15. 15 © 2006 Andreas Haeberlen, MPI-SWS Monarch and TCP are very close Monarch and TCP flow go through the same states Look at aggregate statistics Throughput, latency, loss CWND (packets) RTO (milliseconds) Time (seconds)

16. 16 © 2006 Andreas Haeberlen, MPI-SWS 0% -100%100% Throughput is similar Compare average throughput of Monarch/TCP flows Half of the Monarch flows have <5% relative error Relative error (Monarch/TCP) Fraction of flows Ideal result -50% -25% -75% 50% 75% 25% Monarch vs. TCP Why? TCP vs. TCP 100% 80% 60% 40% 20% 0%

17. 17 © 2006 Andreas Haeberlen, MPI-SWS How often does it make mistakes? Earlier: Inaccuracies if upstream losses are present Monarch can detect upstream losses How often does that actually happen? Datasets: Broadband: 4,805 cable and DSL hosts in 11 ISPs Router: 697 Internet routers

18. 18 © 2006 Andreas Haeberlen, MPI-SWS Monarch is accurate for many Internet paths Most flows were not affected by upstream loss or upstream reordering ResultBroadbandRouter Confirmed13,16884.2 %2,31783.5 % Inaccurate1,1307.2 %1645.9 % Indeterminate1,3448.6 %29510.6 % Traces total15,642100.0 %2,776100.0 %

19. 19 © 2006 Andreas Haeberlen, MPI-SWS Outline Overview How Monarch works Evaluation Applications Summary

20. 20 © 2006 Andreas Haeberlen, MPI-SWS Monarch is useful in many ways Evaluating transport protcols Example: How well do different congestion control algorithms work for DSL and cable? Inferring network properties Example: Characterize networks that researchers do not have access to Other examples are in the paper

21. 21 © 2006 Andreas Haeberlen, MPI-SWS Congestion control over broadband Results: Long last-mile queues are prevalent Reno and BIC do not keep RTT under control; Vegas does Linux default setting caused highest queueing delay! But it still does not achieve a higher throughput RTT (ms) CWND (packets) Time (seconds) RenoBIC Vegas 500 400 300 200 100 0 0 1 234 0 1 234 0 1 234 80 60 40 20 0 Slow start Congestion avoidance RTT >300ms! RTT >400ms! Low RTT!

22. 22 © 2006 Andreas Haeberlen, MPI-SWS TCP throughput over DSL and cable Fraction of flows Throughput (kbps) Ameritech BellSouth BTOpenWorld PacBell Qwest SWBell Charter Chello Comcast Roadrunner Rogers Cable ISPs: DSL ISPs: ISP #1 ISP #2 ISP #3 ISP #4 ISP #5 ISP #6 ISP #7 ISP #8 ISP #9 ISP #10 ISP #11

23. 23 © 2006 Andreas Haeberlen, MPI-SWS Summary Monarch emulates transport protocols over the Internet at large Monarch enables designers to evaluate their protocols in a realistic environment Monarch is accurate over a variety of Internet paths Examples: DSL and cable

24. 24 © 2006 Andreas Haeberlen, MPI-SWS Thank you! Source code and data sets are available at: http://monarch.mpi-sws.org/ Questions?