1. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 1 How do transport protocols affect applications & The relative importance of different protocol properties Panel Discussion Richard Hughes-Jones The University of Manchester www.hep.man.ac.uk/~rich/ then “Talks” www.hep.man.ac.uk/~rich/

2. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 2 Panellists u Pascale Primet INREA, France u Ralph Niederberger Research Center Juelich, Germany u Tim Sheppard u Katsushi Kobayashi National Institute Adv. Industrial Science & Technology, Japan u Michael Welzl University of Innsbruck, Austria

3. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 3 Some Areas for Discussion u What is the interaction between Application and Transport Protocol? u What is the relative importance of fairness vs throughput? rtt fairness (OK what is fairness?) mtu fairness TCP friendliness u How to AIMD rate fluctuations relate to stability & sharing? Stability of Achievable Throughput u Does provable stability of protocols matter? u Is the computational complexity of a protocol important? u What is the relative importance of convergence time? Link utilisation (by this flow or all flows) u Should there be a bias towards "mice“? – Applications u Is conceptual simplicity of the protocol important?

4. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 4 Action of the transport protocol - help or hindrance to the application ?

5. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 5 Remote Compute Farms: Application Req-Resp CERN-Manc Round trip time 20 ms Web100 hooks for TCP status 64 byte Request green 1 Mbyte Response blue TCP in slow start 1st event takes 19 rtt or ~ 380 ms TCP Congestion window gets re-set on each Request TCP stack RFC 2581 & RFC 2861 reduction of Cwnd after inactivity Even after 10s, each response takes 13 rtt or ~260 ms Transfer achievable throughput 120 Mbit/s peak Event rate very low Application not happy!

6. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 6 VLBI Application Protocol u Data wave front send to Correlator uVLBI signal wave front

7. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 7 Visualising CBR/TCP uWhen packet loss is detected TCP: Reduces Cwnd Halves the sending rate uExpect a delay in the message arrival time Message number / Time Packet loss Delay in stream Expected arrival time at CBR Arrival time Stephen Kershaw

8. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 8 CBR/TCP: UKLight JBO-JIVE-Manc Timely data arrival uMessage size: 1448 Bytes uWait time: 22 us uData Rate: 525 Mbit/s uRoute: JB-UKLight-JIVE- UKLight-Man uRTT ~27 ms uTCP buffer 32M bytes uBDP @512Mbit 1.8Mbyte uEstimate catch-up possible if loss < 1 in 1.24M

9. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 9 And now for the protocols …

10. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 10 SC2004 Disk-Disk bbftp ubbftp file transfer program uses TCP/IP uUKLight: Path:- London-Chicago-London; PCs:- Supermicro +3Ware RAID0 uMTU 1500 bytes; Socket size 22 Mbytes; rtt 177ms; SACK off uMove a 2 GByte file uWeb100 plots: uStandard TCP uAverage 825 Mbit/s u(bbcp: 670 Mbit/s) uScalable TCP uAverage 875 Mbit/s u(bbcp: 701 Mbit/s ~4.5s of overhead) uDisk-TCP-Disk at 1Gbit/s

11. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 11 Transport Protocols u TCP Reno; HS-TCP; Scalable; H-TCP; C-TCP; BIC; CUBIC; LCTP u XCP u UDP Some applications NEED this form of delivery u RTP / RTSP Lots of streaming applications available now u DCCP u multicast

12. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 12 DCCP: Datagram Congestion Control Protocol uUnreliable No re-transmissions uHas modular congestion control Can detect congestion and take avoiding action Different algorithms can be selected – ccid TCP-like TCP Friendly Rate Control uDCCP is like UDP with congestion control uDCCP is like TCP without reliability uApplication uses Multi-media – send new data instead of re-sending useless old data Applications that can choose data encoding & transmission rate e-VLBI – discussing a special ccid uRFCs 4340, CCIDs RFC 4341 4342 ue-VLBI considering a ccid: UDP with congestion detection – API extension Detect potential problems with other network users – unexpected route changes

13. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 13 Fairness and Throughput Larger MTU is faster ! Smaller RTT is faster !

14. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 14 uLow performance on fast long distance paths AIMD (add a=1 pkt to cwnd / RTT, decrease cwnd by factor b=0.5 in congestion) Net effect: recovers slowly, does not effectively use available bandwidth, so poor throughput Unequal sharing Rate fluctuations, Stability & Sharing TCP Reno single stream Congestion has a dramatic effect Recovery is slow Increase recovery rate SLAC to CERN RTT increases when achieves best throughput Les Cottrell PFLDnet 2005 Remaining flows do not take up slack when flow removed

15. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 15 Which Protocol for my Network

16. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 16 Transports for LightPaths uHost to host Lightpath One Application No congestion Lightweight framing uLab to Lab Lightpath Many application share Classic congestion points TCP stream sharing and recovery NEEDED Advanced TCP stacks

17. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 17 Transports for Academic Networks uMany different technologies – often low Bandwidths uCautious/conservative Transport Protocols Standard TCP Linux & BIC Microsoft & C-TCP uHigh Bandwidth Backbones But care needed with Access links – Countries and Campus Many Application flows uNote the Digital Divide uRoles for Advanced TCP stack and other transports. Transports for Global Internet

18. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 18 Summary: Some Areas for Discussion u What is the interaction between Application and Transport Protocol? u What is the relative importance of fairness vs throughput? rtt fairness (OK what is fairness?) mtu fairness TCP friendliness u How to AIMD rate fluctuations relate to stability & sharing? Stability of Achievable Throughput u Does provable stability of protocols matter? u Is the computational complexity of a protocol important? u What is the relative importance of convergence time? Link utilisation (by this flow or all flows) u Should there be a bias towards "mice“? – Applications u Is conceptual simplicity of the protocol important?

19. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 19 Thanks to the Panellists u Pascale Primet INREA, France u Ralph Niederberger Research Center Juelich, Germany u Tim Sheppard u Katsushi Kobayashi National Institute Adv. Industrial Science & Technology, Japan u Michael Welzl University of Innsbruck, Austria

20. PFLDnet, Marina Del Ray, 7-9 Feb 2007, R. Hughes-Jones Manchester 20 CBR/TCP: Catch-up? uIf Throughput NOT limited by TCP buffer size / Cwnd maybe we can re-sync with CBR arrival times. uNeed to store CBR messages during the Cwind drop in the TCP buffer uThen transmit Faster than the CBR rate to catch up Message number / Time Packet loss Delay in stream Expected arrival time at CBR Arrival time Stephen Kershaw