Network-aware OS DOE/MICS ORNL site visit January 8, 2004 ORNL team: Tom Dunigan, Nagi Rao, Florence Fowler, Steven Carter Matt Mathis Brian.

Slides:

Advertisements

Similar presentations

Click to edit Master title style Click to edit Master text styles –Second level Third level –Fourth level »Fifth level 1 List of Nominations Whats Good.

Advertisements

Appropriateness of Transport Mechanisms in Data Grid Middleware Rajkumar Kettimuthu 1,3, Sanjay Hegde 1,2, William Allcock 1, John Bresnahan 1 1 Mathematics.

Storage System Integration with High Performance Networks Jon Bakken and Don Petravick FNAL.

Using NetLogger and Web100 for TCP analysis Data Intensive Distributed Computing Group Lawrence Berkeley National Laboratory Brian L. Tierney.

FAST TCP Anwis Das Ajay Gulati Slides adapted from : IETF presentation slides Link:

ORNL Net100 status July 31, UT-BATTELLE U.S. Department of Energy Oak Ridge National Laboratory ORNL Net100 Focus Areas (first year) –TCP optimizations.

CUBIC : A New TCP-Friendly High-Speed TCP Variant Injong Rhee, Lisong Xu Member, IEEE v 0.2.

Congestion Control An Overview -Jyothi Guntaka. Congestion  What is congestion ?  The aggregate demand for network resources exceeds the available capacity.

1 Web Server Performance in a WAN Environment Vincent W. Freeh Computer Science North Carolina State Vsevolod V. Panteleenko Computer Science & Engineering.

Maximizing End-to-End Network Performance Thomas Hacker University of Michigan October 5, 2001.

Congestion Control on High-Speed Networks

High-performance bulk data transfers with TCP Matei Ripeanu University of Chicago.

MULTIMEDIA TRAFFIC MANAGEMENT ON TCP/IP OVER ATM-UBR By Dr. ISHTIAQ AHMED CH.

Transport Level Protocol Performance Evaluation for Bulk Data Transfers Matei Ripeanu The University of Chicago Abstract:

All rights reserved © 2006, Alcatel Accelerating TCP Traffic on Broadband Access Networks  Ing-Jyh Tsang 

The Effects of Systemic Packets Loss on Aggregate TCP Flows Thomas J. Hacker May 8, 2002 Internet 2 Member Meeting.

Development of network-aware operating systems Tom Dunigan

Transport Layer 4 2: Transport Layer 4.

Experiences in Design and Implementation of a High Performance Transport Protocol Yunhong Gu, Xinwei Hong, and Robert L. Grossman National Center for Data.

Maximizing End-to-End Network Performance Thomas Hacker University of Michigan October 26, 2001.

UDT: UDP based Data Transfer Yunhong Gu & Robert Grossman Laboratory for Advanced Computing University of Illinois at Chicago.

NORDUnet 2003, Reykjavik, Iceland, 26 August 2003 High-Performance Transport Protocols for Data-Intensive World-Wide Grids T. Kelly, University of Cambridge,

1 Project Goals Project Elements Future Plans Scheduled Accomplishments Project Title: Net Developing Network-Aware Operating Systems PI: G. Huntoon,

High-speed TCP  FAST TCP: motivation, architecture, algorithms, performance (by Cheng Jin, David X. Wei and Steven H. Low)  Modifying TCP's Congestion.

1 Overview of IEPM-BW - Bandwidth Testing of Bulk Data Transfer Tools Connie Logg & Les Cottrell – SLAC/Stanford University Presented at the Internet 2.

HighSpeed TCP for High Bandwidth-Delay Product Networks Raj Kettimuthu.

Requirements for Simulation and Modeling Tools Sally Floyd NSF Workshop August 2005.

1 BWdetail: A bandwidth tester with detailed reporting Masters of Engineering Project Presentation Mark McGinley April 19, 2007 Advisor: Malathi Veeraraghavan.

Rate Control Rate control tunes the packet sending rate. No more than one packet can be sent during each packet sending period. Additive Increase: Every.

NET100 Development of network-aware operating systems Tom Dunigan

UT-BATTELLE U.S. Department of Energy Oak Ridge National Laboratory Net100 PIs: Wendy Huntoon/PSC, Tom Dunigan/ORNL, Brian Tierney/LBNL Impact and Connections.

TCP performance Sven Ubik FTP throughput capacity load ftp.uninett.no 12.3 Mb/s 1.2 Gb/s 80 Mb/s (6.6%) ftp.stanford.edu 1.3 Mb/s 600.

Network-aware OS DOE/MICS Project Review August 18, 2003 Tom Dunigan Matt Mathis Brian Tierney

NET100 … as seen from ORNL Tom Dunigan November 8, 2001.

NET100 Development of network-aware operating systems Tom Dunigan

Network Emulation for the Study and Validation of Traffic Models, Congestion and Flow Control in TCP/IP Networks Cheryl Pope Lecturer Department of Computer.

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Web100 Basil Irwin & George Brett.

An Introduction to UDT Internet2 Spring Meeting Yunhong Gu Robert L. Grossman (Advisor) National Center for Data Mining University.

Network-aware OS DOE/MICS Project Final Review September 16, 2004 Tom Dunigan Matt Mathis Brian Tierney ORNL.

TERENA Networking Conference, Zagreb, Croatia, 21 May 2003 High-Performance Data Transport for Grid Applications T. Kelly, University of Cambridge, UK.

Compound TCP in NS-3 Keith Craig 1. Worcester Polytechnic Institute What is Compound TCP? As internet speeds increased, the long ‘ramp’ time of TCP Reno.

Web100/Net100 at Oak Ridge National Lab Tom Dunigan August 1, 2002.

1 Wide Area Network Emulation on the Millennium Bhaskaran Raman Yan Chen Weidong Cui Randy Katz {bhaskar, yanchen, wdc, Millennium.

TCP transfers over high latency/bandwidth networks Internet2 Member Meeting HENP working group session April 9-11, 2003, Arlington T. Kelly, University.

Thoughts on the Evolution of TCP in the Internet (version 2) Sally Floyd ICIR Wednesday Lunch March 17,

NET100 Development of network-aware operating systems Tom Dunigan

TCP transfers over high latency/bandwidth networks & Grid DT Measurements session PFLDnet February 3- 4, 2003 CERN, Geneva, Switzerland Sylvain Ravot

UT-BATTELLE U.S. Department of Energy Oak Ridge National Laboratory Net100: developing network-aware operating systems New (9/01) DOE-funded (Office of.

Final EU Review - 24/03/2004 DataTAG is a project funded by the European Commission under contract IST Richard Hughes-Jones The University of.

INDIANAUNIVERSITYINDIANAUNIVERSITY Status of FAST TCP and other TCP alternatives John Hicks TransPAC HPCC Engineer Indiana University APAN Meeting – Hawaii.

Thoughts on the Evolution of TCP in the Internet Sally Floyd PFLDnet 2004 February 16, 2004.

Network-aware OS ESCC Miami February 5, 2003 Tom Dunigan Matt Mathis Brian Tierney

Network-aware OS DOE/MICS Project Review August 18, 2003 Tom Dunigan Matt Mathis Brian Tierney CSM lunch.

UT-BATTELLE U.S. Department of Energy Oak Ridge National Laboratory Net100 year 1 leftovers (proposal): PSC –none ORNL –router access to SNMP data (besides.

A TCP Tuning Daemon SC2002 November 19, 2002 Tom Dunigan Matt Mathis Brian Tierney

© 2006 Andreas Haeberlen, MPI-SWS 1 Monarch: A Tool to Emulate Transport Protocol Flows over the Internet at Large Andreas Haeberlen MPI-SWS / Rice University.

28/09/2016 Congestion Control Ian McDonald (with many other WAND members)

Prepared by Les Cottrell & Hadrien Bullot, SLAC & EPFL, for the

TCP-in-UDP draft-welzl-irtf-iccrg-tcp-in-udp-00.txt

Transport Protocols over Circuits/VCs

TransPAC HPCC Engineer

Prepared by Les Cottrell & Hadrien Bullot, SLAC & EPFL, for the

Wide Area Networking at SLAC, Feb ‘03

Prepared by Les Cottrell & Hadrien Bullot, SLAC & EPFL, for the

Anant Mudambi, U. Virginia

High-Performance Data Transport for Grid Applications

Review of Internet Protocols Transport Layer

Using NetLogger and Web100 for TCP analysis

Presentation transcript:

Network-aware OS DOE/MICS ORNL site visit January 8, 2004 ORNL team: Tom Dunigan, Nagi Rao, Florence Fowler, Steven Carter Matt Mathis Brian Tierney

U.S. Department of Energy Office of Science LBNL/ORNL/PSC Roadmap Net100 overview ORNL contributions Ongoing ORNL work Future research more details at DOE-funded project (Office of Science) $2.6M, 3 yrs beginning 9/01 LBNL, ORNL, PSC, NCAR Net100 project objectives: (network-aware operating systems) measure, understand, and improve end-to-end network/application performance tune network protocols and applications (grid and bulk transfer) emphasis: TCP bulk transfer over high delay/bandwidth nets

U.S. Department of Energy Office of Science LBNL/ORNL/PSC Net100 Objective: speedup network applications “enable” high speed –need buffer = bandwidth*RTT - autotune ORNL/NERSC (80 ms, OC12) need 6 MB –faster slow-start avoid losses –modified slow-start –reduce bursts –anticipate loss (ECN,Vegas?) –reorder threshold speed recovery –bigger MTU or “virtual MSS” –modified AIMD (0.5,1) (Floyd, Kelly) –delayed ACKs, initial window, slow-start increment avoid congestion collapse, be fair (?) … intranets, QoS ns simulation: 500 mbs link, 80 ms RTT Packet loss early in slow start. Standard TCP with del ACK takes 10 minutes to recover!

U.S. Department of Energy Office of Science LBNL/ORNL/PSC Net100 methodology Web100 Linux kernel (NSF) –instrumented TCP stack (IETF MIB draft) Path characterization –Network Tuning and Analysis Framework (NTAF) –both active and passive measurement tools –data base of measurements TCP protocol analysis and tuning (primarily ORNL) –simulation/emulation ns TCP-over-UDP ( atou ) NISTNet –kernel tuning extensions –tuning daemon –evaluation tests

U.S. Department of Energy Office of Science LBNL/ORNL/PSC Net100 results Novel approaches –non-invasive dynamic tuning of legacy applications –out-of-kernel tuning –using TCP to tune TCP –tuning on a per flow/destination based on recent path metrics or policy (QoS) Effective evaluation framework –protocol analysis and tuning –network/application/OS debugging –path characterization tools, archive, and visualization tools Performance improvements –WAD tuned : buffers : 10x AIMD : 2x to 10x delayed ACK : 2x slowstart : 3x reorder : 40x Papers and software available

U.S. Department of Energy Office of Science LBNL/ORNL/PSC Net100 – ORNL contributions User tools –Iperf100/ttcp100 –Applet bandwidth tester –atou (tcp over udp) –Net100 daemon (WAD), traced Kernel extensions –Event notification –AIMD and virtual MSS knobs –HS TCP (Floyd) –Scalable TCP (Kelly) –TCP Vegas –Cray X1 SGI Altix Evaluations –Emulation – NISTNet testbed –Simulation (ns) –Parallel streams, bbftp, pftp, gridFTP –FAST –Non-TCP SCTP UDP (SABUL, Tsunami, FOBS) –Nets: ESnet, Internet2, Europe, GigE cable, ATM, wireless, dialup Interactions –HPSS/Probe (NERSC) –Climate –SLAC –Vendors – Cray, SGI, IBM –Talks/papers, software distribution

U.S. Department of Energy Office of Science LBNL/ORNL/PSC Ongoing Net100 ORNL research –more user-friendly WAD – WAD-lite daemon’s interact and measure to select transport parameters plus optional system manager configuration file (policy) No NTAF –Working with NERSC/ORNL HPSS on Net100 support –Vendor collaboration for Net100 on Cray X1 and SGI Altix –TCP Vegas testing Delay-based congestion avoidance can be configured to compete with standard TCP (Feng) CalTech’s FAST –comparison with other “work arounds” parallel streams non-TCP (SABUL, FOBS, TSUNAMI, RBUDP, UDT, SCTP) Dedicated optical path transports –User-mode Linux and Net100

U.S. Department of Energy Office of Science LBNL/ORNL/PSC TCP Vegas Added Vegas to Linux/Net100 kernel, hires timer Tunable with WAD (alpha/beta) Delay-based congestion avoidance Limits buffer growth Reduces loss ? Compare with FAST More tests ……

U.S. Department of Energy Office of Science LBNL/ORNL/PSC Planned Net100 research –improve ease of use (WAD-lite) –analyze effectiveness/fairness of current tuning options simulation emulation on the net (systematic tests) –additional tuning algorithms slow-start accelerants identify non-congestive loss, ECN? Tuning for dedicated path (lambda/10GigE ) –parallel/multipath selection/tuning –10GigE tests –FreeBSD ports ??? –jumbo frame experiments… the quest for bigger and bigger MTUs