Wide Area Networking at SLAC, Feb ‘03

Slides:

Advertisements

Similar presentations

GSFC to Alaska Performance Results Tino Sciuto Swales Aerospace ESDIS Network Prototype Lab. NASA GSFC Greenbelt, MD.

Advertisements

Web100 at SLAC Presented at the Web100 Workshop, Boulder, CO, August 2002.

Maximizing End-to-End Network Performance Thomas Hacker University of Michigan October 5, 2001.

1 High Performance Active End-to- end Network Monitoring Les Cottrell, Connie Logg, Warren Matthews, Jiri Navratil, Ajay Tirumala – SLAC Prepared for the.

GridPP meeting Feb 03 R. Hughes-Jones Manchester WP7 Networking Richard Hughes-Jones.

High-Performance Throughput Tuning/Measurements Davide Salomoni & Steffen Luitz Presented at the PPDG Collaboration Meeting, Argonne National Lab, July.

PIPE Dreams Trouble Shooting Network Performance for Production Science Data Grids Presented by Warren Matthews at CHEP’03, San Diego March 24-28, 2003.

EDG WP7 Networking Demonstration uDemonstration sending HEP data CERN to SARA Multiple streams of TCP packets Tuned TCP parameters: ifconfig eth0 txqueuelen.

1 High Performance WAN Testbed Experiences & Results Les Cottrell – SLAC Prepared for the CHEP03, San Diego, March 2003

02 nd April 03Networkshop Managed Bandwidth Next Generation F. Saka UCL NETSYS (NETwork SYStems centre of excellence)

The Effects of Systemic Packets Loss on Aggregate TCP Flows Thomas J. Hacker May 8, 2002 Internet 2 Member Meeting.

KEK Network Qi Fazhi KEK SW L2/L3 Switch for outside connections Central L2/L3 Switch A Netscreen Firewall Super Sinet Router 10GbE 2 x GbE IDS.

1 ESnet Network Measurements ESCC Feb Joe Metzger

What we have learned from developing and running ABwE Jiri Navratil, Les R.Cottrell (SLAC)

Maximizing End-to-End Network Performance Thomas Hacker University of Michigan October 26, 2001.

1 Using Netflow data for forecasting Les Cottrell SLAC and Fawad Nazir NIIT, Presented at the CHEP06 Meeting, Mumbai India, February

Data transfer over the wide area network with a large round trip time H. Matsunaga, T. Isobe, T. Mashimo, H. Sakamoto, I. Ueda International Center for.

1 ESnet/HENP Active Internet End-to-end Performance & ESnet/University performance Les Cottrell – SLAC Presented at the ESSC meeting Albuquerque, August.

1 Overview of IEPM-BW - Bandwidth Testing of Bulk Data Transfer Tools Connie Logg & Les Cottrell – SLAC/Stanford University Presented at the Internet 2.

1 High performance Throughput Les Cottrell – SLAC Lecture # 5a presented at the 26 th International Nathiagali Summer College on Physics and Contemporary.

NET100 Development of network-aware operating systems Tom Dunigan

1 Network Measurement Summary ESCC, Feb Joe Metzger ESnet Engineering Group Lawrence Berkeley National Laboratory.

NET100 Development of network-aware operating systems Tom Dunigan

1 Internet End-to-end Monitoring Project - Overview Les Cottrell – SLAC/Stanford University Partially funded by DOE/MICS Field Work Proposal on Internet.

IEPM. Warren Matthews (SLAC) Presented at the ESCC Meeting Miami, FL, February 2003.

1 Passive and Active Monitoring on a High-performance Network Les Cottrell, Warren Matthews, Davide Salomoni, Connie Logg – SLAC

TCP transfers over high latency/bandwidth networks Internet2 Member Meeting HENP working group session April 9-11, 2003, Arlington T. Kelly, University.

Performance Engineering E2EpiPEs and FastTCP Internet2 member meeting - Indianapolis World Telecom Geneva October 15, 2003

30 June Wide Area Networking Performance Challenges Olivier Martin, CERN UK DTI visit.

GNEW2004 CERN March 2004 R. Hughes-Jones Manchester 1 Lessons Learned in Grid Networking or How do we get end-2-end performance to Real Users ? Richard.

Internet Connectivity and Performance for the HEP Community. Presented at HEPNT-HEPiX, October 6, 1999 by Warren Matthews Funded by DOE/MICS Internet End-to-end.

NET100 Development of network-aware operating systems Tom Dunigan

TCP transfers over high latency/bandwidth networks & Grid DT Measurements session PFLDnet February 3- 4, 2003 CERN, Geneva, Switzerland Sylvain Ravot

Final EU Review - 24/03/2004 DataTAG is a project funded by the European Commission under contract IST Richard Hughes-Jones The University of.

1 Evaluation of Advanced TCP stacks on Fast Long-Distance production Networks Prepared by Les Cottrell & Hadrien Bullot, Richard Hughes-Jones EPFL, SLAC.

INDIANAUNIVERSITYINDIANAUNIVERSITY Status of FAST TCP and other TCP alternatives John Hicks TransPAC HPCC Engineer Indiana University APAN Meeting – Hawaii.

1 IEPM / PingER project & PPDG Les Cottrell – SLAC Presented at the NGI workshop, Berkeley, 7/21/99 Partially funded by DOE/MICS Field Work Proposal on.

1 FAST TCP for Multi-Gbps WAN: Experiments and Applications Les Cottrell & Fabrizio Coccetti– SLAC Prepared for the Internet2, Washington, April 2003

Advanced Network Diagnostic Tools Richard Carlson EVN-NREN workshop.

1 Deploying Measurement Systems in ESnet Joint Techs, Feb Joseph Metzger ESnet Engineering Group Lawrence Berkeley National Laboratory.

Toward a Measurement Infrastructure. Warren Matthews (SLAC) Presented at the e2e Workshop Miami, FL, February 2003.

Architecture and Algorithms for an IEEE 802

Measurements on Internet2

Paola Grosso SLAC October

Realization of a stable network flow with high performance communication in high bandwidth-delay product network Y. Kodama, T. Kudoh, O. Tatebe, S. Sekiguchi.

R. Hughes-Jones Manchester

Prepared by Les Cottrell & Hadrien Bullot, SLAC & EPFL, for the

Networking for grid Network capacity Network throughput

TransPAC HPCC Engineer

Milestones/Dates/Status Impact and Connections

High Speed File Replication

Warren Matthews and Les Cottrell (SLAC)

Using Netflow data for forecasting

ESnet Network Measurements ESCC Feb Joe Metzger

TCP Performance over a 2.5 Gbit/s Transatlantic Circuit

Prepared by Les Cottrell & Hadrien Bullot, SLAC & EPFL, for the

High Performance Active End-to-end Network Monitoring

Characterization and Evaluation of TCP and UDP-based Transport on Real Networks Les Cottrell, Saad Ansari, Parakram Khandpur, Ruchi Gupta, Richard Hughes-Jones,

My Experiences, results and remarks to TCP BW and CT Measurements Tools Jiří Navrátil SLAC.

FAST TCP : From Theory to Experiments

Breaking the Internet2 Land Speed Record: Twice

Advanced Networking Collaborations at SLAC

Web100 at SLAC Doug Chang, Warren Matthews, Les Cottrell (SLAC).

IEPM. Warren Matthews (SLAC)

Wide-Area Networking at SLAC

Measuring End-to-end Bandwidth with Iperf using Web100

MAGGIE NIIT- SLAC On Going Projects

PIPE Dreams Trouble Shooting Network Performance for Production Science Data Grids Presented by Warren Matthews at CHEP’03, San Diego March 24-28, 2003.

Prepared by Les Cottrell & Hadrien Bullot, SLAC & EPFL, for the

Presentation transcript:

Wide Area Networking at SLAC, Feb ‘03 Warren Matthews, Les Cottrell, Paola Grosso SLAC/SCS Prepared for BaBar Data Distribution meeting, SLAC February 26 2003 www.slac.stanford.edu/grp/scs/net/talk/babar-dd-deb03.htm

Outline SLAC Connections to the Internet Usage Performance

WAN Connections Stanford (CalREN and Internet2) GigE link OC12 (622Mbps) ESnet These networks connect (peer) with other networks providing SLAC connectivity to the world.

ESnet provides SLACs primary connectivity, This slide taken from the ESnet web site ESnet provides SLACs primary connectivity, Including DoE labs and non-research sites

The California Research and Education Network (CalREN) connects Stanford and SLAC to UC sites and provides Connectivity to the Internet2 Networks. This slide taken from the UCOP web site. OC48 OC12

Abilene Topology This slide taken from Paul Love’s talk at the Internet2 Joint-Tech Meeting in Miami, February 2003.

Performance Monitoring PingER – low impact RTT, loss etc by ping IEPM-BW – higher impact throughput Netflow – passive, characterization of traffic Looking at a combination of inter-packet delay variation (packet pair dispersion) for lightweight available bandwidth normalized to less frequent heavier weight iperf measurements

SLAC Internet Utilization 2002 - Increase by 2.5 in 12 months Most European links now route through GEANT which peers via Internet 2 at the moment. Hence lack of increase in ESnet traffic.

Utilization History

SLAC Link Utilization (from NetFlow) FTP Iperf SSH

Collaborator sites

SLAC and IN2P3 PHYnet (Renater) SLAC IN2P3 Geant Internet2 622Mbps Note difference in iperf or bbcp memory to memory vs. BBFTP or BBCP disk-to-disk throughput (disks?) Geant Internet2 622Mbps 2.5Gbps 1Gbps

SLAC and RAL 1Gbps RAL TVN JAnet ESnet SLAC Geant 622Mbps 2.5Gbps

SLAC and Padova GARR SLAC Geant INFN/Padova Internet2 622Mbps 2.5Gbps

9/01 105 Mbps 30 Streams: SLAC-IN2P3; 102 Mbps 1 Stream CIT-CERN HENP is Learning How to Use Gbps Networks Fully: Factor of 100 Gain in Maximum Sustainable Thruput in 15 Months, On Some US+TransAtlantic Routes 9/01 105 Mbps 30 Streams: SLAC-IN2P3; 102 Mbps 1 Stream CIT-CERN 11/5/01 125 Mbps in One Stream (modified kernel): CIT-CERN 1/09/02 190 Mbps for One stream shared on 2 155 Mbps links 3/11/02 120 Mbps Disk-to-Disk with One Stream on 155 Mbps link (Chicago-CERN) 5/20/02 450-600 Mbps SLAC-Manchester on OC12 with ~100 Streams 6/1/02 290 Mbps Chicago-CERN One Stream on OC12 (mod. Kernel) 9/02 850, 1350, 1900 Mbps Chicago-CERN 1,2,3 GbE Streams, 2.5G Link 11-12/02 FAST TCP: 940 Mbps in 1 Stream California-CERN; 9.4 Gbps in 10 Flows California-Chicago

Improving Performance More bandwidth, less cross traffic Tune TCP/IP parameters Windows, streams, MTU New TCP stacks Tom Kelly's Scalable TCP, Caltech's FAST TCP Sally Floyd's High-Speed TC Can achieve ~1Gbps trans-Atlantic SLAC and collaborators hold the world record (Nov-02) 923Mbps for 6800miles

Other gotchas Linux memory leak Linux TCP configuration caching What is the window size actually used/reported 32 bit counters in iperf and routers wrap, need latest releases with 64bit counters Effects of number of packets in queue for NIC (txqueuelen) Routers do not pass jumbos Some applications have problems with big windows Firewalls

Toward a Monitoring Infrastructure A ubiquitous network monitoring infrastructure Grid Middleware Guide Applications Troubleshooting Resource Allocation Set expectations Putting together a proposal (MAGGIE) to DoE for a high performance Grid network monitoring infrastructure with LBNL, ICSI, PSC & SLAC>

Summary Multiple high-speed connections Well Engineered Networks Optimize routing Backup paths Well Engineered Networks Negligible packet loss, Good Round Trip Times Traffic doubled on ESnet every year since 1990 Need continued upgrades to keep up with demand Transmitting TBytes/day today Room for improvement (bottlenecks at edges) Needs funding & Improved TCP or other protocols TCP works up to Gbits/s rates, but needs wizards today to make it perform

More Information Any Questions ? High-performance Monitoring www-iepm.slac.stanford.edu/bw/ TCP stack evaluation and 10GE testbed www-iepm.slac.stanford.edu/monitoring/bulk/fast/ Measurement reports www.slac.stanford.edu/comp/net/bandwidth-tests/antonia/html/slac_wan_bw_tests.html Land Speed Record www-iepm.slac.stanford.edu/lsr/ Any Questions ?