Presentation is loading. Please wait.

Presentation is loading. Please wait.

ISGC_2005, 27. Apr. 2005 Bruno Hoeft 10Gbit between GridKa and openlab (and their obstacles) Forschungszentrum Karlsruhe GmbH Institute for Scientific.

PublishAshlie McDaniel Modified over 8 years ago

Similar presentations

Presentation on theme: "ISGC_2005, 27. Apr. 2005 Bruno Hoeft 10Gbit between GridKa and openlab (and their obstacles) Forschungszentrum Karlsruhe GmbH Institute for Scientific."— Presentation transcript:

1 ISGC_2005, 27. Apr. 2005 Bruno Hoeft 10Gbit between GridKa and openlab (and their obstacles) Forschungszentrum Karlsruhe GmbH Institute for Scientific Computing P.O. Box 3640 D-76021 Karlsruhe, Germany http://www.gridka.de Bruno Hoeft

2 ISGC_2005, 27. Apr. 2005 Bruno Hoeft Outline LAN of GridKa –Strucure –Prevision of installation in 2008 WAN –History (1G/2003) –Current 10G –Testbed –Challenges crossing multi NREN (National Research and Education Network) –Quality and quantity evaluation of GridKa and openlap network connection –File transfer

3 ISGC_2005, 27. Apr. 2005 Bruno Hoeft IWR 441,442 Tape Storage Main building

4 ISGC_2005, 27. Apr. 2005 Bruno Hoeft Projects at GridKa Atlas (SLAC, USA) (CERN) (FermiLab,USA) calculating jobs with “real” data LHC Experimentenon-LHC Experimente 1,5 Mio. Jobs and 4,2 Mio. hours calculation in 2004

5 ISGC_2005, 27. Apr. 2005 Bruno Hoeft CMS ATLAS LHCb CERN Tier 0 Centre at CERN working groups virtual organizations Germany (FZK) Tier 1 USA (Fermi, BNL) UK (RAL) France (IN2P3) Italy (CNAF) ………. CERN Tier 1 ………. Tier 0 The LHC multi-Tier Computing Model Tier 3 (Institute computer) Tier 4 ( Desktop)      Grid Computing Centre Karlsruhe Tier 2 (Uni-CCs, Lab-CCs) Lab y Uni a Lab i Uni b Lab z Lab x Uni c Uni d Uni e LHC Computing Grid Project - LCG 

6 ISGC_2005, 27. Apr. 2005 Bruno Hoeft DSL has 1024 kbit/s. You can transfer128 kB in a second approx. 40 MB in 5 minutes 1 GBin 2 hours 1 TBin 90 days 1 PB in 248 years 1 PB is the content of 1,4 Million CDs, giving a tower of 1,4 km height ! Can you imagine what a PetaByte is ? PetaByte - 10 15

7 ISGC_2005, 27. Apr. 2005 Bruno Hoeft gradual extention of GridKa resources Apr 2004Okt 2004Apr 2005 % of 2008 Processors68010701.28030 % Computing power / kSI2k580920129012 % Disk [TB]16022027018 % Tape [TB]28037547512 % Internet [Gb/s]210* 10 50 % April 2005: biggest Linux-Cluster within the German Science Society largest Online-Storage at a single installation in Germany strongest Internet connection in Germany available at the Grid with over 100 installations in Europe * Internet connetion for sc

8 ISGC_2005, 27. Apr. 2005 Bruno Hoeft router private network … … C ompute N ode … Ethernet 320 Mbit Ethernet 100 Mbit Ethernet 1 Gbit FiberChannel 2Gbit Ethernet 10 Gbit FS … … FS NAS FS SAN … … switch Network installation Storage direct internet access 1 Gbit/s GridKa PIX L ogin S erver … DFN L ogin S erver router sc nodes … SAN 10Gbit/s s ervice c hallenge only

9 ISGC_2005, 27. Apr. 2005 Bruno Hoeft router 1Gbit/s GridKa PIX L ogin S erver … FS … … private network Storage direct internet access 1 Gbit/s … FS NAS FS DFN … … C ompute N ode … … L ogin S erver switch router Network installation SAN Master Controller RM -Ganglia -Nagios -Cacti switch incl. Management Network Ethernet 320 Mbit Ethernet 100 Mbit Ethernet 1 Gbit FiberChannel 2Gbit Ethernet 10 Gbit 10Gbit/s s ervice c hallenge only sc nodes … SAN

10 ISGC_2005, 27. Apr. 2005 Bruno Hoeft GridKa – WAN connectivity planning 20012002200320042005200620072008 34 Mbps 155 Mbps 2 Gbps 10 Gbps 20 Gbps ? Lightpath to CERN start discussions with DFN/Dante ! - 10G internet connection since Oct 2004 - mid to end of 2005  production - X-Win partner of DFN - light path between CERN and GridKa via X-Win and Geant2

11 ISGC_2005, 27. Apr. 2005 Bruno Hoeft CERN GridFTP server GridFTP server WAN connection and Gigabit test with CERN Géant 10 Gbps DFN 2.4 Gbps GridFTP tested over 1 Gbps Karlsruhe Frankfurt 2x 1 Gbps 98% of 1Gbit Geneva 1000 0

12 ISGC_2005, 27. Apr. 2005 Bruno Hoeft CERN GridFTP server GridFTP server WAN connection 10Gigabit test with CERN Géant 10 Gbps DFN 10 Gbps Karlsruhe Frankfurt Geneva ar-karlsruhe1-ge5-2-700.g-win.dfn.de r-internet.fzk.de cr-karlsruhe1-po12-0.g-win.dfn.de dfn.de1.de.geant.net de.it1.it.geant.net it.ch1.ch.geant.net swiCE2-P6-1.switch.ch swiCE3-G4-3.switch.ch 10 Gbps

13 ISGC_2005, 27. Apr. 2005 Bruno Hoeft Géant

14 ISGC_2005, 27. Apr. 2005 Bruno Hoeft CERN GridFTP server GridFTP server Géant 10 Gbps DFN 10 Gbps Karlsruhe Frankfurt ar-karlsruhe1-ge5-2-700.g-win.dfn.de r-internet.fzk.de cr-karlsruhe1-po12-0.g-win.dfn.de dfn.de1.de.geant.net swiCE3-G4-3.switch.ch de.fr1.fr.geant.net fr.ch1.ch.geant.net MPLS tunnel Geneva WAN connection 10Gigabit test with CERN

15 ISGC_2005, 27. Apr. 2005 Bruno Hoeft CERN GridFTP server GridFTP server Géant 10 Gbps DFN 10 Gbps Karlsruhe Frankfurt -Bandwidth evaluation (tcp/udp) - MPLS via France (MPLS - MultiProtocol Label Switching) - LBE (Least Best Effort) - GridFTP server pool HD to HD Storage to Storage - SRM Geneva WAN connection 10Gigabit test with CERN

16 ISGC_2005, 27. Apr. 2005 Bruno Hoeft Hardware Various Xeon dual 2.8 and 3.0 GHz IBM x-series (Intel and Broadcom NIC) Recently added 3.0 GHz EM64T (800 FSB) Cisco 6509 with 4 10 Gb ports and lots of 1 Gb Storage:Datadirect 2A8500 with 16 TB Linux RH ES 3.0 (U2 and U3), GPFS 10 GE Link to GEANT via DFN (least best effort  ) TCP/IP stack 4 MB buffer 2 MB window size DFN 10Gbit/s s ervice c hallenge only sc nodes … SAN router

17 ISGC_2005, 27. Apr. 2005 Bruno Hoeft Quality Evaluation (UDP streams) 10gkt113 10gkt111 oplapro73 957 Mbit jitter: 0,028; ooo: 0 jitter: 0,021; ooo: 0 957 Mbit LAN ooo - out of order W A N [10gtk111] iperf -s –u [10gtk113] iperf -c 192.108.46.111 -u -b 1000M Transfer 957 Mbits/sec [10gtk113] iperf -s –u [10gtk111] iperf -c 192.108.46.113 -u -b 1000M Transfer 957 Mbits/sec LAN WAN 953 Mbit; jitter: 0,018 ms; ooo: 215 – 0,008% 884 Mbit; jitter: 0,015 ms; ooo: 4239 – 0,018% [oplapro73]> iperf -s -u [10gtk113] iperf -c 192.16.160.13 -u -b 1000M Transfer 953 Mbits/sec [10gtk113] iperf -s -u [oplapro73]> iperf -c 192.108.46.113 -u -b 1000M Transfer 884 Mbits/sec. Reno scalable symmetric = 957 Mbit/sec = 953 Mbit/sec 884 Mbit/sec asymmetric

18 ISGC_2005, 27. Apr. 2005 Bruno Hoeft Bandwidth evaluation (TCP streams) W A N 348Mbit/sec – single stream W A N Reno 10gkt101 – 10gtk105 Reno oplapro71 | Oplapro75 iperf five single steams

19 ISGC_2005, 27. Apr. 2005 Bruno Hoeft 10gkt10[1-5] W A N 5 nodes a 112MByte/sec – 24 parrallel threads Reno iperf Oplapro7[1-5] Reno Bandwidth evaluation time

20 ISGC_2005, 27. Apr. 2005 Bruno Hoeft Wethermap of Géant

21 ISGC_2005, 27. Apr. 2005 Bruno Hoeft Evaluation of max throughput 700 1400 5600 4900 4200 3500 2800 2100 7000 6300 Mbit/s 18:00 20:00 9 nodes each site -8 * 845 Mbit -1 * 540 Mbit higher speed at one stream results in packet loss time

22 ISGC_2005, 27. Apr. 2005 Bruno Hoeft Gridftp sc1 throughput Sc1 – 500MByte sustained 4 th Feb. 056 th Feb. 058 th Feb. 057 th Feb. 05 5 th Feb. 05 4900 4200 3500 2800 2100 1400 700 Mbit/s to /dev/null to hd/SAN 19 Nodes - 15 WorkerNodes* 20MByte IDE/ATA HD - 1 FileServer * 50MByte SCSI HD - 3 FileServer* 50MByte SAN

23 ISGC_2005, 27. Apr. 2005 Bruno Hoeft SC2 approx 1/5 of the load

24 ISGC_2005, 27. Apr. 2005 Bruno Hoeft SC2 five nodes at GridKa gridftp to gpfs

25 ISGC_2005, 27. Apr. 2005 Bruno Hoeft Conclusion GridKa Tier-1 Centre for LHC 4 Production and 4 LHC Experiments multi NREN 10Gbit link up to 7.0Mbit usable Service Challenge to test the whole chain from tape over WAN to tape Diffrent steps just finished SC2 with approx. 1/5 of the CERN aggregated load Lightpath via X-Win (DFN) / Geant2 to CERN will be realised in 2006

26 ISGC_2005, 27. Apr. 2005 Bruno Hoeft Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft

Download ppt "ISGC_2005, 27. Apr. 2005 Bruno Hoeft 10Gbit between GridKa and openlab (and their obstacles) Forschungszentrum Karlsruhe GmbH Institute for Scientific."

Similar presentations

TAB, 03. March 2006 Bruno Hoeft German LHC – WAN present state – future concept Forschungszentrum Karlsruhe GmbH Institute for Scientific Computing P.O.

TAB, 03. March 2006 Bruno Hoeft German LHC – WAN present state – future concept Forschungszentrum Karlsruhe GmbH Institute for Scientific Computing P.O.
GridKa May 2004 Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft Installing dCache into an existing Storage environment at GridKa Forschungszentrum.

GridKa May 2004 Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft Installing dCache into an existing Storage environment at GridKa Forschungszentrum.
Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft LCG-POB, 17.-18.2.2004, Reinhard Maschuw1 Grid Computing Centre Karlsruhe - GridKa Regional/Tier.

Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft LCG-POB, , Reinhard Maschuw1 Grid Computing Centre Karlsruhe - GridKa Regional/Tier.
Service Data Challenge Meeting, Karlsruhe, Dec 2, 2004 Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft Plans and outlook at GridKa Forschungszentrum.

Service Data Challenge Meeting, Karlsruhe, Dec 2, 2004 Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft Plans and outlook at GridKa Forschungszentrum.
T0/T1 – Network Meeting 19.07.2005 Bruno Hoeft T0/T1 – GridKa/CERN Network Forschungszentrum Karlsruhe GmbH Institute for Scientific Computing P.O. Box.

T0/T1 – Network Meeting Bruno Hoeft T0/T1 – GridKa/CERN Network Forschungszentrum Karlsruhe GmbH Institute for Scientific Computing P.O. Box.
LCG Grid Deployment Board, March 2003 Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft Status of GridKa for LCG-1 Forschungszentrum Karlsruhe.

LCG Grid Deployment Board, March 2003 Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft Status of GridKa for LCG-1 Forschungszentrum Karlsruhe.
GridPP meeting Feb 03 R. Hughes-Jones Manchester WP7 Networking Richard Hughes-Jones.

GridPP meeting Feb 03 R. Hughes-Jones Manchester WP7 Networking Richard Hughes-Jones.
02 nd April 03Networkshop 20031 Managed Bandwidth Next Generation F. Saka UCL NETSYS (NETwork SYStems centre of excellence)

02 nd April 03Networkshop Managed Bandwidth Next Generation F. Saka UCL NETSYS (NETwork SYStems centre of excellence)
1 A Basic R&D for an Analysis Framework Distributed on Wide Area Network Hiroshi Sakamoto International Center for Elementary Particle Physics (ICEPP),

1 A Basic R&D for an Analysis Framework Distributed on Wide Area Network Hiroshi Sakamoto International Center for Elementary Particle Physics (ICEPP),
ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University SCOTGRID: A PROTOTYPE TIER-2 CENTRE Steve Thorn Authors: A. Earl, P. Clark, S.

ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University SCOTGRID: A PROTOTYPE TIER-2 CENTRE Steve Thorn Authors: A. Earl, P. Clark, S.
UTA Site Report Jae Yu UTA Site Report 4 th DOSAR Workshop Iowa State University Apr. 5 – 6, 2007 Jae Yu Univ. of Texas, Arlington.

UTA Site Report Jae Yu UTA Site Report 4 th DOSAR Workshop Iowa State University Apr. 5 – 6, 2007 Jae Yu Univ. of Texas, Arlington.
12th November 2003LHCb Software Week1 UK Computing Glenn Patrick Rutherford Appleton Laboratory.

12th November 2003LHCb Software Week1 UK Computing Glenn Patrick Rutherford Appleton Laboratory.
Introduction to U.S. ATLAS Facilities Rich Baker Brookhaven National Lab.

Introduction to U.S. ATLAS Facilities Rich Baker Brookhaven National Lab.
LCG Service Challenge Phase 4: Piano di attività e impatto sulla infrastruttura di rete 1 Service Challenge Phase 4: Piano di attività e impatto sulla.

LCG Service Challenge Phase 4: Piano di attività e impatto sulla infrastruttura di rete 1 Service Challenge Phase 4: Piano di attività e impatto sulla.
TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost TRIUMF SITE REPORT Corrie Kost Head Scientific Computing.

TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost TRIUMF SITE REPORT Corrie Kost Head Scientific Computing.
ScotGRID:The Scottish LHC Computing Centre Summary of the ScotGRID Project Summary of the ScotGRID Project Phase2 of the ScotGRID Project Phase2 of the.

ScotGRID:The Scottish LHC Computing Centre Summary of the ScotGRID Project Summary of the ScotGRID Project Phase2 of the ScotGRID Project Phase2 of the.
1 The D0 NIKHEF Farm Kors Bos Ton Damen Willem van Leeuwen Fermilab, May 23 2001.

1 The D0 NIKHEF Farm Kors Bos Ton Damen Willem van Leeuwen Fermilab, May
Service Challenge Meeting - ISGC_2005, 26. Apr. 2005 Bruno Hoeft 10Gbit between GridKa and openlab (and their obstacles) Forschungszentrum Karlsruhe GmbH.

Service Challenge Meeting - ISGC_2005, 26. Apr Bruno Hoeft 10Gbit between GridKa and openlab (and their obstacles) Forschungszentrum Karlsruhe GmbH.
Data transfer over the wide area network with a large round trip time H. Matsunaga, T. Isobe, T. Mashimo, H. Sakamoto, I. Ueda International Center for.

Data transfer over the wide area network with a large round trip time H. Matsunaga, T. Isobe, T. Mashimo, H. Sakamoto, I. Ueda International Center for.
NL Service Challenge Plans Kors Bos, Sander Klous, Davide Salomoni (NIKHEF) Pieter de Boer, Mark van de Sanden, Huub Stoffers, Ron Trompert, Jules Wolfrat.

NL Service Challenge Plans Kors Bos, Sander Klous, Davide Salomoni (NIKHEF) Pieter de Boer, Mark van de Sanden, Huub Stoffers, Ron Trompert, Jules Wolfrat.

Similar presentations

Ads by Google