Presentation is loading. Please wait.

Presentation is loading. Please wait.

High performance Brocade routers

Published byAudra Small Modified over 9 years ago

Similar presentations

Presentation on theme: "High performance Brocade routers"— Presentation transcript:

0 Communications Services
CERN Data Center Network Changes and Evolution Communications Services David Gutiérrez Co-autors: Carles Kishimoto, Edoardo Martelli Communication Services / Engineering

1 High performance Brocade routers
Outline 2010 2011 2012 2013 1 2 3 High performance Brocade routers External connectivity and Firewall system Network Architecture for Wigner 1

2 Data Center Network 2010 Data Center Racks 841 Systems 11,320 Cores
2011 2012 Brocade deployment Force10 100Gbps tests 100Gbps deployment Data Center Racks 841 Systems 11,320 Cores 57,651 Raw HDD (TiB) 61,137 1G NICs 15,703 10G NICs 390 ToR Switches 584 Consumption DC 2.44 MW Force10 LCG non-blocking Fabric 2.88 Tbps Routers 24 Linecards 248 10Gbps Ports 1100 100Gbps Ports N/A 2 ToR: Top of the Rack

3 DC Network 2010 CORE GPN . . . LCG Force10 router Active Firewall
Passive Firewall Aggregated 10Gbps links Tier1s 10Gbps link CORE Border routers GPN Switching fabric 0.96 Tbps 2.88 Tbps Backbone Distribution . . . Access ToR sw LCG CPU, Disk, Tapes AFS, Mail, Web, … 3 LCG: LHC Computing Grid GPN: General Purpose Network ToR: Top of the Rack 3

4 10Gbps Aggregation issues
Hashing is decoupled from link capacity Flow based hashing Potential network traffic polarization Manageability 10G 4x10G 10G LACP 4 LAG: Link AGgregation ECMP: Equal Cost MultiPath

5 100Gbps fat router interconnects
Technology upgrade where needed Performance and fairness tests 100GBase-LR10 CFP (SMF up to 2Km) Testing 100GbE WAN to: Lyon (RENATER) ~120 km Amsterdam (AMSIX) ~1650 km 5

6 Migration in images 6

7 DC Network today CORE GPN . . . LCG Brocade router Active Firewall
Passive Firewall Aggregated 10Gbps Links LHCONE Tier1s 100Gbps Link CORE Border routers GPN Switching fabric 1.36 Tbps 5.28 Tbps Backbone Distribution . . . Access ToR sw LCG CPU, Disk, Tapes AFS, Mail, Web 7 7

8 Data Center Network today
2012 2013 Data Center 2010 2012 Power 2.9 MW 3.5MW* Racks 841 1070 Systems 11,320 12,483 Cores 57,651 68,385 Raw HDD (TiB) 61,137 97,698 1G NICs 15,703 16,026 10G NICs 390 1,912 ToR Switches 584 662 Consumption 2.44 MW 2.8MW Data Center L3 Switch Brocade LCG non-blocking Fabric 5.28 Tbps Routers 22 Linecards 230 10Gbps Ports 1,280 100Gbps Ports 60 Data Center L2 Switch HP 1Gbps Ports 22,776 4,284 MLXe32 Technical Specs # Non-blocking Fabric 15 Tbps Linecards 32 10Gbps Ports 256 100Gbps Ports 8

9 External connectivity
Outline 2010 2011 2012 2013 2 3 High performance Brocade routers External connectivity Network Architecture for Wigner 9

10 External connectivity changes
Internet Géant2 Internet2 US Peers CIXP LHCONE EXTNET LHCOPN Active Firewall Passive Firewall CORE LCG GPN 10 CIXP: Cern Internet eXchange Point

11 Firewall System Active-Passive
Internet Géant2 Internet2 US Peers CIXP Shared with SWITCH 12Gbps 3.8Gbps 1Gbps 20Gbps 30Gbps 6Gpbs Stateful 130Gbps LHCONE 20Gbps EXTNET LHCOPN Active Firewall Passive Firewall CORE LCG GPN 11

12 Firewall System Active-Active
Internet RENATER 2Gbps 40Gbps Géant2 Internet2 US Peers CIXP Shared with SWITCH 12Gbps 3.8Gbps 1Gbps 20Gbps 130Gbps LHCONE EXTNET LHCOPN 30Gbps 6Gpbs Stateful Active Firewall Active Firewall 30Gbps 10Gpbs Stateful CORE LCG GPN 12

13 Network Architecture for Wigner
Outline 2010 2011 2012 2013 3 High performance Brocade routers External connectivity Network Architecture for Wigner 13

14 LCG Resources 2x100Gbps Geneva Building 513 Budapest Wigner Internet/
GeantIP/ Esnet/I2 Firewall 2x100Gbps CERN Core Network Wigner Core Network LCG GPN LCG 14

15 Autonomous Operation 2x100Gbps Geneva Building 513 Budapest Wigner
Internet/ GeantIP/ Esnet/I2 Internet/ HU access /16 AS198797 2001:1459::/32 Firewall 2x100Gbps Firewall CERN Core Network Wigner Core Network radius dns ntp dhcp LCG GPN LCG GPN 15

16 LHCOPN and LHCONE traffic
Geneva Building 513 Budapest Wigner Internet/ GeantIP/ Esnet/I2 Internet/ HU access /16 AS198797 2001:1459::/32 Firewall Firewall CERN Core Network Wigner Core Network MPLS BGP MPLS: MultiProtocol Label Switching radius dns ntp dhcp LCG GPN LCG GPN LHCOPN LHCONE 16

17 Wigner in numbers Wigner Data Center 2013 2014 Power ~900KW ~1200KW
Racks 90 120 Routers 6 10+Firewall 100Gbps ports 18 Switches 140 210 Servers ~1200 ~1800 L2 Switch 1Gbps ports 3072 4608 10Gbps ports 528 792 17

18 Thank you for your attention Questions?
18

Download ppt "High performance Brocade routers"

Similar presentations

LCG France Network Infrastructures Centre de Calcul IN2P3 June 2007

LCG France Network Infrastructures Centre de Calcul IN2P3 June 2007
T1-NREN Luca dell’Agnello CCR, 21-Ottobre-2005. The problem Computing for LHC experiments –Multi tier model (MONARC) –LHC computing based on grid –Experiment.

T1-NREN Luca dell’Agnello CCR, 21-Ottobre The problem Computing for LHC experiments –Multi tier model (MONARC) –LHC computing based on grid –Experiment.
US CMS Tier1 Facility Network Andrey Bobyshev (FNAL) Phil DeMar (FNAL) CHEP 2010 Academia Sinica Taipei, Taiwan.

US CMS Tier1 Facility Network Andrey Bobyshev (FNAL) Phil DeMar (FNAL) CHEP 2010 Academia Sinica Taipei, Taiwan.
IPv6 at CERN Update on Network status David Gutiérrez Co-autor: Edoardo MartelliEdoardo Martelli Communication Services / Engineering

IPv6 at CERN Update on Network status David Gutiérrez Co-autor: Edoardo MartelliEdoardo Martelli Communication Services / Engineering
CCIN2P3 Network FJPPL 2015

CCIN2P3 Network FJPPL 2015
1 © 2001, Cisco Systems, Inc. All rights reserved. NIX Press Conference Catalyst 6500 Innovation Through Evolution 10GbE Tomáš Kupka,

1 © 2001, Cisco Systems, Inc. All rights reserved. NIX Press Conference Catalyst 6500 Innovation Through Evolution 10GbE Tomáš Kupka,
KIT – University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association STEINBUCH CENTRE FOR COMPUTING - SCC www.kit.edu.

KIT – University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association STEINBUCH CENTRE FOR COMPUTING - SCC
Trial of the Infinera PXM Guy Roberts, Mian Usman.

Trial of the Infinera PXM Guy Roberts, Mian Usman.
A Scalable, Commodity Data Center Network Architecture.

A Scalable, Commodity Data Center Network Architecture.
Transatlantic Connectivity in LHCONE Artur Barczyk California Institute of Technology LHCONE Meeting Washington DC, June 13 th, 2011 1.

Transatlantic Connectivity in LHCONE Artur Barczyk California Institute of Technology LHCONE Meeting Washington DC, June 13 th,
Energy Aware Network Operations Authors: Priya Mahadevan, Puneet Sharma, Sujata Banerjee, Parthasarathy Ranganathan HP Labs IEEE Global Internet Symposium.

Energy Aware Network Operations Authors: Priya Mahadevan, Puneet Sharma, Sujata Banerjee, Parthasarathy Ranganathan HP Labs IEEE Global Internet Symposium.
Tier 2 Prague Institute of Physics AS CR Status and Outlook J. Chudoba, M. Elias, L. Fiala, J. Horky, T. Kouba, J. Kundrat, M. Lokajicek, J. Svec, P. Tylka.

Tier 2 Prague Institute of Physics AS CR Status and Outlook J. Chudoba, M. Elias, L. Fiala, J. Horky, T. Kouba, J. Kundrat, M. Lokajicek, J. Svec, P. Tylka.
Agenda Network Infrastructures LCG Architecture Management

Agenda Network Infrastructures LCG Architecture Management
CMS Data Transfer Challenges LHCOPN-LHCONE meeting Michigan, Sept 15/16th, 2014 Azher Mughal Caltech.

CMS Data Transfer Challenges LHCOPN-LHCONE meeting Michigan, Sept 15/16th, 2014 Azher Mughal Caltech.
Status Report on Tier-1 in Korea Gungwon Kang, Sang-Un Ahn and Hangjin Jang (KISTI GSDC) April 28, 2014 at 15th CERN-Korea Committee, Geneva Korea Institute.

Status Report on Tier-1 in Korea Gungwon Kang, Sang-Un Ahn and Hangjin Jang (KISTI GSDC) April 28, 2014 at 15th CERN-Korea Committee, Geneva Korea Institute.
Fermi National Accelerator Laboratory, U.S.A. Brookhaven National Laboratory, U.S.A, Karlsruhe Institute of Technology, Germany CHEP 2012, New York, U.S.A.

Fermi National Accelerator Laboratory, U.S.A. Brookhaven National Laboratory, U.S.A, Karlsruhe Institute of Technology, Germany CHEP 2012, New York, U.S.A.
| www.geant.org BoD over GÉANT (& NRENs) for FIRE and GENI users GENI-FIRE Workshop Washington DC, 17th-18th Sept 2015 Michael Enrico CTO (GÉANT Association)

| BoD over GÉANT (& NRENs) for FIRE and GENI users GENI-FIRE Workshop Washington DC, 17th-18th Sept 2015 Michael Enrico CTO (GÉANT Association)
Swiss Internet Exchange Association Wouter van Hulten InterXion.

Swiss Internet Exchange Association Wouter van Hulten InterXion.
CIXPPaolo Moroni (Slide 1) SwiNOG (18-10-2000) CERN Internet Exchange Point l In Geneva, Switzerland, on the Franco-Swiss border l Co-located with the.

CIXPPaolo Moroni (Slide 1) SwiNOG ( ) CERN Internet Exchange Point l In Geneva, Switzerland, on the Franco-Swiss border l Co-located with the.
Connect communicate collaborate LHCONE L3VPN Status Update Mian Usman LHCONE Meeting Rome 28 th – 29 th Aprils 2014.

Connect communicate collaborate LHCONE L3VPN Status Update Mian Usman LHCONE Meeting Rome 28 th – 29 th Aprils 2014.

Similar presentations

Ads by Google