1. INFN CNAF TIER1 Network Service
Bologna,
Network Service status and evolution
Stefano Zani

2. WAN Connectivity CERN CERN FZK 10 Gb/s GARR GARR 1Gb/s In 1 Year
Enterasys
ER16
Old-CORE
Extreme
BD 10808
NEW-CORE
CNAF-CERN (LHC)
10Gb/s
T0-T1
10 Gb/s
General purpose
Wan Access
GARR
POP Hosted at CNAF
2x10Gb/s
Cisco
7600
General Purpose
WAN
ACCESS
Router
CNAF(LHC)
T1-T1
FZK
2° 10Gb/s T1-T1 Connectivity and T0-T1 backup
GARR
POP Hosted at CNAF
1Gb/s
General Purpos
Wan Access
Cisco
7600
General Purpose
WAN
ACCESS
Router
CNAF-CERN (LHC)
10Gb/s
T0-T1
SSR 8600
General Purpose
WAN
ACCESS
Router
In 1 Year
Enterasys
ER16
Old-CORE
2x10Gb/s
Extreme
BD 10808
NEW-CORE

3. LAN Elements Tier1’s “BRICK” 42U Rack up to 36-37 Core Switches:
servers
CORE
Switch-Router
Core Switches:
VLANs (802.1Q)
Routing (Wire speed)
ACLs (Firewall)
N x 1Gb/s
Load Sharing
Rack Aggregation Switches:
VLANs (802.1Q) single port granularity
48 Gigabit Ports
10Gb/s Ready
Only few mission critical
Servers are connected
Directly to the Core.
The network is segmented in VLANs
Example: LGC Experiments VLAN, KVM Vlan, PowerVlan, etc..
All routing activity is performed at wire speed by the core
Switches. .

4. Core Switch-Routers
2 Core Switches: Expedition ER16 and Black Diamond 10808
ER16 (96xOptical Gb, 2x10Gb) – FULL!
2 CPU Modules (Redundancy)
Redundant Power Supply
BD10K(128xOptical Gb Ports, 64 Copper Gb,12x10Gb/s) – 3 FREE SLOTS
1Router (Wan Access)
Cisco 7600 (Sup7203B) 4x10Gb/2 + 4x1Gb
2 Sup (Redundancy)

5. 28 Extreme Networks Summit 400
Aggregation Switches
28 Extreme Networks Summit 400
48 Gigabit Ethernet Ports (2-4 Optical Fiber)
2x 10Gb/s ports (4 on 28)
160 Gb/s MAX throughput (NON Blocking)
Routing and ACL evaluation at Wire speed
12 Enterasys Matrix E1
48 Fast Ethernet Ports + 2 Optical Fiber Gigabit Ports
3 Cisco 3550
48 Fast Ethernet Ports + 2 Optical Fiber Gigabit

6. Current Network general layout
SSR8600
sw-04-03
St.1(gi.3.1,gi.5.1)
sw-04-01
sw-04-02
3(3-4)
5(5-6)
7(7-8)
LHCBSW1
9(9-10)
49(49-50)
St.1(gi.5.1,gi.6.1)
29(29-32)
SW-04-06
SW-04-07
SW-04-08
SW-04-09
SW-04-10
SW-03-06
FarmSWF1
St.2(gi.1.1,gi.2.1)
St.3(gi.1.2,gi.2.2)
St.4(gi.1.3,gi.2.3)
St.5(gi.1.4,gi.2.4)
St.6(gi.1.5,gi.2.5)
St.7(gi.1.6,gi.2.6)
St.9(gi.1.8,gi.2.8)
St.15(gi.15,16.1,2)
gi.3.2
192.16
/30
Matrix M5
Sez. Di Bologna
sw-06-06
St.12(gi.5.4,gi.6.4)
SW-03-07
SW-05-01
SW-05-02
SW-05-03
SW-05-04
SW-05-05
SW-03-08
SW-03-09
St.17(gi.15.4,gi.16.4)
St.18(gi.15.5,gi.16.5)
St.19(gi.15.6,gi.16.6)
St.20(gi.15.7,gi.16.7)
St.21(gi.15.8,gi.16.8)
St.22(gi.12.6,gi.13.6)
St.23(gi.12.7,gi.13.7)
SW-06-01
St.24(gi.12.8,gi.13.8)
St.16(gi.15.3,gi.16.3)
St.25(gi.10.1,gi.11.1)
HP Babar
St.13(gi.5.5,gi.5.8,gi.6.5)
SW-08-04
SW-03-10
SW-03-11
St.28(gi.10.5,gi.11.5)
St.29(gi.10.6,gi.11.6)
St.30(gi.5.2,gi.6.2)
SW-04-04
SW-04-05
St.31(gi.10.7,gi.11.7)
St.32(gi.5.6,gi.6.6)
SW-06-08
2,3
gi.12.1
SW-08-05
1/1
7600
FarmSWG2 1
7:3-8:3
7:4-8:4
7:5-8:5
7:6-8:6 BD
FarmSWG4
7:1-8:1
(5/2,6/1)
St.34(gi.3.1,gi.4.1)
St.36(gi.3.3,gi.4.3)
SW-08-03
gi.12.4
gi.12.3
SW-AC
St.35(gi.3.2,gi.4.2)
St.37(gi.3.4,gi.4.4)
gi.12.2
SW (sc)
1:1,2:1
St.33
SW-03-02
SW-03-03
SW-03-04
SW-03-05 Bo
GARR
gi.6.8
FarmSWG1
SW-08-08
SW-08-07
Cat6500
St.11(gi.5.3,gi.6.3)
SW-06-05
(xg.7.1,xg.14.1)
FarmSWF2
FarmSWG3
TIER1 – Network
10Gb/s

7. Monitoring
The Local Area Network is tuned up to make the Wide Area Bandwidth available to the main services without loss of performances due to bottle necks.
Network Monitoring Tools:
MRTG :Every port of every switch is monitored with MRTG
NAGIOS: All the network infrastructure is controlled with NAGIOS ( and SMS alarms on critical events)
NTOP: An NTOP server is used for troubleshooting or Instant analysis of the traffic.

8. T1 Security
ACLs Are applied (in ASIC) on the WAN interfaces of the Switch Routers and between different internal networks (More than 500 rules!).
IDS systems (argus) are in test phase.
Logs are centrally managed and log analysis tools are in study. (Now only simple parsing scripts are used in production).

9. 10 Gb/s on Wan (CERN-CNAF)
Sperimentation
10 Gb/s on Wan (CERN-CNAF)
Tests on 10 Gb/s on LAN and WAN (Recent tests between CERN and CNAF reached 7.5 Gb/s from a single host!)
Service challenges support
Network configuration and tuning activity for Service Challenges (Make bandwidth available from WAN to disk and Tape servers).

10. Evolution and milestones
Before LHC data taking starts…
Better maintenance contracts on Core Switches with on-site intervention in 4-8 hours and guaranteed problem solving (or device substitution) in 1 day. (negotiation in progress…)
Second 10 Gb/s For T1-T1 and Backup for T1-T0 Connection (in 2007).
Core Switch acquisition (in substitution of the Enterasys ER16) (In one or two years .. Depending on the growth of the center)

11. Manpower People: Stefano Zani (Staff) Riccardo Veraldi (Staff)
Donato De Girolamo (Temporary)
Giuseppe Sansonetti (Temporary) only 50%
Every member of the Network Service has to be able to operate on the Routing Switching and firewalling infrastructure (3 of us).
One more person is necessary to guarantee continuity on service availability (holiday time, Conferences, etc..)