1. K. Schauerhammer, K. Ullmann (DFN)
Architecture for the T2 network for LHC data evaluation - Proposed technical solution -
K. Schauerhammer, K. Ullmann (DFN)
Meeting „Requirements for LHC data analysis“, Göttingen, 24 January 2011

2. Contents Status LHC networking
Requirements for T2 access bandwidth and connectivity
Intl. T2 network: proposed architecture
The approach in Germany: HEPPI
Funding issues
Next steps

3. Status LHC Networking
T0 – T1/T1-T1 Traffic: LHCOPN for LHC data delivery from CERN (T0) to T1
Data and compute model changes: no longer a strict hierarchical structure T1-T2
Problem to be solved: How to organise access from T2/T3 centres to LHC data?

4. Requirements for Tier-2 access bandwidth and connectivity (I) (Bos / Fisk, 09/10)
Minimal
Small Tier-2s, well suited for end-use analysis
Nominal
Nominal sized Tier-2s, big analysis samples can be updated regularly
Leadership
Large Analysis Centers, supporting many users, frequent cache turnovers
Connectivity
Shared, best effort connectivity, no guaranteed bandwidth between each of the T2s 1
Gb/s 5
Gb/s 10
Gb/s

5. Requirements for Tier-2 access bandwidth and connectivity (II) (Bos / Fisk, 09/10)
All LHC experiments, but in the first place ATLAS and CMS, would benefit greatly from better connected T2s
The Leadership T2s are mostly in Europe and North America and need 10 Gb/s to connect to other T1s and T2s
Nominal T2s need a 5 Gb/s (access) connection to the same infrastructure
All (other) T2s should at least have 1 Gb/s access bandwidth (Minimal)
The infrastructure needs to be flexible to allow easy changes and expansion
T2s outside Western Europe and Northern America need a special approach

6. Intl. T2 network: proposed architecture (I)
EU-US capacity:
Several 10G links - demand oriented
RENATER
GARR
ESNET/
USLHCNet
GEANT
Capacity reserved for LHC and according to demand
T1 centre
DFN
Other NRENs
T2 (or T3)

7. Intl. T2 network: proposed architecture (II) – T2 network and LHCOPN
Separate networks but interconnected via T1s

8. Intl. T2 network: proposed architecture (III)
Requirements of K. Bos / I. Fisk paper (09/10) achieved
Common (federated) backbone structure with a separate bandwidth slice within Europe, US and demand driven connectivity EU-US; good connectivity to the rest of the world (on IP level)
Capacity upgrades in parts of this backbone (Geant+transatlantic+ESNet/USLHCNet) according to needs and demands easily implementable

9. Intl. T2 network: proposed architecture (IV)
Financing of this structure is clearly a national matter – the federated architecture takes this into account
The architecture provides a scalable system, i.e. other ntl. LHC communities can easily join through their NRENs.
Architecture compatible with „normal“ IP structure
The system is relatively easy and fast to implement with known workflows (in the NRENs)
The system is connected with LHCOPN through the T1s however not violating the LHCOPN AUP

10. Intl. T2 network: proposed architecture (V) - next steps
Detailed planning has started with all actors
Goal: to do a final design until spring 2011 and have an early implementation during summer 2011 already (having in mind that the experiment‘s pressure may increase very soon)

11. The plan for the German HEP community:
High Energy Physics Private Infrastructure
- HEPPI -

12. Example 1 (DFN-GEANT access)

13. Example 2 (DFN backbone)

14. Example3: FRA-AAC (2 x 10GE)

15. Example3: FRA-AAC (2 x 10GE)

16. Approach in DE: HEPPI (I) (High Energy Physics Private Infrastructure)
Sites to be initially connected to HEPPI:
T1 : KIT
T2 s: DESY HH, RWTH Aachen, GSI Darmstadt, LMU/MPI München, Uni Wuppertal, Uni Göttingen, Uni Freiburg
Optional: plus about 13 T3 centres
Design goal: VPN with excellent global connectivity and access bandwith determined finally by every T2.

17. Approach in DE: HEPPI (II)
10 Gbps
1 Gbps

18. Approach in DE: HEPPI (III)
X-WiN
Uni LAN
HEPPI VPN
see *1)
Uni LAN
T2-LAN HR XR
DFN Internet KR T3 T3 T3
HR – XR : BGP

19. Approach in DE: HEPPI features (IV)
T2s connected to HEPPI with dedicated lines (10, 5 or 1Gbps) which can be real physical access lines or a VLAN in an existing access line of the institution.
T3 centers connected to the DFNInternet cloud (i.e. through their normal institutional access mechanism) and their traffic pattern will be monitored. At some point in time they may connect to HEPPI as well when the traffic pattern requires that step.

20. Approach in DE: HEPPI features contd. (V)
Concept extendable in respect to T3s for example and in respect to capacity which is organized separate from the DFNInternet capacity
fulfills the specification given in the Bos / Fisk paper as the external connectivity is organized through BGP
HEPPI will be connected to Geant (again logically through BGP) with 10Gbps in the beginning which extends its reachability on the global scale via BGP.

21. Approach in DE: HEPPI features contd. (VI)
HEPPI will be connected to Geant (logically through BGP) with 10Gbps in the beginning which extends its reachability to the global scale via BGP
Transatlantic capacity must be organized appropriately to other networks similar to „HEPPI“ in North America and specifically to ESNet/USLHCNet
upgrade of transatlantic capacity within a short time frame

22. Funding issues Costs roughly 1 M€/a for the start configuration
Costs must be covered nationally
DFN can provide several options for payments (non exhaustive list):
Option a) X-WiN like (per institutional access)
Option b) larger lump sum plus reduced yearly contribution according to option (a)
Option c) through one single institution
Option d) ...

23. Next steps (HEPPI)
T2s should classify themselves according to the requirements paper
T2s should either order the access to HEPPI themselves or ask DESY to organise that on their behalf
First deployment should be made soon in order to have a first version available for summer 2011