Establishing End-to-End Guaranteed Bandwidth Network Paths Across Multiple Administrative Domains The DOE-funded TeraPaths project at Brookhaven National Laboratory investigates the combination of DiffServ-based LAN QoS with WAN MPLS tunnels and dynamic circuits in establishing true end-to-end (host-to-host) virtual paths with bandwidth guarantees. These virtual paths can be associated with specific data flows and are managed through advance reservations, providing the capability of network resource scheduling. Scheduling network resources is critical in a data-intensive computing environment such as that of the ATLAS experiment. WAN Minimum Best Effort traffic Dynamic bandwidth allocation Shared dynamic class(es) Dynamic microflow policing Mark packets within a class using DSCP bits, police at ingress, trust DSCP bits downstream Dedicated static classes Aggregate flow policing Shared static classes Aggregate and microflow policing TeraPaths Site A Site B Site C Site D WAN 1 WAN 2 WAN 3 service invocation data flow peering WAN chain WAN WAN web services TeraPaths 1 2 3 The TeraPaths software can partition a site’s available network bandwidth into multiple classes of service with statically or dynamically assigned bandwidth. Virtual paths are stitched together by directly configuring end site LANs and interfacing with WAN services to provision a suitable, “DSCP-friendly” path between these end sites. WAN border router MPLS tunnel ingress/egress router WAN switch border router The project has successfully developed techniques to configure LAN devices while satisfying site security requirements. TeraPaths can utilize both L3 and L2 connections. L2 connections (dynamic circuits) require proper VLAN configuration at the end sites and Policy Based Routing (PBR) within a site’s LAN for VLAN selection. A secondary effect of dynamic circuit utilization is per-data flow route selection in addition to guaranteed bandwidth. The TeraPaths software is fully distributed. A site instance comprises a set of web service modules. TeraPaths services are accessible via both a web interface and an API, which can be used by 3rd party clients. current US ATLAS T2 sites Internal Services Public Services Web Interface Admin Module NDC • • • Database protected network API remote local WAN Services proxy CLI s/w client LAN/MPLS TeraPaths resource manager MPLS requests traffic identification: addresses, port #, DSCP bits grid AAA Bandwidth Requests & Releases OSCARS ingress / egress LAN QoS M10 data transfer management monitoring GridFtp & dCache/SRM SE network usage policy ESnet remote Remote LAN QoS requests The TeraPaths testbed currently includes QoS-enabled subnets at BNL and at the University of Michigan and is being expanded to all US ATLAS Tier 2 sites. These sites are interconnected through ESnet and Internet 2. The testbed enables valuable networking research for the benefit of the Physics community and beyond. CHEP 2007