Supporting Advanced Scientific Computing Research Basic Energy Sciences Biological and Environmental Research Fusion Energy Sciences High Energy Physics Nuclear Physics Chin Guok ESnet OSCARS

Multi-Layer Network Architecture View QoS MPLS Layer2 Control Layer1 Control VLANs SONET Optical IP MPLS Layer3 Services Layer2 Services Layer1 Services Grid Middleware Applications Application, Middleware Management Application, Middleware security AA PlaneControl Plane Data Plane Service Plane Management Plane Manage ment Plane AA Manage ment Plane Manage ment Plane Layer 3 Layer 2 Layer 1 Application, Middleware Layer Security

Capabilities Addressed by OSCARS QoS MPLS Layer2 Control Layer1 Control VLANs SONET IP MPLS Layer3 Services Layer2 Services Layer1 Services Grid Middleware Applications Application, Middleware Management Application, Middleware security AA PlaneControl Plane Data Plane Service Plane Management Plane Manage ment Plane AA Manage ment Plane Manage ment Plane Layer 3 Layer 2.5Layer 2Layer 1.5 Layer 1 Application, Middleware Layer Security Optical

OSCARS Service Plane Reservations Requested WBUI or WS-API (using DICE IDCP) Bandwidth request in granularity from 1Mbps to 10Gbs Start/End time with granularity of 1 min Service Handoff Layer 3 Service Accepts/delivers IP packets from/to user unaltered (e.g. preserves DSCP bits, etc) Layer 2 Service Accepts/delivers Ethernet VLAN packets from/to user unaltered (e.g. preserves VLAN ID, etc)

OSCARS Data Plane MPLS LSP MPLS LSP is configured from ESnet PE-to-PE device with strict hops based on reserved path computed Bi-directional MPLS LSP (for Layer 2 service) can be asymmetric MPLS packets are tagged (exp bit) to use a specific QoS queue to guarantee bandwidth and isolate traffic from “regular” IP traffic Layer 3 VC Service IP packets filtered on ESnet PE ingress interface according to user provided IP flow specification Filtered packets injected into MPLS LSP Layer 2 VC Service Uses EoMPLS to inject Ethernet VLAN (user negotiated ID) packets from PE ingress interface into MPLS LSP

OSCARS Control Plane OSPF-TE Topology discovery and distribution RSVP-TE Provisioning and signaling MPLS-TE Label switching LDP Label distribution to support EoMPLS QoS Configuration of distinct QoS queue across ESnet WAN to guarantee bandwidth and isolate VC traffic

OSCARS AA Plane Authentication User/Password for WBUI (over SSL) Signed (X.509) SOAP messages for WS-API (over SSL) Authorization Permission to create/modify/cancel reservations per user level Site coordinator role permits modify/cancel reservations for any VC that terminates at corresponding site regardless of requesting user

OSCARS Current (v0.5) Implementation User User App AAAS Authentication Authorization Auditing Subsystem Notification Broker API Resv API WBUI Web Based User Interface OSCARS Core - Reservation Management - Path Computation - Scheduling - Inter-Domain Communications PSS Path Setup Subsystem - Network Element Interface NS Notification Subsystem HTTPS HTTPS (SOAP) RMI SSHv2 ESnet Public WebServer (Proxy) WS Interface IDC InterDomain Controller Source Sink SDN IP IP Link SDN Link ESnet WAN SDN Notification Call-back Event API Well defined inter-module interfaces Exchange of static topology information PCE integrated into OSCARS Core ESnet IDC (OSCARS)

Network Mechanisms Underlying OSCARS Best-effort IP traffic can use SDN, but under normal circumstances it does not because the OSPF cost of SDN is very high Sink MPLS labels are attached onto packets from Source and placed in separate queue to ensure guaranteed bandwidth. Regular production (best-effort) traffic queue. Interface queues SDN IP IP Link SDN Link RSVP, MPLS, LDP enabled on internal interfaces standard, best-effort queue high-priority queue LSP between ESnet border (PE) routers is determined using topology information from OSPF-TE. Path of LSP is explicitly directed to take SDN network where possible. On the SDN all OSCARS traffic is MPLS switched (layer 2.5). explicit Label Switched Path SDN Link Layer 3 VC Service: Packets matching reservation profile IP flow-spec are filtered out (i.e. policy based routing), “policed” to reserved bandwidth, and injected into an LSP. Layer 2 VC Service: Packets matching reservation profile VLAN ID are filtered out (i.e. L2VPN), “policed” to reserved bandwidth, and injected into an LSP. ESnet WAN bandwidth policer AAAS Ntfy APIs Resv API WBUI OSCARS Core PSSNS OSCARS IDC PCE Source