1. OSCARS Roadmap Chin Guok (chin@es.net)
Energy Sciences Network
Lawrence Berkeley National Laboratory
Feb 6, 2009
Networking for the Future of Science

2. OSCARS Status
Development of a community approach that will support end-to-end virtual circuits in the R&E environment is coordinated by the DICE (Dante, Internet2, Caltech, ESnet) working group (which involves many more organizations than the original group)
Each organization potentially has their own InterDomain Controller approach (though the ESnet/Internet2 OSCARS code base is used by several organizations (flagged OSCARS/DCN)
The DICE group has developed a standardized InterDomain Control Protocol (IDCP) for specifying the set up of segments of end-to-end VCs
While there are several very different InterDomain Controller implementations, they all speak IDCP and support compatible data plane connections
The following organizations have implemented/deployed systems which are compatible with the DICE IDCP:
Internet2 Dynamic Circuit Network (OSCARS/DCN)
ESNet Science Data Network (OSCARS/SDN)
GÉANT2 AutoBahn System
Nortel (via a wrapper on top of their commercial DRAC System)
Surfnet (via use of above Nortel solution)
LHCNet (OSCARS/DCN)
Nysernet (New York RON) (OSCARS/DCN)
LEARN (Texas RON) (OSCARS/DCN)
LONI (OSCARS/DCN)
Northrop Grumman (OSCARS/DCN)
University of Amsterdam (OSCARS/DCN)
DRAGON (U. Maryland/MAX) Network
The following "higher level service applications" have adapted their existing systems to communicate via the user request side of the IDCP:
LambdaStation (FermiLab)
TeraPaths (Brookhaven)
Phoebus (UMd)

3. OSCARS Production VCs (as of 01/27/2009)

4. OSCARS Evolution Phase 1 Phase 2 Phase 3 (Current) Phase 4
Proof of concept
Intra-domain virtual circuit (VC) services
Phase 2
Inter-domain interoperability
Pre-production ESnet VC services
Phase 3 (Current)
Productionalizing OSCARS (IDC)
Phase 4
Extending service offerings
Framework for research
Explore “on-ramp” technologies

5. Network Mechanisms Underlying OSCARS
Layer 3 VC Service: Packets matching reservation profile IP flow-spec are filtered out (i.e. policy based routing), “policed” to reserved bandwidth, and injected into an LSP.
Layer 2 VC Service:
Packets matching reservation profile VLAN ID are filtered out (i.e. L2VPN), “policed” to reserved bandwidth, and injected into an LSP.
LSP between ESnet border routers is determined using topology information from OSPF-TE. Path of LSP is explicitly directed to take SDN network where possible.
On the SDN Ethernet switches all traffic is MPLS switched (layer 2.5).
Best-effort IP traffic can use SDN, but under normal circumstances it does not because the OSPF cost of SDN is very high
SDN
SDN
SDN
SDN Link
SDN Link
RSVP, MPLS, LDP
enabled on
internal interfaces
Sink
Label Switched Path
IP Link
Source IP IP IP
IP Link
ESnet
WAN
bandwidth
policer
high-priority queue
MPLS labels are attached onto packets from Source and
placed in separate queue to ensure guaranteed bandwidth.
standard, best-effort queue
AAAS
Ntfy APIs
Resv API
WBUI
OSCARS Core
PSS NS
OSCARS
Server
PCE
Regular production traffic queue.
Interface queues

6. OSCARS Initial Implementation
IDC
InterDomain
Controller
User App
Proof of concept
User
ESnet Public
WebServer (Proxy)
Source
Sink
SDN IP
IP Link
SDN Link
ESnet
WAN
WBUI
Web Based User Interface
Reservation API
WS Interface
OSCARS Core
Reservation Management
Path Computation
Scheduling
Inter-Domain Communications
PSS
Path Setup Subsystem
Network Element Interface
AAAS
Authentication Authorization Auditing Subsystem
ESnet IDC
(OSCARS)
HTTPS
HTTPS (SOAP)
Function Calls
SSHv2

7. OSCARS Current (ver 0.5) Implementation (2Q09)
IDC
InterDomain
Controller
User App
Well defined inter-module interfaces
Exchange of static topology information
PCE integrated into OSCARS Core
User
ESnet Public
WebServer (Proxy)
Source
Sink
SDN IP
IP Link
SDN Link
ESnet
WAN
Notification Call-back Event API
WBUI
Web Based User Interface
Notification Broker API
Resv API
WS Interface
OSCARS Core
Reservation Management
Path Computation
Scheduling
Inter-Domain Communications NS
NotificationSubsystem
PSS
Path Setup Subsystem
Network Element Interface
AAAS
Authentication Authorization Auditing Subsystem
ESnet IDC
(OSCARS)
HTTPS
HTTPS (SOAP)
RMI
SSHv2

8. OSCARS Future Implementation (4Q09)
Exchange of dynamic topology information
includes time dimension
PCE separated from OSCARS Core
PCEs can be daisy changed
allows PCE to be pluggable
facilitates a research framework for collaboration
IDC
InterDomain
Controller
User App
User
ESnet Public
WebServer (Proxy)
Source
Sink
SDN IP
IP Link
SDN Link
ESnet
WAN
Notification Call-back Event API
WBUI
Web Based User Interface
Notification Broker API
Resv API
WS Interface
OSCARS Core
Reservation Management
Scheduling
Inter-Domain Communications NS
NotificationSubsystem
PSS
Path Setup Subsystem
Network Element Interface
AAAS
Authentication Authorization Auditing Subsystem
PCE
Path Computation Engine
ESnet IDC
(OSCARS)
HTTPS
HTTPS (SOAP)
RMI
SSHv2

9. Depends on the following factors
SDN “On Ramp” Options
There are a number of ways to move flows from the ESnet IP cloud to the SDN
Depends on the following factors
Control: ESnet admin or site admin or user
Advanced reservations or on-demand
Manual vs. automatic
End-to-end vs. border to border
Level of expertise needed
Layer 1 vs layer 2 vs layer 3 circuit
ESnet Confidential

10. Example: ESnet traffic engineering to avoid hot-spots
Off-load congested IP circuits and traffic engineer around hot-spots
Transparent to site networking
Detailed monitoring data of ESnet traffic is needed to determine optimal solution

11. Example: LHC Tier 1 – Tier 2 data movement
Site configures BGP between CE routers to use SDN VC if available
User initiates request to OSCARS for SDN VC

12. Example: Fusion Simulation at NERSC writing results to ORNL over GPFS
Site admin/user requests VC from any IDC
Local domain controller (e.g. OSCARS/LambdaStation/Terapaths) performs VC setup
Typically scheduled in advance

13. Example: Climate data movement from ANL to LLNL
User points to closest Phoebus host
Use Phoebus to forward data to SDN
Phoebus host requests OSCARS for SDN VC on behalf of the user
Phoebus host could be part of ESnet or site infrastructure

14. Example: APS user sending data to home institute
Router forwards packets to monitoring device (e.g. Juniper MS PIC, Bro host, etc)
Monitoring device identifies candidate flows for SDN, requests SDN VC
Monitoring devices could be part of ESnet or part of site infrastructure