1. Data Plane Testing on USN, ESnet, and Internet2 Networks
ESCC/Internet2 Joint Techs Summer Meeting
July 16, 2007
Fermi Lab
Batavia, Illinois
Tom Lehman (USC/ISI)
Nasir Ghani (Tennessee Tech)
Chin Guok (ESnet)
Nagi Rao (ORNL)
John Vollbrecht (Internet2)
John Moore (MCNC)
Hybrid MLN
US Dept. of Energy
Office of Science

2. Outline Dataplane testing configuration Dataplane test data
Dataplane simulation
Dataplane test conclusions
Future Work

3. Hybrid Network Data Planes
Basic premise of hybrid networks is the availability of both best effort routed service and deterministic dedicated resource paths, i.e., circuits
There are many technologies available over which to construct these circuits
IP router-based Multiprotocol Label Switching (MPLS) Label Switched Paths (LSPs) “circuits”
Ethernet VLAN based “circuits”
SONET/SDH TDM “circuits”
Wavelength Division Multiplexing (WDM) “circuits”

4. What Data Plane Technologies to Use?
What do you want to do with your circuits?
Dedicated bandwidth connections for deterministic file transfers?
Dedicated bandwidth & low jitter for instrument control or interactive applications?
Connector backhaul to your IP Network?
Traffic engineering of your IP Network?
Dynamic router-to-router circuits for traffic cut thru?
Computer to Computer communications?
Processor to memory? block data storage system access?
Setting up application specific topologies to create & optimize distributed application or data storage systems?

5. Data Plane Testing
Test characteristics/performances of “circuits” constructed via different technologies; and also “end-to-end paths” constructed via concatenation of individual circuits
Questions
What is difference between the different technologies?
How well does the concatenation/stitching work?
How well does policing/shaping work at the edge?
What happens to a flow that is policed/shaped at the ingress edge by the time it exits the egress edge?

6. Data Plane Testing Data Paths Across: ESnet USN Abilene
Internet Network
DRAGON

7. Test Equipment Spirent AX4000 - Hardware based Traffic Source and Sink
External CDMA Clock allows for synchronized timestamps
Spirent AX4000
10 Gbps with OC192
POS / BERT / 10GbE
Two Gigabit Ethernet

8. Data Collection
Approximately 75 individual tests (generally a unique path)
tests were sourced from 1 Gbps interfaces, some from 10 Gbps
some of the measured flows had cross traffic introduced
Tests generally included 9 measured data collection runs
64, 500, 8000 byte MTU
100, 500, 800 Mbps for 1 Gbps paths
1, 5, 8 Gbps for 10 Gbps paths
For each test run, the following data was collected:
average datarate
total packet loss
average latency
jitter profile (histogram)
transfer delay (histogram)

9. HOPI-Abilene-UltraScience Net-ESnet Test
Histograms
packet inter-arrival
packet end-to-end delay
Spirent source  Washington-Force 10  Washington-Juniper T640  Chicago-Juniper T640 Chicago-Force 10 Chicago-Glimmer Glass  Chicago-Force 10 Chicago-Cisco 6509  Seattle-Juniper T640  Sunnyvale-Juniper T640 Sunnyvale-Force 10  Sunnyvale-CDCI  Seattle-CDCI  Chicago-CDCI 
Chicago-Force 10  Chicago-Juniper T640  Washington-Juniper T640 Washington-Force 10  Spirent receiver

10. Circuit Description Example
The formal description of this extended inter-network path:
Circuit type:
usn [ethernet:tdm:ethernet]:i2dsn [ethernet:tdm:ethernet]:esnet [ethernet:pscq:ethernet]:usn[ethernet:tdm:ethernet]
Circuit path:
usn [ORNL:CHIN]:i2dsn [CHIN:WASH]:esnet [WASH:CHIN]:usn [CHIN:STTL:SUNV]

11. OPNET ModelerTM Environment “In-House” Development
Modeling & Simulation
OPNET ModelerTM Environment
Overview
Discrete event simulation
GUI interface, high re-use
Full C/C++ interface
Hierarchical modeling:
Subnet-node-link-process
“In-House” Development
MPLS/GMPLS control:
RSVP-TE, OSPF-TE, PCE
Layer 2/3 data plane:
IP/MPLS, VLAN
Full Layer 1 support:
DWDM, SONET,GFP
Model any networks

12. Test Plans, Reports, and Data Repositories
Test Report and Plans
 DataPlane Testing and Analysis
Raw data repositories

13. Summary/Conclusions
All of the tested networking technologies (PSC, L2SC, TDM, LSC) and networks (ESnet, USN, Abilene, HOPI, DRAGON) performed well both individually and when concatenated together
There are some key differences observed between the various networking layer technologies when driven at or close to bottleneck capacity
QoS techniques applied to router MPLS or Ethernet switched paths exhibited notably different delay behaviors versus dedicated circuit-paths (TDM)
TDM-based infrastructures is most germane for applications requiring stringent guarantees on latency, jitter, and bandwidth protection
Inter-layer cross-connections can be achieved in a reasonable manner by “stitching” together different network layer technologies.
Ethernet VLANs presents the least problematic demarc (automated techniques needed to coordinate VLAN tag space)
Future Work
impact of ingress traffic "burstiness” on end-to-end delay and loss profiles, i.e., both for reference and interfering cross-traffic streams.
best techniques for ingress policing and transit node QoS
vendor interoperability testing
additional network testing

14. Questions & Comments ? Tom Lehman tlehman@east.isi.edu
Thank-You
Questions & Comments ? Tom Lehman