1. 1 Applications for dynamically shared GMPLS networks Outline Quick summary of CHEETAH project "Business" orientation Technical details of CHEETAH Malathi Veeraraghavan University of Virginia mvee@virginia.edu Sept. 24, 2007

2. Router-to-router leased Ethernet- SONET-Ethernet or SONET circuits (red) Backbone network (e.g., Abilene) Enterprise networks WAN-access router Regional (metro) network Enterprise networks WAN-access router Regional (metro) network e.g., CD-CIs e.g., T640s if colocated in same PoP if not colocated in same PoP

3. Server-to-server circuits (rather than router-to-router) Cheetah studies: Focused on the use of circuits from server to server Since servers only have Ethernet NICs, the circuits were all Ethernet-SONET-Ethernet circuits Focused on enabling dynamic sharing of circuits Leased lines between servers would likely be unjustifiable (from cost perspective)

4. High-speed justification For router-to-router circuits, "high-speed" is required because of aggregation For server-to-server, our justification was for file transfers Higher the rate, faster the transfer

5. Applications we developed for experimentation with GMPLS networks Given that a significant % of file transfers involve the Web, we experimented with two Web based file-transfer applications:  Simple Web client to Web server transfers  Goal: Use GMPLS network without changing Web client or Web server software  Problem: GMPLS networks need to stretch end-to-end  Web proxy servers located at core-network PoPs  Goal: If GMPLS network can be only deployed in the core initially, deploying proxies allows even non- connected end hosts located in enterprises to benefit from core GMPLS network's high speeds.

6. Quick summary of Cheetah project Deployed a wide-area experimental SONET GMPLS network Three PoPs: Raleigh, Atlanta, ORNL Intercity OC192s purchased from NLR and ORNL Colo services purchased from MCNC, SLR, ORNL GbE interface cards for server connectivity Located 2 to 3 servers + GMPLS switch (SN16000) at each PoP Developed core software RSVP-TE client for the server Circuit-TCP for transport protocol on circuits Modified Web applications to interface with the RSVP-TE client to request circuit setup before transfers, and release after Ran our Cheetah core software on HOPI Interconnected Cheetah to HOPI testbed

7. Outline check Outline Quick summary of CHEETAH project  "Business" orientation  Revenues: potential market - applications  Costs Technical details of CHEETAH

8. Business orientation Choose applications [for development, experimentation and demonstration] after taking into account "business" considerations So we started by classifying applications suitable for different types of GMPLS network deployments and services

9. Services & applications (for "dynamic circuit" networks) Leased linesTCP/IPCoarse Grained Sharing (CGS) Fine Grained Sharing (FGS) Coarse Grained Sharing High-bandwidth circuits, AND "Long" holding times Need Book-Ahead (BA) support in the control-plane (scheduling or advance reservations) Fine Grained Sharing Moderate-BW circuits, and/or Short holding time Immediate-Request (IR) mode sufficient in the control-plane. GMPLS networks

10. 10 Services & applications (for "dynamic circuit" networks) DCS-network scope Bandwidth- sharing modes Dynamic circuit services in the core network ONLY Dynamic circuit services are intraregional Dynamic circuit services involve regional and core networks Coarse Grained Sharing ISP router-to- router Long- Distance (LD) leased lines ASP server-to- server LD lines? Disaster- recovery (DR) Server- replication WAN access- link rate change Business interconnect eScience applications Video-conferencing Distance-learning Fine Grained Sharing Web services (proxy, CDN) IPTV/video distribution (CDN) Inter-SMTP server Software-on- the-web Backup-storage WAN access- link rate change Business interconnect Video-telephony Row/column headings: define service types Entries in the body cells: applications Blue: router-to-router

11. One sample point To support the case for providing GMPLS network based dynamic circuit services between PoPs MCI network has 2500 PoPs throughout North America and 2000 around the globe! Are there SMTP servers, CDN servers and other applications servers that need interconnectivity?

12. Video and Content Delivery Network (CDN) The rise of You-tube and video is often cited as a reason for growth in bandwidth and network equipment sales CDN example providers: Akamai CDN servers placed in PoPs Requests from clients served from closest CDN server Use high-speed GMPLS networks in the core to move files between CDN servers

13. Catch? Akamai does a trade with regional Research-and-Education Networks (RENs) Place servers in regional REN PoP Regional REN pays for collocation costs (power, space, remote hands-and-eyes) Regional REN gains by cutting the required rate for the circuit it purchases for IP connectivity from core IP service provider

14. "Dynamic CDN" CDN service is comparable to "leased line" service A web service provider enters into an agreement with a CDN provider to serve out its content What about small-to-moderate sized enterprises? Can they recruit CDN servers located at a few PoPs if they expect a sudden surge of traffic to their web servers (e.g., slashdot phenomenon)? If so, use dynamically setup high-speed circuit to copy over the whole web structure (esp. with databases) to dynamically recruited CDN servers

15. Storage Three types of applications: Disaster recovery (DR): backup of critical data Server replication: e.g., of web servers (to allow for quick switchover in case of failures) Backup storage: of ordinary enterprise users' data

16. DR and server replication Typically, only these two types require network connectivity outside the enterprise Small-to-moderate sized enterprises only require intraregional DCS services (if used) general rule of thumb: 75-mile distance of backup site hence listed in column 2 of services/applications classification table Fortune-500 companies with multiple locations require DCS across regional AND core hence listed in column 3 of services/applications classification table

17. DR and server replication Requirements Few endpoints or users initiating these apps. Few transfers a day Is IP-routed network sufficient?

18. Backup storage, on the other hand "Backup storage" application If a new "storage ASP" emerged, which sold backup storage services for "all" data in enterprises, then given  the large number of employees,  who could initiate backup at any time if they want to save an important file as they make modifications, could justify needing high-speed DCS networks Is bandwidth cheaper than HR costs to hire engineers to maintain backup storage at each enterprise?

19. "Blue" vs "black" applications in table ApplicationsListed in Blue Listed in Black EndpointsRouter-to- router Server-to- server Target market for "encroachment" Leased line services IP services Volume and priceLow volume; High per- unit price High volume Low per-unit price

20. Router-to-router circuits Services (Verizon): Provide network administrator web portal access to explicitly request an increase in leased-line rate e.g., if GbE interface used, but rate capped with VLAN rate-limiting, allow for rate limit to be increased (signaling if leased line realized through SN16000s). Software that reads SNMP MIBs to monitor usage on leased line, and automatically issue signaling request for bandwidth increase Both ideas: aggregate traffic based increase/decrease requests

21. Per-transfer based increases Even if link is lightly loaded, a single file transfer delay can be reduced by increasing the bandwidth of the bottleneck (lowest-rate) link. e.g., an enterprise has an OC3 WAN access link. Even if this link is lightly loaded, this becomes the max. rate that any single file transfer can enjoy. By dynamically increasing this rate for a few seconds, user can enjoy a higher transfer rate. Need tools to determine if WAN access link is the bottleneck link on an end-to-end path, and then increase rate.

22. Costs Started by seeing Internet2 fee structure http://www.internet2.edu/network/fees.html

23. Why GMPLS in core network? Because high-speed interface cards cost less in SONET switches than in IP routers For high switching capacity nodes, which are mainly required in the core.

24. What is the major component of cost? Service provider costs: Same for IP-routed and SONET networks HR costs Bandwidth costs Differ: Equipment costs:  Mainly line card costs If bulk of the costs are in HR and bandwidth, then equipment cost differentials become less significant

25. Summary Opportunity to increase potential market for GMPLS switches We have access to three GMPLS testbeds on which we can test applications and gain experience with R&E users Internet2's DCS, HOPI, Cheetah Choose application(s) carefully with due consideration of business aspects Looking for support: Student HR support to implement "glue" software to make applications run on GMPLS networks, and to build usage base Cheetah testbed annual maintenance charges

26. Outline check Outline Quick summary of CHEETAH project "Business" orientation  Revenues: potential market - applications  Costs  Technical details of CHEETAH

27. Cheetah concept Hybrid architecture: an IP-routed network AND a GMPLS network Use dynamically setup circuits for file transfers Send small files on IP-routed path and use GMPLS network only for large files: call-setup overhead CHEETAH: Circuit-switched High-speed End-to-End ArcHitecture GMPLS network

28. CHEETAH End-host Software Circuit-TCP: TCP minus congestion control Optical connectivity service (uses DNS servers) Determines which path to use: IP-routed or Circuit

29. CHEETAH End-host Software RSVP-TE client software architecture Connection Admission Control: check if bandwidth is available on the UNI from the host to the switch (multiple VLANs) Configure IP routing and ARP table since remote host is reached directly on the newly setup circuit

30. 30 CHEETAH testbed Long-distance OC192s purchased from NLR and ORNL Collocation services purchased from MCNC in NC, SLR in Atlanta Zeldas and wukong/wuneng: Linux Dell PCs

31. 31 Interconnection of CHEETAH to US-wide HOPI experimental testbed HOPI CHEETAH HOPI: Hybrid Optical/Packet Infrastructure: Internet2 supported testbed Force10 E600s used to dynamically setup and release VLANs (virtual circuits)

32. 32 10GbE GbE Force10 Tech. transfer: CHEETAH control plane software modified for HOPI CCPM: CHEETAH Control-Plane Module OSPFD RSVPD Force10 programming module CCSA: CHEETAH Client System Agent RSVPD CHEETAHD Circuit-requestor Circuit setup procedure losa-pc1: Use circuit-requestor to initiate setup to sttl-pc1 sends PATH meesage losa-CCPM: Route computation, CAC, VLAN ID assignment sttl-CCPM: Route extract, Local CAC and VLAN ID check sttl-pc1: Configures VLAN, programs ARP and route tables Sends back RESV message sttl-ccpm: Programs sttl-Force10 for that VLAN losa-ccpm: Programs losa-Force10 for that VLAN; sets rate policing losa-pc1: Configures VLAN, programs ARP and route tables LOSA 10GbE GbE Force10 CCSA CCPM pc1 pc2 pc3 Internet STTL CCSA CCPM pc1 pc2 pc3

33. 33 Setup a circuit from losa-pc1 to sttl-pc1

34. 34 Automatic configuration on the end host

35. 35 Setup multiple circuits to the same remote end host Request exceeding the available bandwidth is rejected.

36. Internet2's new Dynamic Circuit Services (DCS) network Yellow nodes: Ciena CD-CI SONET switches Blue nodes: Juniper T640 IP routers Courtesy: Rick Summerhill (2006)

37. Testbeds Three "GMPLS" wide-area testbeds are available for testing and demonstrating new applications for GMPLS networks Cheetah HOPI Internet2's DCS network

38. Control messages via Internet Application: WebFT Web server Web client Web Server (e.g. Apache) CGI scripts (download.cgi & redirection.cgi URL Response WebFT sender OCS APIRD API RSVP-TE API C-TCP API Web Browser (e.g. Mozilla) WebFT receiver RSVP-TE API C-TCP API Data transfers via a circuit OCS daemon RD daemon RSVP-TE daemon RSVP-TE daemon Cheetah end-host software APIs and daemons Cheetah end-host software APIs and daemons PROBLEM: Need GMPLS networks to be deployed within regional and enterprise networks, not just the core

39. 39 Application: circuit-aware web proxy servers IP-routed network Core-only GMPLS network Web client Web server Original HTTP messages HTTP messages HTTP and ICP messages HTTP messages squid A web proxy software package: squid "Circuit-aware" by integrating RSVP-TE & CTCP Dynamic circuit setup triggered by web client request Use of circuits transparent to human users Use Internet path while circuit is being setup