1. 1 SINET3 L1-Ondemand Service Interface MIddleware, 27 th APAN Meeting March 3 rd, 2009 Motonori Nakamura, Shigeo Urushidani National Institute of Informatics (NII)

2. SINET3: Science Information Network 3  SINET3 is the new Japanese academic backbone network launched in April 2007 for more than 700 universities and research institutions.  It has 63 edge and 12 core nodes and deploys Japan’s first 40 Gbps lines between Tokyo, Nagoya, and Osaka. 2 10 Gbps 622 Mbps : 40 Gbps : 10 to 20 Gbps : 1 to 20 Gbps : Core Node : Edge Node Japan’s first 40 Gbps (STM256) lines Los Angeles New York Tokyo Nagoya Osaka

3. 3 Service Features in SINET3  SINET3 emphasizes four service aspects: transfer layer, virtual private network (VPN), quality-of-service (QoS), and bandwidth on demand.  It provides all services on a single network platform, and users can freely choose the best transfer layer services for their applications. ServicesExamples ★ Multiple Layer Services L3 (IP), L2 (Ethernet), & L1 (dedicated line) ★ Enriched VPN Services Support for collaborative research among distant sites with closed user group environment ★ Enhanced QoS Services Support for performance-sensitive applications ★ Bandwidth-on-demand (BoD) Services Support for data-intensive applications

4. 4  SINET3 provides bandwidth-on-demand (BoD) services as part of layer-1 services.  Users can specify the destinations, duration, bandwidth with granularity of about 150Mbps, and route option, via simple Web pages.  BoD server receives path setup requests from users, calculates the appropriate routes, schedules accepted reservations, and triggers layer-1 path setup. 1 Gbps (13:00-14:00) 2 Gbps (17:00-18:00) 1 Gbps (15:00-16:00) Bandwidth on Demand (BoD) Services User Web-based Interface (Destination, Duration, Bandwidth, & Route option) On-demand layer-1 path Layer-1 path setup trigger SINET3 Layer-1 BoD Server Tokyo Hokkaido Osaka Fukuoka

5. 5 Architecture for BoD Services L2 MUX GMPLS control and management plane L1SW Layer-1 BoD Server GMPLS  BoD server receives reservation requests, performs path calculation, schedules accepted requests, and triggers layer-1 path setup to source layer-1 switch.  Source layer-1 switch sets up layer-1 path toward destination using GMPLS.  BoD server changes path bandwidth for L2/L3 traffic by LCAS via L1-OPS as needed. User L2 MUX IP Router Path setup trigger Hitless bandwidth change by LCAS Destinations, Duration, Bandwidth, & Route Option Path setup request Scheduling Path control Route calculation Resource management Front-end IP Ethernet On-demand L1-OPS

6. 6 Service Parameters of L1 BoD Services  BoD server allows users to specify connection style + destinations, duration, bandwidth, & route option via Web-based interface. VPNExtranet Public Connection Style + Destinations : VPN-A : Non-VPN : VPN-B Pre-configured interfaces Duration - Start Time & - Finish Time (in 15 minute intervals) Bandwidth GE STM-16 STM-64 GE 10GE VC-4-7v VC-4-17v VC-4-Av VC-4-Bv 1 ≤ A ≤ 7 1 ≤ B ≤ 64 STM-64 Lambda (Full bandwidth)Bandwidth-specified VC-4 Granularity (approx. 150 Mbps) Route Option - “Minimum Delay” or - “Unspecified”

7. L1SW Considerations on Path Calculation  BoD server selects path (route and links) by taking into account following conditions. (1) Each link has different available bandwidth for L1 services which varies over time. (2) Each link has different delay which is a fixed value. (3) There are parallel links between core nodes. (4) There are multiple routes between source and destination nodes Link Bandwidth Available bandwidth for L1 services (1) Available bandwidth for L1 L2/L3 Traffic Pattern (3) Parallel Links (4) Multiple Routes L1SW Router Link Aggregation & Load Balancing L1 Path VCAT MonTueWedThuFriSatSun HiroshimaKyoto OsakaNagoyaTokyo1 Fukuoka Matsuyama Kanazawa Tokyo2 VCAT 1.05 Gbps (VC-4-7v) 0.6 Gbps (VC-4-4v) 0.45 Gbps (VC-4-3v) Tokyo1TsukubaSendai Kanazawa Tokyo2 Sapporo 12ms 7ms 3ms5ms 1ms (2) Delay 7

8. Doshisha Univ. L1SW Backbone Topology and Current BoD User Sites : Core L1SW : Edge L1SW Hiroshima L1SW Kyoto L1SW Fukuoka L1SW Kanazawa L1SW Tokyo2 L1SW Sapporo L1SW Tsukuba L1SW Sendai L1SW Matsuyama L1SW Nagoya L1SW-1 Osaka L1SW-2 Osaka L1SW-1 Tokyo1 L1SW-1 Tokyo1 L1SW-2 Tokyo1 L1SW-3 Nagoya L1SW-2 NAOJ L1SW Osaka Univ. L1SW NII L1SW KEK L1SW Kyushu Univ. L1SW Yamaguchi Univ. L1SW Hokkaido Univ. L1SW NIFS L1SW SINET3 has 16 core layer-1 switches and 63 edge layer-1 switches, and has multiple routes and parallel links between core layer-1 switches 8

9. Sample Reservation Screen 9

10. createSNC REQ Path registration REQ Path registration RESP Path setup REQ createSNC RESP Path setup CMPLD Path info retrieving REQ Path info retrieving RESP Notification (create CMPLD) Path setup RESP L1-BoD Server L1-OPSL1SW deleteSNC REQ Path release REQ deleteSNC RESP Path release CMPLD Path deregistration REQ Path deregistration RESP Notification (delete CMPLD) Path release RESP getSNC REQ getSNC RESP Path info retrieving REQ Path info retrieving RESP Interface Between BoD Server and L1-OPS CORBA (TMF-814) TL1 10

11. 11 Current Projects using L1 BoD Services (1)  Three projects (eVLBI, high-quality remote backup, and new video communication) are using L1 BoD services. High-quality remote backup projecteVLBI project : 2.4 Gbps : 0.15G to 1 Gbps : L1 Switch Detected Fringe (June 12 th ) Yamaguchi Gifu Tsukuba Tomakomai Hokkaido Univ. Kyushu Univ. Osaka Univ. NII&NTT (Tokyo) NAOJ (Tokyo) NAOJ: National Astronomical Observatory of Japan * VLBI: Very Long Baseline Interferometory

12. Current Projects using L1 BoD Services (2)  t-Room --- a room-sharing video system that allows people to simultaneously experience "distant space" and "remote time“. Users feel as if they are in the same room.  Folding the spaces of Kyoto, Atsugi, present, past onto the space where you are overlapping spaces (rooms) and overcoming time and space constraints. Present Local Room 1 Room 2 Room 3 Past Kyoto Present Atsugi “Monolith” Building Module: side view (left) and front view (right). 195 cm 142 cm 47 cm 65’’ LCD PanelPC s HDV Camera Effective Screen Size: 142 cm x 80 cm Pathway 65’’ LCD Panel HDV Camera 3.0 m Monolith 12

13. Examples of Path Setup/Release Time 0 1 2 3 4 5 6 1234567891011121314 0 1 2 3 4 5 6 7 8 9 10 11 1234567891011121314 [min] The number of transit switches [min] The number of transit switches Path setup/release time (17) Setup (in parallel) Setup (in series) Release (in parallel) Release (in series) (7) (4) (1) (7) (4) (2) (1) 1.05 Gbps (setup) 600 Mbps (setup) 150 Mbps (setup) 1.05 Gbps (release) 600 Mbps (release) 150 Mbps (release) (a) e-VLBI project(b) High-quality remote backup project (17) Tsukuba - NAOJ Gifu - NAOJ Yamaguchi - NAOJ NII – Hokkaido Univ. Osaka Univ. – Hokkaido Univ. Kyushu Univ. – Hokkaido Univ.  Setup(release) time was defined as the difference between the time at that BoD server sends “create(delete)SNC REQ” and the time at that it receives “notification (create(delete) CMPLD)”.  We first created each path in series after receiving “notification (create CMPLD)” but we refined the mechanism to create paths in parallel right after receiving “createSNC RESP.” 13

14. Layer-1 BoD Server More General Architecture for BoD Services L2 MUX GMPLS Control and Management Plane L1SW GMPLS L1-OPS Router PC HTTP(S) Server CORBA Reservation-based Service (Destinations, Duration, Bandwidth, & Route Option) Admission control, Scheduling Path and bandwidth control Path calculation Resource management Front-end BoD Users Signalling-based Service (Destination and Bandwidth) L2 MUX Hitless Bandwidth Change by LCAS Path for L2/L3  We are planning to provide GMPLS-UNI-based services in addition to reservation-based services. Forwarding adjacency (FA) paths are preliminary established to manage the services.  BoD server receives the information of GMPLS-UNI paths via L1-OPS. If GMPLS-UNI paths are established on unexpected routes, BoD server forcibly tears down them. IP Ethernet GMPLS-UNI 14

15. Open Issues  Admission control toward full-scale operations If the total requested bandwidth exceeds the available bandwidth of a link, we try to rearrange pre- assigned paths for “unspecified” routes to accommodate as many paths as possible. If the rearrangement fails, the BoD server informs the network operators about the situation. We seek negotiated solutions whereby network operators change the bandwidth and duration among users while we limit the number of users of the BoD services. We need an effective admission control algorithm that fairly selects from among the requests.  Improvement of layer-1 path setup/release times We would like to improve the path setup/release times but this depends on the specifications of vendor products.  Dissemination of BoD services to new scientific research areas We would like to explore new scientific research areas which effectively utilize the properties (low delay, no delay variance, and no packet losses ) of on-demand layer-1 paths. 15

16. References 1.S. Urushidani, J. Matsukata, K. Fukuda, S. Abe, Y. Ji, M. Koibuchi, S. Yamada, K. Shimizu, T. Takeda, I. Inoue, and K. Shiomoto, “Layer-1 bandwidth on demand services in SINET3,” IEEE Globecom 2007, Dec. 2007. 2.S. Urushidani, K. Fukuda, Y. Ji, S. Abe, M. Koibuchi, M. Nakamura, S. Yamada, K. Shimizu, R. Hayashi, I. Inoue, and K. Shiomoto, “Resource allocation and provision for bandwidth/networks on demand in SINET3,” 2 nd IEEE International Workshop on Bandwidth on Demand, April 2008. 3.S. Urushidani, S. Abe, Y. Ji, K. Fukuda, M. Koibuchi, M. Nakamura, S. Yamada, R. Hayashi, I. Inoue, and K. Shiomoto, “Design of versatile academic infrastructure for multilayer network services,” IEEE Journal on Selected Areas in Communications, April 2009 (to appear). 4.S. Urushidani. K. Shimizu, R. Hayashi, H. Tanuma, K. Fukuda, Y. Ji, M. Koibuchi, S. Abe, M. Nakamura, S. Yamada, I. Inoue, and K. Shiomoto, “Implementation and evaluation of layer-1 bandwidth-on-demand capabilities in SINET3,” IEEE ICC2009, Jun. 2009 (to appear). 16