1. 1 Applications and Cheetah Outline eScience vs. commercial networks Three modes of bandwidth sharing large-m small-m, long-held calls small-m, short-duration calls Applications Malathi Veeraraghavan University of Virginia mv5g@virginia.edu MCNC meeting, April 10, 2006

2. 2 Circuit/Virtual-Circuit technologies eScience networksCommercial networks Raison d'etre High-throughput connectivity between a few facilities Moderate-throughput connectivity between millions of users Key goal QoS guaranteesBandwidth sharing (different style from TCP based sharing) Call duration Long-heldShort-duration ReachEnd-to-endPartial (HOV lanes)

3. 3 Goal of eScience networks On previous slide, we said "QoS guarantees" But usage of the term "networks" implies bandwidth sharing If bandwidth is not shared, we have a "link" Leased line service

4. 4 What's the difference? leased line service service provider is given enough lead time to add interface modules to meet request eScience networks being designed today above is not true requests are handled by an automated scheduler and granted or denied within a short lead time this is bandwidth sharing this is "book-ahead" or "advance reservations"

5. 5 GMPLS control plane How useful is it to these two types of networks? Purpose of GMPLS control plane Bandwidth (BW) sharing (dynamic& distributed) Provisioning (threading the circuits/VCs) BW sharing mode Immediate request (can't specify a future call- initiation time or call holding time in protocols) Call accepted or rejected: "call blocking"

6. 6 Understanding call blocking mode of BW sharing Input parameters Link capacity: e.g. 10Gbps Express this as m channels If per-call BW is 1Gb/s, m = 10 m is comparable to the number of bank tellers Traffic load measure of call arrival rate (calls/sec) and mean call holding time (seconds/call) Measure of success of BW sharing mode Call blocking probability (% of calls blocked) Link utilization

7. 7 Strong dependence on m: if m=10, to run link at an 80% utilization level, call blocking probability will be a high 23.62% Call blocking probability vs. m Utilization m is a measure of high-throughput vs. moderate-throughput difference between eScience and commercial networks For high-throughput, m is small

8. 8 Dependence on call duration? (eScience: long-held; commercial: short-duration) Consider traffic load Typically, if mean call duration (holding time) is high, call arrival rate is low Traffic load is a product of these two parameters For a given call blocking probability, link utilization and m, the required traffic load is a fixed value Dependence on call duration unclear

9. 9 But BW sharing does depend on mean call holding time if m is small Mean waiting time is proportional to mean call holding time Can afford to have a queueing based solution if calls are short Large m Moderate throughput Small m Short callsLong calls Bank tellerDoctor's office High throughput immediate-request with call blocking call queueingbook-ahead

10. 10 eScience: small-m, long-held Does a book-ahead BW sharing mechanism help with the high call blocking probability problem? We analyzed and simulated two types of Book- Ahead (BA) mechanisms BA-n: caller specifies n call-start time options BA-all: caller is willing to accept any open time slot Parameters: K: reservation time horizon (in timeslots): 2000 Two call classes:  class-1 holding time: 100 (h 1 ); 10% of calls (r 1 )  class-2 holding time: 300 (h 2 ); 90% of calls (r 2 )

11. 11 Call blocking probability (m=10, K=2000, l=2, h 1 =100, h 2 =300, r 1 =0.1, r 2 =0.9) Xiangfei Zhu, xz4p@virginia.edu

12. 12 Utilization (m=10, K=2000, l=2, h 1 =100, h 2 =300, r 1 =0.1, r 2 =0.9) Xiangfei Zhu, xz4p@virginia.edu

13. 13 Observations BA-n/BA-all performs better than IR eg., if we want to achieve 80% utilization, the call blocking probabilities using different mechanisms are IR, 22.6% BA-3, 8.3% BA-5, 4% BA-10, 2% BA-all, almost 0% BA-1 performs worse than IR because of the interaction between the two call classes if class-1 calls reserve timeslots with gaps, class-2 calls are denied Xiangfei Zhu, xz4p@virginia.edu

14. 14 Third type of BW sharing Calls with small-m, short-duration Call queueing mode Have a couple of ideas, but need to study delayed start distributed queueing at callers (like CSMA-CD)

15. 15 Status check Outline eScience vs. commercial networks Three modes of bandwidth sharing large-m small-m, long-held calls small-m, short-duration calls  Applications

16. 16 Applications (small-m, long-held) Cheetah project supports TSI (Terascale Supernova Initiative) Very large dataset (TB) transfers Ensight for remote visualization Other apps with high-throughput long-duration calls (good for book-ahead) Other eScience applications e-Learning (small-USBs at each desk - better quality) Access Grid, Videnet video conferencing, teleimmersion Distributed Grid computing - MPI apps, e.g. blast (recruit clusters - 6 hours) IPTV/VOD/HDTV/Triple-play video streaming (entertainment) Asynchronous Storage Area Network support (nightly backups)

17. 17 Applications Large-m applications High-quality video-telephony (10Mbps) at every desk (3 min. average durations) 100MB to GB range file transfers: through web (Red Hat Linux distribution) GridFTP, PVFS, NFS/XFS web caching Remote storage: LORS IBP Depots, synchronous SAN operations Gaming (warcraft, battlenet) currently written for low-BW; need good Graphics cards if higher-BW is available can more information be moved with simpler lower- cost graphics cards for a lower-lag experience? OfficeLive, "network is the computer" model, old dumb-terminal model lower admin costs Others? Suggestions? Small-m (high-throughput), short-duration calls 100MB to GB range file transfers Want to give 1Gbps per transfer to lower delays

18. 18 Status check Outline eScience vs. commercial networks Three modes of bandwidth sharing large-m small-m, long-held calls small-m, short-duration calls Apps for large-m  video-telephony web downloads and caching

19. 19 Video telephony Paul Sijben writes: "Today, video telephony usually implies that a group of people gather around a table and watch a TV showing a similar group of people around another table. Personal video telephony usually means watching postage stamp sized people in a PC screen, whose image is refreshed occasionally." Also, "As has been known for quite some time, the nuances of face, body and arm gestures add a wealth of information to communication." Can we exploit higher-speeds (10Mbps, OC1, OC3 rates) for better TV-like video-telephony? Delay requirement: 150ms one-way Compress less, use more bandwidth for lower latency and higher quality

20. 20 Sony SNC-RZ30 Camera Ethernet output: 640 x 480 @ 30 FPS ~ 8Mb/s using Motion-JPEG Built-in web server Total system latency ~109ms (90-160ms observed) Timer courtesy of www.Auvidea.com Tyson Baldridge, tyson@virginia.edu

21. 21 Video Products Group – VPG5720 Maps video signal onto a Sonet OC3 I/O Connections: Video: SDI or NTSC/PAL Audio: AES/EBU or analog A/V Sync within 10mS Unidirectional: need a TX & RX module at both endpoints. Need one chassis per endpoint: VPG6200 Esoteric solution, but ideal for testing best-case performance Any experience? Tyson Baldridge, tyson@virginia.edu

22. 22 Other issues Automating "camera-man," "director" roles Movement sensor based positioning of camera? Lighting issues Eye contact Placement in offices LCD projectors on to walls? Multiple camera feeds - hence the director role? Not really teleimmersion but a better experience to improve remote communications (save energy costs, less travel!) Suggestions?

23. 23 Status check Outline eScience vs. commercial networks Three modes of bandwidth sharing large-m small-m, long-held calls small-m, short-duration calls Apps for large-m video-telephony  web downloads and caching

24. 24 File transfer application Two CHEETAH solutions Web file transfers across CHEETAH (end-to-end) Web caching (partial-path circuits)

25. 25 Web File Transfer on CHEETAH Consists of a software package, called WebFT Leverages CGI for deployment without modifying web client and web server software Integrated with CHEETAH end-host software APIs to provide use of CHEETAH network in a mode transparent to users Xiuduan Fang, xf4c@virginia.edu

26. 26 WebFT Architecture Web server Web client Web Server (e.g. Apache) CGI scripts (download.cgi & redirection.cgi URL Response WebFT sender OCS APIRD API RSVP-TE API C-TCP API Web Browser (e.g. Mozilla) WebFT receiver RSVP-TE API C-TCP API Control messages via Internet Data transfers via a circuit OCS daemon RD daemon RSVP-TE daemon RSVP-TE daemon Xiuduan Fang, xf4c@virginia.edu Cheetah end-host software APIs and daemons Cheetah end-host software APIs and daemons OCS: Optical connectivity service RD: Routing Decision C-TCP: Circuit TCP

27. 27 Experimental Testbed and Results for WebFT--cont. Zelda3 and Wukong: Dell machines, running Linux FC3 and ext2/3, with RAID-0 SCCI disks RTT between them: 24.7ms on the Internet path, and 8.6ms for the CHEETAH circuit. CHEETAH Network Internet zelda3 NIC I NIC II wukong NIC I NIC II IP routers Sycamore SN16000 MCNC, NC Sycamore SN16000 Atlanta, GA Xiuduan Fang, xf4c@virginia.edu

28. 28 Experimental Results for WebFT--cont. The web page to test WebFT Test parameters: Test.rm: 1.6 GB, circuit rate: 1 Gbps Test results throughput: 680 Mbps, delay: 19 s Xiuduan Fang, xf4c@virginia.edu

29. 29 Web caching (partial-path circuits) Uses the web proxy software package, squid, to make CHEETAH accessible to non- CHEETAH hosts. Improves web performance by Breaking up the long-distance connectionless (CL) path into a partial circuit through CHEETAH, and two low-RTT CL sub-paths Leverages web caching protocols Xiuduan Fang, xf4c@virginia.edu

30. 30 The Framework for using web caching and partial-path circuits Internet CHEETAH Application Gateway (CAG) CHEETAH Application Gateway (CAG) Web client Web server Original HTTP messages HTTP messages HTTP and ICP messages HTTP messages squid Xiuduan Fang, xf4c@virginia.edu

31. 31 Experimental Testbed Configure caching hierarchy such that zelda1 ’ s NIC II is wuneng ’ s parent. Configure CAGs to cache file with the size < 4 GB RTT for the Internet path: ballstein and wuneng: 14.6 ms RTT for the CHEETAH path wuneng and zelda1: 8.9 ms Xiuduan Fang, xf4c@virginia.edu

32. 32 Experimental Results for Web Partial CO Transfer Web server parameterstotal RTT (ms) through CHEETAH file size (MB) delay (s) through name location RTT (ms) withCHEETAH, cached? Intern et path zelda1ballstein NoYes www.kernel.orgAshland, Oregon 61.686.014.6 + 8.9 + 61.6 = 85.1 48331870 internap.dl.sour ceforge.net Atlanta, GA 6.032.014.6 + 8.9 + 6.0 = 29.5 11314036520 www.cs.virginia. edu Charlot tesville, VA 25.0<114.6 + 8.9 + 25.0 = 48.5 113503014 Xiuduan Fang, xf4c@virginia.edu

33. 33 Future Work Decide whether to use a partial-path circuit by examining the client ’ s geographic location relative to server Use squid to connect multiple circuit/VC networks to further reduce RTT Test the scalability and reliability of squid Model, analyze and measure performance improvement by using a partial-path circuit Xiuduan Fang, xf4c@virginia.edu

34. 34 Thanks for listening! Conclusions: Add MPLS and/or SONET to create large-m (moderate- BW) mode of sharing Enable partial-path circuits - through Policy Based Routing to isolate flows (trigger signaling from host) Avoiding BW partitioning for IR and BA is hard because IR calls have unlimited holding time while BA needs specified holding time Centralized scheduler per domain is acceptable if BA load is low Suggestions, comments, thoughts? email address: mv5g@virginia.edu

35. 35 Centuar FastIron FESX448 1G Compute-0-4 152.48.249.6 Orbitty Compute Nodes 1G OC192 GbE 1-8-33 1-8-34 1-8-35 1-8-36 1-6-1 1-6-17 1-8-37 MCNC Catalyst 7600 H H H H H 1G 1-7-1 Compute-0-3 152.48.249.5 Compute-0-2 152.48.249.4 Compute-0-1 152.48.249.3 Compute-0-0 152.48.249.2 1G Wukong 152.48.249.102 1-8-38 1-7-17 cheetah-nc 3x1G VLAN OC192 1-6-1 1-6-17 10GbE 1-7-1 GbE 1-7-33 1-7-34 1-7-35 1-7-36 1-7-37 1-7-38 1-7-39 1G Zelda1 10.0.0.11 H H H 1G Zelda2 10.0.0.12 Zelda3 10.0.0.13 1G Zelda4 10.0.0.14 H H Zelda5 10.0.0.15 2x1G MPLS tunnels 1G Cheetah-atl OC-192 lamda 10GbEGbE 1-7-33 1-7-34 1-7-35 1-7-36 Cheetah-ornl 1-7-11-6-1 OC192 X1(E)UCNS 1GFC 1G Juniper T320 Juniper T320 1G Force10 E300 switch ORNL Atlanta NC Direct fibers VLANs MPLS tunnels Wuneng 152.48.249.103 H 1-8-39 H 1G UVa Catalyst 4948 WASH HOPI Force10 WASH Abilene T640 NCSU M20 2x1G MPLS tunnels CUNY Foundry NYC HOPI Force10 1G UVa host H CUNY host H 1G UVa CUNY CHEETAH Network Xuan Zheng, xuan@virginia.edu

36. 36 Current status Thanks to HOPI: UVA and CUNY connectivity to CHEETAH almost done Software completed RSVP-TE end host clients C-TCP transport protocol OCS - DNS based solution to check connectivity C-VLSR for Ethernet switches WebFT application Current focus: TSI support + growth of network + large-m applications