University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Application-Level Network Performance / Measurement Tools Jason Leigh, Oliver Yu, Alan Verlo, Linda Winkler

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Tele-Immersion is the synthesis of Virtual Reality, video conferencing, and advanced computation

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Example of a Data Intensive Tele-Immersion Application TIDE: Tele-Immersive Data Explorer Collaborative Large Scale Data Visualization In collaboration with National Center for Data Mining General framework for collaborative visualization of massive data-sets Current data-set is ozone data from NOAA

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Example of a Data Intensive Tele-Immersion Application CIBRView: Collaborative Image Based Rendering Viewer Cosmology Hydrodynamic code by Julian Borrill, LBNL/NERSC shows theoretical condensation of diffuse matter into string- like formations during early stages of universe evolution Accesses volume data: 512x256x256x 256 frames ~ 40Gig data- sets Generates image slices that are distributed to collaborating clients Sent about 500, 1M slices/files from Chicago to Japan using parallel TCP. It was also the application over which we tested DiffServ.

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) BWLatency Sensitive Jitter Sensitive Reliability Reqrmnt BurstinessPacket Size Protocol Avatar (virtual representation) LowYY ConstantSmallRTP / UDP Audio conference MedYYLowConstantMedRTP / UDP Video conference HighYYLowConstantLargeRTP / UDP Realtime state updates LowYYMedShort bursts SmallFEC Non-realtime state updates LowNNHighApp dependent SmallTCP Small bulk data (image files) MedNNHighMedium bursts LargeTCP/CPTCP Large bulk data (raw data sets) HighNN Very long bursts LargePTCP / RBUDP Streaming bulk data (high quality audio/video/bit map/polygons) HighYN Long bursts LargeSRBUDP / AFEC Characteristics of Tele-Immersion Flows

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Motivation for Application-level networking tools Bandwidth is becoming increasingly available. Networking QoS is still under research and difficult to deploy and use. It is not as easy as flipping a switch. Network QoS is not only about bandwidth, its about latency and jitter. Applications today are not ready to use the extra bandwidth even if available. Application developers have to be increasingly network savvy in order to be able to convert application requirements to networking services. Need to make advanced networking easier for the average application developer. Need to provide a higher level application framework to keep pace with network advances.

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Advanced Data Transport Techniques for Tele-Immersion Maybe we can compensate for latency: –Reliable Low-Latency Data Transfer for Tele- Immersion Even if you had QoS could you really take advantage of it? –High Throughput Techniques for Tele- Immersion

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Reliable Low-Latency Data Transfer for Tele-Immersion In Tele-Immersion it is desirable to be able to transmit state information with minimum latency and jitter while preserving reliability Rather than use TCP which uses acknowledgments to obtain reliability, try UDP augmented with error correction codes: Forward Error Correction

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) ` FEC greatest benefit is in small packets. Larger packets impose greater overhead. As redundancy decreases FEC approaches UDP. goal

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) G o a l

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Packet Loss over UDP vs FEC/UDP between Chicago & Amsterdam 50Mbps UDP or FEC +50Mbps UDP congestion Packet Loss UDP1.90% FEC0.05% UDP with congestion17.40% FEC with congestion4.15%

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) High Throughput Techniques for Tele- Immersion In Tele-Immersion it is desirable to share data files as rapidly as possible Even if you had QoS, you couldnt take advantage of it Long Fat Network problem: an ftp session will max out at 3.5Mbps over a 100Mbps link between Chicago and Amsterdam (and Switzerland) 2 Techniques: –Parallel TCP Socket Striping –Reliable Blast UDP

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Parallel Sockets : 64K Window Size Amsterdam (SARA) to Chicago (EVL) Found it difficult to achieve more than 50Mbps on a 100Mbps link. Have been able to achieve 80Mbps on rare occasions.

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Reliable Blast UDP (RBUDP) RBUDP - An old idea that may be useful now that networking bandwidth is increasing Use UDP for bulk data transmission rather than TCP If bandwidth can be guaranteed by QoS – reliability will be high- chances of errors will be few

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) RBUDP Performance (EVL to SARA)

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) RBUDP Performance (EVL to CERN)

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) DiffServ results over a testbed between EVL and Argonne National Lab 25Mbps 80Mbps ANL EVL 42Mbps 100Mbps fore back x x xx Cisco7507 Bandwidth recovery good Latency recovery not good Packet loss double + background+ DiffServ 150ms 1-way latency

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Collaborative Coordination Experiments between Chicago and Singapore Tightly coordinated collaborative interaction task between 2 remote users 200ms RTT is the threshold where performance begins to suffer 200ms RTT with 0 jitter is same as 10ms RTT with 7ms jitter So DiffServ is not suited for Tele-Immersion

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) G2 is C++ toolkit for building Tele-Immersive applications with special emphasis on networking G2 is the Grids main tele-immersion library Networking: –UDP, TCP, Multicast, HTTP. –UDP reflector and multicast bridge. –TCP reflector. –Remote procedure calls. –32 and 64bit Remote file I/O. –Parallel 32 & 64 bit TCP socket striping for high throughput data delivery. –FEC. –Client/Server distributed shared memory persistent database. –Threading, Mutual Exclusion. –Built-in Instrumentation of networking services.

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Tools for higher level application development: –Audio streaming –Articulated Avatars –VR navigation –VR menus –Speech recognition with IBM ViaVoice –Collaborative application shell to jumpstart development –Network visualization tools

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) QoS Internet Monitoring Tool QoSIMoto Provides real time viewing of CAVERNsoft data streams Visualizes bandwidth, latency, jitter of multiple network flows Accepts Netlogger compatible format

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) These techniques are needed more than ever in STAR LIGHT Being able to dedicate lambdas will ease our latency and jitter problems but we still cant beat the speed of light. Lambda switching requires more intelligence at the edges to perform traffic shaping, bandwidth management, error correction. Need to provide high level API for applications to select Lambdas and define which application flow will go over the lambda. Long Fat Network problems do not go away with GMPLS. Need to provide a higher level application programmers framework to keep pace with network advances. We need more than just sockets( ) API.

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) What new things can we do with STAR LIGHT? Streaming uncompressed high resolution stereoscopic 3D movies (1024x768x24bitsx30fps) ~ 1.05 Gbps Imagine magnifying this to multi-tiled displays for ultra high resolution displays such as CAVEs and Active Murals

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) What new things can we do with STAR LIGHT? Distributed Tera-snap: Perform a tera-mining correlation between distributed databases and generate a visual overview 1 PC can absorb ~500Mbps => min 4.6 hours to perform a tera-snap 20 PCs can absorb data and produce image composites in a min of 13 mins using 10Gbps

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) What new things can we do with STAR LIGHT? Digital Continuums: Distributed collaboratories with linked tiled, tele-immersive and mobile displays enhanced with cluster computing and data mining services

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Thanks Thanks SARA (Amsterdam), CERN (Switzerland), IHPC (Singapore) for graciously participating in these network experiments For more info: –