Logistical Networking Micah Beck, Research Assoc. Professor Director, Logistical Computing & Internetworking (LoCI) Lab Computer Science Department University of Tennessee SciDAC Kickoff Mtg Collaboratory & Networking Jan 17, 2002

Logistical Optimization of Distributed Applications Principle Investigators –Micah Beck –James S. Plank –Jack Dongarra –Rich Wolksi (UCSB) Research Staff –Alex Bassi –Terry Moore –Graziano Obertelli (UCSB) Funding –DoE SciDAC High Performance Network –NSF Next Generation Software Graduate Students –Scott Atchley –Erica Fuentes –Ding Jin –Sharmila Kancherla –Wo Ling –Xiang Li –Stephen Soltesz (ug) –Martin Swany (UCSB) –Sathish S. Vadhiyar –Yong Zheng

Why “Logistical” Networking Analogy to military or industrial logistics Distributing goods requires not only highways but also warehouses (depots) Data transmission is the highway Storage servers are the depots “Scalably sharable” network storage Enabling new users and applications

Internet Backplane Protocol (IBP) Servers that make allocation of primitive “byte arrays” available to clients Byte arrays are not blocks (more abstract) –Network capabilities (primitive security) –Variable extents Byte arrays are not files (weaker semantics) –Size & duration are limited –“Volatile” allocations –Best effort reliability and availability –No directory structure, accounting –No caching, replication An end-to-end approach to network storage

Application Areas Source routing Bandwidth adaption Reducing ( BW  delay) Reliable multicast Content Distribution Data Grids Remote access to structured data Managing computation state (NetSolve caching) Temporary storage Very large data sets Collaborative computing & visualization

Dimensions in Communication Routing (spatial)Storage (temporal) Logistical Networking

PIs: Micah Beck, Jack Dongarra, James S. Plank / Tennessee; Rich Wolski / UCSB Impact and Connections  IMPACT:  Improved performance and scalability of data-intensive distributed application  Greater ease of and lower cost of deployment of new wide area data management strategies  Dramatically improved flexibility in data-intensive collaboration  CONNECTIONS:  SciDAC: Net100, Data Grid, Scalable Systems, Data Mgt, Computational Science (e.g. Climate, Supernovas)  Base:Network Monitoring, Data Grid, Transport Protocols, Storage Res. Mgt., IQ-Echo, Milestones/Dates/Status –IBP applications demonstrated at SC’01 –exNode support in NetSolve –Reliability/performance coscheduling alpha –Allocation policy simulation – Initial generalized caching infrastructure –Initial logistical overlay network on ESNet –Wide-area logistical peering mechanisms and policies –Resolution for highly volatile storage resources –Experimental IBP architectures –Large scale measurement and simulations Novel Ideas  Storage is too cheap to hoard.  Storage can be a scalably shared network resource.  Logistical Networking gives applications and middleware uniform control over buffering and routing of data.  Data storage and data transport can be viewed as points on a spectrum of data management mechanisms.  Monitoring and prediction can replace reservation as a means of scheduling storage resources.  End-to-end networking principles can apply to storage. Logistical Networking: Developing a communicative infrastructure with persistence Tasks: -develop/deploy network storage depots -develop layered storage stack & tools -develop/validate scheduling techniques -optimize application performance loci.cs.utk.edu Date Prepared: 1/10/02 Logistical Networking – SciDAC High Performance Networking MICS Program Manager: Thomas Ndousse 6-12mos 12mos 12-18mos 18-36mos