Logistical Networking as an Advanced Engineering Testbed Micah Beck, Assoc. Prof. & Director Logistical Computing & Internetworking (LoCI) Lab Internet2 Spring Member Meeting April 10, 2003

Panel Participants Gabriella Paolini, GARR (by phone) Topic: IPv6 Geoff Hayward, Yotta Yotta Topic: Wide Area Storage Networking Jim Ferguson, NCSA Topic: Web 100 / TCP Tuning

Funding Dept. of Energy SciDAC National Science Foundation ANIR UT Center for Info Technology Research Logistical Networking Research at UTK University of Tennessee Micah Beck James S. Plank Jack Dongarra University of California, Santa Barbara Rich Wolski

What is Logistical Networking A scalable mechanism for deploying shared storage resources throughout the network A general store-and-forward overlay networking infrastructure A way to break transfers into segments and employ heterogeneous network technologies on the pieces

Why “Logistical Networking” Analogy to logistics in distribution of industrial and military personnel & materiel Fast highways alone are not enough  Goods are also stored in warehouses for transfer or local distribution Fast networks alone are not enough  Data must be stored in buffers/files for transfer or local distribution

The Network Storage Stack Applications Logistical File System Logistical Tools L-Bone IBP Local Access Physical exNode Our adaption of the network stack architecture for storage Like the IP Stack Each level encapsulates details from the lower levels, while still exposing details to higher levels

IBP: The Internet Backplane Protocol Storage provisioned on community “depots” Very primitive service (similar to block service, but more sharable) Goal is to be a common platform (exposed) Also part of end-to-end design Best effort service – no heroic measures Availability, reliability, security, performance Allocations are time-limited! Leases are respected, can be renewed Permanent storage is to strong to share!

Models of Sharing: Logistical Networking Moderately valuable resources Storage, server cycles Sharing enabled by relative plenty Internet-like policies Loose access control No per-use accounting Primary design goal: scalability Application autonomy Resource transparency Burdens of scalability The End-to-End Principles Weak operation semantics Vulnerability to Denial of Service

Data Movers Module implementing standard point-to- multipoint transfer between IBP allocations Uniform API allows independence from the underlying data transfer protocol Not every DM can apply to every transfer Caller responsible for determining validity Current options: Multi-TCP, Multi-UDP (reliable), UDP Multicast (unreliable)

The Network Storage Stack The L-bone: Resource Discovery & Proximity queries IBP: Allocating and managing network storage (like a network malloc) The exNode: A data structure for aggregation LoRS: The Logistical Runtime System: Aggregation tools and methodologies

The Logistical Backbone (L-Bone) LDAP-based storage resource discovery. Query by capacity, network proximity, geographical proximity, stability, etc. Periodic monitoring of depots. 10 Terabytes of shared storage. (with plans to scale to a petabyte...)

L-Bone: January 2003 Current Storage Capacity: 10 TB

The Network Storage Stack The L-bone: Resource Discovery & Proximity queries IBP: Allocating and managing network storage (like a network malloc) The exNode: A data structure for aggregation LoRS: The Logistical Runtime System: Aggregation tools and methodologies

The exNode The Network “File Descriptor XML-based data structure/serialization Map byte-extents to IBP buffers (or other allocations). Allows for replication, flexible decomposition of data. Also allows for error-correction/checksums Arbitrary metadata.

ExNode vs inode exNode inode IBP Allocations the network local system disk blocks kernel capabilities block addresses user

The Network Storage Stack The L-bone: Resource Discovery & Proximity queries IBP: Allocating and managing network storage (like a network malloc) The exNode: A data structure for aggregation LoRS: The Logistical Runtime System: Aggregation tools and methodologies

Logistical Runtime System Basic Primitives: Upload, Download, Augment, Refresh End-to-end Services Checksums, Encryption, Compression

Download Movie

Multithreaded Transfers

Routed/Multipath

Point-to-Multipoint

Heterogeneous Multicast

Caching/Staging

Further Advanced Capabilities IBP over IPv6 Dual stack depot Specialized DataMovers UDP (SABUL, Tsunami) Fiber Channel over IP Non-standard TCP Stacks Web 100 Future: FAST?

Panel Participants Gabriella Paolini, GARR (by phone) Topic: IPv6 Geoff Hayward, Yotta Yotta Topic: Wide Area Storage Networking Jim Ferguson, NCSA Topic: Web 100 / TCP Tuning

Conclusions IBP is a global testbed for advanced network engineering Transfer rates routinely exceed 100Mbps New Data Movers under development can reach current applications Dedicated depots can support global testbed for kernel modificaitons