1. Wide Area Data Sharing with Logistical Networking Micah Beck, Assoc. Prof. & Director Logistical Computing & Internetworking (LoCI) Lab Computer Science Department mbeck@cs.utk.edu End-to-End Workshop, MiamiFeb 5, 2003

2. »Funding Dept. of Energy SciDAC National Science Foundation ANIR UT Center for Info Technology Research »LoCI Lab developers Logistical Networking Research »University of Tennessee Micah Beck James S. Plank Jack Dongarra »University of California, Santa Barbara Rich Wolski

3. The Data Sharing Problem »Large data objects created as byproducts of common operations »A large community of potential collaborators that might need access to the data »Asynchrony between collaborators (especially when in different time zones) »No single administrative domain »No centrally managed resource pool (DB or FS) »Control of access to data is necessary

4. The Internet in Collaboration »Providing bandwidth resources on demand »Network community is fluid, loosely organized »Any two endpoints can communicate »User authentication, security are managed by endpoints »But: there is no persistence, hence no direct support for asynchronous collaboration!

5. Logistical Networking in Collaboration »Adding persistence of data to the network while maintaining other important properties: Resources available to any community member »10TB now, 50TB by 2003, 100TB-1PB goal No centralized administrative domain »Each allocation is individually managed No central management of resource pool »Allocations are time limited »Storage reclaimed when they expire! Access control, security managed by endpoints.

6. The Network Storage Stack Applications Logistical File System Logistical Tools L-Bone IBP Local Access Physical exNode Our adaption of the network stack architecture for storage Like the IP Stack Each level encapsulates details from the lower levels, while still exposing details to higher levels

7. IBP: The Internet Backplane Protocol »Storage provisioned on community “depots” »Very primitive service (similar to block service, but more sharable) Goal is to be a common platform (exposed) Also part of end-to-end design »Best effort service – no heroic measures Availability, reliability, security, performance »Allocations are time-limited! Leases are respected, can be renewed Permanent storage is to strong to share!

8. Models of Sharing: Logistical Networking »Moderately valuable resources Storage, server cycles »Sharing enabled by relative plenty »Internet-like policies Loose access control No per-use accounting »Primary design goal: scalability Application autonomy Resource transparency »Burdens of scalability The End-to-End Principles Weak operation semantics Vulnerability to Denial of Service

9. The Network Storage Stack The L-bone: Resource Discovery & Proximity queries IBP: Allocating and managing network storage (like a network malloc) The exNode: A data structure for aggregation LoRS: The Logistical Runtime System: Aggregation tools and methodologies

10. The Logistical Backbone (L-Bone) »LDAP-based storage resource discovery. »Query by capacity, network proximity, geographical proximity, stability, etc. »Periodic monitoring of depots. »Currently10 Terabytes of shared storage. 50 TB awarded, 100TB proposed Our goal is 1PB global total

11. L-Bone: January 2003

12. The Network Storage Stack The L-bone: Resource Discovery & Proximity queries IBP: Allocating and managing network storage (like a network malloc) The exNode: A data structure for aggregation LoRS: The Logistical Runtime System: Aggregation tools and methodologies

13. The exNode »The Network “File Descriptor »XML-based data structure/serialization »Map byte-extents to IBP buffers (or other allocations). »Allows for replication, flexible decomposition of data. »Also allows for error-correction/checksums »Arbitrary metadata.

14. ExNode vs inode exNode inode IBP Allocations the network local system disk blocks kernel capabilities block addresses user

15. ExNode Mobility XML Serialization The exNode serialization is a portable soft link

16. The Network Storage Stack The L-bone: Resource Discovery & Proximity queries IBP: Allocating and managing network storage (like a network malloc) The exNode: A data structure for aggregation LoRS: The Logistical Runtime System: Aggregation tools and methodologies

17. Logistical Runtime System »Basic Primitives: Upload, Download, Augment, Refresh »End-to-end Services Checksums, Encryption, Compression »Other Things We Can Do Routing through an intermediate depot to reduce IP RTT, speeding up TCP transfers Overlay multicast using either multiple TCP streams or IP multicast at tree nodes »What’s missing? Management by Applications!

18. Upload

19. Augment

20. Download

21. Routing through Intermediate Depots

22. IBP Enables Data Intensive Collaboration »Large files can be uploaded to nearby depots, then managed by movement between depots End systems are not involved in long distance transfers »Data can be moved near to distant collaborator without being downloaded into their end system Direct access to collaborators private storage is not required »Depot-to-depot transfers can take advantage of multithreading, UDP transfer, Net/Web 100, other high-performance optimizations

23. The Next Step: Computation! »Depots can store data, but cannot compute, e.g. Recomputing checksums for stored data would help maintain redundancy Operations such as XOR required to recover redundantly stored data in case of loss »The Network Functional Unit is an extension of the depot that operates on stored data NFU operations are limited, cannot access data outside of depot Management of “process state” must be performed at end systems.

24. LoCI Lab Online http://loci.cs.utk.edu »IBP server and clients for Unix/Linux/OS X Additional clients for Java, Win32 »Logistical Runtime System libraries and tools Run under Unix/Linux/OS X natively Ported to Windows under Cygwin Includes visualization (Tcl/tk) Web interface »Logistical Backbone resource discovery server Unix/Linux/OS X only »Publications, documentation, L-Bone status