1 Mobile Management of Network Files Alex BassiMicah Beck Terry Moore Computer Science Department University of Tennessee

2 Logistical Networking Technology for shared network storage that can scale in terms of –the size of the user community, –the aggregate quantity of storage that can be allocated, and –the breadth of distribution of service nodes across network borders Parallels layered IP networking stack

3 Bottom-up Design Philosophy The lowest globally accessible network layer in the network storage stack should –Enable scalable Internet-style resource sharing –Expose underlying storage resources in order to maximize freedom at higher levels Standard network storage systems fails in some degree to satisfy one or both criteria.

4 Internet Backplane Protocol (IBP) Servers that make allocation of primitive “byte arrays” available to clients Byte arrays are not blocks (more abstract) –Network capabilities (primitive security) –Variable extents Byte arrays are not files (weaker semantics) –Size & duration are limited –“Volatile” allocations –Best effort reliability and availability –No directory structure, accounting –No caching, replication

5 The Logistical Networking Project at the University of Tennessee Primary UT Investigators –Micah Beck –James S. Plank UT Research Staff –Alex Bassi –Terry Moore Funding –DOE Next Generation Internet & SciDAC –NSF Next Generation Software Logistical Computation and Internetworking (LoCI) project w/ Birman, Dongarra, Wolski UT Graduate Students –Wo Ling –Yong –Xiang –Erica Fuentes –Anthony Burton

6 Collaborating Projects Network Weather Service (UCSB) –Rich Wolski –Martin Swany NetSolve (UT ICL) –Jack Dongarra –Michelle Miller L-Bone & LoFS (UT) –James S. Plank Application Level Scheduling (UCSD) –Fran Birman –Henri Casanova –Graziano Obertelli Tamanoir (Université Claude Bernard Lyon 1 ) –Laurant Lefavre

7 Building on IBP Many applications assume file semantics –Unbounded size & duration –High reliability & availability –Caching & replication In a layered architecture, these are implemented through aggregation and additional intelligence at the next level

8 Enter the exNode In the Unix file system, blocks are aggregated into files using the intermediate node data structure, or inode We implement file semantics on top of IBP using the external node (exNode) data structure –Aggregation of IBP byte array allocations –Additional metadata to enable management

9 ExNode vs inode exNode inode IBP Allocations the network local system disk blocks kernel capabilities block addresses user

10 The exNode is a Set A set of mappings and associated metadata A mapping associates a data extent and a temporal extent with a (set of) storage resource(s). Minimal container API –constructor & destructor –add mapping –enumerate Lookup functions –get byte extent –get event list

11 The exNode is Mobile Code! (files are structured processes) Mappings are location-independent XML serialization enables interoperability The state of file system processes can be captured in an abstract, portable form Mobile files can be freed of directory, operating system, accounting boundaries

12 Example: IBP Mail sender receiver SMTP IBP write IBP read IBP copy exNode

13 File System Services Indirection to preserve integrity Temporal and data extent aggregation Reliability through RAID, replication Performance through locality

14 Active File Management Active Services –Active Probing to Maintain Fault Tolerance –Lease Renewal –Defragmentation –Asynchronous Transfer Management Defragmentation

15 Conclusions The Internet Backplane Protocol provides a fabric for network state management Scalable and ubiquitous sharing are enabled by weak semantics & minimal accounting The exNode & serialization provide a mobile abstraction of storage aggregation Active file services can span OS & FS boundaries (Log Structured Logistical FS)