1. NeST: Network Storage Technologies
Building an I/O Appliance on Commodity Systems
John Bent, Remzi Arpaci-Dusseau and Miron Livny

2. Problem Statement
Appliances are attractive because they are robust, reliable, available and especially because they are easy to use. To fulfill these criteria, traditional network appliances impose policy decisions on their users and are built either as kernel modules or upon specially designed kernels.
“How to build a portable, configurable I/O appliance?”

3. Goals
Provide stable, robust and easy to use network storage on a variety of platforms
Allow configuration of server modules to fit application needs
Maintain high server throughput under heavy load
Degrade client performance gracefully as concurrency increases

4. Concurrency Architectures
Nest Structure
Protocols
Name
Space
Static
Configuration
Administrative
Interface
Concurrency Architectures
Runtime
Adaptation
Storage
Management
Consistency
Semantics

5. Protocols
Nest currently supports the following client requests in “nest-speak”:
Send/receive file
Send/receive partial file
Query statistics
Directory operations
Remove file
Timestamp request
Future support: ftp, http, nfs, WiND fs.

6. Concurrency Architectures
Non-blocking single process (NOB)
Pre-allocated pool of processes (POP)
Pre-allocated pool of threads (POT)
All concurrency architectures use one control thread to accept and dispatch client requests. This approach will more easily allow centralized decision making as regards admission control, authentication and quality of services guarantees.

7. Other Modules Static configuration Namespace Runtime adaptation
Build the most appropriate I/O appliance based on knowledge of the host system and the application needs
Namespace
Current: flat namespace
Future: hierarchical
Runtime adaptation
Bandwidth monitoring and other flows of information may prove useful in consistently maximizing system throughput

8. Even More Modules Consistency semantics Storage management
Well defined behavior is necessary in order to allow concurrent client access to the same file
Storage management
Current: ext2 FS on linux
Future: raw disk management
Administrative interface
An “easy to use” tool must have a nice administrative interface. This will probably be added through the http protocol.

9. Performance
Server bandwidth in each of the concurrency architectures remains constant as the number of clients increases. Client latency and bandwidth are also affected correctly as client concurrency grows.

10. Future Future Work Future Applications Quality of service
Admission control, authentication
Hybrid concurrency architectures
Static configuration, runtime adaptation, namespacing, storage management, consistency semantics
Future Applications
Condor checkpoint server
Local staging area for Condor jobs which have flocked over the WAN
Storage “brick” for WiND
Lots and lots and lots and lots more