1. Nick LeRoy & Jeff Weber Computer Sciences Department University of Wisconsin-Madison condor-admin@cs.wisc.edu http://www.cs.wisc.edu/condor/nest Managing Storage with NeST

2. www.cs.wisc.edu/condor Overview of NeST › NeST: Network Storage Technology › Lightweight: Configuration and installation can be performed in minutes. › Multi-protocol: Supports Chirp, GridFTP, NFS, HTTP  Chirp is NeST’s internal protocol › Secure: GSI authentication › Allocation: NeST negotiates “mini storage contracts” between users and server.

3. www.cs.wisc.edu/condor Why storage allocations ? › Users need both temporary storage, and long-term guaranteed storage. › Administrators need a storage solution with configurable limits and policy. › Administrators will benefit from NeST’s autonomous reclamations of expired storage allocations.

4. www.cs.wisc.edu/condor Storage allocations in NeST › Lot – abstraction for storage allocation with an associated handle  Handle is used for all subsequent operations on this lot › Client requests lot of a specified size and duration. Server accepts or rejects client request.

5. www.cs.wisc.edu/condor Lot types › User / Group  User: single user (user controls ACL)  Group: shared use (users control ACL) › Best effort / Guaranteed  Best effort: server may purge data if necessary. Good fit for derived data.  Guaranteed: server honors request duration. › Hierarchical: Lots with lots (“sublots”)

6. www.cs.wisc.edu/condor Lot operations › Create, Delete, Update › MoveFile  Moves files across lots › AddUser, RemoveUser  Lot level access control  List of users allowed to request sub-lots › Attach / Detach  Performs NeST lot to path binding

7. www.cs.wisc.edu/condor Functionality: GT4 GridFTP GridFTP Server Disk Module Disk Storage globus-url-copy Sample Application (GSI-FTP)

8. www.cs.wisc.edu/condor Functionality: GridFTP +NeST GridFTP Server NeST Module Disk Storage NeST Server NeST Client Chirp Handler globus-url-copy (Lot operations, etc.) (File transfers) (chirp)(GSI-FTP) (File transfer) (chirp)

9. www.cs.wisc.edu/condor GridFTP with NeST (Cont) GridFTP Server NeST Module Disk Storage NeST Server Chirp Custom Application (Lot operations, etc.) (chirp) (File transfers) (GSI-FTP) (File transfer) (chirp) GT4NeST

10. www.cs.wisc.edu/condor Stork NeST Sample Work DAG NeST AllocateJob Xfer InXfer OutRelease Stork Condor

11. www.cs.wisc.edu/condor Release Status › http://www.cs.wisc.edu/condor/nest › v0.9.7 expected soon  v0.9.7 Pre 1 just released, and in VDT 1.3.3  Just bug fixes as found -> v0.9.7 › v1.0 expected later this year › Currently supports Linux, will support other O/S’s in the future.

12. www.cs.wisc.edu/condor Roadmap › Undergo performance tests with Stork › Continue hardening code base › Expand supported platforms  Solaris & other UNIX-en › Add SRM front-end › Bundle with Condor

13. www.cs.wisc.edu/condor Questions ? › Demo on Wednesday  Room 4289, CS building 1:00pm – 4:00pm  More information available at  http://www.cs.wisc.edu/condor/nest