1. Networked Storage Technologies Douglas Thain University of Wisconsin thain@cs.wisc.edu GriPhyN NSF Project Review 29-30 January 2003 Chicago

2. 229 Jan 2003 Douglas Thain, University of Wisconsin thain@cs.wisc.edu Unreliable Internet The Problem of Remote I/O Storage Job Remote CPUs Survive disconnections. Hide high latencies. Hide bursty throughput. Audit progressive results. Ensure consistency between job and storage. Arbitrate between users. Make it easy.

3. 329 Jan 2003 Douglas Thain, University of Wisconsin thain@cs.wisc.edu NeST Turns Raw Storage into a Storage Appliance Storage NeST Allocable Auditable Authentic Accessible Appl Cmd Tool Web Browser User-Level Adapter Appl OS Kernel Admin or Owner Chirp NFS HTTP GridFTP DaP Match Maker ClassAds

4. 429 Jan 2003 Douglas Thain, University of Wisconsin thain@cs.wisc.edu DaP Makes Data Transfer a Managed Job Storage NeST DaP Storage NeST Submit, Query, Remove Data Mvmt Queue AllocationActivation Transfer

5. 529 Jan 2003 Douglas Thain, University of Wisconsin thain@cs.wisc.edu Building the Grid Storage Job Remote CPUs NeST DaP GridFTP Transfer Chirp Reservation Adapter Chirp Input Kangaroo Output Chirp Output Status and Supervision Policy and Control Requests

6. 629 Jan 2003 Douglas Thain, University of Wisconsin thain@cs.wisc.edu Ph.D. Research Enabled by Griphyn l NeST: Network Storage Technologies –John Bent –http://www.cs.wisc.edu/condor/nest l DaP: Data Placement Manager –Tefvik Kosar –http://www.cs.wisc.edu/condor/dap l Distributed I/O using Grid Services –Douglas Thain –http://www.cs.wisc.edu/~thain l Grid Security –Ian Alderman l Too many MS and BS students to list!

7. 729 Jan 2003 Douglas Thain, University of Wisconsin thain@cs.wisc.edu Future Work: l I/O - CPU Specialization in Workloads –Automatically provision a cluster with the correct number of storage/worker nodes. l DaP / DAG integration –Convergence of technologies for reliable data scheduling and reliable job scheduling. l Error Management –What happens when something goes wrong? Backup/retry/pause/inform? l Security –An online CA to issue task-specific certificates just- in-time for work to be done.

8. 829 Jan 2003 Douglas Thain, University of Wisconsin thain@cs.wisc.edu Publications Enabled by Griphyn l “Architectural Implications of Pipeline and Batch Sharing in Scientific Workloads”, UW-CS-TR 1463, 2003, also in review. –http://www.cs.wisc.edu/condor/doc/profiling-tr.pdf l “The Case for Sparse Files”, UW-CS-TR 1464, 2003. –http://www.cs.wisc.edu/~thain/library/sparse.pdf l “Error Management in the Pluggable File System”, UW-CS-TR 1448, 2002. –http://www.cs.wisc.edu/condor/doc/pfs-tr.pdf l “Flexibility, Manageability, and Performance in a Grid Storage Appl”, HPDC 2002. –http://www.cs.wisc.edu/condor/nest/papers/nest-hpdc02.pdf l “Error Scope on a Computational Grid: Theory and Practice”, HPDC 2002. –http://www.cs.wisc.edu/condor/doc/error-scope.pdf l “Exploiting Gray-Box Knowledge of Buffer-Cache Management”, USENIX 2002. –http://www.cs.wisc.edu/wind/Publications/dust-usenix02.pdf l “Gathering at the Well: Creating Communities for Grid I/O”, SC 2001. –http://www.cs.wisc.edu/condor/doc/community-sc2001.pdf l “The Kangaroo Approach to Data Movement on the Grid”, HPDC 2001. –http://www.cs.wisc.edu/condor/doc/kangaroo-hpdc10.pdf

9. 929 Jan 2003 Douglas Thain, University of Wisconsin thain@cs.wisc.edu The Real Value: “Why don’t you go to down to visit Fermi next Wednesday?”