Working together towards a next generation storage element Surya D. Pathak Advanced Computing Center for Research and Education

Outline Existing Storage Elements: A quick review Where do we go from here? –A proposed direction for the next generation storage element –Wish List Efforts at ACCRE –Backward compatibility tools –Shared storage backend

State of Art in Storage Elements (SE) SRM-DCACHE-ENSTORE

SRM-DCACHE-ENSTORE Achievement Highlights –First working storage element –Space and Quota management (concept of pools) –PNFS  NFS2 –Enstore File System Check Caveats –Mirroring, Striping, Encryption of data –DCACHE is not firewall friendly

State of Art in Storage Elements (SE) LOCI-IBP Effort Applications LoRS (Logistical Runtime System) E X NODE IBP Internet Backplane Protocol Local Access Layer Physical Layer Network Storage Stack L-Bone Logistical Backbone LoDN Client LoRS Client

LOCI effort Achievement Highlights –Mirroring, Striping and encryption of data –Agnostic to location specific details –Completely Firewall friendly (verified) –Very easy to set up, dependencies are minimal Caveats –No explicit space or quota management system –No formal authentication/accounting/auditing functionality

Where do we go from here? Proposed Next Generation SE TAPE LoDN ClientSRM-IBP client tools Disk Cache ENSTORE DISK HSM Support LoCI Infrastructure Firewall Solve the nagging firewall problem Mirroring, Striping, Encryption DCache/IBP Depot both provide disk cache functionalities Enstore provides robust Error checking and file checking

Next Generation SE Wish List Proposed ActivityRemarksBuilt on Concepts from 1.Space and Quota Management Permanent, Durable, Volatile Pools and user quota enforcement SRM-DCACHE effort 2.PNFS  NFS4Virtual file system (Extend Attributes to support, location, permissions, encryption, pools…) SRM-DCACHE Effort 3.LOCI Tape AwareIncorporate Space management and quota management functions into LoCI tools. Make it tape aware such that user can micro-control the data he/she wants to store LoCI 4.Mirroring, Striping, EncryptionUse LoCI tools to achieve the functionality LoCI

Next Generation SE Wish List Proposed ActivityRemarksBuilt on Concepts from 5.Bi-directional CommunicationFirewall problem is taken care of. LoCI straddles the Firewall LoCI 6.AuthenticationImportant from security point of view Also to enforce space and quota management OSG? S SRM-DCACHE? 7.Auditing and AccountingA must for enforcing spaces and quotas OSG/GRID3? SRM-DCACHE? 8.Backward CompatibilitySuch that existing client tools such that SRM/DRM clients can integrate seamlessly into the new framework ACCRE 9.Shared Backend StorageUsing Agent technology to dynamically perform data transfers between IBP depots connected to common tape/disk libraries ACCRE

Efforts at ACCRE Backward compatibility of existing elements IBP-DCACHE-ENSTORE based storage element Call Translator SRMCP(..) SRM calls IBP calls

Efforts at ACCRE: Shared Storage Backend Slave Depot Tape/Disk Slave Depot Master Depot Write Data Efficient Data Transfer Resource Discovery Software Agent Use Software Agents to implement a simple contract net protocol I am free I am busy

Efforts at ACCRE: Shared Storage Backend Slave Depot Tape/Disk Slave Depot Master Depot Read Data Software Agent Use Software Agents to implement a simple contract net protocol I am busy I am free

Project Management Program of work? (Who and what) Schedules and Milestones Mechanics and technical issues Critical Paths Tests and demos Documentation Guys lets figure this all out by lunch tomorrow!!!!