Computing Sciences Directorate, L B N L 1 SC 2003 Storage Resource Managers: Essential Components for the Grid Arie Shoshani Staff: Alex Sim, Junmin Gu, Alex Romosan, Viji Natarajan Scientific Data Management Group Lawrence Berkeley National Laboratory

Computing Sciences Directorate, L B N L 2 SC 2003 Outline What are Storage Resource Managers - MotivationWhat are Storage Resource Managers - Motivation General Analysis Scenario and the use of SRMsGeneral Analysis Scenario and the use of SRMs SRM functionalitySRM functionality Real examples of working SRMsReal examples of working SRMs Advantages of using SRMsAdvantages of using SRMs Conclusions and Future WorkConclusions and Future Work

Computing Sciences Directorate, L B N L 3 SC 2003Motivation Grid architecture needs to include reservation & scheduling of:Grid architecture needs to include reservation & scheduling of: Compute resources Storage resources Network resources Storage Resource Managers (SRMs) role in the data grid architectureStorage Resource Managers (SRMs) role in the data grid architecture Shared storage resource allocation & scheduling Especially important for data intensive applications Often files are archived on a mass storage system (MSS) large scientific collaborations (100’s of clients) – opportunities for file sharing File replication and caching may be used Need to support non-blocking (asynchronous) requests

Computing Sciences Directorate, L B N L 4 SC 2003 A multi-file request to a Disk Resource Manager... client File Transfer Service Disk Cache file transfer requests network DRM Disk Cache... Disk Cache File Transfer Service multi-file request file access client Tape System Client-SRM Communication

Computing Sciences Directorate, L B N L 5 SC 2003 Accessing Remote Storage Resource Managers Tape System Disk Cache file transfer requests network DRM Disk Cache... Disk Cache multi-file request file access... client DRMHRM SRM-SRM Communication

Computing Sciences Directorate, L B N L 6 SC 2003 General Analysis Scenario MSS Request Executer Storage Resource Manager Metadata catalog Replica catalog Network Weather Service logical query network client... Request Interpreter request planning A set of logical files Execution plan and site-specific files Client’s site... Disk Cache Disk Cache Compute Engine Disk Cache Compute Resource Manager Storage Resource Manager Compute Engine Disk Cache Requests for data placement and remote computation Site 2 Site 1 Site N Storage Resource Manager Storage Resource Manager Compute Resource Manager result files Execution DAG : Uniform SRM Interface : Uniform SRM Interface

Computing Sciences Directorate, L B N L 7 SC 2003 File movement functionality: srmGet, srmPut, srmReplicate SRMClient Client-FTP-get (pull) Client-FTP-put (push) srmGet/srmPut SRM-FTP-put (push) SRMClient SRM/ No-SRM SRM-FTP-get (pull) srmReplicate SRM/ No-SRM FTP-get

Computing Sciences Directorate, L B N L 8 SC 2003 SRM is a Service (OGSA, CORBA, C++, Java, …) SRM functionalitySRM functionality Manage space Negotiate and assign space to users Manage “lifetime” of spaces Manage files on behalf of a user Pin files in storage till they are released Manage “lifetime” of files Manage action when pins expire (depends on file types) Manage file sharing Policies on what should reside on a storage resource at any one time Policies on what to evict when space is needed Get files from remote locations when necessary Purpose: to simplify client’s task Manage multi-file requests A brokering function: queue file requests, pre-stage when possible Provide grid access to/from mass storage systems HPSS (LBNL, ORNL, BNL), Enstore (Fermi), JasMINE (Jlab), Castor (CERN), MSS (NCAR), …

Computing Sciences Directorate, L B N L 9 SC 2003 Types of SRMs Types of storage resource managersTypes of storage resource managers Disk Resource Manager (DRM) Manages one or more disk resources Tape Resource Manager (TRM) Manages access to a tertiary storage system (e.g. HPSS) Hierarchical Resource Manager (HRM=TRM + DRM) An SRM that stages files from tertiary storage into its disk cache SRMs and File transfersSRMs and File transfers SRMs DO NOT perform file transfer SRMs DO invoke file transfer service if needed (GridFTP, FTP, HTTP, …) SRMs DO monitor transfers and recover from failures TRM: from/to MSS DRM: from/to network

Computing Sciences Directorate, L B N L 10 SC 2003 SRM works with other SRMs as well as legacy systems by using GridFTP DRM Disk Cache Disk Cache Disk Cache Disk Cache Berkeley ChicagoLivermore HRMGridFTP FTP Disk Cache Request Interpreter Request Manager DRM GridFTP client server Logical Request Data Path Control path Legend:

Computing Sciences Directorate, L B N L 11 SC 2003 Tomcat servlet engine Tomcat servlet engine MCS Metadata Cataloguing Services MCS Metadata Cataloguing Services RLS Replica Location Services RLS Replica Location Services SOAP RMI MyProxy server MyProxy server MCS client RLS client MyProxy client GRAM gatekeeper GRAM gatekeeper CAS Community Authorization Services CAS Community Authorization Services CAS client NCAR-MSS Mass Storage System HPSS High Performance Storage System HPSS High Performance Storage System DRM Storage Resource Management DRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management gridFTP server gridFTP server gridFTP server gridFTP server gridFTP server gridFTP server gridFTP server gridFTP server openDAPg server openDAPg server gridFTP Striped server gridFTP Striped server LBNL LLNL USC-ISI NCAR ORNL ANL DRM Storage Resource Management DRM Storage Resource Management disk Earth Science Grid

Computing Sciences Directorate, L B N L 12 SC 2003 Uniformity of Interface  Compatibility of SRMs SRM Enstore JASMine Client USER/APPLICATIONS Grid Middleware SRM DCache SRM CASTOR SRM Disk Cache

Computing Sciences Directorate, L B N L 13 SC 2003 Screen Dump of Demo at Fermi Booth

Computing Sciences Directorate, L B N L 14 SC 2003 Where do SRMs belong in the Grid architecture?

Computing Sciences Directorate, L B N L 15 SC 2003 SRMs provide a brokering service by supporting multi-file requests

Computing Sciences Directorate, L B N L 16 SC 2003 DataMover: SRMs use in ESG and PPDG for Robust Muti-file replication for Robust Muti-file replication HRM-COPY (thousands of files) SRM-GET (one file at a time) GridFTP GET (pull mode) stage files archive files Network transfer Get list of files From directory Recovers from file transfer failures Anywhere Disk Cache DataMover (Command-line Interface) HRM (performs writes) LBNL/ ORNL Disk Cache HRM (performs reads) NCAR NCAR-MSS Recovers from staging failures Recovers from archiving failures Web-based File Monitoring Tool

Computing Sciences Directorate, L B N L 17 SC 2003 Web-Based File Monitoring Tool Shows: -Files already transferred - Files during transfer - Files to be transferred Also shows for each file: -Source URL -Target URL -Transfer rate

Computing Sciences Directorate, L B N L 18 SC 2003 File tracking helps to identify bottlenecks Shows that archiving is the bottleneck

Computing Sciences Directorate, L B N L 19 SC 2003 File tracking shows recovery from transient failures Total: 45 GBs

Computing Sciences Directorate, L B N L 20 SC 2003 File tracking shows network slowdown and recovery Total: 53 GBs

Computing Sciences Directorate, L B N L 21 SC 2003 Types of Spaces and Files Space reservation servicesSpace reservation services Spaces and files: volatile, durable, permanent Lifetime, action at end of lifetime Volatile – SRM owned, files can be removed if space needed Durable – files cannot be removed, but administrator notified Permanent – can be removed by owner only Directory servicesDirectory services Usual unix semantics any type of files in directory Access control servicesAccess control services Support owner/group/world permission Can only be assigned by owner File sharing for read-only files check with source for shared file permission File sharing for updatable files check with “master copy” for time of last update

Computing Sciences Directorate, L B N L 22 SC 2003 SRM Methods File Movement srm(Prepare)Get: srm(Prepare)Put: srmReplicate: Lifetime management srmReleaseFiles: srmPutDone: srmExtendFileLifeTime: Terminate/resume srmAbortRequest: srmAbortFile srmSuspendRequest: srmResumeRequest: Space management srmReserveSpace srmReleaseSpace srmUpdateSpace srmCompactSpace: srmGetCurrentSpace: FileType management srmChangeFileType: Status/metadata srmGetRequestStatus: srmGetFileStatus: srmGetRequestSummary: srmGetRequestID: srmGetFilesMetaData: srmGetSpaceMetaData:

Computing Sciences Directorate, L B N L 23 SC 2003 Summary: advantages of using SRMs Synchronization between storage resourcesSynchronization between storage resources Pinning file, releasing files Allocating space dynamically on as “needed basis” Insulate clients from storage and network system failuresInsulate clients from storage and network system failures Transient MSS failure Network failures Interruption of large file transfers Facilitate file sharingFacilitate file sharing Eliminate unnecessary file transfers Support “streaming model”Support “streaming model” Use space allocation policies by SRMs: no reservations needed Use explicit release by client for reuse of space Control number of concurrent file transfersControl number of concurrent file transfers From/to MSS – avoid flooding MSS and thrashing From/to network – avoid flooding and packet loss

Computing Sciences Directorate, L B N L 24 SC 2003 Ongoing and Future Work Ongoing workOngoing work Developing Standard SRM interfaces Particle Physics Data Grid (PPDG) project LBNL, TJNAF, FNAL European Data Grid (EDG) project WP2 - data management WP5 – mass storage Deployment LBNL, BNL, ORNL, TJNAF, FNAL, CERN, (SE-England) Use of SRM by other agents Storage Resource Broker (SDSC) calling HRM to Stage files from HPSS GridFTP invoking HRM New Spec completed (SRM V2.1) directory management File/directory file movement dynamic space management Future workFuture work Access authorization – community access service (CAS) “On-demand” space allocation, accounting, and charging Replica management – invoke SRMs and RLS as a single service Request executer (e.g. DAGMAN) to invoke SRMs SRMs over NeST (Network STorage)