1. A. Sim, CRD, L B N L 1 OSG Applications Workshop 6/1/2005 OSG SRM/DRM Readiness and Plan Alex Sim / Jorge Rodriguez Scientific Data Management Group Computational Research Division Lawrence Berkeley National Laboratory and High Energy Physics Group Dept. of Physics University of Florida

2. A. Sim, CRD, L B N L 2 OSG Applications Workshop 6/1/2005 Service Description Brief overview of the DRM ServiceBrief overview of the DRM Service core service Provides core functionality Movement of files into and out of DRM cache Web Service components by Java wrapper Based on SRM v.1.1 specification interoperable with other SRMs DRM dependenciesDRM dependencies Functions on top of ordinary UNIX file systems: local, NFS… Binaries available for RH7.x, RH8 and RHEL3 Statically linked and self contained Requires Java 1.4.x+ for web services components Self contained except Java VM Data movement services such as Gridftp

3. A. Sim, CRD, L B N L 3 OSG Applications Workshop 6/1/2005Installation Installation:Installation: DRM is included in VDT 1.3.x DRM version 1.2.7 in the current VDT Startup script, SXXsrm to start/stop/check processes Can be deployed with everything else from VDT… Can also be deployed as a stand-alone VDT will handle dependencies

4. A. Sim, CRD, L B N L 4 OSG Applications Workshop 6/1/2005Configuration Configuration:Configuration: Recommend to install and run under a non-privileged user e.g. srm Service provider recommends this for security reasons Should use a separate (SRM) service certificate Default configuration is provided Java based configuration generator is provided for customization Manually modified configuration parameters will also work

5. A. Sim, CRD, L B N L 5 OSG Applications Workshop 6/1/2005 Installation Testing Testing:Testing: srm-tester is provided Testing DRM installation and its proper running Testing compatibility of any SRMs based on SRM spec v.1.1 This tool can be used for SRM/dCache or any other SRM installations Testing interoperability of two SRMs based on SRM spec v.1.1 E.g. It checks the interoperability of SRM/DRM and SRM/dCache.

6. A. Sim, CRD, L B N L 6 OSG Applications Workshop 6/1/2005 Client tools for SRM/DRM Java based srmcp from FNAL based on SRM v.1.1 can be usedJava based srmcp from FNAL based on SRM v.1.1 can be used For general SRM v.1.1 based file movement requests: get, put, copy DRM specific client commands can also be usedDRM specific client commands can also be used Works only for DRMs drm-copy, drm-get, drm-put, drm-status, drm-ping, drm-ls, drm-release, drm-remove Java based srm-copy can also be usedJava based srm-copy can also be used Included in coming DRM v.1.2.8 For general SRM v.1.1 and SRM v.2.1.1 access Works for any versions of SRMs : based on v.1.1 or v.2.1.1 File movement functionalities get, put, copy ( for servers based on v.1.1 and v.2.1.1) directory management capabilities (for SRM servers based on v.2.1.1) ls, mkdir explicit space reservations (for SRM servers based on v.2.1.1) reserveSpace

7. A. Sim, CRD, L B N L 7 OSG Applications Workshop 6/1/2005 Disk Cache HRM Disk Cache HRM Files request (thousands of files) Disk Cache Disk Cache Disk Cache DRM Disk Cache Disk Cache MSS DRM Disk Cache Disk Cache HRM (performs writes) HRM (performs writes) HRM (performs writes) GridFTP HTTP(s) FTP BBFTP servers Client Disk Cache SRM dCache Disk Cache Disk Cache Disk Cache Disk Cache Disk Cache Disk Cache Disk Cache CORBA Name services Web services gateway SRMCP DRM-COPY DRM-GET DRM-PUT DRM-LS DRM-REMOVE DRM-RELEASE DRM-PING DRM-STATUS Other SRMs Disk Cache Disk Cache Disk Cache Disk Cache Storage Overview of the service architecture

8. A. Sim, CRD, L B N L 8 OSG Applications Workshop 6/1/2005 Sample Application Level Test Scenario (1) Simplified application test :Simplified application test : 1.stage files into DRM cache from remote SRM/DRM, SRM/dCache or any file repositories (with supported transport protocols – gsiftp, ftp, http, https or bbftp) 2.Execute jobs that access “staged in” files in DRM cache via POSIX calls

9. A. Sim, CRD, L B N L 9 OSG Applications Workshop 6/1/2005 Scenario (1) DRM Disk Cache DISK CACHE Disk Cache HRM Disk Cache DRM Disk Cache Disk Cache SRM/dCache Disk Cache Disk Cache Disk Cache Disk Cache Disk Cache Other SRMs Disk Cache Disk Cache Disk Cache Disk Cache Storage HRM (performs writes) HRM (performs writes) HRM (performs writes) GridFTP HTTP(s) FTP BBFTP services Disk Cache Client 1. File Request 3. Job execution with file access when ready 2. Bring files in

10. A. Sim, CRD, L B N L 10 OSG Applications Workshop 6/1/2005 Sample Application Level Test Scenario (2) Simplified application test :Simplified application test : 1.stage files into DRM cache from remote SRM/DRM, SRM/dCache or any file repositories (with supported transport protocols – gsiftp, ftp, http, https or bbftp) 2.Execute jobs that access “staged in” files in DRM cache via POSIX calls 3.Job creates an output file on local storage ($WNTMP or $TMP) 4.Job moves the output file from local storage into the local DRM cache 5.Job moves the output file from DRM cache to remote SRM managed or remote archival location

11. A. Sim, CRD, L B N L 11 OSG Applications Workshop 6/1/2005 Scenario (2) DRM Disk Cache DISK CACHE Disk Cache HRM Disk Cache DRM Disk Cache Disk Cache SRM/dCache Disk Cache Disk Cache Disk Cache Disk Cache Disk Cache Other SRMs Disk Cache Disk Cache Disk Cache Disk Cache Storage HRM (performs writes) HRM (performs writes) HRM (performs writes) GridFTP HTTP(s) FTP BBFTP services Disk Cache Client 1. File Request 3. Job execution with file access when ready 4. Space Request 5. Puts the output file into the desginated DRM space when ready 2. Bring files in 6. Send files out

12. A. Sim, CRD, L B N L 12 OSG Applications Workshop 6/1/2005 Other Application Level Tests Scientific Applications (willing or able to participate)Scientific Applications (willing or able to participate) Provide document recommendations for use of DRM services in job work flow Make DRM enabled sites available for application tests Schedule with application administrators Monitor activity by site and app admins.

13. A. Sim, CRD, L B N L 13 OSG Applications Workshop 6/1/2005 Service Monitoring Monitor performance of storage activityMonitor performance of storage activity DRM has information of current activities in its cache as well as logs of all past activities Will need input from Service Provider as to what to monitor, what to measure, and what to report

14. A. Sim, CRD, L B N L 14 OSG Applications Workshop 6/1/2005 Documents and Support Document linksDocument links http://sdm.lbl.gov/srm-dist Support of the serviceSupport of the service We (LBNL SDM group) will support all SRM/DRM related inquiries All support requests to srm@lbl.gov

15. A. Sim, CRD, L B N L 15 OSG Applications Workshop 6/1/2005Plans Where we are todayWhere we are today DRM v.1.2.7 in current VDT Documents for DRM v.1.2.7 on the web DRM v.1.2.8 to release in a week or two Next stepsNext steps Documents to include use cases Having more participations from OSG sites for installation and testing Testing other use cases if provided Enhancements if any Bug fixing if found