Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego dCache in OSG 1.0 and SRM 2.2 Ted Hesselroth Fermilab

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego dCache 1.8 Features Server Software in VDT  dCache 1.8  Bestman 2.2 New clients and client versions in VDT OSG Storage Activities for dCache 1.8 Create Installation Package Test servers and clients, with feedback to developers Prepare to support standard use cases, deployments

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego dCache 1.8 Features SRM 2.2 Chimera beta (not supported by OSG in ’08)  In server RPM, optionally invoked  Scalability, File sizes > 2GB, ACLs, Needed from NFS 4.1 FTP 2 in grid doors Multiple files per connection Checksum from client Direct connection to pool for both reads and writes Supported in srmcp 1.8

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego dCache 1.8 Features (cont.) Performance  Option: Use Berkeley DB in Pools  Caching of authorization in SRM  Caching of policy files in gPlazma  Adjustments to Tomcat, postgres use by SRM Support for log4j Expanded checksumming Pools mounting pnfs is optional

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego SRM v1 Features A standard interface among storage implementations Protocol Negotiation Internal Space Reservation Load Balancing Internal Checksum management Pinning 3 rd party transfers

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego SRM v2.2 Features Space Management functions  srmReserveSpace  srmReleaseSpace  Directory Functions  srmLs, srmMkdir, srmRm, srmMkDir, srmRmDir  Permission Functions  rwx-rwx-rwx

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego SRM v2.2: AccessLatency and RetentionPolicy From SRM v2.2 WLCG MOU  The agreed terminology is: TAccessLatency {ONLINE, NEARLINE} TRetentionPolicy {REPLICA, CUSTODIAL}  The mapping to labels ‘TapeXDiskY’ is given by: Tape1Disk0: NEARLINE + CUSTODIAL Tape1Disk1: ONLINE + CUSTODIAL Tape0Disk1: ONLINE + REPLICA TFileType {VOLATILE, DURABLE, PERMANENT}

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego dCache 1.8 and WLCG Input to SRM 2.2 Specification Extensive Testing by GSSD  Tested multiple SRM implementations  SRM 2.2 features needed by LCG, load testing  Weekly phone conference with dCache developers  All dCache code issues from GSSD have been fixed Edinburgh Tier 2 Workshop  Extensive presentation by Timur Perelmutov  Slides on srm.fnal.gov

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Site Involvement Test Stands involved in GSSD  FZK, SARA, GridKa, IN2P3, NGDF, BNL, FNAL, DESY Early Adopters – in production  NGDF, FZK, DESY, IN2P3, SARA, FNAL Tier 1's – November-December, 2007 Tier 2's – January-February, 2008

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Glue Schema 1.3 Storage Element  ControlProtocol SRM  AccessProtocol gsiftp  Storage Area Groups of Pools VOInfo  Path  Working on GIPs, dialog StorageElement ControlProtocol AccessProtocol StorageArea VOInfo

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego SRM v2.2 Clients Three sets under consideration  lcg-utils  LBL clients  Fermilab clients Will test and evalulate support issues

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Upgrading from dCache 1.7 RPM Upgrade only  No need for offline conversions  Pools will convert control files at startup VDT Installation scripts  Can merge new and current dCacheSetup  Some features need modified PoolManager.conf Space Manager, including Opportunistic use Replica Manager Merge?

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Details of Space Reservation Attributes of a space reservation Authorized group and role (creation time only) Access Latency Retention Policy Link Group ID Size Lifetime Description

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Link Groups: Policies for space reservations Authorization to create Allowed File Attributes CustodialAllowed=false, OnlineAllowed=true, … Pool Selection Rules  Client IP ranges  Area in namespace being accessed  Type of transfer View as “container”. Pools comprise space-reservable area.

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Pool Selection with Space Reservation No Space Reservation  Pool selected from links not in any link group With Space Reservation  Link group selected based on file attributes Multiple matches: link group selection is random  Pool selected from links in selected link group Multiple matches: pool selection based on cost function Authorization and file system permissions are honored

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego SrmSpaceManager admin commands reserve [-vog=voGroup] [-vor=voRole] [-acclat=AccessLatency] [-retpol=RetentionPolicy] [-desc=Description] [- lgid=LinkGroupId] [-lg=LinkGroupName] # create a new reservation release [ ] # release the space reservation identified by # release existing reservation ls [-l] # list reservations and link groups update link groups # trigger update now, which is otherwise performed every 3 min listFilesInSpace # what are the files already written into this space listInvalidSpaces

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Schedule – RC1 and RC2 Release Candidate 1 – December 14  Space Reservation Release Candidate 2 – January 18  Replica Manager  Opportunistic Storage  Static GIP for glue schema 1.3  Gratia Probes

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Schedule – OSG 1.0 OSG 1.0 – February 15  Information Services  SRM-dCache Toolkit

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Appendix: Installing dCache 1.8 with the VDT Package

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego The VDT Package for dCache RPM-based  Multi-node install # wget \ preview/2.0.4/vdt-dcache-SL4_ tar.gz # wget \ preview/2.0.4/vdt-dcache-SL4_ tar.gz # tar zxvf vdt-dcache-SL4_ tar.gz # cd vdt-dcache-SL4_ /preview

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego The Configuration Dialog Queries  Distribution of “admin” Services Up to 5 admin nodes  Door Nodes Private Network Number of dcap doors  Pool Nodes Partitions that will contain pools Because of delegation, all nodes must have host certs. # config-node.pl

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego The site-info.def File “admin” Nodes  For each service, hostname of node which is to run the service Door Nodes  List of nodes which will be doors  Dcap, gsidcap, gridftp will be started on each door node Pool nodes  List of node, size, and directory of each pool  Uses full size of partition for pool size # less site-info.def

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Customizations DCACHE_DOOR_SRM_IGNORE_ORDER=true SRM_SPACE_MANAGER_ENABLED=yes SRM_IMPLICIT_SPACE_MGR=no SRM_LINK_GROUP_AUTH_FILE REMOTE_GSI_FTP_MAX_TRANSFERS=2000 DCACHE_LOG_DIR=/opt/d-cache/log # config-node.pl Copy site-info.def into install directory of package on each node.

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego The Dryrun Option Does not run commands. Used to check conditions for install. Produces vdt-install.log and vdt-install.err. #./install.sh --dryrun On each node of the storage system.

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego The Install Checks if postgres is needed  Installs postgres if not present  Sets up databases and tables depending on the node type. Checks if node is pnfs server  Installs if not present  Creates an export for each door node #./install.sh On each node of the storage system.

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego The Install, continued Unpacks dCache rpm Modifies dCache configuration files  node_config  pool_path  dCacheSetup If upgrade, applies previous settings to new dCacheSetup Runs /opt/d-cache/install/install.sh  Creates links and configuration files  Creates pools if applicable  Installs srm server if srm node

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego If authorization fails or is denied, attempts next method Authorization - gPlazma Cell dcachesrm-gplazma.policy: # Switches" saml-vo-mapping="ON" kpwd="OFF" grid-mapfile="OFF" gplazmalite-vorole-mapping="OFF" # Priorities saml-vo-mapping-priority="1" kpwd-priority="3" grid-mapfile-priority="4" gplazmalite-vorole-mapping-priority="2“ … # SAML-based grid VO role mapping mappingServiceUrl=" horizationServicePort" vi etc/dcachesrm-plazma.policy

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego The saml-vo-mapping Method Acts as a client to GUMS GUMS returns a username. Lookup in storage-authzdb follows for uid, gid, etc. Provides site-specific storage obligations /etc/grid-security/storage-authzdb: authorize uscms157 read-write / /pnfs/fnal.gov/data / authorize osg read-write / /pnfs/fnal.gov/data/public /

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Starting dCache # bin/dcache-core start On each “admin” or door node. # bin/dcache-pool start On each pool node. Starts JVM (or Tomcat, for srm). Starts cells within JVM depending on the service.

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Check the admin login # ssh –l admin –c blowfish –p adminnode.oursite.edu On each pool node. (local) admin > cd SrmSpaceManager (SrmSpaceManager) admin > info (SrmSpaceManager) admin > help (SrmSpaceManager) admin > ls (SrmSpaceManager) admin >.. (local) admin > (local) admin > cd SrmSpaceManager (SrmSpaceManager) admin > info (SrmSpaceManager) admin > help (SrmSpaceManager) admin > ls (SrmSpaceManager) admin >.. (local) admin > On any machine with ssh. Can “cd” to dCache cells and run cell commands Scriptable, also has jython interface and gui.

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Validating the Install On client machine with user proxy Test a local -> srm copy, srm protocol 2. $ /opt/d-cache /srm/bin/srmcp --debug=true - srm_protocol_version=2 file:////home/tdh/tmp/test1 srm://fapl037.fnal.gov:8443/srm/managerv2?SFN=/pnfs/fnal.go v/data/test1

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Additional Validation Other client commands  srmls  srmmv  srmrm  srmrmdir  srm-reserve-space  srm-release-space See Neha's talk and the web page

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Register your Storage Element Fill out form at View the results at Affiliation: OSG SitesLast TestLast test runs Archive TTU_bestman _09_00 2, 5, 7, 14, Archive NERSC_bestman _09_12 2, 5, 7, 14, Archive UCSD_dcache _09_12 2, 5, 7, 14, Archive

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Advanced Setup: Space Reservation On machine logged in to admin ssh interface Make a space reservation using the admin interface. (SrmSpaceManager) admin > reserve -acclat=ONLINE - retpol=REPLICA -desc=osgdemo -lgid= A space reservation token will be assigned.

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Using a space reservation Use the space token in the command line. /opt/d-cache /srm/bin/srmcp --debug=true - srm_protocol_version=2 -space_token=3 file:////home/tdh/tmp/test1 srm://fapl037.fnal.gov:8443/srm/managerv2?SFN=/pnfs/fnal.go v/data/test2 Or, implicit space reservation may be used. Command line options imply which link groups can be used. -retention_policy= -access_latency=

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Advanced Setup: Restricting root paths On node with pnfs mounted Restrict reads/writes to a namespace. # cd /pnfs/fnal.gov/data # mkdir public # chmod 777 public # cd /pnfs/fnal.gov/data # mkdir public # chmod 777 public /etc/grid-security/storage-authzdb: authorize uscms157 read-write / /pnfs/fnal.gov/data/public / On node running gPlazma

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Advanced Setup: Opportunistic Storage Create pools in PoolManager.conf. Assign them to a link, “public-link”. psu create pool fapl037_3 psu create pool fapl037_4 psu create pgroup public psu addto pgroup public fapl037_3 psu addto pgroup public fapl037_4 psu create link public-link any-store world-net any-protocol psu set link public-link -readpref=10 -writepref=10 - cachepref=10 -p2ppref=-1 psu add link public-link public psu create pool fapl037_3 psu create pool fapl037_4 psu create pgroup public psu addto pgroup public fapl037_3 psu addto pgroup public fapl037_4 psu create link public-link any-store world-net any-protocol psu set link public-link -readpref=10 -writepref=10 - cachepref=10 -p2ppref=-1 psu add link public-link public On node running dCache domain

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Advanced Setup: Opportunistic Storage (cont.) Create a link group and add “public-link” to it. Set the attributes of the link group. psu create linkGroup public-link-group psu addto linkGroup public-link-group public-link psu set linkGroup custodialAllowed public-link-group true psu set linkGroup onlineAllowed public-link-group true psu set linkGroup nearlineAllowed public-link-group true psu set linkGroup replicaAllowed public-link-group true psu set linkGroup outputAllowed public-link-group false psu set linkGroup attribute public-link-group HSM=None psu create linkGroup public-link-group psu addto linkGroup public-link-group public-link psu set linkGroup custodialAllowed public-link-group true psu set linkGroup onlineAllowed public-link-group true psu set linkGroup nearlineAllowed public-link-group true psu set linkGroup replicaAllowed public-link-group true psu set linkGroup outputAllowed public-link-group false psu set linkGroup attribute public-link-group HSM=None On node running dCache domain

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Making a space reservation On client machine with user proxy Space token (integer) is obtained from the output. $ /opt/d-cache /srm/bin/srm-reserve-space -- debug=true -desired_size= guaranteed_size= retention_policy=REPLICA - access_latency=ONLINE -lifetime=60 -space_desc=osgdemo2 srm://fapl037.fnal.gov:8443 /etc/LinkGroupAuthorization.conf: LinkGroup reserved-link-group cms/Role=* atlas/Role=* LinkGroup public-link-group */Role=*

Ted Hesselroth, OSG Site Administrators Meeting, December 13, 2007 Abhishek Singh Rana and Frank Wuerthwein UC San Diego Using the space reservation Use the space token in the command line. /opt/d-cache /srm/bin/srmcp --debug=true - srm_protocol_version=2 -space_token=5 file:////home/tdh/tmp/test1 srm://fapl037.fnal.gov:8443/srm/managerv2?SFN=/pnfs/fnal.go v/data/public/test3 Namespace is restricted by /etc/grid-security/storage-authzdb. Pool selection is restricted by LinkGroupAuthorization.conf and rules in PoolManager.conf.