1. Storage Classes report GDB Oct 4 2006 Artem Trunov Artem.trunov@cc.in2p3.fr

2. Pre-GDB meeting Short report on Storage Classes WG progress Discussion on storage classes (and userSpaceTokenDescription) Details of the implementation of storage classes in storage systems Castor, dCache. Experiment’s use cases how to map them to existing SRM and storage functionality Site’s implementations RAL, CERN.

3. Main focus – three storage classes Tape0Disk1 Only disk, no tape Tape1Disk0 Disk cache in front of tape, system managed. Tape1Disk1 Disk cache in front of tape, files pinned on disk

4. Storage Classes – pictorial view Tape disk Tape disk Tape disk Tape0Disk1 Tape1Disk0 Tape1Disk1 Files are on disk. Explicitly managed by the VO Tape backend not required, but if used, files are guaranteed to be on disk When disk is full, srm put() will return an error Files are transferred to the WAN (transfer) pool (by definition), but don’t stay on disk System-managed disk cache A File can be stage in System managed in both cases, but user-specified pin time can be set or taken into account Files are pinned on disk by the system and guaranteed to be on disk. WAN pool basically becomes just a technical intermediate pool used for transfers - File does not stay on disk (doesn’t have a pin) - A file has some pin time on disk - File stays in disk

5. Storage Classes – basic use cases Three storage classes are really what experiments want, and how they mostly want to do their data management: Tape1Disk0, Long Term Inactive Use case: transfer of raw data from CERN. It needs to be archived on tape, but not kept on disk after archiving. It will be later staged in for reprocessing. Tape1Disk1, Long Term Active Use case: transfer of aod data produced on site or elsewhere for custodial storage (first copy at producer’s site). It needs to be archived on tape, and kept on disk for user’s analysis. Tape0Disk1, Short Term Active Use case: temporary output of production data, which doesn’t need to be archived to tape. Use case: replica of aod data for which the site is not a custodian (Atlas). Plus, experiments will need to request data stage in, purge from disk, remove permanently from disk and tape.

6. Storage classes vs. spaceTokenDescription Storage Classes are internal concept in the storage. Users need to pass spaceTokens with transfer requests, that are them mapped to storage classes in the storage system. ATLAS_RAW, ATLAS_ESD There is always a default space, for which a space token is not passes. Tape1disk0 at T1s with MSS backend Tape0Disk1 at sites without MSS backend.

7. Thought to make the life simpler Map those storage classes to VO name space subdirs. /store/CSA06 -> Tape1Disk0 /store/unmerged -> Tape0Disk1 Also map COS to VO name space subdirs for tape optimization Tape1disk0 and tape0disk1 are default for tape-backed space and disk-only space correspondingly. Only tape1disk1 needs to be specially passed with transfer requests If this class is not used – the rest is easy. Would work nice in dCache, but implementation in Castor is in favor of always using spaceTokens. Will still profit from well structured VO name space.

8. Details dCache relies on storage groups associated with directories for mapping to COS or families in MSS Meaning that need to enforce structured VO name space to ensure that data with different access pattern goes to different set of tapes. Castor maps storage class directly to svcClass, pools. Tape1Disk0 classes is default. Tape0Disk1 in Castor is still backed up on tape, so effectively becomes Tape1Disk1. Tape1Disk1 in Castor can be setup such that it will still purge less used files (RAL). This is to prevent system meltdown and failing transfers – satiations where human involvement is needed. In both Castor and dCache changing from Tape1Disk0 to Tape1Disk1 does not bring a file on disk, but rather marks in as such in metadata, and files are staged in on first access or explicitly by srmBringOnline. ! – so need to use this class carefully Class transitions Tape1Disk0 Tape1Disk1 – OK T0 T1 – not supported. To archive a file on tape, a VO needs to retransfer a file with T1Dx class

9. Experiments Presented use cases in terms of Storage Classes Atlas mapped everything in T0D1 and T1D0 But could also make use of T1D1 when it’s available Alice will use only T1D0 and T0D1 CMS de facto using only T1D0 and T0D1, but the strategy and details are still to be refined. LHC foreseen use of T1D1 besides other classes.

10. Use of Tape1Disk1 class on theexample of one LHCb use case Use case: need to transfer RAW to a T1, archive to tape and have it stayed on disk for 48 hours for reprocessing jobs Solution 1: Use Tape1Disk1 class with transfer request. Issue a class transition function Tape1Disk1 to Tape1Disko after reprocessing However it’s a bit fragile, since VO is required to keep track of files/classes. May work but, but may also bring troubles. Solutoin 2: Use Tape1Disk0 class with transfer request. Then use srmBringOnline with pintime (or without) to make sure a file will stay on disk for required time. No ‘clean up’ is necessary – files are system-managed. Of course, may not guarantee that 100% of files are on disk for reprocessing jobs, but still a good compromise solution. Given some problems Tape1Disk1 class may bring, it’s use is in general discouraged

11. Next Will continue working on storage classes and experiment’s use cases Next meeting on the topics in again before the next GDB.