1. DPM Name Server (DPNS) Namespace Authorization Location of physical files DPM Server Requests queuing and processing Space Management SRM Servers v1.1, v2.1 and v2.2 Database (DPM and DPNS) MySQL and Oracle are supported High scalability and availability DPM and DPNS scalability is directly linked to the database (MySQL or Oracle) scalability All servers can be load balanced, except the DPM one Security in DPM GSI Permissions – Ownership ACLs VOMS support Virtual uids/gids (based on VOMS groups and roles) SRM v2.2 Admin srmPing Space Management srmReserveSpace, srmReleaseSpace,... Permissions and ACLs srmSetPermission, srmGetPermission,... Transfer srmPrepareToPut, srmPrepareToGet,... Namespace srmLs, srmMkdir The DPM (Disk Pool Manager) is a secure, lightweight and highly scalable system to manage storage on disks DPM head node Disk Servers The Disk Pool Manager (DPM) Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 The Disk Pool Manager Data Management in gLite Status and Next Steps file DPM disk servers /vo /dpm /domain /home DPM head node CLI, C API, SRM-enabled client, etc. data transfer … Quotas srmCopy NFSv4.1 support Medical Data Management Encryption hep-service-dpm@cern.ch New features http / https In the namespace Secondary groups support On disk pools Secondary groups intentionally not supported at pool level (so that the group or sub-group who actually uses the space gets accounted for it) But possibility to: (1) restrict a pool to a list of groups (2) add and remove groups later on Web access to DPM storage via http and https The DPM Architecture Largest DPM Storage 80 TB (Glasgow) Largest DPM Name Server million entries (Oracle) Thanks to Gidon Moont (GridPP – Imperial College – London) See http://gridportal.hep.ph.ic.ac.uk/rtm/http://gridportal.hep.ph.ic.ac.uk/rtm/ DPM target in EGEE Small & medium-sized sites What do sites like? Easy installation Easy administration DPMs in production 125 instances 138 VOs supported http://twiki.cern.ch/twiki/bin/view/LCG/DataManagementTop (CERN) Akos FROHNER, Jean-Philippe BAUD, David SMITH, Sophie LEMAITRE, Krzysztof NIENARTOWICZ, Lana ABADIE, Rémi MOLLON (1) dpm-addpool --poolname AtlasPool --def_filesize 1G --group atlas, atlas/Role=production (2) dpm-modifypool --poolname AtlasPool --group +atlas/Role=lcgadmin,- atlas/Role=production GridFTP Server Secure RFIO Server The Disk Servers host the actual physical files No bottleneck: direct data transfer from/to the Disk Servers Easy to guarantee space for a given VO Easy space reservation Space can easily be reserved (1) and released (3) by users (limited) administrators (unlimited) (1) dpm-reservespace --gspace 200G --group atlas --token_desc atlas_ESD (2) lcg-cr --vo atlas file:higgs.root --st atlas_ESD –d dpm.cern.ch (3) dpm-releasespace --token_desc atlas_ESD The user space token created in (1) by the DPM administrator can be used by any user in the atlas group. Ex. with lcg_util (2) $ export DPNS_HOST=dpm.cern.ch $ export DPM_HOST=dpm.cern.ch $ rfcp file01 /dpm/cern.ch/home/atlas/data/file01 $ rfrm /dpm/cern.ch/home/atlas/file01 Usage example: