DPM Status & Roadmap Ricardo Rocha ( on behalf of the DPM team ) EMI INFSO-RI

CLIENT DPNSDPMSRMHTTPNFS GRIDFTP RFIOHTTPNFSXROOT HEAD NODE DISK NODE(s) FILE METADATA OPS FILE ACCESS OPS DPM Overview RFIO HTTP NFS XROOT

DPM Core 1.8.2, Testing, Roadmap

DPM – Highlights Improved scalability of all frontend daemons – Especially with many concurrent clients – By having a configurable number of threads Fast/Slow in case of the dpm daemon Faster DPM drain – Disk server retirement, replacement, … Better balancing of data among disk nodes – By assigning different weights to each filesystem Log to syslog GLUE2 support

DPM Core – Testing Activity Improved validation & testing – Collaboration with ASGC for this purpose (thanks!) – Hammercloud tests running regularly – They started with a 400 core setup, we looked at the issues, now moving to 1000 cores to increase load Example run – To be used extensively for stress testing – Covering all components: DPM, RFIO, GRIDFTP, NFS, HTTP, … Results will benefit other sites too

DPM Core – Testing HC using GridFTP Thanks to ShuTing for the plots ( preliminary results ) HC using RFIO Example GridFTP vs RFIO

DPM Core - Testing Big contribution from openlab student – Martin Hellmich, University of Edinburgh Detailed analysis of DPM internals – Detecting bottlenecks in specific transfer / access phases Example… but we have a lot more results which we are now investigating

DPM Core – Roadmap Package consolidation: EPEL compliance Fixes in multi-threaded clients Replace httpg with https on the SRM Improve dpm-replicate (dirs and FSs) GUIDs in DPM Synchronous GET requests Reports on usage information Quotas Accounting metrics HOT file replication

Beta Components HTTP/DAV, NFS, Nagios, Puppet, Perfsuite, Catalog Sync, Contrib Tools

Beta Components: Overview Faster releases – Monthly releases since June Separate yum repository Already in use by several sites – Including sites in the UK

Beta Components: PerfSuite Overview

Performance Suite Set of tools to easily trigger bunches of tests – With different configurations Common wrapper, many tests Existing suites – POSIX Transfers: RFIO, NFS – GET/PUT Transfers: HTTP, GSIFTP – ROOT – More coming… Used for most results presented later

Performance Suite Set of tools to easily trigger test bunches – With different configurations Common wrapper, many tests Existing suites – POSIX Transfers: RFIO, NFS – GET/PUT Transfers: HTTP, GSIFTP – ROOT – More coming… Used for most results presented later Sample Configuration test_rfcp(c:5,s:{1M 2M 4M 8M 16M 32M 64M 128M 256M 512M 1G})x3 test_nfs(m:/mnt/nfs41,c:5,s:{1M 2M 4M 8M 16M 32M 64M 128M 256M 512M 1G})x3

Beta Components: HTTP / DAV Overview, Performance, Roadmap

HTTP / DAV: Overview CLIENT LFC DPM HEAD DPM DISK GET GET / PUT REDIRECT DATA

HTTP / DAV: Overview CLIENT LFC DPM HEAD DPM DISK GET GET / PUT REDIRECT DATA

HTTP: Client Support curlbrowser OSAny GUINOYES CLIYESNO X509YES ProxiesYESOnly IE so far RedirectYES PUTYESNO Recommendation: browser/curl for GET, curl for PUT Chrome Issue 9056 submitted for proxy support

DAV: Client Support TrailMixCadaverDavlibShared Folder DavFS2NautilusDolphin OSFirefox < 4*nixMac OS XWindows*nixGnomeKDE GUIYESNOYES N/AYES CLINOYESNO N/ANO X509YES NOYES NO Proxies?NO YESNO RedirectYESNOYESNot PUTNO YES Updated analysis based on initial one from dCache Recommendation: Cadaver for *nix, Windows explorer

HTTP vs GridFTP: Multiple streams Not explicit in the HTTP protocol But needed for even higher performance – Especially in the WAN So we added it, with some semantics – Small wrapper around libcurl – PUT with ‘0 bytes’ && null content-range == end of write Submitted patch to libcurl to allow ssl session reuse among parallel requests

HTTP vs GridFTP: 3 rd Party Copies Implemented using WEBDAV COPY Requires proxy certificate delegation – Using gridsite delegation, with a small wrapper client Requires some common semantics to copy between SEs (to be agreed) – Common delegation portType location and port – No prefix in the URL ( just / )

HTTP vs GridFTP: 3 rd Party Copies Example of FTS usage

Ongoing Evaluation HTTP / DAV: Performance No difference detected in LAN with different number of streams – But early results do show a big difference on the WAN lcg-cp configured to use gridftp File registration & transfer times considered in both cases Xeon 4 Cores 2.27GHz 12 GB RAM 1 Gbit/s links

HTTP / DAV: Issues & Roadmap Towards a first production release – Testing with large number of concurrent clients – Finish up the WAN performance tests And after that – Further testing of 3 rd party copy with larger files – Finish validation against other implementations – Validate usage via ROOT – Improved GET on the LFC – PUT support on the LFC (?)

Beta Components: NFS 4.1 / pNFS Overview, Performance, Roadmap

NFS 4.1/pNFS: Why? Industry standard (IBM, NetApp, EMC, …) Free clients (with free caching) Strong security (GSSAPI) Parallel data access Easier maintenance … But you know all this by now…

NFS 4.1/pNFS: Overview CLIENT METADATA SERVER DISK SERVER(s) OPEN 1 LAYOUTGET 2 3 GETDEVICEINFO 4 OPEN 5 READ / WRITE 6 CLOSE 7

NFS4.1 / pNFS: Client pNFS support in linux kernel from >= nfs-utils >= Latest Fedora and Debian Sid have it We provide packages for EL5 – Enabled pNFS in the elrepo mainline kernel – nfs-utils and AFS module we package ourselves

NFS4.1 / pNFS: Performance IOZONE Results Ongoing Evaluation Server – Xeon 4 Cores 2.27GHz – 12 GB RAM – 1 Gbit/s links Client – Dual core – 2 GB RAM – 100 Mbit/s link

NFS4.1 / pNFS: Performance NFS vs RFIO Ongoing Evaluation RFIO read misbehaving in this test… investigating Server – Xeon 4 Cores 2.27GHz – 12 GB RAM – 1 Gbit/s links Client – Dual core – 2 GB RAM – 100 Mbit/s link 8 KB block sizes

NFS4.1 / pNFS: Issues & Roadmap Towards a first production release – Tests with a faster network link – Testing with a larger number of concurrent clients – WAN testing – Enable bigger block sizes And after that – X509 certificate support Still not figured out… needs a strong focus – Further validation with other implementations

Beta Components: Even more… Puppet, Nagios, Contrib, Catalog Sync

Even more components… Catalog Synchronization – Check Fabrizio’s talk next Monday (EGI Forum Lyon) DPM Admin contrib package – Contribution from GridPP – Now package and distributed with the DPM components – Nagios monitoring plugins for DPM – Available now – Puppet templates – Available now in beta –

Conclusion fixes many scalability and performance issues – But we continue testing and improving Popular requests coming in next versions – Accounting, quotas, easier replication Beta components getting to production state – Standards compliant data access – Simplified setup, configuration, maintenance – Metadata consistency and synchronization And much more extensive testing – Performance test suites, regular large scale tests