30-31 Jan 2003J G Jensen, RAL/WP5 Storage Elephant Grid Access to Mass Storage

30-31 Jan 2003J G Jensen, RAL/WP5 Overview ControlInfoData Xfer Storage Element MSS 2 MSS 1 Grid clients

30-31 Jan 2003J G Jensen, RAL/WP5 Interfaces Control Pin/unpin files (reserve for reading) Create new file (reserve for writing) Modify metadata... Information Get amount of free space Protocols supported by the SE... Data Transfer

30-31 Jan 2003J G Jensen, RAL/WP5 HTTPS –Reliable, well known protocol –Looking at extensions for Grid security Web services –Rapid development –Language independent (but some languages are more independent than others…) GridFTP –Extension of well known FTP protocol, complicated –Achieves good transfer performance LDAP WP3 information producer –Currently provides only status of daemons Worker nodes can access their local SE via NFS –We will investigate using SlashGrid instead – RFIO is trivial to add Protocols

30-31 Jan 2003J G Jensen, RAL/WP5 Portability RPMs have been or can be built for RedHat 6.2, 6.3, 7.2, 7.3 –Support for RedHat 8.x soon SE builds from source on both Debian x86 stable and testing and on other Linux. TODO: Test on Solaris - ?

30-31 Jan 2003J G Jensen, RAL/WP5 Client App API SE HTTP library SSL socket library Apache Tomcat Axis RMANMAN SE core C Client SE Web Services Client Axis libraries Java Clients The design of the SE follows a layered model with a central core handling all paths between client and MSS. Core is flexible and extensible making it easy to support new protocols, features and MSS

30-31 Jan 2003J G Jensen, RAL/WP5 RManager SE CORE Handler 1 Handler 2 Handler 3 Handler 4 Response returned to user MSS Each handler performs a simple task. The sequence of handlers depends on type of request, the type of storage system, local configurations, etc. Rmanager (request manager) is responsible for running handlers This architecture is extremely flexible and allows SE to easily meet diverse requirements for both clients and backend MSS.

30-31 Jan 2003J G Jensen, RAL/WP5 Interfaces to SE ControlInformationData Transfer HTTPS GridFTP Web S. Done Nearly done Planned Done Not planned Note: diagram shows existing interfaces, not implemented functionality

30-31 Jan 2003J G Jensen, RAL/WP5 SE functionality For MSS with many features, the SE will be a thin layer For MSS with few features (or disk), SE will provide missing functionality Functionality CASTOR ADS Disk = SE

30-31 Jan 2003J G Jensen, RAL/WP5 Security Access control: –Currently uses Andrew McNabs GACL –Will also use VoMS (and CAS?) –Fine grained access control – files and dirs all have ACLs but ACLs can also be shared between files/dirs GSI –Will use G-HTTPS –Currently uses Mike Jones patch to mod_ssl to accept proxies

30-31 Jan 2003J G Jensen, RAL/WP5 SRM Collaboration SRM – storage resource manager – collaboration with LBNL, Fermi, Jefferson, CERN Particle Physics Mass Storage System interface –Mostly control interface; data transfer done through GridFTP Old SRM specification version 1.0 – IDL, web service..? –Some naughtiness: server callbacks New specification: interoperability through WSDL –Is WSDL sufficient for interoperability? –Complicated space reservation semantics

30-31 Jan 2003J G Jensen, RAL/WP5 Non-PP Grids There exist people who will use Grids for non-PP, non-HEP projects! Examples from DataGrid: –Earth observations (WP9): will write handlers for their own MSS. EO applications have many (10 6 ) small (2-30 kB) files –Medical Imaging (WP10): will do the same but WP10 have more challenging requirements Confidentiality – data must be enciphered when it goes out to the Grid Key will be held in a Grid database on trusted site CrossGrid is / will be based on DataGrid software

30-31 Jan 2003J G Jensen, RAL/WP5 Deployment SEs are deployed and working at –CERN, talking to CASTOR using RFIO –RAL, talking to ADS and disks SE soon to be deployed for testing and development at INSA Lyon –INSA SE will talk to DICOM (medical images) Integration with testbed in March, tested by applications in May

30-31 Jan 2003J G Jensen, RAL/WP5 Concluding Remarks Current interfaces are flexible and can be expanded to support new commands and features, or add support for new protocols Core is very flexible and it is easy to add new functionality Were making the SE easier to install and maintain – will be installable by sysadmins rather than developers!