1. 1 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh Deployment of Storage Resource Broker at CCLRC for E-science Projects Ananta Manandhar Data Management Group CCLRC Daresbury Laboratory

2. 2 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh Overview SDSC SRB overview Use of SRB in EScience Projects Eminerals/Ematerials Project CERN CMS SRB support UK National Grid Service – SRB Federation Questions

3. 3 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh Storage Resource Broker The SDSC Storage Resource Broker is a client server middleware that virtualizes data space by providing a unified view to multiple heterogeneous storage Resources over the network. It is a software that sits in between users and resources and provides a storage service by managing users, file locations, storage resources and metadata information SRB Space

4. 4 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh SRB Logical Structure DB SRB View Scientist Disk Farm Tape Farm RDBMS Hides the physical location of the file by providing a logical view to the users files which is arranged in a UNIX like hierarchical structure The logical directory structure and the physical file locations are maintained in a Database. Also other persistent information such as User accounts, Participating resources and file meta data are stored in it.

5. 5 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh Oracle ClientSRB Server MES SRB Server Oracle RDBMS SRB Vault SRB Physical Structure SRB Server Storage Space Storage Driver SRB Server Storage Space Storage Driver SRB Server Storage Space Storage Driver User

6. 6 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh SRB Communication a)SRB Client sends request for file to SRB server. b)SRB Server contacts MCAT Enabled Server (MES). c)MES translates query into SQL and sends to database hosting MCAT d)Database query returned to MES e)Location of file etc returned to SRB Server A. f)SRB Server A contacts SRB Server B hosting data file. g)Data file transferred to user. User File Request User MCAT enabled Server MCAT Database SRB Server a b c d e f g

7. 7 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh SRB Access Interfaces Scommands Unix file system like interface Versions available for Unix, Dos and Mac inQ – Windows Explorer style interface Version available for Windows My SRB Web Browser interface Client access API C API JARGON – Java API to SRB. MATRIX – SRB Workflow management system

8. 8 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh SRB User Interfaces Scommands – primary interface to SRB Bash like commands for interacting with SRB. Versions available for Unix, Windows and Mac Can also be used for scripting for batch processing.

9. 9 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh SRB User Interfaces InQ – Windows Explorer Style interface to SRB Primary highlights includes Support for drag and drop between Windows mounted filesystems Provisions for access control enforcements, file replication, metadata entry and metadata query.

10. 10 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh SRB User Interfaces My SRB Interface – Web browser interface to SRB Web based interface to SRB space. Works through port 80 and hence works around firewall issues.

11. 11 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh SRB Admin Tool For managing: Users Domains Resources Collaborating Machines Collaborating Zones (version 3 series)

12. 12 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh Behind the scenes Behind the scenes SRB provides many other functionalities in managing files and resources Supports grouping of multiple physical resources into a logical resource. Support for direct Client Server parallel file transfers for performance improvements Support for bulk transfer of multiple small files into SRB server Supports grouping of multiple files into ‘containers’ which is then manageable for insertion and retrieval from Mass Storage systems. Fine Grained Access Control Meta Data Query and File Replication between resources

13. 13 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh Eminerals Project UK e-Science project for investigating the environment from the molecular level Current collaborating Institutions Cambridge Royal Institution University College London Bath Reading CCLRC Daresbury Laboratory e-Minerals: Environment from the Molecular Level

14. 14 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh eMinerals SRB Domain JinxElektra DL databases SRB Application Server CCLRCFS CambsLakecambridgeVault UCLLake Storage Vaults CCLRC Cambridge UCL ScommandsInQ Web Browser MySRBData Portal SRB

15. 15 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh eMinerals integration with Data Portal Web Browser Data Portal HPC Portal SRB Oracle DB1Oracle DB2 SRB Tools Files maybe inserted and retrieved using SRB tools Also files may be accessed via the data portal if user Data Portal is also given access privileges The Data portal may utilize different meta data to locate files

16. 16 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh SRB Space (Used as a Project file system) Use of SRB on Eminerals Use of SRB with Condor/Globus at Cambridge Condor Client Tools/ DAG Clients Globus 2.4 Head Node Computing Cluster Scientist @ Cambridge Scientist @ UCL Condor G interface Condor Pool Code Repository Linux Solaris Data Repository

17. 17 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh CMS SRB CMS is one of the Large Hadron Collider (LHC) particle accelerator experiments at CERN. For CMS DC04 SRB was used at RAL - UK IN2P3 - France Karlsruhe - Germany CERN - Switzerland MCAT hosted at RAL

18. 18 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh SRB Disk Server CERNSRB Disk Server1 RALSRB Disk Server2 RAL Oracle Database Cluster at DL running SRB MCAT SRB MCAT Server Disk Server at IN2P3 Tape Servers at IN2P3 Disk Servers at FZK Atlas Storage Driver Atlas 1 PB Tape Robot and 5 TB Disk Cache at RAL Oracle Database Cluster at RAL running SRB MCAT

19. 19 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh File Transfer Agents were implemented at CMS for transferring files from source to the T1 centers. MSS RAL T1 RAL Export Buffer MSS IN2P3 T1 IN2P3 Export Buffer T1 FZK Export Buffer Data Generator CERNT1 CentersTransfer Management DB

20. 20 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh MCAT Enable Server Databases are clustering on each site and mirrored between Daresbury and Rutherford Appleton Databases CCLRC DL Databases CCLRC RAL Oracle Data Guard MES and Database Application Server Oracle Client

21. 21 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh Statistics Roughly about 260,000 connection were seen per day It is currently stores about 48 TB of data 1 million data files 70 Physical Resources 18 Countries

22. 22 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh National Grid Service Data Nodes RAL – 20 node data cluster with 20TB RAID Manchester – 20 node data cluster with 20TB RAID Computational Nodes Oxford – 64 node dual Xeon cluster Leeds – 64 node dual Xeon cluster Services Computational service File management service Database service The National Grid Service is a new initiative at UK to provide computing resources to the e-Science community

23. 23 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh SRB Federation DB SRB Enables an SRB Network to recognize the presence of another SRB Network and be able to interact with it. The overall federation can still be viewed at one single namespace whereby each participating Zone is a child node of the root node. This keeps the learning curve very low and travelling from one Zone to another is like changing directory. To go to Zone B: # Scd /B/home/ananta.Adomain To copy a file back to Zone A # Scp foo.txt /A/home/projectX DB SRB DB SRB Trust Relation Trust Relation Trust Relation Zone B Zone C Zone A

24. 24 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh Why Federation As Federation enables SRB zones to recognize and interact with other zones it opens many possibilities in the types of interaction Federation Enables Fragmentation of MCAT Database An MCAT fragment coming down due to network or system failure does not bring the entire SRB down. The other sections continue to provide the server leaving only one section temporarily inaccessible. With federation, meta data can be made to reside closer to the resource by hosting a MCAT server closer to the resource. This enables reducing network latency, thus may enable an overall improvement in performance when an SRB network spans across large geographical distance. Federation enables local administrative control of resources enabling the notion of Virtual Organizations (VO) and also make it easy for a new VO to join the network It also enables easy replication and transfer of documents between VO

25. 25 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh SRB deployment on NGS RAL MCAT SRB Access Point 2 SRB Access Point 1 Oracle Cluster Oxford Access Point Manchester MCAT SRB Access Point DL Access Point Oracle Cluster Leeds Access Point RAL Zone Manchester Zone RAL Domain Oxford Domain Leeds Domain Manchester Domain Trust Relationship

26. 26 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh RAL Configuration ….. Oracle 9i RAC (Nodes 1.. 8) MCAT Server (SRB v3.1) Oracle Client Oxford SRB RAL SRB RAL Zone Manchester Zone Trust Relation User / Process

27. 27 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh Use of GSI on NGS SRB Verification of the authenticity of the user is left to the certification authority rather than by the use of password stored in the MCAT MCAT maps the user’s DN to its internal Access control mechanism for providing reliable service. Benifit: User can be authenticated to any participating MCAT server without contacting its local MCAT. A computational process may work on behalf of the user provided that the user has delegated its proxy certificate. Enables single sign-on for all applications as they all recognize the same authentication framework.

28. 28 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh /ral/home/ematProj/bin /ral/home/ematProj/data /man/home/ematProj/bin /man/home/ematProj/data Ematerials Domain at RAL Ematerials Domain at Man SRB utilization by Projects on NGS Synchronization of Project Users, Replication of files Job Execution at Oxford Computing Farm Oxford computing Farm Temporary file storage at Oxford SRB Vault Leeds computing Farm Temporary file storage at Leeds SRB Vault Job Execution at Leeds Computing Farm User / Process

29. 29 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh Benefits for Projects Immediate Benefits for projects Better reliability – One of the site being inaccessible will not bring down the system Improved performance – Reduction in network latency in WAN environments Interoperability with other Grid applications deployed on NGS by the use of same authentication mechanism (GSI) Easy sharing of data within the project and between projects Provision of storage and computing resource

30. 30 eScience Grid Environments- 8-12 th May 2004 NESC - Edinburgh Questions..