1. INFSO-RI-508833 Enabling Grids for E-sciencE www.eu-egee.org University of Coimbra GSAF Grid Storage Access Framework Salvatore Scifo INFN of Catania EGEE User Forum Manchester, UK - 10th-11th May 2007

2. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 2 Partnership Grid Storage Access Framework –The project is carried out by a cooperation between the INFN and the IR&T engineering s.r.l. (a SME located in Catania, Italy). –The context of this work is the e-Infrastructure Trinacria Grid Virtual Laboratory Project and the ADAT Project (“Archivi Digitali Antico Testo”), aiming at the implementation of a Digital Archive for Cultural Heritage Data (antique manuscripts) with Digital Repository based on EGEE grid services. Resources –INFN  S. Scifo (s.scifo@ct.infn.it)s.scifo@ct.infn.it  A. Calanducci (a.calanducci@ct.infn.it)a.calanducci@ct.infn.it  On behalf of the Gilda Team –IR&T engineering (http://www.irt-engineering.it)http://www.irt-engineering.it  V. Milazzo (v.milazzo@irt-engineering.it)v.milazzo@irt-engineering.it  A. Magrì (a.magri@irt-engineering.it)a.magri@irt-engineering.it

3. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 3 Web Integration Requirements Main objectives of web application –Infrastructure side  Organize and handle big amounts of information  Share documents among several organizations –Security side  Define and apply Access Control Policies –Development side  Build application without specific technical knowledge of the adopted infrastructure (high level API)  Build and maintain dynamic web content (simple tools to manage repositories for provisioning purposes) –User side  Manage Groups and Users (administrative user profiles)  Manage Digital Resources (authoring user profiles)  Access, search and retrieve file/data easily (end user profiles)

4. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 4 Grid as Digital Repository Repository Virtualization provided by –interfaces to manage DATA –interface to manage METADATA Data Management capabilities –Large and numerous file handling also in distributed environments –Ubiquity: data access independently by their location Security capabilities –Centralized access control mechanism based on x.509 certificates Systems capabilities –Availability, Scalability, Fault Tolerance

5. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 5 Classic Web Application Data Presentation Layer consists of all graphical interfaces that make user able to interact with application Data Business Layer collects all software components that implement the behavior of the given application Data Access Layer is made up by software components that allow application to manage data (ascii files, xml files, digital object, metadata, SQL data) Data Access Layer components interact to several types of data sources –File System (for data stored into files) –Relational Database Management System (for data organized into SQL tables)

6. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 6 Grid Web Application Grid environment porting a spects –files are stored inside a Storage Element (SE) –files can be replicated on several SEs for ubiquity, security and sharing needs; relationship among locations of files, replicas and theirs identifiers are kept within a specific File Catalogue Service –for each file is possible to associate descriptive metadata arranged through a specific Metadata Catalogue Service Technical Approach –replace Data Access Layer with an appropriate interface that permits:  business components to manage data stored within the DMS  presentation objects to search and retrieve data from DMS

7. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 7 Designers point of view Development of applications (web or desktop) is not easy This fragmentariness forces software engineers and web designers to consider a vertical architecture Application must take care about the atomicity, coherence and synchronization of data manipulation –Grid Data Services are independent each from others –They work in a “stand a lone” mode –Any kind of coherence is ensured

8. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 8 Users point of view User can use only Command Line Tools These tools are installed on specific machines called User Interface (UI) and located inside the Grid network boundaries Users encounter several problem about net access User has a personal UI –who does ensure its security? All logical relationships among data and metadata must be kept in his mind

9. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 9 GSAF solution GSAF is an Object Oriented Framework –built on top of the Grid Metadata Service and Grid Data Service and exposes classes and related methods for applications located above Main objective –hide the complexity and the fragmentation of the several underlying APIs –grouping functional requirements shared among applications –ensure atomicity among different data manipulation

10. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 10 GSAF System Architecture

11. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 11 GSAF Functional Requirement Managing Metadata Schemas Managing ACLs to access Metadata Managing ACLs to access Data Uploading file to the SE Browsing Metadata Catalogue \ File Catalogue Search file by Metadata Deleting file

12. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 12 GSAF Web Interface GSAF Web Interface to manage data and their metadata remotely –Initially, the main target of this application was to be a natural tester of the framework basic functionalities –it represents a useful tool to administrate the Grid Storage through internet Web Interface is the easiest approach –for new users which don’t have specific knowledge of the Grid environment. –no syntax rules are required and users don’t loose the high level view of data neither of metadata schemas. –immediate interaction thanks to comfortable and friendly driven procedures that make training and learning faster. –web application needs only a simple internet connection than it avoids any dependencies from the Grid UI machines.

13. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 13 GSAF Web Interface

14. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 14 ADAT project

15. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 15 Use Cases ADAT Project –embeds GSAF within the Digital Archive Software Physics Department of the University of Catania (PI2S2 project) –aims to implement a Grid Oriented Digital Archive for DICOM images. BM Portal project (Bio-Lab, DIST University of Genoa ) –embeds GSAF framework as a plug-in GILDA Team –adopts the GSAF web interface for dissemination and training purposes.

16. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 16 Outlook File Replica support VOMS Integration ACLs at Disk Pool Manager Level –for coherence between File Catalogue permissions and DPM permissions Transaction Manager –Serialization levels –Transaction pattern  Execute()  Commit()  Rollback() All we need to integrate applications….

17. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 17 Conclusions GSAF means –Useful API to develop Grid Storage based applications –Useful and simple web interface to access Data Management Services remotely extreme flexible, multi platform and multi user –to be a cross application domain plug-in comfortable usage of the Web Interface –to be a simple Content Management Tool to manage data remotely candidate at the EGEE Respect Program –to become a recommended external software for the EGEE middleware

18. Enabling Grids for E-sciencE INFSO-RI-508833 Manchester, 10-11 May 2007 18 References GSAF wiki pages –https://grid.ct.infn.it/twiki/bin/view/TRIGRID/GSAFhttps://grid.ct.infn.it/twiki/bin/view/TRIGRID/GSAF Amga Web Interface wiki pages –https://grid.ct.infn.it/twiki/bin/view/TRIGRID/AMGAWIhttps://grid.ct.infn.it/twiki/bin/view/TRIGRID/AMGAWI AMGA Service and Java API –http://project-arda-dev.web.cern.ch/project-arda-dev/metadata/index.htmlhttp://project-arda-dev.web.cern.ch/project-arda-dev/metadata/index.html GFAL Java API –http://grid-deployment.web.cern.ch/grid-deployment/gis/GFAL/gfal.3.html –https://grid.ct.infn.it/twiki/bin/view/GILDA/APIGFALhttps://grid.ct.infn.it/twiki/bin/view/GILDA/APIGFAL LFC Java API –http://wiki.egee-see.org/index.php/SEE-GRID_File_Management_Java_APIhttp://wiki.egee-see.org/index.php/SEE-GRID_File_Management_Java_API IR&T engineering s.r.l. –http://www.irt-engineering.comhttp://www.irt-engineering.com Trigrid VL –http://www.trigrid.ithttp://www.trigrid.it