WP2: Data Management Gavin McCance University of Glasgow

2/18 GridPP, September 2002Gavin McCance, University of Glasgow Overview Current status of components Testbed 1.2 strategy Testbed 2 strategy Meta-data: Spitfire OptorSim

3/18 GridPP, September 2002Gavin McCance, University of Glasgow Current software status Replication: edg-replica-catalog – deployed GSI version available. Graphical User Interface available. Replication: edg-replica-manager – deployed Atomic replication operations. This component should be used for data management tests. Replication: GDMP – deployed More complex subscription-based functionality. Has been tested and is proven to work. Can be abused to provide the same functionality as edg-replica- manager but it is overkill and much harder to do. Metadata Storage: SQL DB Service Spitfire Browser – deployed Web browser based access to metadata GSI enabled + authorisation Optimisation: Replica Selection OptorSim – working

4/18 GridPP, September 2002Gavin McCance, University of Glasgow …Current software status Replication: Replica Location Service – deployable Joint Globus – EDG development Unit and performance tests successful Distributed Replica Catalog service Graphical User Interface available. Security: edg-security module – in testing Security for other Web Services Component Metadata Storage: SQL Database Service Spitfire – in testing Web Service interface over SOAP Remote Procedure Call Security through edg-security and proper local authorisation Complimentary service to web based Spitfire Browser Replication: Replica Manager Reptor – in testing and development Optimization: Replica Selection Optor – in testing and development Replication: Replica Metadata Catalog Repmec – in testing and development Security: edg-voms – in development, joint effort with WP6, WP7

5/18 GridPP, September 2002Gavin McCance, University of Glasgow Strategy for 1.2 Integrate a single-instance RLS into TB1.2 (replacement for edg-replica-catalog) Integrate RLS into edg-replica-manager and GDMP Focus users away from GDMP and towards edg-replica-manager for simple data management use cases. Increment stability and usability of edg-replica-manager Configuration Simplify configuration and build system of deployed components Unify build for all components, adopt edg-standards Testing Large effort on unit testing and functionality tests inside WP2 for all components. Close interaction with ITeam and the new testing group

6/18 GridPP, September 2002Gavin McCance, University of Glasgow Towards testbed 2 Focus away from GDMP Introduction of Reptor (refactorization of GDMP) Initially with identical functionality Incremental and controlled addition of requested functionality Use R-GMA for information gathering and publishing edg-replica-manager will stay as a command-line interface to Reptor, easy migration for users Security: work towards new security model with VOMS and local authorization. Documentation: Guides Examples, How-to Tutorials

7/18 GridPP, September 2002Gavin McCance, University of Glasgow Overview Current status of components Testbed 1.2 strategy Testbed 2 strategy Metadata: Spitfire OptorSim

8/18 GridPP, September 2002Gavin McCance, University of Glasgow Metadata interfaces Spitfire Metadata interface will handle general metadata e.g. calibration data, bookkeeping data Replica Metadata interface (RepMec) is an convenient extra service interface that will handle metadata which is keyed on Logical File Names. Both technical and application metadata

9/18 GridPP, September 2002Gavin McCance, University of Glasgow Spitfire Split into 2 independent products now. Spitfire-browser (web pages) Spitfire service itself (web services)

10/18 GridPP, September 2002Gavin McCance, University of Glasgow Spitfire browser The client is a web browser (optionally with a GSI certificate loaded) The service sits in front of a relational database. The service exposes specific views and operations of a database via web page forms.

11/18 GridPP, September 2002Gavin McCance, University of Glasgow Web form which sends user request to predefined template Any operation is possible although typical use would be simple select or simple update

12/18 GridPP, September 2002Gavin McCance, University of Glasgow After querying the result-set is returned and can be formatted as necessary GSI authentication & authorisation; full integration with WP2 security module

13/18 GridPP, September 2002Gavin McCance, University of Glasgow Spitfire web service WSDL defined Remote Procedure Call (RPC) interface to permit secure access to database. Sits in front of RDBMS. Currently available for Java, C, C++. Beta just released. GSI authentication and authorisation via WP2 security module

14/18 GridPP, September 2002Gavin McCance, University of Glasgow Spitfire for meta-data access RDBMS Application in GAUDI framework GANGA Services Spitfire Web Service Spitfire Client library Component::getCalibration( …, blah ) Spitfire::select( …, WHERE cal1=blah) Registry Secure web service connection

15/18 GridPP, September 2002Gavin McCance, University of Glasgow Build system WP2 working to extend build and configuration system for web services + servlet software Currently ~absent from testbed docu Merge with other WP approaches Common standard for deploying Web Services

16/18 GridPP, September 2002Gavin McCance, University of Glasgow OptorSim Optimise data access and replication based on an economic model View files as digital assets which can be bought and sold for profit Let nodes on the grid interact according to an economic model and thus optimise the system implicitly rather than explicitly

17/18 GridPP, September 2002Gavin McCance, University of Glasgow Results of OptorSim 0.4 Sequential Flat Random Unitary Walk Gaussian Access Pattern Basic Economic Model

18/18 GridPP, September 2002Gavin McCance, University of Glasgow …Optimisation to-do Looking at access patterns from SAM for tuning optimisation algorithms Improving OptorSim with a proper simulation of job times More realistic simulation of the eco- model process Application to the real-life Optor module (>=TB2.0)