1 CHEP 2000, 10.02.2000Roberto Barbera Tests of data management services in EDG 1.2 ALICE Off-line Week, 09-13.09.2002.

Slides:



Advertisements
Similar presentations
1 CHEP 2000, Roberto Barbera The ALICE Inner Tracking System Off-line Software CHEP 2000, (
Advertisements

CHEP 2000, Roberto Barbera NA3, NA4, and NA5 activities Milano, Università di Catania and INFN Catania - Italy ALICE Collaboration.
CHEP 2000, Roberto Barbera Roberto Barbera (*) GENIUS: a Web Portal for the GRID Meeting Grid.it, Bologna, (*) work in collaboration.
DataTAG WP4 Meeting CNAF Jan 14, 2003 Interfacing AliEn and EDG 1/13 Stefano Bagnasco, INFN Torino Interfacing AliEn to EDG Stefano Bagnasco, INFN Torino.
WP2: Data Management Gavin McCance University of Glasgow November 5, 2001.
ATLAS/LHCb GANGA DEVELOPMENT Introduction Requirements Architecture and design Interfacing to the Grid Ganga prototyping A. Soroko (Oxford), K. Harrison.
Data Management Expert Panel. RLS Globus-EDG Replica Location Service u Joint Design in the form of the Giggle architecture u Reference Implementation.
1 CHEP 2000, Roberto Barbera Recent grid activities at INFN Catania (*) HEPiX/HEPNT 2002, Catania, (*) work in collaboration with.
Workflows over Grid-based Web services General framework and a practical case in structural biology gLite 3.0 Data Management Hands-on David García Aristegui.
Workflows over Grid-based Web services General framework and a practical case in structural biology gLite 3.0 Data Management David García Aristegui Grid.
EGEE is a project funded by the European Union under contract IST Grid Data Management Hands-on Simone Campana LCG Experiment Integration and.
CHEP 2000, Roberto Barbera The AliEn GENIUS interface ALICE Offline Week, CERN, Università di Catania and INFN Catania - Italy ALICE.
Author - Title- Date - n° 1 GDMP The European DataGrid Project Team
1 CHEP 2000, Roberto Barbera Roberto Barbera (*) Grid monitoring with NAGIOS WP3-INFN Meeting, Naples, (*) Work in collaboration with.
Grid and CDB Janusz Martyniak, Imperial College London MICE CM37 Analysis, Software and Reconstruction.
DataGrid is a project funded by the European Union CHEP 2003 – March 2003 – Title – n° 1 Grid Data Management in Action Experience in Running and.
GRID DATA MANAGEMENT PILOT (GDMP) Asad Samar (Caltech) ACAT 2000, Fermilab October , 2000.
5 November 2001F Harris GridPP Edinburgh 1 WP8 status for validating Testbed1 and middleware F Harris(LHCb/Oxford)
Alice off-line meeting Alberto Colla Cern, October 3, 2005 AliEn How-To Alice off-line meeting Cern, October 3, 2005 Alberto Colla (Alice off-line Calibration.
KNMI Applications on Testbed 1 …and other activities.
3 Sept 2001F HARRIS CHEP, Beijing 1 Moving the LHCb Monte Carlo production system to the GRID D.Galli,U.Marconi,V.Vagnoni INFN Bologna N Brook Bristol.
INFSO-RI Enabling Grids for E-sciencE The GENIUS Grid portal Tony Calanducci INFN Catania - Italy First Latin American Workshop.
CHEP 2000, Giuseppe Andronico Grid portal based data management for Lattice QCD data ACAT03, Tsukuba, work in collaboration with A.
K.Harrison BNL, 7 May 2002 Overview of GANGA – First steps towards GANGA – Outline of required functionality – Some existing technology – Conclusions.
Author - Title- Date - n° 1 Partner Logo EU DataGrid, Work Package 5 The Storage Element.
Author - Title- Date - n° 1 Partner Logo WP5 Summary Paris John Gordon WP5 6th March 2002.
Introduction to HDFS Prasanth Kothuri, CERN 2 What’s HDFS HDFS is a distributed file system that is fault tolerant, scalable and extremely easy to expand.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE middleware: gLite Data Management EGEE Tutorial 23rd APAN Meeting, Manila Jan.
First attempt for validating/testing Testbed 1 Globus and middleware services WP6 Meeting, December 2001 Flavia Donno, Marco Serra for IT and WPs.
Stephen Burke – Data Management - 3/9/02 Partner Logo Data Management Stephen Burke, PPARC/RAL Jeff Templon, NIKHEF.
University of Bristol 5th GridPP Collaboration Meeting 16/17 September, 2002Owen Maroney University of Bristol 1 Testbed Site –EDG 1.2 –LCFG GridPP Replica.
Data Management GridPP and EDG Gavin McCance University of Glasgow May 9, 2002
1 Andrea Sciabà CERN Critical Services and Monitoring - CMS Andrea Sciabà WLCG Service Reliability Workshop 26 – 30 November, 2007.
DGC Paris WP2 Summary of Discussions and Plans Peter Z. Kunszt And the WP2 team.
Jens G Jensen RAL, EDG WP5 Storage Element Overview DataGrid Project Conference Heidelberg, 26 Sep-01 Oct 2003.
Managing Data DIRAC Project. Outline  Data management components  Storage Elements  File Catalogs  DIRAC conventions for user data  Data operation.
Data Management The European DataGrid Project Team
Further aspects of EGEE middleware components INFN, Catania EGEE is funded by the European Union under contract IST
Data Management The European DataGrid Project Team
Author - Title- Date - n° 1 Partner Logo WP5 Status John Gordon Budapest September 2002.
Testing the HEPCAL use cases J.J. Blaising, F. Harris, Andrea Sciabà GAG Meeting April,
Stephen Burke – Sysman meeting - 22/4/2002 Partner Logo The Testbed – A User View Stephen Burke, PPARC/RAL.
Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES The AliEn File Catalogue Jamboree on Evolution of WLCG Data &
EGEE-II INFSO-RI Enabling Grids for E-sciencE Architecture of LHC File Catalog Valeria Ardizzone INFN Catania – EGEE-II NA3/NA4.
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) Algiers, EUMED/Epikh Application Porting Tutorial, 2010/07/04.
FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America LFC Server Installation and Configuration.
CERN IT Department CH-1211 Genève 23 Switzerland t CMS SAM Testing Andrea Sciabà Grid Deployment Board May 14, 2008.
Scuola Grid INFN, Trieste, 1-12 Dic Managing Confidential Data in the gLite Middleware – The Secure Storage.
Oxana Smirnova, Jakob Nielsen (Lund University/CERN)
Java API del Logical File Catalog (LFC)
INFNGRID Monitoring Group report
Moving the LHCb Monte Carlo production system to the GRID
The European DataGrid Project Team
gLite Data management system overview
Grid2Win: Porting of gLite middleware to Windows XP platform
The GENIUS portal Roberto Barbera University of Catania and INFN
The gLite API – Part II Giuseppe LA ROCCA ACGRID-II School
POOL File Catalog: Design & Status
Hands-On Session: Data Management
Data Management in Release 2
Stephen Burke, PPARC/RAL Jeff Templon, NIKHEF
Data Management Ouafa Bentaleb CERIST, Algeria
AMGA Web Interface Vincenzo Milazzo
Data services in gLite “s” gLite and LCG.
The EU DataGrid Data Management
Grid Data Replication Kurt Stockinger Scientific Data Management Group Lawrence Berkeley National Laboratory.
The GENIUS portal and the GILDA t-Infrastructure
Architecture of the gLite Data Management System
gLite Data and Metadata Management
The EU DataGrid Fabric Management Services
Presentation transcript:

1 CHEP 2000, Roberto Barbera Tests of data management services in EDG 1.2 ALICE Off-line Week,

2 Roberto Barbera Outline Data management services in EDG 1.2 GDMP 3.0 EDG RC Interface and EDG Replica Manager A useful user interface Live demo Summary and conclusions Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration ALICE Off-line Week,

3 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration ALICE Off-line Week, Data management services GDMP 3.0 (gdmp_*) Globus Replica Manager EDG Replica Catalogue Interface (edg_rc_*) EDG Replica Manager (edg-replica-manager-*) UI/GUI wrappers

4 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration Select a VO export EDG_VO=alice ; source /etc/profile.d/edg-apps.sh the variables $GDMP_CONFIG_FILE and $RC_CONFIG_FILE are correctly set Publish a file/directory gdmp_ping [-a] [-V ] –S [-r ] (Un)subscribe a SE gdmp_host_subscribe [-u] [-c] [-V ] –S -r Copy a local file to a SE globus-url-copy file:/// gsiftp:// / / / GDMP 3.0 (1) ALICE Off-line Week,

5 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration Register a file/directory on the local catalogue gdmp_register_local_file –R [–S ] [-V ] -p / / gdmp_register_local_file –R [–S ] [-V ] -d / / Publish the local catalogue on the RC gdmp_publish_catalogue –C [-V ] [-S ] Replicate a file between two SE’s gdmp_replicate_get –C [-V ] –S -r gdmp_replicate_put –C [-V ] –S -r Remove a file locally and from the RC gdmp_remove_local_file –p / [-V ] [-S ] GDMP 3.0 (2)

6 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration Pros: no need to transmit a proxy with a job can manage both single files and entire directories works in a client-server mode; commands can be issued from UI’s and WN’s and go directly to SE’s Cons: if one of the two services dies the connection breaks registration and publication are asynchronous operations in the next versions of the DataGrid m/w will be replaced by Reptor which will use the Replica Manager API’s Documentation: GDMP 3.0 (3) ALICE Off-line Week,

7 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration Add and register a local file on the RC edg-replica-manager-copyAndRegisterFile –l -s / / -d / / / [-e] Get the PFN(s) of a LFN edg_rc_getPhysicalFileNames –l Replicate a file between two SE’s edg-replica-manager-replicateFile –l -s / / / -d / / / [-e] Delete a file (both LFN and PFN) edg-replica-manager-deleteFile –l -s / / / EDG RC Interface and EDG Rep. Manager (1)

8 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration Pros: there is no client-server architecture and the commands are executed immediately all the needed commands are there Cons: the syntax is a bit ugly to remember (“-” in edg-replica-manager-* and “_” in edg_rc_*) the error messaging/trapping/handling is quite poor the current version can manage only single files and not entire directories Documentation: Guide.html Guide.html ALICE Off-line Week, EDG RC Interface and EDG Rep. Manager (2)

9 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration Developed by Andrea Sciabà from CMS on top of edg-replica- manager-* and edg_rc_* commands All commands can be found in one single TAR ball: /afs/infn.it/project/cms/grid/edg-utils.tar Set the default VO once and for all with the command: ln –s $RC_CONFIG_FILE $HOME/.rcrc The following commands are available: edg-ls edg-cd edg-pwd edg-l2p edg-cat edg-put edg-rm edg-copy (will be implemented soon) Documentation: edg- -h gives an usage help of A useful User Interface

10 Roberto Barbera Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration ALICE Off-line Week,

11 Roberto Barbera Conclusions and plans Dipartimento di Fisica dell’Università di Catania and INFN Catania - Italy ALICE Collaboration Compared to 1.1.4, data management in EDG version 1.2 has done a big step forward in a very short amount of time Things are not ideal: commands are still quite fragile (poor error trapping and handling) and the syntax is ugly but now all the basic functionality is there and one can really see the grid storage as an extension of the local file system Good UI/GUI are already available and really make the life of non-expert users much easier It’s now time to stress the EDG RC and measure its performances Next tests will now concern MSS interface in EDG ALICE Off-line Week,