Java API del Logical File Catalog (LFC) Valeria Ardizzone INFN Catania Martina Franca (TA), 16.11.2007

What is a file catalog File Catalog SE SE SE LFC Features: Mantaining mappings between LFN(s),GUID and SURL(s). Best performance and less security problem than old RLS. It supports transactions, roll-back, sessions m bulk queries. It is a unique catalog where LFN is the main key. Hierarchiacal name-space for LFNs. Symlink to main LFN. System metadata. User metadata (but only a single string entry). File Catalog SE SE gLite UI SE

The LFC (LCG File Catalog) It keeps track of the location of copies (replicas) of Grid files LFN acts as main key in the database. It has: Symbolic links to it (additional LFNs) Unique Identifier (GUID) System metadata Information on replicas One field of user metadata

lcg-cr –vo gilda –l lfn:/ …. LFC high level tool A file is considered to be a Grid file if it is both physically present in a SE and registered in the file catalog. In general high level tools like lcg utils will ensure consistency between files in the SEs and entries in the File Catalog. LFC MyData SE Data lcg-cr –vo gilda –l lfn:/ ….

LFC commands Summary of the LFC Catalog commands lfc-chmod Change access mode of the LFC file/directory lfc-chown Change owner and group of the LFC file-directory lfc-delcomment Delete the comment associated with the file/directory lfc-getacl Get file/directory access control lists lfc-ln Make a symbolic link to a file/directory lfc-ls List file/directory entries in a directory lfc-mkdir Create a directory lfc-rename Rename a file/directory lfc-rm Remove a file/directory lfc-setacl Set file/directory access control lists lfc-setcomment Add/replace a comment

LFC low level tool SE LFC Usage of low level tools, for both data transfer and catalog entries management could cause inconsistencies between SEs physical files and catalog entries; what would imply the corruption of GRID files. This is why the usage of low level tools is strongly discouraged unless really necessary. LFC MyData SE Data lcg_cp( ….) lfc_createg()

File naming convention The Grid Unique IDentifier (GUID), which identifies a file uniquely: guid:_unique string_ guid:93bd772a-b282-4332-a0c5-c79e99fc2e9c The Logical File Name (LFN) or User Alias, which can be used to refer to a file in the place of the GUID: lfn:_anything_you_want_ lfn:/grid/gilda/valeria/myfile.txt The Storage URL (SURL), also known as Physical File Name (PFN), which identifies a replica in a SE: sfn | srm://_SE_hostname_/_some_string_ sfn://_SE_hostname_ _SE_Accesspoint_ _VO_path_ _filename_ srm://aliserv6.ct.infn.it/dpm/ct.infn.it/home/gilda/generated/2006-09-24/….. The Transport URL (TURL), which is a valid URI with the necessary information to access a file in a SE : protocol://_some_string_ gsiftp://tbed0101.cern.ch/flatfiles/SE00/dteam/generated/2004-02-26/file3596e86f-

Java api code examples Several environment variables need to be set before to start: export LCG_GFAL_INFOSYS=bdii.gs.ba.infn.it:2170 export LCG_CATALOG_TYPE=lfc export LFC_HOST=lfc.gs.ba.infn.it This class is intended for use with gLite3.0 grid middleware and current version uses LFC API and GFAL API C libraries. The example are created using the LFC java API tool developed in the contest of the SEE-GRID2 project.

Example: Listing a catalogue directory(1) > cat repmngr.properties e.o, LFest Java API properties. This file's location needs to be in classpath. # # Name of Virtual Organisation VO = gilda # List of available Storage Elements separated by space character SEList = se-dpm1.gs.ba.infn.it se-dcache1.gs.ba.infn.it se-dpm2.gs.ba.infn.it se-dcache2.gs.ba.infn.it

Example: Listing a catalogue directory(2) LFCDataStorage is an implementation of DataStorageInterface which provides information and access to data storage resources of a grid. LFCDirectoryItem class provide methods for retrieveing directory information and data management. ... DataStorageInterface dsi = new LFCDataStorage(); DirectoryItem di = new LFCDirectoryItem(args[0],null,dsi);

Example: Listing a catalogue directory(3) Using ItemIterator class allows iteration of grid catalogue items, where a item can be any catalogue entry. ItemIterator iter = di.itemIterator(); while(iter.moreChildren()) { Item nextItem = iter.next(); System.out.println( type+ nextItem.getName()+"\t"+ (new LFCFileMode(nextItem.getFileMode()))+"\t"+ nextItem.getUID()+"\t"+ nextItem.getGID()+"\t"+ nextItem.getSize()+" bytes"); System.out.println( "User: "+nextItem.getUser()+"\n"+ "Group: "+nextItem.getGroup()+"\n"+ "Parent: "+nextItem.getParent().getPathName()+"\n"+ "Comment: "+nextItem.getComment()+"\n"); ...

Practicals Download the follow archive which contains the java source for your exercises: wget --no-check-certificate http://repository.gs.ba.infn.it/twiki/pub/GILDA/LFCJA VAAPI/LFC_JAVA_API.tar.gz Wiki page for the exercises: http://repository.gs.ba.infn.it/twiki/bin/view/GILDA/LFCJAVAAPI

LFC C API LFC cli and C API documentation http://grid-deployment.web.cern.ch/grid-deployment/documentation/LFC_DPM/lfc/html/ Example usage C API: lfc_listreplica - list replica entries for a given file #include <sys/types.h> #include “lfc_api.h" struct lfc_filereplica *lfc_listreplica (const char *path, const char *guid, int flags, lfc_list *listp) A C program listing the replicas fo a given sfn could look like: int flags; lfc_list list; struct lfc_filereplica *lp; flags = CNS_LIST_BEGIN; while ((lp = lfc_listreplica (sfn, NULL, flags, &list)) != NULL) { flags = CNS_LIST_CONTINUE; /* process the entry */ ..... } (void) lfc_listreplica (sfn, NULL, CNS_LIST_END, &list);

References GILDA -- Data Management quickstart https://repository.gs.ba.infn.it/twiki/bin/view/GILDA/DataManagement SEE-GRID2 LFC JAVA API http://wiki.egee-see.org/index.php/SEE- GRID_File_Management_Java_API LFC Documentation https://twiki.cern.ch/twiki/bin/view/LCG/LfcAdminGuide#LFC_CLI_an d_API EGEE Middleware Support https://twiki.cern.ch/twiki/bin/view/EGEE/EGEEMiddlewareSupport