1 November 17, 2005 FTS integration at CMS Stefano Belforte - INFN-Trieste CMS Computing Integration Coordinator CMS/LCG integration Task Force leader (where CMS/FTS integration happen) Pretending to be someone else

2 November 17, 2005 Apologies Really I should not be giving this talk Daniele Bonacorsi in CNAF/Bologna has been working on FTS integration in CMS ➨ Daniele had a last minute schedule override last week ➨ he could not make it to this workshop, nor prepare slides FTS integration happens via CMS PhEDEx tools ➨ Other main PhEDEx developers are either in vacation or UK and could not make it to come with short notice So you have to listen to someone who does not really know what he is talking about. Put together this talk from reports by Daniele to the CMS/LCG integration task force

3 November 17, 2005 PhEDEx CMS developed its own data-movement tool ➨ PhEDEx PhEDEx is a high level tool, incorporating data transfer and data placement for datasets and data routing policies. PhEDEx interfaces with several possible file transfer tools, providing sophisticated error handling and retry mechanism on top of those ➨ Globus-Url-Copy ➨ srmcp Integration of FTS has been always possible, but never was a priority with respect to making SC3 work. Data transfer was supposed to be the tool to exercise sites capability to move and serve data, not the goal.

4 November 17, 2005 Adding FTS Daniele Bonacorsi in CNAF-Bologna started integrating FTS into PhEDEx as file transfer agent in mid-October, after having had this in his wish list since August. Possibility to work earlier on FTS limited by competition with Service Challenge 3 activities, that have required almost full time attention by Daniele and other PhEDEx developers In October Daniele effectively removed CNAF from CMS-SC3 operations in order to dedicate effort to FTS integration in PhEDEx Foreseen operation mode is to hand list-of-files to PhEDEx for movement

5 November 17, 2005 FTS for CMS 1 FTS client installed at CNAF (CMS-Tier1) FTS client successfully used to move data to CMS-Tier2’s ➨ CNAF to Bari (limited by 100MBit/sec network) ➨ CNAF to Legnaro (LNL) on 1Gbit/sec network Learning how to use FTS went smoothly, few minor issues found and fixed by FTS developers (Tomcat restart e.g.) Thin glue layer developed around FTS for usage in PhEDEx ➨ Some “inconvenience” in usage was found and reported to FTS developers ➨ Good feedback, but most of desired functionalities will be only in FTS new version 1.4

6 November 17, 2005 FTS for CMS 2 Daniele completed the necessary ground work and started using an FTS-capable version of PhEDEx last week Then he had to disappear due to exams in Italy, same reason prevents him from being here today CMS still plans to add FTS into PhEDEx in next weeks Single most noteworthy lacking features: ➨ checksum (or at list file size) check after transfer Other desiderata: ➨ FTS stays in “waiting” even if file already exists at destination and SRMput fails ➨ Would like to have full timing info for each transfer: request, start, end

7 November 17, 2005 File Placement CMS Computing TDR specifies the CMS data management strategy ➨ Natural data organization Files (order of GigaBytes) Datasets (tens to hundreds of TeraBytes) An intermediate container introduced for Data Management convenience ➨ Fileblock (order of Terabyte) CMS will have a catalog of Fileblocks locations (replicas) ➨ DLS = Data Location Service List of files in a Fileblock to be retrieved from a global Dataset Bookkeeping Systems (DBS) PhEDEx will manage Fileblocks movements (using FTS where convenient) and register copies in DLS

8 November 17, 2005 Conclusion CMS will let its regional centers (Tier1, Tier2) to pick the data transfer tool. We will not mandate it. Will support srmcp, FTS, possibly Globus-Url-Copy. Expect this tool to be the boundary of responsibility between CMS and the sites/LCG. This wish is not so realistic since transfers involve two parties so may not be able to point the finger to one site. Work is in progress for FTS, and proceeds quickly (pending SC3 effort and other external disturbances). No problem foreseen at present for FTS usage on all sites who ask so.