A Test Framework for Movement of MC Data using SAM & SRM Interface Manoj K. Jha (INFN- Bologna) Antonio Cuomo, Donatella Lucchesi, Gabriele Compostella, Simone Pagan (INFN – Padova) Doug Benjamin (Duke University) Robert Illingworth (Fermilab) 8 th April, 2008 CDF Offline Meeting

2 Overview  Test Framework  Composition of files  Monitoring Parameters  Time Plot  Transfer Rate  Cumulative Data Plot  Conclusion & Next Step  Issues

Data Movement: Test Framework cdfsam15 FNAL SRM V2 srm://cmssrm.fnal.gov:8443 UCSD SRM V2 srm://t2data2.t2.ucsd.edu:8443 srmcp A process is running on “cdfsam15” which create dummy files of random sizes. The random number generator is being biased for creating files of larger sizes (> 1 GB). We are using WAN for transfer of files using srmcp. The no. of channels is 50 at UCSD SRM end.

Composition of Files TypeFile Size (fsize) Afsize <= 10 MB B10 MB < fsize <= 100 MB C100 MB < fsize <= 1GB D1GB < fsize <= 2GB E2GB < fsize <= 3GB Ffsize > 3GB 4 The dummy files generated in sizes from 1 MB to 4GB. Number in the pie chart indicates the number of files generated corresponding to that category from 3 rd to 8 th April, 2008.

Monitoring Parameters: Transfer Time 5  Initializations time of SRM is roughly of the order of 10 sec.

Monitoring Parameters: Transfer Rate 6  Transfer rate increases with increase in file size.  Maximum achievable transfer rate is around 60 MB/sec  User analysis jobs will fall in category B while MC jobs in C & D B D C

Monitoring Parameters: Transfer Time and Rate Comparison of transfer time for diff. categories of file. Comparison of transfer rate for diff. categories of file.

Monitoring Parameters: Cumulative Flow of Data/hr 8 File Size Time  Transfer rate of 25 GB/hr is achievable using WAN.  Transfer rate of 2 TB/hr is achievable using “FermiLight Path Circuits”. For more information, please visit

Monitoring Parameters: Cumulative flow of data/day 9  Around 600 GB/day of data can be transferred using WAN.  For data transfer of higher rate, we would have to consider other options like “FermiLight Path Circuits”.

Conclusion & Next Step  Conclusion:  SRM initialization time is of the order of 10 sec.  Date transfer rate increases with file size.  Maximum transfer rate achievable is around 60 MB/sec  Transfer rate of 25 GB/hr and 600GB/day is achievable using WAN.  Next Steps  Study the monitoring parameters for  Trans Atlantic movement of data.  Data movement between worker nodes and SRM  As a function of various number of requests arriving at SRM  Write a CDFNote explaining the implementation of framewrok within the CAF 10

Issues !  Authentication:  Presently, the user’s proxy is being generated on robotic head node. Users cannot take the proxy outside from the head node. What should be the mechanism for accessing the files on SRM, since users cannot access his/her proxy credentials ?  Addition of more SRM managed storage element sites will help us in speedy development of this work.  Compatibility of SAM with SRM version 2  Present version of SRM in SAM is 1  Latest version of SRM is 2. SRM version 2 is not backward compatible with SRM version 1.  Present version of SAM needs some development such that it would be compatible with SRM version 2 11

Acknowledgement ! We would like to thank Jon Bakken (from FNAL), Frank Wuerthwein and Abhishek Singh Rana (from UCSD) for providing resources on SRMs ! 12