1. A Test Framework for Movement of MC Data using SAM & SRM Interface 2 nd March, 2008 CDF Offline Meeting

2. 2 Overview  Test Framework  Composition of files  Monitoring Parameters  Time Plot  Transfer Rate  Cumulative Data Plot  Query Map  Conclusion & Next Step  Wish List !

3. Data Movement: Test Framework Data from SAM, dccp fcdflnx1cdfsam15 FNAL SRM V2 srm://cmssrm.fnal.gov:8443 UCSD SRM V2 srm://t2data2.t2.ucsd.edu:8443 scp srmcp

4. Composition of Files 4 A SAM data file is assosciated with its meta data file.

5. Composition of Files: File Size 5 No. of Files Meta Data File Size in Bytes Data File Size in Mega Bytes File Size  Meta data files are of the order of few Kilo bytes.  Data files are the order of several Mega Bytes.

6. Monitoring Parameters: Transfer Time 6 No. of Files Meta Data Transfer Time in Sec Data Transfer Time in Sec Time Taken  Meta data files (few Kilo bytes) take around 14 seconds.  Data files takes (several Mega bytes) around 34 seconds.

7. Monitoring Parameters: Evolution of files with time 7 Time Meta Data File Size in Bytes Data File Size in Mega Bytes File Size  A dip in the right hand side plot (18:32 – 19:02) is being balanced by copying meta data files during this period.

8. Monitoring Parameters: Transfer Rate 8 No. of Files Meta Data Tr. Rate in Bytes/sec Data File Tr. Rate in MBs/sec Transfer Rate  Transfer rate for larger size files is much higher than smaller size files.

9. Monitoring Parameters: Cumulative Flow of Data 9 File Size Time  More than ~30 Giga Bytes/hr of data has been transferred.  More test is needed to confirm a robust transfer rate.

10. Monitoring Parameters: Query Map 10  Not easy to draw any conclusions from the above plot.  But, it will be helpful to display quality data for multiple sites versus time. Each box represents a fixed duration worth of transfers.

11. Conclusion & Next Step  Conclusion:  Larger the file size, higher the efficiency of the framework.  Next Steps  Different sizes of files will be used  For files of other than above different sizes, joining files with “tar” is one option. But, it is too much cpu consuming.  In the next offline meeting, we will show plots on monitoring parameters for trans Atlantic movement of data. 11 DataSetEach File Size (in MB) 1.atop48500 2.bhel0d737 3.b0dsdka850 4.ftop1b1050 5.bhmu0i1350

12. Wish List !  Addition of more SRM managed storage element sites for studying the performance of the framework.  Compatibility of SAM with SRM version 2  Present version of SRM in SAM is 1  Latest version of SRM is 2. SRM version 2 is not backward compatible with SRM version 1.  Present version of SAM needs some development such that it would be compatible with SRM version 2  A rough idea of the time frame ( under which the above issues will be resolved) will be helpful in speedy development of this work ! 12