1 Object Level Physics Data Replication in the Grid Koen Holtman Caltech/CMS ACAT’2000, Fermilab October 16-20, 2000

2 Introduction Object replication: Replicate (parts of) particular events (from a big store to a smaller store) Doing this in a convenient way is important to make tier 3-4 regional centre systems useful Got prototype object replication tool working now Uses Grid middleware (Globus), Objectivity Replicates objects from big data store to desktop machine Demo at SC’2000 Will do some integration into CMS physics analysis system (ORCA)

3 Data model Physics objects: read-only (versioning, not overwriting) Objects are organised as a sparse (SQL) Table Rows=events, event ID=`numbering domain + 64 bit integer' Columns=object types+versions, ID = a string (or URL) Software tools know how to resolve from row IDs+column IDs to the objects Can do this quickly in an iterator

4 Most basic use case Replicate a subset of objects to user desktop machine Do not ship objects but (database) files with object sets Works as follows…. Note: since 2 weeks ORCA also supports something like this

5 More complex use case Replicate some more objects This is where tool support (and strong catalog technology) really starts to become useful! Catalog uses per-file object indices sorted on the 64 bit number of the corresponding event Can do fast set operations

6 Use of Globus middleware Globus components currently used: Communication with server: GSI-authenticated TCP/IP connection (re-using code from GDMP) Shipping files: GSI authenticated FTP Will directly benefit from future improvements in FTP space Good experience using Globus components so far Intend to use of additional Globus components in future, when extending the prototype Object Replication Server Object Replication Client GSI authenticated connection GSI FTP

7 Some performance results Replicating 1900 objects of 100KB each With both client and server on my desktop Linux machine: 5.8 MB/s Replicating to another building: 0.72 MB/s (network bound)

8 Conclusion Current status: Prototype object replication tool is working Future plans: Integration with GDMP Integration with CMS ORCA Whether this will move beyond the prototype stage will depend on CMS production needs Prototype will function as input to architecture discussions in Grid efforts (files vs. objects) Prototype development will move from client- server to N-party scenario