BaBar and the GRID Tim Adye CLRC PP GRID Team Meeting 3rd May 2000

3 May 2000Tim Adye2 Status of Objectivity (and it’s replacement) Short-term GRID-like requirements Network tuning for bulk data transfer Meta-data Medium-term Distributed MC Production Longer-term Distributed analysis

3 May 2000Tim Adye3 Status of Objectivity Objectivity is used for BaBar Reconstruction and MC Production. MC Production at SLAC and a few external sites (also private MC generation elsewhere) RAL production on csflinux under development Objectivity used for analysis at SLAC and IN2P3 New “KanGA” format now replaces Objectivity for analysis only Converted from Objectivity data at SLAC Just the “microDST” quantities KanGA based on ROOT IO same usage / analysis framework as Objy Smaller, faster, cheaper, and much easier to export For microDST analysis, Objectivity gives little added functionality

3 May 2000Tim Adye4 Status at RAL No longer exporting Objectivity data to RAL Still have 0.9 TB on disk and in Datastore IN2P3 now have 2-3 TB Exports to Rome stopped just after us Still need Objectivity for local development, MC Production, private MC, etc. 335 GB (and growing rapidly) of KanGA data

3 May 2000Tim Adye5 Network Tuning Currently copy GB a night KanGA (ROOT) files “Plain binary” files, but complex directory structure Currently no meta-data exported SLAC -> RAL -> “tier 2” sites IC / Manchester / Bristol / Edinburgh / RHBNC / Rome Naïve copy tools (ftp, scp, rsync) don’t work well for bulk transfer Currently need to use multiple ( rsync ) processes copying in parallel Should be improvable with tuning Or maybe some new tool, eg. GLOBUS ftp? Need authentication SLAC doesn’t accept FTP BaBar-UK developer: TJA

3 May 2000Tim Adye6 Meta-data KanGA bookkeeping uses Oracle at SLAC Also works with mySQL Allows complex searches eg. “all on-peak real data files processed with release 8.6.2c” Doesn’t know about KanGA files mirrored elsewhere Not accessible from outside SLAC Have to run query on SLAC machine, copy result to UK, and filter out files not here. Solved with Oracle 8? Need to develop data management system that knows about file locations Must be able to work with SLAC system and offer similar functionality Shouldn’t rely on everyone buying / installing Oracle (LDAP?) BaBar-UK developers: Alessandra Forti, TJA

3 May 2000Tim Adye7 Distributed MC Production RAL production on csflinux Farm under development Uses prebuilt binaries Dedicated Objectivity Federation Shipped back to SLAC in Objectivity format BaBar-UK developer: Steve O’Neale From JIF, 6-9 BaBar sites will acquire Linux MC “small-holdings” Total CPUs (32 dual CPU/site) distributed geographically fractions of people for support / development at each site

3 May 2000Tim Adye8 Distributed MC Production Developments These will soon be real and together make a significant resource Already have central expertise and facilities for OO DB output at RAL prototype dedicated connections to Manchester and IC Still must be built into a “proto-GRID” This is an ideal opportunity to develop something real and useful Eg. tools for submitting, monitoring, and managing jobs on this distributed system with output direct to OODMBS, possibly centralised at RAL on the timescale of 1 year

3 May 2000Tim Adye9 Long-Term developments Currently data mirrored RAL -> Universities “Skim” selections made according to local interest Could benefit from GRID-like analysis environment Particularly if RAL CPU becomes heavily saturated