11th September 2002Tim Adye1 BaBar Experience Tim Adye Rutherford Appleton Laboratory PPNCG Meeting Brighton 11 th September 2002

Tim Adye2 Talk Outline New (more distributed) BaBar computing model and its impact, especially on RAL User experience Transfers SLAC -> RAL Transfers UK -> SLAC

11th September 2002Tim Adye3 New Computing Model Goal is to spread computing load much more around the collaboration Simulation production is already highly distributed Small-scale analysis already performed at Universities (9 in UK) and Regional Centres (eg. RAL) Now have three new “Tier A” centres Lyon – Objectivity (database) analysis (since last year) RAL – Kanga (ROOT MicroDST) analysis (from May 02) Padova – Reprocessing (commissioning) RAL has relieved SLAC of all Kanga analysis Each site requires large data transfers from and to SLAC

11th September 2002Tim Adye4 BaBar CPU Usage at RAL Tier A

11th September 2002Tim Adye5 User Experience Now have users throughout US and Europe Interactive experience is generally excellent “ Connecting to RAL and working at RAL was very fast, as fast as at SLAC. ” – Uriel Nauenberg, University of Colorado at Boulder. AFS access between UK and SLAC is still slow SLAC -> RAL AFS, RAL+UK -> SLAC AFS Maybe an intrinsic property of AFS with slower RTT

11th September 2002Tim Adye6 Bulk transfers SLAC -> RAL Production at SLAC has come in bursts Reprocessing old data Kanga production is the final (and relatively simple) step in a long processing chain We have had no problem keeping up with steady state Sometimes delays of 1-2 weeks to catch up with a burst in production This is quite acceptable So far, >19 TB copied from SLAC and on disk at RAL 15 TB since January RAL is now the primary Kanga repository, so others in UK/Europe/US will copy from us So far modest: most << 1TB per site (10-20 sites)

11th September 2002Tim Adye7 Kanga Data transfers bbftp SLAC -> RAL

11th September 2002Tim Adye8 Transfer Rate per bbftp session (2-20 streams each)

11th September 2002Tim Adye9 Bandwidth Transfer rate of 5-25 Mbit/s is obviously much less than SLAC RAL 622 Mbit/s link Actually get up to 50 Mbit/s by using multiple sessions (on different servers) Probably several effects limit us Only run a few (1-5) simultaneous sessions More is cumbersome to manage Currently use only a couple of 100 Mbit/s servers at RAL Firewall problems with bbftp limit servers we can use Will soon have dedicated import/export Gbit servers bbftp doesn’t handle small files very efficiently Typical file sizes MB Will use bbcp, but requires a bug fix (on Andy’s list) Not a problem at the moment

11th September 2002Tim Adye10 Transfers UK -> SLAC UK now performs ~75% of total BaBar Simulation Production Mostly at University sites, though RAL is starting up Objectivity output files sent back to SLAC Size reduced by a factor of 8 after dropping intermediate files no longer required for most analysis Output then skimmed, converted, and re-exported to Tier A and C sites

140 TB Transferred To SLAC ! 1 Aug 01 1 Sep 02 Raw/sim not since Apr02 rate drop: x8 8 TB 4 TB 10 TB 20 TB By week By site Simulation Transfers to SLAC (all sites)

11th September 2002Tim Adye12 Transfer rate limited by local+SLAC infrastructure and number of bbftp streams (currently 3) Gaps in transfers show we are keeping up – CPU is main limit Full plots at Transfer rate UK Simulation Production A typical UK site

11th September 2002Tim Adye13 Summary New computing model is heavily reliant on network Especially UK to/from SLAC User experience is good but is there anything we can do about AFS? Bulk transfer from and to SLAC currently limited by local infrastructure Nevertheless, we are easily keeping up with SLAC and UK farm production