Adapting SAM for CDF Gabriele Garzoglio Fermilab/CD/CCF/MAP CHEP 2003

Overview SAM for CDF: why ? Goals of the pilot project The path (and pitfalls...) to integration (summer 02) Current status and future vision

History End Pre-Pilot Project: UK starts an evaluation of the needs of CDF for distributed computing: the Grid and SAM April Pilot Project: adaptation of SAM for CDF August Pilot SAM Deployment complete: move toward deployment of SAM for the collaboration

Why considering SAM ? Collaborating institutions can provide local computing resources to process data. An example: estimated size of the datasets UK institutes wish to access PhysicsPhysics Trigger SetNo. EvtsSecondary Data Size (GB) B Lifetime B oscillations CP violation Central J/Psi Displaced Vertex 12 M 1,000 M 1, ,000 W/Z Higgs SUSY B Physics High Pt leptons Inclusive electrons Inclusive muons 2 M 50 M 14 M 200 5,000 1,400 SUSY Calibrations High Et photons58 M5,800 HiggsZ 0 ---> b bbar6 M600 SAM was/is being actively developed and integrated with Grid middleware (SAM-Grid)

What SAM Provides SAM is a Data Handling System (project started in 1997) used in production by DZero (see Lee Lueking’s talk) Main characteristics –data movement and caching –meta-data catalogue –bookkeeping of analysis projects –set of tools for users and administrators

Highlights of the mapping between DFC and SAM Problem: how do we map the CDF (DFC) and DZero (SAM) views of the data ? DFC: files organized in Datasets which contain Filesets SAM: provides virtual files + metadata parameters (datastreams, data tiers, applications, …) DFC has the concept of Books to implement group/user-specific metadata and resource management. The complete mapping between the DFC and SAM on CDF note 6169.

Goals of the pilot phase (by summer 2002) 1.Supporting 5 “remote” groups to do data analysis 2.Enabling access to datasets of interest: read access to secondary data + read/write to higher order data 3.Production quality availability of the system: key machines maintained 24x7 4.Controllable limited impact on the CDF Mass Storage System (Enstore)

Areas of work  Designing and implementing an architecture  Adding /Adjusting features to SAM  Design / Load CDF SAM Database  Enable CDF clients to access SAM  Installation / Configuration Group Coordination during development Support/Shifter Organization

Architecture this architecture was designed with the goals in mind. 2 MSS: –CDFen: 1-ary/2-ary data via DCache –STKen: higher order; 5TB tapes

Architecture 1 routing station for >2-ary data: –1TB disk cache –1GBs connectivity 1 FNAL analysis station: –dual 1GHz Pentium –160 GB Disk.

Architecture SAM services on a sun machine (like DZero) supporting development/ integration/ production DB. PC were the offline CDF Oracle machines.

Feature adjustment Integration of SAM with DCache: SAM transports files to local caches from the weakly authenticated ftp door of DCache Enable Enstore “discipline” module to limit access to SAM (see Don Petravick’s talk for more on FNAL MSS) Enable direct SAM station to station file transfer: implement better file routing in SAM

Database Use development/integration/production databases for schema evolution Periodically read data from the DFC and translate them to the SAM schema: a java based program, Predator, runs every 3 hours to keep SAM up-to- date Loading the db using the SAM interface served as proof of principle but turned out to be too slow

Integration with the CDF analysis framework: AC++ AC++ does not use exception handling SAM was developed for DZero, whose analysis framework support exceptions SAM communication is based on CORBA and the idl interfaces used exception To manage communication, AC++ I/O modules fork a CDF Project Protocol Converter (CPPC). CPPC is a finite state machine that communicates with SAM via CORBA and with AC++ via pipes CPPC was generalized to allow communication with multiple processes The UI required a dataset definition + project name

Installation and Configuration Version management resulted to be a sensitive issue DZero used to maintain the current cooperating product versions tagging them as “current” in the UPS/UPD repository To promote independency of software upgrades, CDF needed a different way of tagging consistent cooperating versions Cooperating versions were hardcoded in SAM installation script.

Current status Today, CDF uses SAM for physics at Oxford and Karlsruhe Currently Testing SAM on the FNAL CAF (see Frank Wuerthwein’s talk) Next step is a tighter integration of SAM with DCache.

Future vision: the SAM-Grid The integration of SAM with Grid middleware to enable Job handling and Information Monitoring (JIM) was demonstrated at SC2002 in November (see Igor Terekhov talk on JIM; Fedor Ratnikov’s talks on JIM and DCAF; Stefan Stonjek’s talk on SAM-Grid at sc2002) SAM was integrated with CDF DCAF and deployed at ~10 sites around the world The production version of SAM-Grid is going to be deployed for DZero in April, we look forward to CDF