LHCbDirac and Core Software

LHCbDirac and Core SW Core Software workshop, PhC2 Running Gaudi Applications on the Grid m Application deployment o CVMFS on all Tier1s (but Gridka) and a few Tier2s o Pushing for it (GDB) o Other sites: mostly NFS or GPFS P Installation done with install_project in dedicated jobs o If SW not available, local installation in the job P Support for dual installation (shared / local) mandatory o Other use case: running ROOT in LHCbDirac P Need to set up environment 1. Tell user to “SetupProject LHCbDirac ROOT” 2. Set it up internally: currently broken by dual installation changes o LHCbDirac is also deployed on the Grid and CVMFS P Not used (yet) by jobs, but by users m Application support o Can we be more generic, i.e. support any Gaudi Application? o Support random application? o Support for just ROOT: SetupProject ROOT?

LHCbDirac and Core SW Extra packages, AppConfig m First sight, all OK now o Versions fixed in step manager m Dynamic SQLite installation (see yesterday’s talk from MCl) o Transparent? m Additional options (set up by LHCbDirac) o Input dataset o Output dataset o Special options P Setting time stamps, defining MsgSvc format o Should LHCbDirac know about them? o Should AppConfig contain templates with just placeholders P E.g. file name replaced by actual name by Dirac, e.g. d to be replaced with _0012.bhadron.dst P Mechanical operation rather than “knowledge” Core Software workshop, PhC3

LHCbDirac and Core SW Job finalization m LHCbDirac needs information from the application o Success / failure ! o Bookkeeping report P Nb of events processed P Nb of events per output stream, GUID (also on catalog) P Memory used, CPU time? o Production system P Files successfully processed P Failed files d Event number of crash? m Most (all) information now in XML summary reports o XML summary browsing implemented (Mario) P Needs thorough testing in jobs (already tested with many cases) o Get rid (!) of AnalyseLogFile… m Any specific requirements for MC simulation? o Should info be added to BK? Core Software workshop, PhC4

LHCbDirac and Core SW Bookkeeping m Is there enough information? m Can it be used for producing reports (e.g. performance benchmark)? m Accessing metadata from within a job: o Use case: determine the DDDB to be used for MC o Would BK sustain the load? o Where to query: jobs, BK GUI, ganga? P What if more than one in the dataset? m User job bookkeeping? o Is it worth investing? o Definition of requirements Core Software workshop, PhC5

LHCbDirac and Core SW Step manager and production requests m Is the interface adequate? o Reco / Stripping: production manager mostly o MC: WG representatives, Gloria m Is it a limitation that the Condition should be in the BK for creating a request? o E.g. prepare a reconstruction production before data is available? o Can this be avoided? m Is the request progress monitoring OK? m Currently when enough events are in BK, one kills jobs and deletes files o Is this useful? Should we just let go and flush? P Disk extra usage vs wasted CPU Core Software workshop, PhC6

LHCbDirac and Core SW Tools for users m Not directly for Core Software but… education! m How to get a proper PFN? o Too many users of PFN:/castor/cern.ch/… P Triggers a disk 2 disk copy to a smallish pool (overload) P Should be root://castorlhcb.cern.ch/ o Currently available: P PFNs from BK GUI P CLI tools: to be improved to get directly the PFN at a site d Resurrect genXMLCatalog and include in LHCbDirac d Documentation and Education! m How to replicate datasets? o To local disk o To shared SE o Tools exist but are not streamlined nor documented Core Software workshop, PhC7

LHCbDirac and Core SW Software tools m Synergy between LHCbDirac and Core Software o Eclipse o SVN vs git P Why are branches / merging a nightmare for LHCbDirac and not for Core SW? o Savannah vs JIRA o Benefit from the CS and Applications’ experience P Is LHCbDirac that different? P Is packaging a bonus or a nuisance? Monolithic vs packages P Is getpack useful? How to couple it with Eclipse? P Setup the environment from Eclipse working area m Can one use SetupProject to get LHCbDirac on WNs? o LHCb private pilot jobs o Any particular requirement? m For LHCbDirac services and agents: o How to get a controlled Grid environment without doing GridEnv? Core Software workshop, PhC8