Daniele Spiga PerugiaCMS Italia 14 Feb ’07 Napoli1 CRAB status and next evolution Daniele Spiga University & INFN Perugia On behalf of CRAB Team

Daniele Spiga PerugiaCMS Italia 14 Feb ’07 Napoli2 OutLine Introduction Today status CrabServer evolution Motivations Implementation First results Schedule Conclusions & plans

Daniele Spiga PerugiaCMS Italia 14 Feb ’07 Napoli3 CRAB CMS remote analysis builder: Allow users to run analysis jobs on the grid The aim of crab is to open the grid to the masses: CMS user in addition to the development of his own analysis code must only specify few things to CRAB then it take care to: Interface with user working environment (SCRAM-CMSSW) Interface with the Data Management (search where requests data are) Interact with the grid middleware performing: job submission job status monitoring on demand Manage the produced output (copy back, into Storage Element, castor) Prepare all the stuff to run on remote WN (local environment configuration) Other functionalities: kill, resubmit, postMortem….

Daniele Spiga PerugiaCMS Italia 14 Feb ’07 Napoli4 Today Status F.Fanzago, S.Lacaprara Release in production : CRAB_1_4_2 Fully working with CMSSW, DataManagment (dbs/dls), gLite(bulk submission) Release in preparation: CRAB_1_4_3  support submission from scl4 to scl* CRAB_1_5_0  news: support BOSS API usage improve: grid interaction (submission, check status, getoutput) Work In Progess: Integration with DBS2 Publication of CRAB produced data to local scope DBS/DLS G.Codispolti

Daniele Spiga PerugiaCMS Italia 14 Feb ’07 Napoli5 CrabServer Evolution Actual Job flow CRAB working on UI CRABSERVER Crab Server Job Flow Automate as much as possible resubmission, error handling, output retrieval…. More scalability

Daniele Spiga PerugiaCMS Italia 14 Feb ’07 Napoli6 Implementation strategy Client side: same CRAB interface, same command line and same installation procedure Server side Adopt ProdAgent general architecture, adding all needed stuff re-use as much as possible all what was done during these years!!

Daniele Spiga PerugiaCMS Italia 14 Feb ’07 Napoli7 Client User must just properly configure CRAB and use it as usual crab –create same as crab standalone crab –submit submit crab project to the server and NOT to the grid crab –status show a summary (prepared by the server) of the status of the whole task crab -getoutput retrieve from the server the output crab.cfg : The client ship to the server the user proxy too New configuration keys

Daniele Spiga PerugiaCMS Italia 14 Feb ’07 Napoli8 Server The server adopt the general PA architecture: Components are implemented as independent agents, communication is performed through an asynchronous and persistent message service Plus a grid-Ftp server DropBoxGuardian Check the dropBox for new CRAB user task to be managed. ProxyTarAssociator Associate the task to the right user proxy CrabWorker submit jobs to the grid (lcg/osg) in the CRAB style JobTraking Track every single job (via BOSS) ErrorHandler Perform a basic error handler JobSubmitter Resubmit single job if needed TaskTracking Keep general informations about all tasks under execution. Notification Notify the user when the task is ready via D.Spiga F.Farina M.Cinquilli W.Bacchi A.Dorigo C.Kavka M.Corvo New components re-used components

Daniele Spiga PerugiaCMS Italia 14 Feb ’07 Napoli9 First results: First client version ready /afs/cern.ch/cms/ccs/wm/scripts/Crab/] –submit the task to the server –retrieve the output –(task status check on going) –Ship user proxy to the server in secure way Server installed and set up at cern: crabdev1.cern.ch –Submit tasks to the grid for the user –Track the jobs and resubmit if needed –Retrieve the produced output from the grid –Prepare the finished task –notify the user Two days ago the server components were integrated and started working together with success!!

Daniele Spiga PerugiaCMS Italia 14 Feb ’07 Napoli10 Next few days Schedule Commit code on CVS discussion with CMS management on going Finalize the first prototype for O(4) users (beta tester) continue tests to check all works fine Setup a CNAF replicate the cnaf (discussion with D.Bonacorsi) End of February

Daniele Spiga PerugiaCMS Italia 14 Feb ’07 Napoli11 Medium term Plans Wait for the feedback from the beta tester Add support for the other needed functionalities not yet implemented in the basic workflow: 1.active interaction with DM (DBS/DLS publication) 2.status shown also via web page available on server 3.killing of jobs/task by user 4. server disk space management test on server functionalities and scalability June ‘07

Daniele Spiga PerugiaCMS Italia 14 Feb ’07 Napoli12 Summary & Conclusions Ready for sl4 transition Under development the support for publication on DBS/DLS of user output Used also with real data (TAC data: see Nicola’s talk) Users support, still same problem: which CMS user support role? Just ask/remember us to answer the users? CRAB stand-Alone:

Daniele Spiga PerugiaCMS Italia 14 Feb ’07 Napoli13 Summary & Conclusions Basic workflow is ready First server for beta tester almost ready Developed code to be commited on cvs Preliminary twiki page available: A lot of work to have a Server-Client functionalities available to users for june07!!! CRAB Server: