1. LHCb-INFN Computing for the years 2003-2005 CSN1, Perugia, November 11, 2002 Domenico Galli, Bologna

2. Status of LHCb-INFN Computing, 2 Domenico Galli Outline LHCb Constraints: Experiment milestones in 2003-2005 which require computing power. Software: SICbMC/Geant-3, Gaudi, Gauss/Geant-4, Giga, Brunel, DaVinci. Grid integration: Ganga. Computing Model: Tier-1, (Tier-2), Tier-3, Tier-4 functionalities. Last Bologna/CNAF Farm improvement: Analysis facility.

3. Status of LHCb-INFN Computing, 3 Domenico Galli LHCb Constraints

4. Status of LHCb-INFN Computing, 4 Domenico Galli LHCb Constraints LHCb TDR: September 2003. L0/L1 trigger TDR: September 2003. L23 trigger TDR: Q4/2003. Statistics increase of a factor of 2. Computing TDR: Q4/2004. Gauss/Geant-4 large scale testing and tuning: After LHCb design and TDR delivery.

5. Status of LHCb-INFN Computing, 5 Domenico Galli MC Production Plan (2002-2003) Nov 22: End software improvement. Nov 22 – Dec 16: Brunel final commissioning. Dec 16 – Jan 13: Pre-production (~3 Mevents). Jan 8 – Jan 27: Data quality tests. Jan 22 – Feb 4: Prepare production version. Feb 4 – May 4: Final MC production (~15 Mevents). Summer: Possible reprocessing. Sep 9: TDR submission (LHCb, Trigger).

6. Status of LHCb-INFN Computing, 6 Domenico Galli Software

7. Status of LHCb-INFN Computing, 7 Domenico Galli Software Status Present LHCb production software: Monte Carlo: SICbMC: Geant-3/FORTRAN Reconstruction: Brunel: OO/Gaudi/C++ Analysis: DaVinci: OO/Gaudi/C++ Present LHCb development software: Monte Carlo: Gauss: OO/Geant-4/Gaudi/C++ A first version of the whole simulation chain using Gauss/Geant-4 is now working. Starting to study the response of the detectors in detail.

8. Status of LHCb-INFN Computing, 8 Domenico Galli GAUDI: the Framework LHCb Collaboration is convinced of the importance of the architecture since long time. Sep 1998 – project started, GAUDI team assembled. Brunel (reconstruction) & DaVinci (analysis) use GAUDI. “framework is an artefact that guarantees the architecture is respected” to be used in all the LHCb event data processing applications including: high level trigger, simulation, reconstruction, analysis. Build high quality components and maximize reuse. The proposed LCG architecture is not very different from the GAUDI architecture (see RTAG architectural “blueprint”). the component model, role of interfaces, plug-in, basic framework services, interactive services, etc. are very similar.

9. Status of LHCb-INFN Computing, 9 Domenico Galli The GAUDI Framework Converter Algorithm Event Data Service Persistency Service Data Files Algorithm Transient Event Store Detec. Data Service Persistency Service Data Files Transient Detector Store Message Service JobOptions Service Particle Prop. Service Other Services Histogram Service Persistency Service Data Files Transient Histogram Store Application Manager Converter Event Selector

10. Status of LHCb-INFN Computing, 10 Domenico Galli GAUDI Architecture: Design Criteria Framework contains real code. Implementations of class methods, not only interfaces. Clear separation between data-type and actor-type (algorithms) objects. Three basic types of data: event, detector, statistics. Clear separation between persistent and transient data. Computation-centric architectural style. focus is on the transformation of objects that are interesting to the system. User code encapsulated in few specific places: algorithms and converters. All components with well defined interfaces and as generic as possible.

11. Status of LHCb-INFN Computing, 11 Domenico Galli GAUDI: Collaboration with Other Experiments ATLAS also contributing to the development of GAUDI Open-Source style, experiment independent web and release area. Other experiments are also using GAUDI: HARP, GLAST, OPERA Encouragement to put more quality into the product. Better testing in different environments (platforms, domains,…). Shared long-term maintenance.

12. Status of LHCb-INFN Computing, 12 Domenico Galli GAUDI: Changes to Comply with LCG The proposed LCG architecture is not very different from the GAUDI architecture. No big problem in adopting the concrete LCG software when available Unavoidable code changes will be required but the end-user code is well isolated. The “end-user” physicist should not see any difference. The Algorithm code stays unchanged. Most probable changes in the component configuration (JobOptions).

13. Status of LHCb-INFN Computing, 13 Domenico Galli GAUDI: Changes to Comply with LCG (III) LCG Pool LCG DDDD LCG Pool AIDA Other LCG services LCG CTS LCG CTS HepPDT LCG CTS

14. Status of LHCb-INFN Computing, 14 Domenico Galli Gauss: Transition to Geant 4 Geometry Input: XML database. A version available for all the detectors in LHCb. All detectors are in the new framework (GAUSS: Geant-4 simulation). Gaudi-Geant4 interface (GiGa: GEANT4 Interface for Gaudi Applications). Input events: From Pythia or other similar programs through the HEPMC interface into GEANT4. Starting to study the response of the detectors in detail. Need large scale testing and tuning (after LHCb design and TDR delivery). MC transition to Geant4/C++ in production foreseen for 2004.

15. Status of LHCb-INFN Computing, 15 Domenico Galli Grid Integration

16. Status of LHCb-INFN Computing, 16 Domenico Galli Ganga: Gaudi/Athena and Grid Alliance ATLAS and LHCb develop applications within a common framework: Gaudi/Athena. Both collaborations aim to exploit potential of Grid for large-scale, data-intensive distributed computing. Simplify management of analysis and production jobs for end-user physicists by developing tool for accessing Grid services with built-in knowledge of how Gaudi/Athena works.

17. Status of LHCb-INFN Computing, 17 Domenico Galli Ganga: Gaudi/Athena and Grid Alliance GAUDI Program GANGA GUI JobOptions Algorithms Collective & Resource Grid Services Histograms Monitoring Results

18. Status of LHCb-INFN Computing, 18 Domenico Galli General requirements for GANGA The user will interact with a single application integrating all stages of job life-time. He will be able to restore his or her workspace (list of files, tools state, jobs in preparation) at the beginning of each session. The GUI will be similar to work with, for both the Grid and a local network. It will be similar to the mailing system (e.g. Outlook Express), with jobs taking role the mails. The goal is to perform configuring/running Gaudi job as easy as sending a mail. Interface access not only from the computer with the Grid UI program running, but also from a remote “thin” client. The aim is to have a first release of Ganga for the end of the year.

19. Status of LHCb-INFN Computing, 19 Domenico Galli Ganga Prototyping Embedded Python interpreter Tree of user jobs Job options for selected job

20. Status of LHCb-INFN Computing, 20 Domenico Galli Computing Model

21. Status of LHCb-INFN Computing, 21 Domenico Galli Tier-2 Computer Centers Network bandwidth increase and grid software integration make the resources location transparent for the end-users (the physicists performing analysis jobs). LHCb-Italy plans to store computing resources in the places in which is available manpower for system design, management and administration (not for physical analysis). Need of Tier-2 Computer Centers not foreseen for LHCb-Italy (at least at present).

22. Status of LHCb-INFN Computing, 22 Domenico Galli Tier-3 Computer Centers Not thought for Monte Carlo production but can be used as a booster for peak needs. 2 Functionalities: As buffer-cache for the analysis data between Tier-1 (AOD storage) and Tier-4 (user desktop/interactive analysis). As parallel interactive analysis facility (using JAS/RMI or ROOT/PROOF, like PIAF facility at CERN since 1993). The size in CPU-power and disk storage need to be determined on the basis of the simulation of the data flow between Tier-1 and Tier-4. Preliminary test on Firenze Farm. On-the-field test in high level trigger studies foreseen.

23. Status of LHCb-INFN Computing, 23 Domenico Galli Tier-3 as Buffer-Cache Tier-4Tier-3Tier-1 AOD Catalog AOD retrieve Request load Look-upregister Data present on local storage Data not present on local storage

24. Status of LHCb-INFN Computing, 24 Domenico Galli ROOT/PROOF (Parallel Root Facility) Traditional Master/Slave approach root proof slave proof slave proof slave node1 node2 node3 node4 proof slave proof master Cint ROOT C++ command line interface is usable by C++ gurus, but not by most of physics.

25. Status of LHCb-INFN Computing, 25 Domenico Galli JAS/RMI (Remote Method Invocation) Server calls registry to associate a name with a remote object. Client looks up the remote object by its name in the server’s registry and then invokes a method on it. client Web server server registry server registry RMI URL RMI URL

26. Status of LHCb-INFN Computing, 26 Domenico Galli Possible JavaSpaces implementation Based on Linda coordination language (Yale University). Programming = Computation + Coordination Uncoupling senders and receivers. Intrinsic adaptive load balancing (on heterogeneous resources too). Intrinsic robustness.

27. Status of LHCb-INFN Computing, 27 Domenico Galli Last Bologna/CNAF Farm improvement

28. Status of LHCb-INFN Computing, 28 Domenico Galli Bologna/CNAF LHCb Farm Architecture Manager Node NAS 1 TB RAID5 Analysis Station 1 Analysis Station 12 PVFS Striped Disk Array (1 TB) Analysis Station 13 Analysis Station 14 Analysis Station 15 Gateway Fast Ethernet Switch Public VLAN Private VLAN Uplink MC Prod. node 1 MC Prod. node 40

29. Status of LHCb-INFN Computing, 29 Domenico Galli High Performance I/O System I/O parallelization system successfully tested and put in production PVFS (Parallel Virtual File System). Striping of data files among local disks of several I/O servers (ION). Scalable System (maximum throughput ~ 100 Mbit/s x number of IONs) DaVinci 1 DaVinci 2 DaVinci 80 ION 1 ION 2 ION 12 MGR I/O servers Meta data server Network Ntuple

30. Status of LHCb-INFN Computing, 30 Domenico Galli Benchmark Results on B  Analysis 80 DaVinci processes reading from PVFS (2000 events per job) 2288 files (500 OODST events each) x 120 MB 75 MB out of 120 MB are actually retrieved by the algorithm 167 GB read from the network and processed in 4600 s

31. Status of LHCb-INFN Computing, 31 Domenico Galli Farm Monitor Tool Interactive. Based on java applet (presentation logic)/java servlet (data selection logic) technology and Jakarta Tomcat. Transfers data (not graphics). Completely configurable using XML. Developed together with CNAF.

32. Status of LHCb-INFN Computing, 32 Domenico Galli Extra slides

33. Status of LHCb-INFN Computing, 33 Domenico Galli Software Structure Basic Framework Foundation Libraries Simulation Framework Reconstruction Framework Visualization Framework Applications... Optional Libraries Other Frameworks Applications built on top of frameworks and implementing the required physics algorithms. Various specialized frameworks: visualization, persistency, interactivity, simulation, etc. A series of basic libraries widely used: STL, CLHEP, etc. Main framework

34. Status of LHCb-INFN Computing, 34 Domenico Galli GAUDI: Changes to Comply with LCG (II) LHCb model of describing the Event Model with GOD (Gaudi Object Description) XML files will continue to work. We can generate the code to populate the LCG Object dictionary, which then will be used by POOL to provide object persistency (based on ROOT I/O). The “end-user” physicist should not see any difference. The Algorithm code stays unchanged. Most probable changes in the component configuration (JobOptions).

35. Status of LHCb-INFN Computing, 35 Domenico Galli Gauss application Geant4 JobOpts Geant4 (GiGa) Digi Alg JobOpts Digit MCDigit Geometry HepMC MCParticle MCVertex MCHit Generator Detector Simulation Cnv GiGa Pythia etc Int.face

36. Status of LHCb-INFN Computing, 36 Domenico Galli GiGa structure Geant4 Algorithm Event Service Persistency Service Data Files Algorithm Transient Event Store Detec. Service Persistency Service Data Files Transient Detector Store Application Manager GiGa Service G4 Hits G4 Kine GiGaKine Conversion Service Cnv Converter Other Services Action GiGaHits Conversion Service G4 Geom GiGaGeom Conversion Service

37. Status of LHCb-INFN Computing, 37 Domenico Galli Production Components Edit Prod.Mgr Work flow Editor Production Editor Instantiate Workflow Job request Status updates Production Center Production data Scripts Production DB Production Server Bookkeeping info Bookkeeping Updates

38. Status of LHCb-INFN Computing, 38 Domenico Galli Current Production Scheme n Job Batch farm bbftp Storage Castor Bookkeeping info Production center Submit job Log files Histo files Data files Transfer data BK files Production Worker scripts Local Prod. manager Job scripts Central Prod. manager

39. Status of LHCb-INFN Computing, 39 Domenico Galli Production Agent n Job Batch farm Production Agent bbftp Storage Castor Job request Job status update Bookkeeping info Production center Job status update Submit job Log files Histo files Data files Transfer data BK files Production Worker scripts Check data

40. Status of LHCb-INFN Computing, 40 Domenico Galli Agent Advantages Actively asks for the work to be done: no idle “forgotten” resources; Runs locally at a production center: no problems with write access to local file system; Automates most of the routine production tasks: software updates; submit jobs; transfer data; update bookkeeping;

41. Status of LHCb-INFN Computing, 41 Domenico Galli Required Functionality (I) Job preparation and configuration Resource booking Job submission User can choose between Grid and local resource management system Job monitoring and control GUI for the resource browsing Virtual Organisation active services Computing Elements Storage Elements Query existing files in the Grid GUI for data management tools e.g., Dataset registration to the Grid (used by Production Manager) Copy file from a Computing Element to a Storage Element Replication of files

42. Status of LHCb-INFN Computing, 42 Domenico Galli Required Functionality (II) Job preparation and configuration: Determine job requirements in terms of software products needed: executables, libraries, databases, etc. Get access to the Job Configurations DB: Common configurations could be stored in a database and retrieved using high-level commands User would have possibility of modifying settings and storing personalised configurations in his/her own area Perform job configuration: select algorithms to run and set properties specify input event data, requested output, etc Provide graphical tools for editing default Job Options files. Contact the Gaudi Bookkeeping Database and the Grid Replica Catalogue to obtain the list of Logical File Names (LFNs) from high-level physics selection criteria. Automated generation of JDL scripts for job submission.

43. Status of LHCb-INFN Computing, 43 Domenico Galli Design of GANGA Two ways of implementation have been discussed: Based on one of the general-purpose grid portals (not tied to a single application/framework): Alice Environment (AliEn). Grid Enabled Web eNvironment for Site-Independent User Job Submission (GENIUS) Grid access portal for physics applications (Grappa). Simulation for LHCb and its Integrated Control Environment (SLICE). Based on the concept of Python bus (P. Mato): use different modules whichever are required to provide full functionality of the interface use Python to glue this modules, i.e., allow interaction and communication between them A new development using Python software bus is better suited to the aims of ATLAS and LHCb.

44. Status of LHCb-INFN Computing, 44 Domenico Galli Ganga Prototyping (Current State) GUI is created using wxPython extension module. Access to the Gaudi Job Configuration DB is implemented with the xmlrpclib module. User can browse and create Job Options files using this DB. Serialization of objects (user jobs) is implemented with the Python pickle module. Python interpreter is embedded into the GUI and allows user to configure interface from the command line GRID stuff is under development at the moment and is oriented on EDG testbed 1.2.

45. Status of LHCb-INFN Computing, 45 Domenico Galli General Requirements for the Architecture Simplicity of implementation Portability (platform independence) Rich functionality Modularity, which allows for Extensibility Should provide interactivity

46. Status of LHCb-INFN Computing, 46 Domenico Galli Python Bus Design Server Bookkeeping DB Production DB EDG UI PYTHON SW BUS XML RPC server XML RPC module GANGA Core Module OS Module Athena\ GAUDI GaudiPython PythonROOT PYTHON SW BUS GUI Job Configuration DB Remote user (client) Local Job DB LAN/WAN GRID LRMS

47. Status of LHCb-INFN Computing, 47 Domenico Galli Ganga Prototyping: Towards the First Release The aim is to have a first release of Ganga for the end of the year. GANGA will be able to handle the configuration, submission (to LSF) and monitoring of a single Gaudi/Athena application. The GUI will be similar to the mailing system (e.g. Outlook Express), with jobs taking role the mails. The goal is to perform configuring/running Gaudi job as easy as sending a mail. The first release will work (at least) with Atlfast and DaVinci.

48. Status of LHCb-INFN Computing, 48 Domenico Galli Data Organization (GAUDI) Event Raw RecPhy VeloCalo TracksHits Event Cand RAWESDAOD versions Event MyTrk Phy Private

49. Status of LHCb-INFN Computing, 49 Domenico Galli Gaudi Model to Access Event Data Gaudi Bookkeeping DB Dataset Event 1 Event 2 … Event 3 Dataset Event 1 Event 2 … Event 3 File Event 1 Event 2 … Event N Files RAW2-1/1/2008 RAW3-22/9/2007 RAW4-2/2/2008 … Dataset Event 1 Event 2 … Event 3 Dataset Event 1 Event 2 … Event 3 Event tagcollctn Tag 15 0.3 Tag 22 1.2 … Tag M8 3.1 Collection Set B->ππCandidates (Phy) B-> J/Ψ(μ + μ - ) Candidates … EventTagColl table DataSets table DataSet (file) EventTag collection

50. Status of LHCb-INFN Computing, 50 Domenico Galli Architectural Styles General categorization of systems [1]: user-centricfocus on the direct visualization and manipulation of the objects that define a certain domain data-centricfocus upon preserving the integrity of the persistent objects in a system computation-centricfocus is on the transformation of objects that are interesting to the system [1] G. Booch, “Object Solutions”, Addison-Wesley 1996