Applications Area Preparation of Phase II Planning Pere Mato 22 on March 2005

23/03/052 Project Focus  Deliver the common physics applications software  Organized to ensure focus on real experiment needs –Experiment-driven requirements and monitoring –Architects in management and execution –Open information flow and decision making –Participation of experiment developers –Frequent releases enabling iterative feedback  Success defined by experiment validation –Integration, evaluation, successful deployment

23/03/053 Applications Area Organization AA Manager AliceAtlasCMSLHCb Architects Forum Application Area Meeting PEBLHCC SC2 External Collaborations Geant4ROOT Workplans Quartery Reports Reviews Resources LCG AA Projects PROJECT A EGEE Chairs Decisions WP1WP2 PROJECT B WP1WP2 PROJECT D WP1 WP2 WP3...

23/03/054 Current AA Projects  SPI – Software process infrastructure (A. Aimar) –Software and development services: external libraries, savannah, software distribution, support for build, test, QA, etc.  SEAL – Core Libraries and Services (P. Mato) –Foundation class libraries, math libraries, framework services, object dictionaries, python services, etc.  POOL – Object Persistency Framework (D. Duellmann) –Storage manager, file catalogs, event collections, relational access layer, conditions database, etc.  PI - Physicist Interface (V. Innocente) –Analysis services (AIDA), Analysis environment (CINT, Python)  SIMU - Simulation project (G. Cosmo) –Simulation framework, physics validation studies, MC event generators, participation in Geant4, Fluka.

23/03/055 AA Organization  Application Area Meetings (AAM) –Informal forum of exchange of information between the AA projects and experiments, etc. »Status reports, results, new ideas, evaluations, new requirements, general discussions, etc. –Encourage presentations from the projects and experiments –Every two weeks on 17:30 –Each meeting should have a “theme” defined well in advance

23/03/056 AA Organization (2)  Architects Forum Meetings –Formal decision and action taking meeting –Consists of the experiment architects, AA projects leaders, computing coordinators with an standing invitation and other invited participants –Experiments participate directly in the planning, management, and architectural and technical direction of AA activities –Public minutes

23/03/057 AA Organization (3)  Work Plans –AA Projects must prepare yearly work plans –Opportunity to re-think strategic decisions, change of direction, introduce new work packages –Approved by PEB  Quarterly reports –To monitor progress of the projects –Scrutinized by SC2 committee and generation of feedback  Reviews –Internal AA reviews and LHCC reviews

23/03/058 ROOT Collaboration  User-Provider relationship with LCG AA –Defined in the Blueprint RTAG  Close and effective collaborative effort between POOL and ROOT teams  AA internal review recommended an evolution towards a closer cooperation  Planned the convergence in a number of areas –Mathematical libraries, dictionaries, POOL reference support, etc. –Currently in these areas most work is being done on the LCG side; objectives are not reflected in ROOT plans and work commitments

23/03/059 Programme of work Preparation  Process initiated some weeks ago  Had a round of discussions with PH management, project leaders, experiment architects, experiment representatives, etc.  Presentations to the experiments –CMS last week, ATLAS April 11th  The goal is to have a final plan with most details to be presented and reviewed at the Second AA internal review (March 30 th – April 1 st )  PEB approval sometime in April

23/03/0510 Core Data Management Software Organization SimulationDistributed AnalysisSoftware Support and Infra- structure ReflexMath I/O PyROOT Zip Geometry L.A HistogramsPhysics Util OS binding Fitters Viewers GUIBuild Delivery of Geant4 VMC EvtGen(s) FC RAL(imp) Ganga Proof Physics Validation CondDB PlugMgr MCDB GDML DDD GSL Cintex gccxml PyReflex pythonCINTBoostQt Trees Collections Manage Servers Development tools External Libraries Savannah QA/testing Code Management FlukaPythiaCastorOracleGLite DBTrees StorageSvc

23/03/0511 Changes in AA Projects  The proposed changes the Applications Area are: –SEAL and ROOT projects merge –Some redefinition of SPI role –Some adaptations of POOL required –PI libraries absorbed by client projects –SIMULATION project basically unchanged

23/03/0512 SEAL + ROOT project merge  Both SEAL and ROOT projects have a big overlap –The objectives are very similar –Avoid duplication by construction  Single TEAM to provide the core and framework software  Why does it look very attractive? –Optimization of resources –Better “coherency” vis-à-vis our clients (experiments) –ROOT activity fully integrated in the LCG organization –Ease long term maintenance and evolution of a single product set

23/03/0513 What that means in practice?  Single team lead by Rene Brun  Combined program of work, single deliverable –Initially the union of what is available in SEAL+ROOT –Continuation of the convergence work in Dictionary, MathLibs, etc. –Towards a single set of functionalities in a time scale of 1-2 years  User-level compatibility MUST be maintained –Adiabatic transition for the LHC collaborations  Software evolution decided by the new project team –The “WHAT” is agreed with the experiments –The “HOW” is mostly left to the developers with input from experiment core people –Distributed responsibility (sub-projects)  Open team/project nature –Encourage external participation

23/03/0514 Known difficulties  Cultural merge –Team members with different backgrounds –Different ways of developing software –Need to establish a “common” culture  Compromises  Client merge –Possible conflicts between LHC requirements and other ROOT users requirements –Client adaptability to changes –Platforms support, licensing issues, etc.  Study case by case  How to achieve a common set of functionalities –Detailed plans should be developed by the new Team

23/03/0515 Current ROOT structure & libs

23/03/0516 Dictionary MathLibs Scripting Foundation SealUtil Framework SEAL packages and dependencies rootboostzlibpcreuuidpythongccxml SealBase SealKernel SealServices PluginManager Reflection ReflectionBuilder SealZip SealIOTools SealSTL SealCLHEP SealROOT clhep PyLCGDict PyReflex Minuit PyBus FML MathCore GSL Cintex

23/03/0517 SEAL + ROOT Migration  Adiabatic changes towards experiments –Experiments need to see libraries they use currently will evolve from current usage today towards a unique set  Details be planned in the Programme of Work –Will be extra tasks in order to complete migration newROOT Libraries SEAL Libraries ROOT Libraries SEAL Libraries ROOT Libraries 1 deliverable but some duplication still 2 deliverables 1 deliverable and no duplication time now ~August 2005 ~January 2006

23/03/0518 SEAL+ROOT: Foreseen Actions  Prepare a number of action plans to propose/discuss with the experiments  Foundation libraries –Workout a proposal to merge SEAL/Foundations with ROOT base  Dictionary –Mini workshop proposed in (May 2->6) to discuss CINT/Reflex/Cintex  I/O system –Discussions with POOL about RootStorage simplification, RAL, Object-Relational storage, TTreeSQL  Mathematical libraries –Preparing a strategy with TMath, GSL, Fitting, MathCore, … and future of CLHEP  Others

23/03/0519 Rene’s proposed project structure Ilka Antcheva Maarten Ballinjtin MIT 80% Bertrand Bellenot ALCAN 50% Bruno Belbute 100% Marek Biskup Rene Brun Philippe Canal FNAL Olivier Couet Christophe Delaere BE 10% Valeri Fine BNL 10% Markus Frank LHCb 50% Gerri Ganis Andrei Gheata ALICE 80% Mihaela Gheata unpaid 50% Masa Goto JP ??% Kris Gulbrandsen MIT 80% Ivana Hrivnocova ORSAY 10% Guenter Kickinger 100% Anna Kreshuk Wim Lavrijsen LBL 50% Sergei Linev GSI 60% Jose Lo 75% Pere Mato Richard Maunder Lorenzo Moneta Axel Naumann FNAL Eddy Offermann RENTEC 50% Valeriy Onuchin Timur Pocheptov JINR 80% Fons Rademakers Stefan Roiser Andras Zsenei +xrootd BaBar  BASE  BASE: System & CVS & DOC & Releases & Test suite & newsgroups Fons, Philippe, Pere, Bertrand, Ilka, Axel, Jose, Rene  DICTINT  DICTINT: CINT, Dictionary, Interpreters Masa, Philippe, Markus, Stefan, Wim  IO  IO: Basic I/O & Trees Markus, Philippe  PROOF  PROOF: PROOF & xrootd & GRIDs Fons, Andreas, Maarten, Derek, Gerri, Marek, Guenter+ BaBar  MATH  MATH: Maths,Histograms,Functions, Linear Alg Lorenzo, Andras, Anna,Eddy  GUI Ilka, Valeriy, Bertrand, Valeri, Fons  GRAPHICS  GRAPHICS: 2-D & 3-D graphics & Geometry Olivier, Richard, Andrei, Mihaela, Timur, Bruno

23/03/0520 Example1: MATH work package  Responsibility for this work package: –Basic Mathematical functions »TMath and SEAL MathCore –Functions and Fitting »Parameteric function classes (TF1,...) »Minuit, Fumili, Linear and Robust fitters, quadratic prog. –Random Numbers –Linear Algebra –Physics Vector  Also, but not considered now: –Histograms –Statistics (confidence level ) –Neural Net, multivariate analysis, etc..

23/03/0521 SEAL, CLHEP and ROOT Math Libraries

23/03/0522 Proposed new Math Structure

23/03/0523 Example2: DICTINT work package  Goal is to have a single API and set of data structures for I/O and interpreters (CINT and Python)  Hope to converge on a C++ data structures taking advantage of Reflex and the current redesign of CINT by Masa.  A very important workshop has been scheduled for May 2->7 to discuss the integration of Reflex and CINT.

23/03/0524 Dictionaries : situation today X.h X.xml XDictlcg.cxx REFLEX DS CINT DS rootcint lcgdict gccxml XDictcint.cxx CINT API REFLEX API ROOT Root meta C++ CINT cintex

23/03/0525 Dictionaries : situation in the future X.h Reflex/Cint DS new rootcint Gccxml2reflex/cint XDictcint.cxx CINT/Reflex API ROOT Root meta C++ CINT Python

23/03/0526 SEAL+ROOT: Status  Rene and the new ROOT+SEALteam working hard in the preparation of the programme of work  Many discussions ongoing  More details during the AA internal review

23/03/0527 SPI: Redefinition of its role  A number of “services” of common interest will continue to be run –Savannah, external libraries, etc.  Helping projects and LHC experiments to provide/maintain the software development infrastructure –CVS servers, build server, etc.  Direct participation in the software development projects –Librarians, release managers, documentation, toolsmiths, QA

23/03/0528 POOL: Some adaptations required  Domain of expertise in data persistency, data management, deployment in the Grid and (relational) databases in general  No major changes in the structure are proposed  Two differentiated parts –POOL as a product –COOL (conditions database)  Proposed to move the ROOT storage manager implementation to new SEAL+ROOT project  Study the collections and their relations with ROOT trees  Started discussions about the Relation Access Layer (RAL) organization

23/03/0529 PI: absorbed by client projects  The proposal is that the project is discontinued as such  Make an inventory of existing libraries and study their usage by experiments –If not used then abandon library –If used by a single experiment (or framework) move the library in question to the experiment (or framework) –Incorporate remaining parts to SEAL+ROOT project

23/03/0530 SIMU: Basically unchanged  Domain of expertise in event generators and detector simulation  No changes in the structure are proposed  Current subprojects –Simulation Framework –Geant4 –Fluka –Physics Validation –Generator Services  Added new subproject Garfield  Encourage to (re)use the core software and software development infrastructure –Interactivity, persistency, analysis, etc.

23/03/0531 Staffing Requirements Estimates SPI SEAL+ROOT POOL PI Simulation Total The total staffing levels estimated as required, and being planned:

23/03/0532 Staffing Sources CERN base21.2 LCG special contributions17.7 Staff at CERN Experiments Total

23/03/0533 Summary  Started to plan second phase of Applications Area  The major proposed change for this new phase is the merge of ROOT and SEAL projects  Adjustments proposed in SPI, POOL  No major changes foreseen in SIMU  Detailed work plans are being prepared –Hard deadline is the AA internal review (next week)  This is the current status of things. Further discussions with experiments are going to be needed to polish many details