Blueprint RTAG Status Torre Wenaus, BNL/CERN SC2 Meeting July 5, 2002

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 2 Blueprint RTAG Status  Current RTAG page:  Linked from the applications page  Activity so far:  Meetings Wed June 12, Fri June 14  ‘Opening statement’ type talks and discussions  Between these meetings: a long dialogue on architecture and design  Particularly on the role and architecture of ROOT, following on from a proposal made by Rene on June 14  All day meeting Wed July 3  Response/reaction to Rene’s proposal and the dialogue  Some resolved issues and decisions  Domain decomposition  On to developing the architectural blueprint

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 3 The main software areas GRID middleware RDBMS run/file catalogs Object persistencyv 2-d, 3-d graphics GUI Toolkits Math Libs Statistics Detector Geometry Event Generators Dectector Simulation Object persistency Histograming Fitting Event Models Folders Event Display Ntuple analysis Interpreters DAQ Online System services ETC... Rene Brun

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 4 Framework with Object bus Object bus: Object dictionary Data Interface (I/O): Functional Interface User Applicat ions Higher level framewor k services Higher level framewor k services Experiment framework Higher level framewor k services Higher level framewor k services Higher level framework services User Applications Rene Brun

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 5 Python as a Software Bus  Python is an object-oriented scripting language commonly used in a wide variety of domains  But, it could also be seen as framework where you can plug easily “extension modules” in binary form implemented using other languages.  Very easy and natural to interface to C++ classes (C++ API)  Python should only be the “glue” between modules developed in C++ or other languages  The interface (API) for Python extension modules is quite simple and at the same time flexible Pere Mato

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 6 GUI Python as a Software Bus Python math shell gaudipython DatabaseEDG API GUI Very rich set of Python standard modules Several GUI toolkits XML Very rich set specialized generic modules Gaudi Framework rootpython Root Classes PVSS JPE Java Classes LHC modules Gateways to other frameworks Pere Mato

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 7 Architecture: Devil is in the Details  Build independent components: Avoid  Dependencies among components at the same level  Gratuitous and exaggerated re-use One hammer does not fit all screws  global states (even cout)  Exposure of internal relationships (a->b()->c(i)->d(“b”))  assumptions on higher level behavior (lent pointers)  Interfaces that force your environment on user code  Balance inheritance (white box) vs composition (black box)  Distinguish Framework API, Client API and User API These are Architectural issues NOT coding guidelines I do not mind of “#define int float” in your.cc, I mind if in a.h Vincenzo Innocente

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 8 Toward a Project Praxis  Define the global software model  Granularity, role and nature of “Modules” Physical vs logical modules (yesterday at CMS plenary M.Livny concluded asking for staticly linked, check- pointable executables…)  Reuse model of sub-components  Which “glues” have to be used, where and how  Define THE set of basic components  Agree on Metrics to measure modularity  Not only Frameworks, also applications based on them Vincenzo Innocente

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 9 Rene’s Proposal The existing set of ROOT libs is the starting core of the LCG software. Because the system is already widely distributed and used, it guarantees the initial acceptance to a wide community. We invite architects and key developers to review the current organisation of libraries and to propose an evolution if it proves necessary, keeping in mind that this may affect thousands of existing users.

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 10 LCG Software Architecture and ROOT  Architectural principles that CMS, LHCb, ATLAS want to follow were developed in presentations by Pere and Vincenzo and in a long dialogue  Exchanges indicated sharply the different views on design between the ROOT/ALICE team and CMS/LHCb/ATLAS  A clear conclusion, agreed in Wed meeting…  It is not going to work for ROOT to be taken as the starting core of LCG software  The design differences are too great, and the ROOT team is not going to redesign ROOT  What we know is going to happen is that ROOT will be used heavily by the LCG software and all four experiments  And has been taken directly as the core foundation for one

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 11 How will ROOT be used?  While design discussions should continue, we cannot rely on resolving them in order to progress…  We accepted that  We are not going to converge anytime soon on a common approach to architecture and design  What we must take up and resolve is working out an architecturally acceptable way to make use of a big grey (not black) ROOT box  With accommodation on both sides  Changes to ROOT library organization?  More substantive changes to improve factorization?  Dealing constructively with ‘linking in the kitchen sink’ issues  Agreed metrics meaningful for usability, maintability, modularity etc.

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 12 How will ROOT be used?  LCG software developed as a ROOT user will draw on a great ROOT strength: users are listened to very carefully!  Much more carefully than software designers proposing major design changes!!  The ROOT team has been very responsive to needs for new and extended functionality coming from the persistency effort  Drawing on ROOT in a user-provider relationship matches much better the reality of the ROOT development model of a very small number of ‘empowered’ developers  The ROOT development team is small and they like it that way  While ROOT will be used at the core of much LCG software for the foreseeable future, we agree there needs to be a line with ROOT proper on one side and ‘LCG software’ on the other.

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 13 LCG Software Architecture  With the ‘ROOT relationship’ resolved to be not architecture/design wars to be (endlessly) fought, but rather designing how ROOT will be used in a distinct LCG architecture, we can (and we began to) move on to developing the LCG architectural blueprint

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 14 Views I expressed, to no vocal objection  I think that within the LCG software architecture ‘bare ROOT’ should be an integral, trivially accessible part of the architecture  e.g., most obviously, affording the interactive user the ability to easily move between a Python prompt (see coming slide) and a ROOT prompt with access to their object model from ROOT; possible now given the foreign class support in ROOT  As Rene often says: Users vote with their feet  The design and evolution of LCG software should be well attuned to the user, as is ROOT  LCG software we develop that does not use ‘bare ROOT’, or ROOT at all, will live or die on its merits  We should build it so that it lives on its merits, not on life support through architecturally walling off ROOT in some way  Plenty of people have bet against ROOT in the past. They have all been wrong. LCG software should not make this mistake

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 15 Interactivity and software buses  Points of agreement  A common object dictionary will be a key component at the foundation of the architecture  A Python-based interactive environment and ‘software bus’ will be part of the architecture  ROOT and the ROOTCINT interpreter will be trivially accessible from the Python environment and vice versa  The architecture will provide for access to data objects from programmatic and interactive environments throughout the system

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 16 Python in the architecture  Rene requested expression of his views…  Python should not be the primary interactive interface.  Personally I agree; I think that ROOTCINT and Python should essentially be on a level playing field  Will lead to confusion and waste of time by users  Actually I think users will insist on easy availability of their preferred interactive environment, as I expressed on the ‘views’ slide a few slides ago  Python should be available, to take advantage of services for which a Python interface is already available  In the same way, we must encourage cross-language communication  Rene in favour of a public presentation of a complete and coherent framework  I think the outcome of this RTAG should be presented clearly in a public form, eg. in an applications area meeting (with the anticipation of larger than average attendance!)

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 17 Notes on Domain Decomposition - NB user and software provider perspectives on domain decomposition will be different - basic layer - software bus / object dictionary - persistency/data management - object persistency - relational cataloging - event-specific data management - conditions-specific data management - event processing framework - event models - event generation **defer - detector simulation **defer - geometry and materials - persistent representation - transient geometry model - reconstruction - reconstruction - scripting (Python) interpreter tool as software bus - GUI **defer - NB requirements coming from support for picking - ability to put any kind of object into a list of primitives - 2D & 3D graphics **defer - analysis tools **defer - ntuple analysis - math libraries, statistical analysis - histogramming, fitting - grid middleware **defer - meta-services (Pere will elucidate; cf. Gaudi) - utility and foundation libraries - system services. OS interaction - packaging and distribution

Torre Wenaus, BNL/CERN SC2 meeting, July Slide 18 Blueprint RTAG plans  Another series of 3 meetings next week  Into the specific details of techniques and patterns via proposals presented as concrete examples  Refining the domain decomposition; walking through it and identifying candidate implementations; interdependencies  The following week:  Begin assembling the report  Too many absences for meetings  Then more meetings, and on and on…  Will invite some speakers (Tony Johnson,  There is some skepticism within the RTAG that we will make early Sep (we will not make early Aug), but this is our objective  We recognize the importance of getting this done in good time  It has to be done right  I think the RTAG is off to a good start and has made some important, clarifying decisions.