Coherent Software Framework a Proposal LCG meeting CERN- 14 June René Brun ftp://root.cern.ch/root/blueprint.ppt Blueprint RTAGs

Way 1: the component model PROs: In principle easy to add or replace a component because of weak coupling B C D E G F H I In reality, you simply postpone the integration problem if the number of components is > N eg N> 10 L J K J’ ... LCG 14 June Rene Brun Blue print RTAG

The main software areas DAQ Online GRID middleware Event Models Folders ETC... Event Generators RDBMS run/file catalogs Object persistencyv Object persistency Event Display Dectector Simulation System services Histograming Fitting 2-d, 3-d graphics Ntuple analysis Detector Geometry Math Libs Statistics GUI Toolkits Interpreters LCG 14 June Rene Brun Blue print RTAG

Any box connected to many boxes DAQ Online GRID middleware Event Models Folders Event Generators RDBMS run/file catalogs Object persistencyv Object persistency Event Display Dectector Simulation System services Histograming Fitting 2-d, 3-d graphics Ntuple analysis Detector Geometry Math Libs Statistics GUI Toolkits Interpreters LCG 14 June Rene Brun Blue print RTAG

Any box connected to many boxes DAQ Online GRID middleware Event Models Folders Event Generators RDBMS run/file catalogs Object persistencyv Object persistency Event Display Dectector Simulation System services Histograming Fitting 2-d, 3-d graphics Ntuple analysis Detector Geometry Math Libs Statistics GUI Toolkits Interpreters LCG 14 June Rene Brun Blue print RTAG

Any box connected to many boxes DAQ Online GRID middleware Event Models Folders Event Generators RDBMS run/file catalogs Object persistencyv Object persistency Event Display Dectector Simulation System services Histograming Fitting 2-d, 3-d graphics Ntuple analysis Detector Geometry Math Libs Statistics GUI Toolkits Interpreters LCG 14 June Rene Brun Blue print RTAG

Way 2: Framework with Object bus User Applications Higher level framework services Experiment framework Higher level framework services Higher level framework services Higher level framework services Higher level framework services User Applications Object bus: Object dictionary Data Interface (I/O): Functional Interface LCG 14 June Rene Brun Blue print RTAG

LCG 14 June Rene Brun Blue print RTAG

Evidence It takes time to build solid foundations more than 6 years for Geant3, PAW, ROOT The process must be validated by users in different experiments in different configurations In the long term, users prefer general purpose frameworks to experiment-specific frameworks because users work in 3, 4 experiments during the life time of any framework People vote with their feet LCG 14 June Rene Brun Blue print RTAG

ROOT libs structure LCG 14 June Rene Brun Blue print RTAG pcnotebrun) #[1006] cd $ROOTSYS/lib (pcnotebrun) #[1007] ls -l total 26860 1246870 Jun 10 19:58 libCint.so 5468790 Jun 13 18:26 libCore.so 428183 Jun 13 18:30 libEG.so 156149 Jun 13 18:30 libEGPythia.so 164398 Jun 13 18:30 libEGPythia6.so 163824 Jun 13 18:30 libEGVenus.so 320933 Jun 13 18:30 libGX11.so 186870 Jun 13 18:30 libGX11TTF.so 995291 Jun 13 18:28 libGpad.so 1508147 Jun 13 18:28 libGraf.so 1031451 Jun 13 18:28 libGraf3d.so 3505642 Jun 13 18:29 libGui.so 1605028 Jun 13 18:30 libHbook.so 1840708 Jun 13 18:27 libHist.so 332928 Jun 13 18:29 libHistPainter.so 114938 Jun 13 18:29 libHtml.so 170068 Jun 13 18:30 libMC.so 461357 Jun 13 18:27 libMatrix.so 326767 Jun 13 18:29 libMinuit.so 270753 Jun 13 18:30 libMySQL.so 21917 Jun 12 19:52 libNew.so 90902 Jun 13 18:30 libPgSQL.so 345287 Jun 13 18:29 libPhysics.so 199557 Jun 13 18:29 libPostscript.so 500237 Jun 13 18:29 libProof.so 148509 Jun 13 18:30 libRFIO.so 2015746 Jun 12 20:05 libRGL.so 181086 Jun 13 18:29 libRint.so 35410 Jun 12 20:07 libSRPAuth.so 1123930 Jun 13 18:30 libTable.so 317808 Jun 13 18:30 libThread.so 1059588 Jun 13 18:27 libTree.so 369594 Jun 13 18:29 libTreePlayer.so 416115 Jun 13 18:29 libTreeViewer.so 158887 Jun 13 18:30 libX3d.so LCG 14 June Rene Brun Blue print RTAG

Several actions are proposed in the following slides Proposal The existing set of ROOT libs is the starting core of the LCG software. Because the system is already widely distributed and used, it guarantees the initial acceptance to a wide community. We invite architects and key developers to review the current organisation of libraries and to propose an evolution if it proves necessary, keeping in mind that this may affect thousands of existing users. Several actions are proposed in the following slides (non exhaustive list) LCG 14 June Rene Brun Blue print RTAG

Review of libCore components Because libCore is the essential library, it is important to understand the rationale behind its current organisation. We want to keep this core as small as possible. It will not be possible to agree on many other things if we cannot find an agreement on this fundamental component. LCG 14 June Rene Brun Blue print RTAG

{ { { Object Persistency Continue current developments in ROOT I/O Foreign classes Support for STL Improved Trees TLongRefs Interface to Catalogs Large files > 2 Gbytes Implement in TStreamerInfo in interpreted mode what is currently generated by Rootcint for STL containers Collaboration with FNAL and BNL DAQ Online GRID middleware Event Models Folders { Event Generators RDBMS run/file catalogs Object persistencyv Object persistency { Event Display Dectector Simulation System services Collaboration with LCG Histograming Fitting 2-d, 3-d graphics { Ntuple analysis Detector Geometry Collaboration with BNL Math Libs Statistics GUI Toolkits Interpreters LCG 14 June Rene Brun Blue print RTAG

Object Persistency Propose a restricted discussion on the interface to the dictionary following the initial work by Craig. Discussions and prototyping on access to CINT dictionary (feed/retrieve info) from external sources (eg XML/IDL, or other languages). Idem for a restricted group discussing, prototyping LongRefs. Would like to develop a variant of TStreamerInfo class generating XML for low volume exchange of objects. Implementation of the RDBMS layer is an independent project (Dirk) LCG 14 June Rene Brun Blue print RTAG

GRIDS Run/file catalogs Interface with Grid middleware Interface with Alien (see Predrag) PROOF (GRID oriented) Interfaces with RDBMS Oracle, MySQL, Postgres, etc TSQL, TSQLResult Discussions with GriPhyn DAQ Online GRID middleware Event Models Folders Event Generators RDBMS run/file catalogs Object persistencyv Object persistencyv Event Display Dectector Simulation System services Histograming Fitting 2-d, 3-d graphics Expecting close relationship with LCG Ntuple analysis Detector Geometry Math Libs Statistics GUI Toolkits Interpreters LCG 14 June Rene Brun Blue print RTAG

DAQ Online Multi-Threading support Shared memory Sockets/Monitors client/server Network classes System interface/Signals Interpreter Histograming Event Display Browsers/Inspectors Persistency DAQ Online GRID middleware Event Models Folders Event Generators RDBMS run/file catalogs Object persistencyv Object persistencyv Electronic logbook at FNAL well integrated with ROOT Event Display Dectector Simulation System services Histograming Fitting JavaRoot interface possibly interesting 2-d, 3-d graphics Ntuple analysis Detector Geometry Math Libs Statistics GUI Toolkits Interpreters LCG 14 June Rene Brun Blue print RTAG

Object Dictionary Follow the new C++ standard proposals in the area of instrospection, RTTI. Too much emphasys so far on the transient class dictionary. The real difficulty is the dictionary support for automatic schema evolution (persistent views and relationship with current transient classes). We did not discuss enough this essential area. Remove as much as possible automatic generated code in favour of dynamic interpretation. LCG 14 June Rene Brun Blue print RTAG

CINT, Python, Java, C# CINT: Smooth transition from interpreted code to compiled code dynamically unlinked/linked : root > .x script.C (interpreted) root > .x script.C++ (compiled) Facilitate automatic interfaces to Python and Java Current implementations are slow Root dictionary could be exploited to improve run time. DAQ Online GRID middleware Event Models Folders Event Generators RDBMS run/file catalogs Object persistencyv Object persistencyv Event Display Dectector Simulation System services Histograming Fitting 2-d, 3-d graphics Ntuple analysis Detector Geometry Math Libs Statistics GUI Toolkits Interpreters LCG 14 June Rene Brun Blue print RTAG

Interpreters Propose CINT as the main line because the same script can be interpreted or/and compiled. CINT matches very well with the GUI signal/slots. Interpred code can easily be called from compiled code and vice-versa. Propose to extend Pere’s Python interface and support it. Pere’s example with Excel is a good example of use of Python. Interactive analysis with loops filling histograms is not realist. Propose to consolidate the JavaRoot interface from Subir Sarkar. I hope that Tony and Julius will be interested. Investigate an interface with C# LCG 14 June Rene Brun Blue print RTAG

Event models: Folders/Tasks Some convergence with Gaudi to be exploited Whiteboard communication Tasks = Algorithms Folders <--> Trees to be developed Class 1 Class 2 Class 3 Class 4 Class 5 Class 6 Class 7 Class 8 Det Task Event Models Folders/tasks LCG 14 June Rene Brun Blue print RTAG

Event Models Encourage discussions on the transient event store organisation and tools to facilitate the export/import to persistent containers like the ROOT trees. This is in my mind an important item: role of collection classes, whiteboard communication style. How to work (transient/persistent view) with external class libs (eg G4) collections. LCG 14 June Rene Brun Blue print RTAG

GUI Toolkits Consolidate the TVirtualX interface Complete TVirtualX/Qt implementation TVirtualX/Win32-FOX (free/fast on Windows) Export script from a running GUI Build GUI from a given script GUI editor/builder The ROOT event loop has proven to work with all known graphics systems: X11, Xt, Motif, Qt, Open Inventor, etc DAQ Online GRID middleware Event Models Folders Event Generators RDBMS run/file catalogs Object persistencyv Object persistencyv Event Display Dectector Simulation System services Histograming Fitting 2-d, 3-d graphics TGX11 Ntuple analysis TQt Detector Geometry TVirtualX Math Libs Statistics TGWin32 GUI Toolkits TGWin32++ Interpreters LCG 14 June Rene Brun Blue print RTAG

GUIs An important area. Take advantage of the existing expertise and experience with different solutions. Take a long term view and not just what is fashionable today. Non blocking GUIs. GUIs in distributed applications, including Web. We would like to discuss the Carrot project see http://carrot.cern.ch We want to consolidate the existing ROOT GUI We would like a full TVirtualX/Qt implementation (Valery) LCG 14 June Rene Brun Blue print RTAG

with integrated picking 2-D, 3-D graphics Rich set of 2-D and 3-D primitives with integrated picking and context menus + Geometry primitives X3D viewer OpenGL viewer OpenInventor viewer Postscript,SVG TVirtualPad TVirtualPS DAQ Online GRID middleware Event Models Folders TVirtualGL Event Generators RDBMS run/file catalogs TPad Object persistencyv Object persistencyv TCanvas Event Display Dectector Simulation System services Hope to see a cooperation Olivier, Guy Histograming Fitting 2-d, 3-d graphics Ntuple analysis Detector Geometry Math Libs Statistics GUI Toolkits Interpreters LCG 14 June Rene Brun Blue print RTAG

2-d graphics A lot of work to be done, eg; Support for TrueTypeFonts 2 New objects for DAQs, time series, statistics Rethink existing organisation (TVirtualPad) to optimize cooperation. Can we agree on the level of functionality (eg polymorphic graphics containers, picking) before discussing interfaces. LCG 14 June Rene Brun Blue print RTAG

3-D graphics Pros & Cons of existing 3-d viewers Do we assume that 3-D graphics is only on a local PC or also across LANs/WANs? Do we want to duplicate the objects collections between the user application and the renderer? Picking strategy (graphics list or really object oriented)? Can we do the 2-D with 3-D / (as Guy proposes) many 3-D objects to be developed Is a cooperation really impossible? LCG 14 June Rene Brun Blue print RTAG

Event Displays Assume client-server model with access to remote files. We have many very successful implementations LCG 14 June Rene Brun Blue print RTAG

Histograming & Fitting Much more than HBOOK/PAW Fix & var bin size for 1-d, 2-d 3-D Profile 1, 2 & 3D All kinds of projections, slices Errors for all dims Filling with strings (auto sort) time axis associated fitting Random n. generation + auto binning + auto addition support for parallelism DAQ Online GRID middleware Event Models Folders Event Generators RDBMS run/file catalogs Object persistencyv Object persistencyv Event Display Dectector Simulation System services Histograming Fitting 2-d, 3-d graphics Fairly complete no requests for extensions Ntuple analysis Detector Geometry Fitting new ideas collaborators Math Libs Statistics GUI Toolkits Interpreters LCG 14 June Rene Brun Blue print RTAG

Ntuples & Trees analysis PAW-like queries on attributes tree.Draw(“varx”,”sqrt(x*y) <z”) tree.Draw(“event.tracks.GetPt()”) + Tree browser and viewer +MakeClass (generation of skeleton analysis code) +MakeSelector same as MakeClass for PROOF Collaboration with FNAL DAQ Online GRID middleware Event Models Folders Event Generators RDBMS run/file catalogs Object persistencyv Object persistencyv Event Display Dectector Simulation System services Histograming Fitting 2-d, 3-d graphics Ntuple analysis Detector Geometry Math Libs Statistics GUI Toolkits Collaboration with MIT Interpreters LCG 14 June Rene Brun Blue print RTAG

Math Libs & Statistics DAQ Online GRID middleware In ROOT today Can generate random numbers from basic distributions; gaus, poisson, etc from parametric analytic functions 1,2,3-d from histograms, 1,2,3-d DAQ Online GRID middleware In ROOT today TVector2,3 TLorentzRotation TLorentzVector TRandom,2,3 TMatrix TMath TFeldmanCousins TPrincipal TMultidimFit Event Models Folders Event Generators RDBMS run/file catalogs Object persistencyv Object persistency A collection of many algorithms CERNLIB, Numerical Recipees in C/C++ Event Display Dectector Simulation System services Many algorithms classes developed by a huge user community See recent FNAL meeting and effort organized within ACAT Histograming Fitting 2-d, 3-d graphics Ntuple analysis Would like to see an interface to GSL to Numerical Recipees in C++ Collaboration with Fred James, Louis Lyons, Pushpa Bhat Detector Geometry Math Libs Statistics GUI Toolkits Interpreters LCG 14 June Rene Brun Blue print RTAG

Math Libs , Statistics Not much to keep from CLHEP. Assuming that a team will investigate GSL and its interface to interpreters (should be trivial). Would like to see a better integration of the new nice “Numerical Recipees in C++” Plenty of people “outside” willing to help. See ACAT working group or “Confidence Levels” workshops (Fred James, Louis Lyons, Pushpa Bhat, Jim Linnemann, Prosper Harrison, etc) Linear algebra important (including persistent objects) LCG 14 June Rene Brun Blue print RTAG

Detector geometry C++ classes Geometry package MySQL Modelling A very important element Request number 1 at ROOT FNAL workshop 2001 Work in progress with ALICE Huge interest in many experiments I have a complete talk on this Modelling Visualisation Interactivity Where am I? Distance to boundary Closest boundary Persistency DAQ Online GRID middleware Event Models Folders Event Generators RDBMS run/file catalogs Object persistencyv Object persistency We want to make it transparent to Geant3 users Will be used by our Virtual MC Event Display Dectector Simulation System services Geometry package Reconstruction program Simulation Geant3-based Geant4-based Fluka-based C++ classes MySQL Histograming Fitting 2-d, 3-d graphics Ntuple analysis Detector Geometry Math Libs Statistics GUI Toolkits Interpreters LCG 14 June Rene Brun Blue print RTAG

Detector Geometry package I strongly hope we can cooperate on this subject. We have very nice results now. We have concentrated the efforts on the caching techniques: speed is a key element. We support the full detailed geometry on most existing detectors (tested with 37 !), including 4 LHCs. We invite interested people in discussions and real work on the final API, functionality and testing. We know how to interface it to G3. LCG 14 June Rene Brun Blue print RTAG

TGeo performance vs Geant3 Number gtmedi Root Geant3/ gtmedi Root Geant3/ nodes physics physics Root random random Root Gexam1 425 3.08 1.84 1.67 6.60 4.39 1.50 Gexam3 86 2.87 2.15 1.33 3.47 2.50 1.38 Gexam4 12781 2.51 2.20 1.14 12.09 11.18 1.08 Brahms 2649 5.82 3.04 1.91 4.17 1.93 2.16 Tesla 15370 6.56 5.58 1.17 12.95 7.15 1.81 CDF 24422 14.81 4.31 3.43 20.94 5.85 3.57 Minos_near 30988 30.93 20.99 1.47 21.57 13.70 1.57 BTeVEcal 52 1.57 1.08 1.45 1.78 0.73 2.43 BTeV 295310 45.27 25.88 1.75 197.06 26.83 7.34 CMSEcal 251713 5.60 1.81 3.09 5.69 1.74 3.27 CMS 1166310 33.57 8.76 3.83 39.09 24.98 1.56 LHCb 1533488 7.98 6.75 1.18 12.58 2.89 4.35 Alice 3080198 11.50 8.63 1.33 11.45 7.28 1.57 Atlas 29046966 8.90 9.94 0.89 32.48 23.39 1.38 LCG 14 June Rene Brun Blue print RTAG

Detector Simulation Propose to discuss the Virtual MC approach. Goal is to help in comparing different trackers/physics engines. VirtualMC guarantees common input for the geometry common input for the kinematics common output for hits/digits common visualisation LCG 14 June Rene Brun Blue print RTAG

Detector Simulation Geant3 Kinematics Geometry Geant4 Fluka ROOT can provide a solid base for: geometry, visualisation, interactivity, interpreter and persistency This strategy facilitates migration or comparisons with a common input and a common output DAQ Online GRID middleware Event Models Folders Event Generators RDBMS run/file catalogs Object persistencyv Object persistency Geant3 Event Display Dectector Simulation Kinematics Geometry System services Geant4 Histograming Fitting Fluka 2-d, 3-d graphics TVirtualMC Ntuple analysis Geant3.tar.gz includes an upgraded Geant3 with a C++ interface Detector Geometry Math Libs Statistics GUI Toolkits Hits, Digits Geant4_mc.tar.gz includes the TVirtualMC <-->Geant4 interface classes Interpreters LCG 14 June Rene Brun Blue print RTAG

Virtual MC example (with G3) { // Load basic libraries gSystem->Load("libPhysics"); gSystem->Load("libMC"); gSystem->Load("$(MCINSTALL)/lib/Linux-g++/libexample01"); // Load Geant3 libraries // … skipped // MC application Ex01MCApplication* appl = new Ex01MCApplication("Example01", "The example01 MC application"); appl->InitMC("g3Config.C"); appl->RunMC(1); } void Config() g3Config.C { cout << "Create geant3" << endl; new TGeant3("C++ Interface to Geant3"); } LCG 14 June Rene Brun Blue print RTAG

Virtual MC example (with G4) { // Load basic libraries gSystem->Load("libPhysics"); gSystem->Load("libMC"); gSystem->Load("$(MCINSTALL)/lib/Linux-g++/libexample01"); // Load Geant4 libraries // … skipped // MC application Ex01MCApplication* appl = new Ex01MCApplication("Example01", "The example01 MC application"); appl->InitMC("g4Config.C"); appl->RunMC(1); } void Config() { g4Config.C // RunConfiguration for Geant4 TG4RunConfiguration* runConfiguration = new TG4RunConfiguration(); // TGeant4 new TGeant4("TGeant4", "The Geant4 Monte Carlo", runConfiguration); } LCG 14 June Rene Brun Blue print RTAG