Download presentation
Presentation is loading. Please wait.
Published byEdgar Douglas Modified over 9 years ago
1
Design, Development and Evolution of the ROOT system Rene Brun February 21 2010 ACAT2010 Jaipur René Brun, ACAT2010, Jaipur1ROOT: design,development, evolution
2
plan ROOT in 2010 packagesusers How did we reach this situation? after-PAW fights Objectivity era and big confusion about object persistency LCG project and problems with the committee approach the ramping and consolidation years Directions René Brun, ACAT2010, Jaipur2ROOT: design,development, evolution
3
René Brun, ACAT2010, JaipurROOT: design,development, evolution3 OS & compilers Networking Frameworks like ROOT, Geant4 Experiment Software End user Analysis software Hardware 20 MLOC 2 MLOC 0.1 MLOC Hardware Software hierarchy
4
ROOT today a quick review René Brun, ACAT2010, JaipurROOT: design,development, evolution4
5
15 years of ROOT René Brun, ACAT2010, JaipurROOT: design,development, evolution5 1995200020052010 after PAW GEANT 3 PAW-like ROOT Objectivity era LCG projects start PI, SEAL, POOL ROOT mainstream
6
Modular structure? Nice and modular building. But can we use only what we need? René Brun, ACAT2010, JaipurROOT: design,development, evolution6
7
Project Structure CORE I/O GRAF3 D GRAF2D MATH HIST NET Tree s TGEO GUI PROOF xroot d Roofit TMVA EVE reflex CINT VMC g3root Tfluka g4roo t RooStats pyroot 7ROOT: design,development, evolutionRené Brun, ACAT2010, Jaipur
8
framework Like in the Albert Hall Museum, we created base classes (the CORE), doors and windows (IO) and utility libraries on top. Can be seen as a mess or a very functional building. René Brun, ACAT2010, JaipurROOT: design,development, evolution8
9
Interpreters In this talk, I have no time to discuss issues about interpreters like CINT and Pyroot, or the internals of the reflexion system (see my slide about LLVM later). CINT and its reflexion system is the cornerstone of ROOT for the IO system and scripts. CINT was not designed to be a full C++ interpreter. We came several years ago with ACLIC that automatizes the interface with the compiler. We strongly recommend the use of ACLIC for medium/large size scripts. LLVM will simplify even more the problem once the system will be operational. René Brun, ACAT2010, JaipurROOT: design,development, evolution9
10
René Brun, ACAT2010, JaipurROOT: design,development, evolution10 root >.x script.C root > DoSomething(…); root >.x script.C++ root >.x script.C+ gROOT->ProcessLine(“.L script.C+”); gROOT->ProcessLine(“DoSomething(…)”); execute file script.C execute function DoSomething compile file script.C and execute it compile file script.C if file has been modified. execute it same from compiled or interpreted code Interpreter & Compiler integration
11
11 ROOT Math/Stat Libraries ROOT Math/Stat LibrariesTMinuit TFumili Minuit2 Fitting and Minimization Linear Fitter Quadp Extra algorithms Extra Math functions GSL MathMor e Random Numbers MLP Statistical Libraries Limit classes Y Math functions Basic algorithms Math Interfaces MathCore TComplex TRandom Fitting Classes FFTW Extra Libraries Unuran FoamSpectrum SMatrix Linear Algebra TMatrix GenVector Physics Vectors TVector3/TLV RooFit TMVA High Level Analysis Libraries RooStats ROOT: design,development, evolution The new CERNLIB/MAT HLIB With better algorithms René Brun, ACAT2010, Jaipur
12
ROOT 2-D graphics based on 10 years experience with PAW and 15 years of operation with ROOT itself. zillions of tiny new features always requested by users works in batch (canvas.root, canvas.pdf, canvas.gif,..) object editors C++ script generated from canvas (TObject::SavePrimitive) René Brun, ACAT2010, JaipurROOT: design,development, evolution12
13
2-D Graphics 13 New functions added at each new release. Always requests for new styles and new ways to display data. ps,pdf,svg,gif, jpg,png,c,root, etc jpg,png,c,root, etc Move to GL ? ROOT: design,development, evolutionRené Brun, ACAT2010, Jaipur
14
ROOT: design,development, evolution14René Brun, ACAT2010, Jaipur
15
Image processing René Brun, ACAT2010, JaipurROOT: design,development, evolution15
16
GUI Portable GUI with all widgets found in modern UI toolkits. Many enhancements in the GUI classes: browser, html browser, TABs, EVE widgets, FitPanel. GUI builder with C++ code generator. Session recorder tool (class TRecorder) can store and replay a GUI session: All mouse events Keyboard input, including macro execution QT interfaces: a big pain, difficult to maintain with the successive versions of Qt. 16ROOT: design,development, evolutionRené Brun, ACAT2010, Jaipur
17
GUI ROOT: design,development, evolution17
18
ROOT: design,development, evolution18 GUI One of the most demanding areas Must follow technology and not standard systems (gks -> phigs - >Motif ->Qt ->?) GUI based on OpenGL too ! Plug-ins must work on the web (but 2-d and 3-d graphics too!) GUI builder extensions (code generators) GUI must be scriptable and theme-able HELP + language internalisation Context menus help access to documentation René Brun, ACAT2010, Jaipur
19
ROOT 3-D graphics OpenGL based: graphics accelerators detector geometry visualization event displays (EVE) interaction between GL view and original objects (picking/editing) René Brun, ACAT2010, JaipurROOT: design,development, evolution19
20
3-D Graphics Highly optimized GL views in TPad TPad the GL viewer 20 ROOT: design,development, evolution Mathematical objects Detector hits René Brun, ACAT2010, Jaipur
21
ROOT: design,development, evolution21 Graphics (2D-3D) “SURF” “LEGO” TF3 TH3 TGLParametric René Brun, ACAT2010, Jaipur
22
The Geometry Package TGEO The TGeo classes are now stable (thanks ALICE). Can work with different simulation engines (G3,G4,Fluka) (See Virtual Monte Carlo) G3->TGeo, G4->TGeo, TGeo >GDML Used in online systems and reconstruction programs Built-in facilities for alignment Impressive gallery of experiments (35 detectors in $ROOTSYS/test/stressGeometry) 22ROOT: design,development, evolutionRené Brun, ACAT2010, Jaipur
23
ROOT: design,development, evolution23 OpenGL see $ROOTSYS/tutorials/geom René Brun, ACAT2010, Jaipur
24
Event Display : EVE a GL-based package for event displays. Developed in collaboration with ALICE (AliEve) and CMS (FireWorks). Provides all the GUI widgets, browsers, GL infrastructure (far better than the old OpenInventor). Used now by many experiments (see eg FAIRROOT, ILCROOT) to display raw data, MC events or detector oriented visualization. 24ROOT: design,development, evolutionRené Brun, ACAT2010, Jaipur
25
PROOF Massive data analysis with parallel systems. Now in production in many places. Growing success. A lot of developments for a better integration with dataset management systems and to support priority and accounting management. PROOF-LITE is a mini version to run on a multi-core. ROOT: design,development, evolution25 ROOT: one core only with PROOF with PROOF-LITE René Brun, ACAT2010, Jaipur
26
IO and Trees They are the most important components of the system because they have a deep impact on the experiment frameworks and their data handling strategy. Shaping these 2 systems has been a long process. René Brun, ACAT2010, JaipurROOT: design,development, evolution26
27
René Brun, ACAT2010, JaipurROOT: design,development, evolution27 Memory Tree Each Node is a branch in the Tree tr 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 T.Fill() T.GetEntry(6) T Memory
28
Automatic branch creation from object model float a; int b; double c[5]; int N; float* x; //[N] float* y; //[N] Class1 c1; Class2 c2; //! Class3 *c3; std::vector ; TClonesArray *tc; René Brun, ACAT2010, Jaipur28ROOT: design,development, evolution branch buffers
29
branches created from a dynamic list a b c d e a a a a a Each named member of a TList Is a top level branch René Brun, ACAT2010, Jaipur29ROOT: design,development, evolution
30
Top level branches created by user a b c d e Top level branches Are created manually René Brun, ACAT2010, Jaipur30ROOT: design,development, evolution
31
Branches created from STL collections std::vector Use split=199 to reduce The number of branches In case of polymorphic collections std::vector TClonesArray std::vector split = 0 split >0 split <100 split =199 split >0 René Brun, ACAT2010, Jaipur31ROOT: design,development, evolution
32
Trends in Tree structure Maximum split level, ie more branches is better eg, Atlas 9785, CMS 1436, LHCb 1167, Alice >1000 Many branches is good for the future because branch reading can be executed in parallel. many branches is better because the access granularity is better for fast selections. René Brun, ACAT2010, JaipurROOT: design,development, evolution32
33
ObjectWise/MemberWise Streaming abcdabcd a1b1c1d1a2b2c2d2…anbncndn a1a2..anb1b2..bnc1c2..cnd1d2..dn a1a2…an b1b2…bn c1c2…cn d1d2…dn 3 modes to stream an object object-wise to a buffer member-wise to a buffer each member to a buffer member-wise gives better compression René Brun, ACAT2010, Jaipur33ROOT: design,development, evolution
34
Supported Platforms Linux (RH, SLC4, SLC5, Suse, Debian, Ubuntu) gcc3.4,…, gcc4.4 (32 and 64 bits) icc10.1, icc11.1 MacOS X (Intel/PPC, 10.4, 10.5, 10.6) gcc4.0.1, gcc4.2.1,gcc4.4.1, llvm-g++ icc10.1, icc11.1 Windows (XP, Vista) VC++7.1, VC++9 Cygwin gcc3.4, 4.3 Solaris + OpenSolaris CC5.2 gcc3.4, gcc4.4 34ROOT: design,development, evolution Support for many compilers adds robustness at a minor development cost. It facilitates the migration to new compiler versions. René Brun, ACAT2010, Jaipur
35
Distribution Stats 35ROOT: design,development, evolution More than 20000 users 35000 FTP-only distributions of binaries per month. Many more from SVN source or AFS access René Brun, ACAT2010, Jaipur
36
How did we reach the current situation Rien ne sert de courir, il faut partir a point Jean de la Fontaine René Brun, ACAT2010, JaipurROOT: design,development, evolution36
37
one cannot forget the past Systems like ROOT and G4 have been strongly influenced par their predecessors, by politics, by sociology. Like Jumbo jets, large systems that required a lot of manpower evolve with time with the pros and cons of their inertia. Once a system is massively used, people expect backward and forward compatibility. However ROOT in 2010 is very much different of ROOT in 1995 or 2000 or 2005 René Brun, ACAT2010, JaipurROOT: design,development, evolution37
38
Software in HEP René Brun, ACAT2010, JaipurROOT: design,development, evolution38 1970198019902010 cernlib ===============-------------------------………………....... paw---====================-----------------….... zebra---=================------------------------…….. geant1,2 ==-- hbook===============================------------------------…………... geant4----=================== piaf==== geant3 ======================------------------………... ROOT---=================== JAS--===----…... most systems used over 2,3 or 4 decades
39
Main parameters who shaped ROOT Long experience with GEANT and PAW Reuse of components Concept of packages within a framework Optimal for batch, optimal for interactivity User support quasi online René Brun, ACAT2010, JaipurROOT: design,development, evolution39
40
CERNLIB 1970: main.f links with cernlib.a where cernlib is a collection of independent subroutines. 1975: subroutines are grouped into packages identified by the letters of the alphabet, eg Y250 1980: cernlib accepts large packages, eg hbook, minuit,gd3 1985: cernlib is a collection of large packages, mathlib, kernlib,packlib,graflib,pawlib,geant3 René Brun, ACAT2010, JaipurROOT: design,development, evolution40
41
hbook : 1973 20xy first large package in cernlib histogram-oriented system including visualization (from line-printer to graphics) and persistency. interface with Minuit and Fumili interface with zbook in 1975, then zebra in 1983 extended in 1985 to support row-wise ntuples and column- wise ntuples in 1990. the heart of PAW René Brun, ACAT2010, JaipurROOT: design,development, evolution41
42
geant3 : 1980- 20xy successor of geant1 (1975), geant2(1978) includes a powerful geometry package used to model all existing HEP detectors. interfaces with external hadronic shower packages like tatina,gheisha,fluka. includes visualisation: detector cross-sections, event displays. includes IO for geometry and events (zebra) René Brun, ACAT2010, JaipurROOT: design,development, evolution42
43
PAW : 1984 20xy started by a committee (design for the past) quickly reshaped in 1985 with KUIP, HIGZ, ntuples query processor: ntuple/plot 10.px analysis patterns indicates row-wise column-wise storage and access. developments using zebra RZ IO system more and more graphics options René Brun, ACAT2010, JaipurROOT: design,development, evolution43
44
PIAF : 1992-1994 PAW Parallel Interactive Analysis Facility very important step to understand the issues with parallelism with the IO sub system and the implications on the users analysis code. very important to understand the balance between CPU, disk IO, network IO. René Brun, ACAT2010, JaipurROOT: design,development, evolution44
45
software crisis 1992-1994 It was clear in 1992 that our Fortran-based systems will be impacted by new languages and new techniques. Many people thought that Fortran was the future. MOOSE project investigating languages (C++, Eiffel) and techniques (UML, Rose, OO-design) commercial software or home-grown dilemma OODBMS at the horizon René Brun, ACAT2010, JaipurROOT: design,development, evolution45
46
software crisis 1992-1994 -more many computer scientists in the discussion most of them with no experience existing experience with large software neglected as well as feedback coming from experienced physicists west coast syndrome Babar as guinea pig (C++ and OODBMS) Revolution, not Evolution René Brun, ACAT2010, JaipurROOT: design,development, evolution46
47
Towards Objectivity 1994-2000 Gurus predict that OODBMS will replace RDBMS systems in 2000. Post-mortem influence of defunct SSC dream of a central data base accessed world-wide at the bit level for writing and reading. Don’t worry about containers structure, CPU and network performance; the technology will solve your problems. Most physicists trust gurus for reaching Nirvana René Brun, ACAT2010, JaipurROOT: design,development, evolution47
48
1994-2000 Lost in the clouds before the grid and the clouds Experienced physicists busy with LHC exp design. It took many years (of frustration) to move them from good old Fortran to the magic of OO and C++ catastrophic LCB (LHC Computing Board) committee with always the same referees with no experience in the new technology. As a consequence the LCB has no choice but trusting the gurus. Despite early signs of problems, gurus and referees persist in their recommendations. René Brun, ACAT2010, JaipurROOT: design,development, evolution48
49
1995-1998 ROOT is born in the meadows While many people were actively working in committees and expecting magic from Objectivity, a small group of anarchists is exiled in the CERN 2 meadows. free of committees and not spending time only with UML diagrams, but concentrating their work on key issues and delivering frequent releases of prototypes tested in real life experiments (eg NA49). While the first ROOT prototype was a C++ version of PAW, it was rapidly realized that the key problem was with a general object persistency solution much different than the Objectivity solution. René Brun, ACAT2010, JaipurROOT: design,development, evolution49
50
Original Objectives in 95 Continue the tradition started with CERNLIB to develop packages targeted to the needs of our experiments and with a long life time. Eg hbook (36 years), geant3 (28), paw(25): still in use! Focus on data analysis (PAW in the oo world) Open Source model to get contributions and feedback as soon as possible. First ROOT prototype for NA49 in 1995, rapidly followed by many more adopters. Bottom-up (user-driven) instead of top-down (committee) approach. Users workshops. ROOT: design,development, evolution50René Brun, ACAT2010, Jaipur
51
ROOT in 1995 During the first 3 months of 1995 we prototyped several versions of the histograming system, including a version based on templates. In October 1995 we presented a first prototype with histograms, functions, graphics (a la PAW). This prototype included a command line interface supporting very primitive calls to ROOT classes member functions. This prototype generated a lot of attention but also fears. René Brun, ACAT2010, JaipurROOT: design,development, evolution51
52
ROOT in 1996 Instead of our very basic interpreter, we decided to use CINT developed since 1991 by Masa Goto from HP Japan. We used CINT not only as an interpreter, but also as a parser for the ROOT header files to generate automatically a dictionary such that: one could call class member functions from C++ interpreted scripts one could stream objects to a buffer with explicit streamers. ROOT version 1 included also many more classes and better graphics. René Brun, ACAT2010, JaipurROOT: design,development, evolution52
53
ROOT in 1997 The IO Streamer functions generated by rootcint were quite simple to understand and were efficient. However, we realized soon that classes were evolving. We had to introduce class versions and the possibility for users to write their own custom streamers where they could take into account the class schema evolution. At the same time, we made rootcint more general such that it could parse many more classes, in particular non- ROOT classes developed by our experiments guinea-pigs. René Brun, ACAT2010, JaipurROOT: design,development, evolution53
54
ROOT in 1998 We released version 2 with more features (IO, math, graphics and more sophistication in Trees, our main container for event data. In fall 1998 ROOT was selected by FNAL & RHIC as: the persistency solution the data analysis solution This decision generated a big chock-wave in the establishment. It also generated more work for us. René Brun, ACAT2010, JaipurROOT: design,development, evolution54
55
ROOT in 1999 The ROOT Streamers were very efficient. However the library containing the Streamers was required when reading the data sets. While this requirement was not a problem when reading complex object collections and the classes used to write were necessary in the read phase, it was a serious drawback for files with simple Trees like PAW ntuples. At this point we implemented an automatic system to stream data using the information in the dictionary instead of using the generated C++ Streamers René Brun, ACAT2010, JaipurROOT: design,development, evolution55
56
StreamerInfo With the automatic streaming, we also saved to the ROOT file the meta data information describing all classes such that ROOT files are now self-describing. With time and in the following years we continued to optimize this system: to support the full C++ language in general. to support the full C++ language in general. to support more sophisticated cases of class schema evolution. René Brun, ACAT2010, JaipurROOT: design,development, evolution56
57
ROOT vs Objectivity In 99.99% cases people write a file once and read it many times. No need for a locking process when reading. ROOT files are self-described and independent of the location where they are processed. ROOT files are compressed. Typical compression factors for LHC experiments is between 3 and 5. ROOT files are machine independent René Brun, ACAT2010, JaipurROOT: design,development, evolution57
58
ROOT vs Objectivity 2 ROOT does not stream transient data members ROOT can stream collections member-wise, optimizing both speed and disk space. ROOT can read files containing different class versions: automatic class schema evolution. ROOT has a complete query/visualization system. ROOT files can be processed in parallel with PROOF ROOT was processing files orders of magnitude faster ROOT has a powerful caching mechanism allowing to read files very efficiently in wide area networks René Brun, ACAT2010, JaipurROOT: design,development, evolution58
59
Directions René Brun, ACAT2010, JaipurROOT: design,development, evolution59
60
Our Challenges Keep a stable core, improving robustness At the same time follow the technology changes Distributed and parallel computing Multi-core systems, GPU processing (OpenCL) New version of C++, etc CINT to LLVM Improve algorithms (quality and speed) Improve doc, user interface, user support Add support for new communities, LC,etc And, of course, provide a robust and efficient data analysis tool for the LHC experiments. ROOT: design,development, evolution60René Brun, ACAT2010, Jaipur
61
ROOT: design,development, evolution61 Will have to deal with many shared libs Only a small fraction of code used Problem decomposition
62
René Brun, ACAT2010, JaipurROOT: design,development, evolution62 ROOTin1995 ROOTin2010 10 shared libs 200 classes 100 shared libs 2000 classes PAW model Plug-in manager Some Facts
63
René Brun, ACAT2010, JaipurROOT: design,development, evolution63 many_x.f -> many_x.o many_x.o + some_libs.a + many_libs.so -> x.exe x.exe Input.dat (free format) Output.log Objectivity? ROOT ? Zebra file RFIO Executable module in 2000
64
René Brun, ACAT2010, JaipurROOT: design,development, evolution64 x.exe Config.C (interpreter) Output.log ROOT files Oracle Mysql Dcache castor xrootd a.so b.so u.so Shared libs dynamically loaded/unloaded by the plug-in manager LAN Executable module in 2010
65
René Brun, ACAT2010, JaipurROOT: design,development, evolution65 x.exe Config.C (interpreter) Output.log ROOT files Oracle Mysql Cache Proxy manager http a.cxx http b.cxx http u.cxx Local shared libs dynamically Compiled/loaded/unloaded from a source URL WAN x.exe ROOT buffers local cache Multi-threaded Core executor Executable module in 2020 ?
66
The LLVM system We are having big hopes with LLVM (open source project with people from Apple, Adobe, Google, etc). LLVM is a gcc compatible compiler, parser and JIT (Just In Time) compiler. Our idea is that the interpreter will be the compiler itself (C++0x compliant) and offering many additional possibilities like quality and speed for important classes like TFormula, TTreeFormula, etc We have a working prototype today, but a fully operational use of LLVM in ROOT is probably 2 years away. René Brun, ACAT2010, JaipurROOT: design,development, evolution66
67
René Brun, ACAT2010, JaipurROOT: design,development, evolution67 The GRID is a parallel engine. However it is unlikely that you will use the GRID software on your 32-core laptop. There is a lot of work to be done within ROOT to use parallelism internally or making ROOT-based applications more easily parallelizable. Think Top->Down and Bottom->Up Coarse grain: job, event, track Fine grain vectorization Design for Parallelism
68
René Brun, ACAT2010, JaipurROOT: design,development, evolution68 100,000 computers in 1000 locations 5,000 physicists in 1000 locations WAN LAN Towards a distributed system
69
René Brun, ACAT2010, JaipurROOT: design,development, evolution69 Few big users submitting many long jobs (Monte Carlo, reconstruction) They want to run many jobs in one week Many users submitting many short jobs (physics analysis) They want to run many jobs in one hour or less GRID: Users profile
70
Difficult to fit all sizes Like the Taj Mahal, ROOT has many different facets and the visitor may have different impressions depending on his viewpoint and time of day. We try to do our best to satisfy very heterogeneous users. Some use ROOT as a framework, others as a simple tool, others as a set of libraries. René Brun, ACAT2010, JaipurROOT: design,development, evolution70
71
Taj Mahal & ROOT The Taj Mahal combines elements from Persian, Indian, Islamic architectural styles, an integrated complex of structures. Building began around 1632 and was completed around 1653, and employed thousands of artisans and craftsmen. ROOT combines ideas and techniques that proved to be successful in the geant3, zebra and PAW eras. Building began in 1995 and in 2010 is still under construction by many motivated people. René Brun, ACAT2010, JaipurROOT: design,development, evolution71
72
Many Thanks I have been pleased to follow the ACAT meetings since the beginning in Lyon (1989 ?) Many thanks to my colleagues organizing this event. This is my last talk at ACAT as ROOT project leader. I take this opportunity to thank all my colleagues who are working hard in this project and I wish them a healthy future. René Brun, ACAT2010, JaipurROOT: design,development, evolution72
73
Input/Output: Major Steps 73 ROOT: design,development, evolution parallel merge generalized schema evolution member-wise streaming for STL collections member-wise streaming for TClonesArray automatic streamers from dictionary with StreamerInfos in self-describing files streamers generated by rootcint User written streamers filling TBuffer René Brun, ACAT2010, Jaipur
74
CMZ…………………………………CVS………………………………………………………….SVN…………… Major Technical Steps 74 P R O O F Reflex From SEAL to ROOT ROOT: design,development, evolution LLVM CINT7 GL/EVE RooFit TMVA MathLibs Reflex From SEAL to ROOT Trees automatic split GUI based on signal/slot s I/O based on dictionaries in memory Graphics based on TVirtualX CINT René Brun, ACAT2010, Jaipur
75
Testing, Nightly builds Several thousand tests run every night on many platforms. Code checking with a coding convention rule checker Code coverage to be improved, but already much better than any other package reviewed. Frequent valgrinding and profiling ROOT: design,development, evolution75René Brun, ACAT2010, Jaipur
76
PROOF-LITE PROOF-LITE 76 PROOF optimized for single many-core machines Zero configuration setup. No config files and no daemons Workers are processes and not threads for robustness Like PROOF it can exploit fast disks, SSD’s, lots of RAM, fast networks and fast CPU’s Once your analysis runs on PROOF Lite it will also run on PROOF (with exactly the same user code) ROOT: design,development, evolution ROOT: one core onlywith PROOF-LITE René Brun, ACAT2010, Jaipur
77
ROOT: design,development, evolution77 many_x.f -> many_x.o many_x.o + many_libs.a -> x.exe x.exe Input.dat (free format ) Output.log portable Zebra file Executable module in 1990
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.