1. ROOT: Status & Developments1 Aachen 1 February 2005 Ren é.Brun@cern.ch The ROOT Project Status & Developments

2. ROOT: Status & Developments2 ROOT in a nutshell An efficient data storage and access system designed to support structured data sets in very large distributed data bases (Petabytes). A query system to extract information from these distributed data sets. The query system is able to use transparently parallel systems on the GRID (PROOF). A scientific visualisation system with 2-D and 3-D graphics. An advanced Graphical User Interface A C++ interpreter allowing calls to user defined classes. An Open Source Project

3. ROOT: Status & Developments3 ROOT: An Open Source Project The project is developed as a collaboration between : Full time developers: 6 people full time at CERN 1 core developer at FermiLab 1 core developer in Japan (Agilent Technologies) 1 core developer at MIT 1 mathematician at CERN sponsored by a US Finance Company Many contributors spending a substantial fraction of their time in specific areas (> 50). Key developers in large experiments using ROOT as a framework. Several thousand users given feedback and a very long list of small contributions.

4. ROOT: Status & Developments4 Project History Jan 95: Thinking/writing/rewriting/??? November 95: Public seminar, show Root 0.5 Spring 96: decision to use CINT Jan 97: Root version 1.0 Jan 98: Root version 2.0 Mar 99: Root version 2.21/08 (1st Root workshop FNAL) Feb 00: Root version 2.23/12 (2nd Root workshop CERN) Mar 01: Root version 3.00/06 Jun 01: Root version 3.01/05 (3rd Root workshop FNAL) Jan 02: Root version 3.02/07 (LCG project starts: RTAGs) Oct 02: Root version 3.03/09 (4th Root workshop CERN) Feb 04: version 4.00 (5 th Root workshop SLAC) Dec 04: version 4.02 (current stable version) Feb 05: Developing 4.04 to be released in June (LCG/ROOT merge) 10 years !!

5. ROOT: Status & Developments5 The ROOT web pages http://root.cern.ch  General Information and News  Download source and binaries  Howto & tutorials  User Guide & Reference Guides  Roottalk Digest & Forum

6. ROOT: Status & Developments6 Current ROOT structure & libs

7. ROOT: Status & Developments7 CINT: the C++ interpreter

8. ROOT: Status & Developments8 My first session root [0] 344+76.8 (const double)4.20800000000000010e+002 root [1] float x=89.7; root [2] float y=567.8; root [3] x+sqrt(y) (double)1.13528550991510710e+002 root [4] float z = x+2*sqrt(y/6); root [5] z (float)1.09155929565429690e+002 root [6].q root root [0] try up and down arrows See file $HOME/.root_hist

9. ROOT: Status & Developments9 My second session root [0].x session2.C for N=100000, sum= 45908.6 root [1] sum (double)4.59085828512453370e+004 Root [2] r.Rndm() (Double_t)8.29029321670533560e-001 root [3].q root { int N = 100000; TRandom r; double sum = 0; for (int i=0;i<N;i++) { sum += sin(r.Rndm()); } printf("for N=%d, sum= %g\n",N,sum); } session2.C unnamed macro executes in global scope

10. ROOT: Status & Developments10 My third session root [0].x session3.C for N=100000, sum= 45908.6 root [1] sum Error: Symbol sum is not defined in current scope *** Interpreter error recovered *** Root [2].x session3.C(1000) for N=1000, sum= 460.311 root [3].q root void session3 (int N=100000) { TRandom r; double sum = 0; for (int i=0;i<N;i++) { sum += sin(r.Rndm()); } printf("for N=%d, sum= %g\n",N,sum); } session3.C Named macro Normal C++ scope rules

11. ROOT: Status & Developments11 My third session with ACLIC root [0] gROOT->Time(); root [1].x session4.C(10000000) for N=10000000, sum= 4.59765e+006 Real time 0:00:06, CP time 6.890 root [2].x session4.C+(10000000) for N=10000000, sum= 4.59765e+006 Real time 0:00:09, CP time 1.062 root [3] session4(10000000) for N=10000000, sum= 4.59765e+006 Real time 0:00:01, CP time 1.052 root [4].q #include “TRandom.h” void session4 (int N) { TRandom r; double sum = 0; for (int i=0;i<N;i++) { sum += sin(r.Rndm()); } printf("for N=%d, sum= %g\n",N,sum); } session4.C File session4.C Automatically compiled and linked by the native compiler. Must be C++ compliant

12. ROOT: Status & Developments12 Macros with more than one function root [0].x session5.C >session5.log root [1].q void session5(int N=100) { session5a(N); session5b(N); gROOT->ProcessLine(“.x session4.C+(1000)”); } void session5a(int N) { for (int i=0;i<N;i++) { printf("sqrt(%d) = %g\n",i,sqrt(i)); } void session5b(int N) { double sum = 0; for (int i=0;i<N;i++) { sum += i; printf("sum(%d) = %g\n",i,sum); } session5.C.x session5.C executes the function session5 in session5.C root [0].L session5.C root [1] session5(100); >session5.log root [2] session5b(3) sum(0) = 0 sum(1) = 1 sum(2) = 3 root [3].q use gROOT->ProcessLine to execute a macro from a macro or from compiled code

13. ROOT: Status & Developments13 Combining UI and GUI root [0].x session2.C for N=100000, sum= 45908.6 root [1] sum (double)4.59085828512453370e+004 Root [2] r.Rndm() (Double_t)8.29029321670533560e-001 root [3].q

14. ROOT: Status & Developments14 The Histogram Package

15. ROOT: Status & Developments15 The Histogram Classes Structure 1-Dim 2-Dim 3-Dim

16. ROOT: Status & Developments16 Histograms class THStack Long list of functions in TH1 Plenty of drawing options Filling with string variables Automatic binning TH1::SetBuffer TH1::GetRandom h3 = 3*h1 – h2

17. ROOT: Status & Developments17 Filling with strings See tutorials -hlabels1.C -hlabels2.C -cernstaff.C

18. ROOT: Status & Developments18 Peak Finder + Deconvolutions by Miroslav Morach TSpectrum

19. ROOT: Status & Developments19 Fitting TVirtualFitter TFitter TMinuit TH1::Fit bin chisquare bin likelihood TGraph::Fit unbinned chisquare User TTree::Fit unbinned likelihood TFumili

20. ROOT: Status & Developments20 Graphics

21. ROOT: Status & Developments21 Gui/Graphics strategy TVirtualX TGWin32TGX11TGWin32GDKTGQt TVirtualPad TPad User/Root GUI and Graphics classes Applications see only Abstract Interfaces Low level screen graphics and GUI High level pad graphics TG??

22. ROOT: Status & Developments22 A Data Analysis & Visualisation tool

23. ROOT: Status & Developments23 Graphics : 1,2,3-D functions

24. ROOT: Status & Developments24 Full LateX support on screen and postscript TCurlyArc TCurlyLine TWavyLine and other building blocks for Feynmann diagrams Formula or diagrams can be edited with the mouse

25. R.Brun 1 Feb 2005ROOT: Status & Developments ROOT 3D Graphics Simple Box The ROOT basic 3D shapes

26. ROOT: Status & Developments26 Alice 3 million nodes

27. R.Brun 1 Feb 2005ROOT: Status & Developments R. Brun (CERN), O. Couet (CERN), M. Gheata (ISS), A. Gheata (ISS), V. Onoutchine (IFVE), T.Pocheptsov (JINR) ROOT 3D Graphics Text ………………… Atlas

28. ROOT: Status & Developments28 GUI

29. ROOT: Status & Developments29 Available Widgets Complete set of widgets: label, icon, button, check button, radio button, picture button, button box, list box, combo box, list view, icon view, number entry, text entry, text view, text edit, tree view, tab view, scrollbar, slider, menubar, popup menu, cascading menu, statusbar, toolbar, message dialogs, file selection dialog, progress bars, tooltips,...

30. ROOT: Status & Developments30 Basic Widgets

31. ROOT: Status & Developments31 Advanced Widgets Color selector dialog: TGColorDialog

32. ROOT: Status & Developments32 GUI Examples

33. ROOT: Status & Developments33 GUI Examples – Histogram Browser

34. ROOT: Status & Developments34 More GUI Examples – Period System

35. ROOT: Status & Developments35 Object Editors

36. ROOT: Status & Developments36 Histogram Editor

37. ROOT: Status & Developments37 Object Editors (rebinning)

38. ROOT: Status & Developments38 Input/Output

39. ROOT: Status & Developments39 Evolution of ROOT I/O Hand-written Streamers Streamers generated via rootcint Support for Class Versions Support for ByteCount Persistent class Dictionary written to files rootcint modified to generate automatic Streamers Support for Automatic Schema Evolution Can generate code for “DataObjects” classes in a file Support for complex C++ cases Can read files without the classes Persistent Reference pointers Support for foreign classes Full support for STL 3.00 3.01 1995 2005 3.02 3.04 4.02

40. ROOT: Status & Developments40 Object Persistency (in a nutshell) Two I/O modes supported (Keys and Trees). Key access: simple object streaming mode. A ROOT file is like a Unix directory tree Very convenient for objects like histograms, geometries, mag.field, calibrations Trees A generalization of ntuples to objects Designed for storing events split and no split modes query processor Chains: Collections of files containing Trees ROOT files are self-describing Interfaces with RDBMS also available Access to remote files (RFIO, DCACHE, GRID)

41. ROOT: Status & Developments41 ROOT I/O : An Example TFile f(“example.root”,”new”); TH1F h(“h”,”My histogram”,100,-3,3); h.FillRandom(“gaus”,5000); h.Write(); TFile f(“example.root”); TH1F *h = (TH1F*)f.Get(“h”): h->Draw(); f.Map(); Program Writing Program Reading 20010831/171903 At:64 N=90 TFile 20010831/171941 At:154 N=453 TH1F CX = 2.09 20010831/171946 At:607 N=2364 StreamerInfo CX = 3.25 20010831/171946 At:2971 N=96 KeysList 20010831/171946 At:3067 N=56 FreeSegments 20010831/171946 At:3123 N=1 END demoh.C demohr.C

42. ROOT: Status & Developments42 A Root file pippa.root with two levels of directories Objects in directory /pippa/DM/CJ eg: /pippa/DM/CJ/h15

43. ROOT: Status & Developments43 Self-describing files Dictionary for persistent classes written to the file when closing the file. ROOT files can be read by foreign readers (eg JavaRoot (Tony Johnson) Support for Backward and Forward compatibility Files created in 2003 must be readable in 2015 Classes (data objects) for all objects in a file can be regenerated via TFile::MakeProject Root >TFile f(“demo.root”); Root > f.MakeProject(“dir”,”*”,”new++”);

44. ROOT: Status & Developments44 Automatic Schema Evolution

45. ROOT: Status & Developments45 Auto Schema Evolution (2)

46. ROOT: Status & Developments46 File types & Access in 4.02/xx Local File X.xml RFIOChirp CastorDcache Local File X.root http rootd/xrootd Oracle SapDb PgSQL MySQL TFile TKey/TTree TStreamerInfo user TSQLServer TSQLRow TSQLResult TTreeSQL

47. ROOT: Status & Developments47 Memory Tree Each Node is a branch in the Tree tr 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 T.Fill() T.GetEntry(6) T Memory

48. ROOT: Status & Developments48 Tree example Event (write) void demoe(int nevents) { //load shared lib with the Event class gSystem->Load("$ROOTSYS/test/libEvent"); //create a new ROOT file TFile f("demoe.root",”new"); //Create a ROOT Tree with one single top level branch int split = 99; //try also split=1 and split=0 int bufsize = 16000; Event *event = new Event; TTree T("T","Event demo tree"); T.Branch("event","Event",&event,bufsize,split); //Build Event in a loop and fill the Tree for (int i=0;i<nevents;i++) { event->Build(i); T.Fill(); } T.Print(); //Print Tree statistics T.Write(); //Write Tree header to the file } All the examples can be executed with CINT or the compiler root >.x demoe.C root >.x demoe.C++

49. ROOT: Status & Developments49 Tree example Event (read 1) void demoer() { //load shared lib with the Event class gSystem->Load("$ROOTSYS/test/libEvent"); //connect ROOT file TFile *f = new TFile("demoe.root"); //Read Tree header and set top branch address Event *event = 0; TTree *T = (TTree*)f->Get("T"); T->SetBranchAddress("event",&event); //Loop on events and fill an histogram TH1F *h = new TH1F("hntrack","Number of tracks",100,580,620); int nevents = (int)T->GetEntries(); for (int i=0;i<nevents;i++) { T->GetEntry(i); h->Fill(event->GetNtrack()); } h->Draw(); } Rebuild the full event in memory

50. ROOT: Status & Developments50 8 Branches of T 8 leaves of branch Electrons A double-click to histogram the leaf

51. ROOT: Status & Developments51 The Tree Viewer & Analyzer A very powerful class supporting complex cuts, event lists, 1-d,2-d, 3-d views parallelism

52. ROOT: Status & Developments52 Tree Friends tr 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Public read Public read User Write Entry # 8

53. ROOT: Status & Developments53 Tree Friends Root > TFile f1(“tree1.root”); Root > tree.AddFriend(“tree2”,“tree2.root”) Root > tree.AddFriend(“tree3”,“tree3.root”); Root > tree.Draw(“x:a”,”k<c”); Root > tree.Draw(“x:tree2.x”,”sqrt(p)<b”); x Processing time independent of the number of friends unlike table joins in RDBMS Collaboration-wide public read Analysis group protected user private

54. ROOT: Status & Developments54 PROOF and GRID(s) interface

55. ROOT: Status & Developments55 Batch/Interactive models Batch Production Simulation reconstruction Interactiv e Chaotic analysis Interactiv e batch model Need experiment framework +widely available tools Need only widely available tools

56. ROOT: Status & Developments56 GRID: Interactive Analysis Case 1 Data transfer to user’s laptop Optional Run/File catalog Optional GRID software Optional run/File Catalog Remote file server eg rootd Trees Analysis scripts are interpreted or compiled on the local machine

57. ROOT: Status & Developments57 GRID: Interactive Analysis Case 2 Remote data processing Optional Run/File catalog Optional GRID software Optional run/File Catalog Remote data analyzer eg proofd Trees Commands, scripts histograms Analysis scripts are interpreted or compiled on the remote machine

58. ROOT: Status & Developments58 GRID: Interactive Analysis Case 3 Remote data processing Run/File catalog Full GRID software Run/File Catalog Remote data analyzer eg proofd Trees Commands, scripts Histograms,trees Trees slave Analysis scripts are interpreted or compiled on the remote master(s)

59. ROOT: Status & Developments59 Parallel ROOT Facility The PROOF system allows: Parallel analysis of trees in a set of files Parallel analysis of objects in a set of files Parallel execution of scripts on clusters of heterogeneous machines Its design goals are: Transparency, scalability, adaptability Prototype developed in 1997 as proof of concept, full version nearing completion now

60. ROOT: Status & Developments60 Parallel Script Execution root Remote PROOF Cluster proof TNetFile TFile Local PC $ root ana.C stdout/obj node1 node2 node3 node4 $ root root [0].x ana.C $ root root [0].x ana.C root [1] gROOT->Proof(“remote”) $ root root [0] tree->Process(“ana.C”) root [1] gROOT->Proof(“remote”) root [2] chain->Process(“ana.C”) ana.C proof proof = slave server proof proof = master server #proof.conf slave node1 slave node2 slave node3 slave node4 *.root TFile

61. ROOT: Status & Developments61 Google: a good model Make it simple Simple interface Available everywhere Hidden Parallelism Distributed DB Don’t know a priori the data location Fast

62. ROOT: Status & Developments62 I did not talk about

63. ROOT: Status & Developments63 Currently in ROOT 4.02 SQL interface (MySQL, Oracle, postgres,sapdb) Threads Networking Mass Storage interfaces (Dcache,Castor,Chirp..) Mathlibs, Physics vectors Linear Algebra The geometry package Virtual MonteCarlo Interfaces to event generators The GUI Builder Neural Network Image processing PyRoot (The ROOT/Python interface) Carrot: ROOT web interface

64. ROOT: Status & Developments64 Current developments (4.03/4.04) (in June release)

65. ROOT: Status & Developments65 Consolidation Robustness Zillions of tiny details User support

66. ROOT: Status & Developments66 OpenGL interface A lot of effort invested in the GL interface Real Time navigation in complex detectors with millions of physical volumes. Show only what makes sense Invisible, translucent, variable precision rendering Dynamic particles High quality output GUI

67. ROOT: Status & Developments67 Mathlib Consolidation of TMath (now a namespace) Improved algorithms Robust Fitters (from the R package) Outliers, cluster finders Interface with a subset of GSL Collaboration with SEAL/Math

68. ROOT: Status & Developments68 Image processing Collaboration with Sasha Vasko for the development of libAfterImage Fast graphics (also in batch): an alternative to X11. Will be used by Carrot for execution of remote scripts and fast interactive graphics on the web. Could be used for fast image manipulation (better than large scatter plots)

69. ROOT: Status & Developments69 xrootd Joint development with BaBar Robust and efficient distributed file server with load balancing. Could be integrated with Castor. Integration with PROOF Requires consolidation of authentication and security. Implementation of read-ahead for ROOT Trees.

70. ROOT: Status & Developments70 PROOF Add support for all features currently available in ROOT Trees (friends, event lists, selectors) Special (but growing case) of one client, one server with no slaves Stateless connections with multiple PROOF objects in the same ROOT session (long and short queries). View/Import results of batch queries Save ROOT/PROOF session with resume capabilities. Interfaces with GRID services.

71. ROOT: Status & Developments71 GUI Continuous improvements. More & more objects edited via the GUI. GUI objects persistent. Currently the GUI can be saved as a C++ script. We would like to save it as a canvas.root too. More features in the GUI Builder (+ doc)

72. ROOT: Status & Developments72 SQL Interface RDBMS access via a set of abstract base classes TSQLServer, TSQLResult and TSQLRow Concrete implementations for MySQL and Oracle exist TMySQLServer, TMySQLResult and TMySQLRow TOracleServer, TOracleResult and TOracleRow

73. ROOT: Status & Developments73 New TTreeSQL class TTreeSQL tree(const char *db,const char* uid,…) tree.Print(), Browse, Scan, etc tree.Draw(“var1:var2”,”varx <0”) TTree query style converted to SQL Connect to an existing db TTreeSQL tree(“mysql://localhost/test”,”nobody”,”new”); Event *event = new Event; tree.Branch(“top”,”Event”,&event); tree.Fill(); tree.AutoSave(); A TSQLRow is filled and sent to the server Columns created using the normal split algorithm. Blobs created below split. Create the data base on server

74. ROOT: Status & Developments74 CINT Masa Goto is now releasing a new version of CINT with better C++ support, in particular for namespaces and scoping rules. Only the byte code option supported

75. ROOT: Status & Developments75 ROOT and the LCG projects Applications manager Architects forum Applications area meeting Simulation project PI project SEAL project POOL project SPI project decisions strategy consultation ROOT User - provider LCG & ROOT in 2002,2003,2004 But substantial changes this month….

76. ROOT: Status & Developments76 PH/SFT group reorganisation VMC G4 GRID catalogs collections storage dict pgm POOL ROOT SEAL SIMU Merged dict and pgm mathlib move to use New SFT generators

77. ROOT: Status & Developments77 Final remarks 1965: CERNLIB 1975: HBOOK and other packages 1985: start of PAW project 1995: start of ROOT project 2005: ROOT & LCG common developments 2015: ??

78. ROOT: Status & Developments78 ROOT: 2005-  20XX Strong support from CERN/FNAL System will be consolidated, continuously developed. Better and better UI and GUIs, including web. More and more support for parallel distributed computing. What about C++ ?