ROOT: Status & Developments1 Aachen 1 February 2005 Ren The ROOT Project Status & Developments

ROOT: Status & Developments2 ROOT in a nutshell An efficient data storage and access system designed to support structured data sets in very large distributed data bases (Petabytes). A query system to extract information from these distributed data sets. The query system is able to use transparently parallel systems on the GRID (PROOF). A scientific visualisation system with 2-D and 3-D graphics. An advanced Graphical User Interface A C++ interpreter allowing calls to user defined classes. An Open Source Project

ROOT: Status & Developments3 ROOT: An Open Source Project The project is developed as a collaboration between : Full time developers: 6 people full time at CERN 1 core developer at FermiLab 1 core developer in Japan (Agilent Technologies) 1 core developer at MIT 1 mathematician at CERN sponsored by a US Finance Company Many contributors spending a substantial fraction of their time in specific areas (> 50). Key developers in large experiments using ROOT as a framework. Several thousand users given feedback and a very long list of small contributions.

ROOT: Status & Developments4 Project History Jan 95: Thinking/writing/rewriting/??? November 95: Public seminar, show Root 0.5 Spring 96: decision to use CINT Jan 97: Root version 1.0 Jan 98: Root version 2.0 Mar 99: Root version 2.21/08 (1st Root workshop FNAL) Feb 00: Root version 2.23/12 (2nd Root workshop CERN) Mar 01: Root version 3.00/06 Jun 01: Root version 3.01/05 (3rd Root workshop FNAL) Jan 02: Root version 3.02/07 (LCG project starts: RTAGs) Oct 02: Root version 3.03/09 (4th Root workshop CERN) Feb 04: version 4.00 (5 th Root workshop SLAC) Dec 04: version 4.02 (current stable version) Feb 05: Developing 4.04 to be released in June (LCG/ROOT merge) 10 years !!

ROOT: Status & Developments5 The ROOT web pages  General Information and News  Download source and binaries  Howto & tutorials  User Guide & Reference Guides  Roottalk Digest & Forum

ROOT: Status & Developments6 Current ROOT structure & libs

ROOT: Status & Developments7 CINT: the C++ interpreter

ROOT: Status & Developments8 My first session root [0] (const double) e+002 root [1] float x=89.7; root [2] float y=567.8; root [3] x+sqrt(y) (double) e+002 root [4] float z = x+2*sqrt(y/6); root [5] z (float) e+002 root [6].q root root [0] try up and down arrows See file $HOME/.root_hist

ROOT: Status & Developments9 My second session root [0].x session2.C for N=100000, sum= root [1] sum (double) e+004 Root [2] r.Rndm() (Double_t) e-001 root [3].q root { int N = ; TRandom r; double sum = 0; for (int i=0;i<N;i++) { sum += sin(r.Rndm()); } printf("for N=%d, sum= %g\n",N,sum); } session2.C unnamed macro executes in global scope

ROOT: Status & Developments10 My third session root [0].x session3.C for N=100000, sum= root [1] sum Error: Symbol sum is not defined in current scope *** Interpreter error recovered *** Root [2].x session3.C(1000) for N=1000, sum= root [3].q root void session3 (int N=100000) { TRandom r; double sum = 0; for (int i=0;i<N;i++) { sum += sin(r.Rndm()); } printf("for N=%d, sum= %g\n",N,sum); } session3.C Named macro Normal C++ scope rules

ROOT: Status & Developments11 My third session with ACLIC root [0] gROOT->Time(); root [1].x session4.C( ) for N= , sum= e+006 Real time 0:00:06, CP time root [2].x session4.C+( ) for N= , sum= e+006 Real time 0:00:09, CP time root [3] session4( ) for N= , sum= e+006 Real time 0:00:01, CP time root [4].q #include “TRandom.h” void session4 (int N) { TRandom r; double sum = 0; for (int i=0;i<N;i++) { sum += sin(r.Rndm()); } printf("for N=%d, sum= %g\n",N,sum); } session4.C File session4.C Automatically compiled and linked by the native compiler. Must be C++ compliant

ROOT: Status & Developments12 Macros with more than one function root [0].x session5.C >session5.log root [1].q void session5(int N=100) { session5a(N); session5b(N); gROOT->ProcessLine(“.x session4.C+(1000)”); } void session5a(int N) { for (int i=0;i<N;i++) { printf("sqrt(%d) = %g\n",i,sqrt(i)); } void session5b(int N) { double sum = 0; for (int i=0;i<N;i++) { sum += i; printf("sum(%d) = %g\n",i,sum); } session5.C.x session5.C executes the function session5 in session5.C root [0].L session5.C root [1] session5(100); >session5.log root [2] session5b(3) sum(0) = 0 sum(1) = 1 sum(2) = 3 root [3].q use gROOT->ProcessLine to execute a macro from a macro or from compiled code

ROOT: Status & Developments13 Combining UI and GUI root [0].x session2.C for N=100000, sum= root [1] sum (double) e+004 Root [2] r.Rndm() (Double_t) e-001 root [3].q

ROOT: Status & Developments14 The Histogram Package

ROOT: Status & Developments15 The Histogram Classes Structure 1-Dim 2-Dim 3-Dim

ROOT: Status & Developments16 Histograms class THStack Long list of functions in TH1 Plenty of drawing options Filling with string variables Automatic binning TH1::SetBuffer TH1::GetRandom h3 = 3*h1 – h2

ROOT: Status & Developments17 Filling with strings See tutorials -hlabels1.C -hlabels2.C -cernstaff.C

ROOT: Status & Developments18 Peak Finder + Deconvolutions by Miroslav Morach TSpectrum

ROOT: Status & Developments19 Fitting TVirtualFitter TFitter TMinuit TH1::Fit bin chisquare bin likelihood TGraph::Fit unbinned chisquare User TTree::Fit unbinned likelihood TFumili

ROOT: Status & Developments20 Graphics

ROOT: Status & Developments21 Gui/Graphics strategy TVirtualX TGWin32TGX11TGWin32GDKTGQt TVirtualPad TPad User/Root GUI and Graphics classes Applications see only Abstract Interfaces Low level screen graphics and GUI High level pad graphics TG??

ROOT: Status & Developments22 A Data Analysis & Visualisation tool

ROOT: Status & Developments23 Graphics : 1,2,3-D functions

ROOT: Status & Developments24 Full LateX support on screen and postscript TCurlyArc TCurlyLine TWavyLine and other building blocks for Feynmann diagrams Formula or diagrams can be edited with the mouse

R.Brun 1 Feb 2005ROOT: Status & Developments ROOT 3D Graphics Simple Box The ROOT basic 3D shapes

ROOT: Status & Developments26 Alice 3 million nodes

R.Brun 1 Feb 2005ROOT: Status & Developments R. Brun (CERN), O. Couet (CERN), M. Gheata (ISS), A. Gheata (ISS), V. Onoutchine (IFVE), T.Pocheptsov (JINR) ROOT 3D Graphics Text ………………… Atlas

ROOT: Status & Developments28 GUI

ROOT: Status & Developments29 Available Widgets Complete set of widgets: label, icon, button, check button, radio button, picture button, button box, list box, combo box, list view, icon view, number entry, text entry, text view, text edit, tree view, tab view, scrollbar, slider, menubar, popup menu, cascading menu, statusbar, toolbar, message dialogs, file selection dialog, progress bars, tooltips,...

ROOT: Status & Developments30 Basic Widgets

ROOT: Status & Developments31 Advanced Widgets Color selector dialog: TGColorDialog

ROOT: Status & Developments32 GUI Examples

ROOT: Status & Developments33 GUI Examples – Histogram Browser

ROOT: Status & Developments34 More GUI Examples – Period System

ROOT: Status & Developments35 Object Editors

ROOT: Status & Developments36 Histogram Editor

ROOT: Status & Developments37 Object Editors (rebinning)

ROOT: Status & Developments38 Input/Output

ROOT: Status & Developments39 Evolution of ROOT I/O Hand-written Streamers Streamers generated via rootcint Support for Class Versions Support for ByteCount Persistent class Dictionary written to files rootcint modified to generate automatic Streamers Support for Automatic Schema Evolution Can generate code for “DataObjects” classes in a file Support for complex C++ cases Can read files without the classes Persistent Reference pointers Support for foreign classes Full support for STL

ROOT: Status & Developments40 Object Persistency (in a nutshell) Two I/O modes supported (Keys and Trees). Key access: simple object streaming mode. A ROOT file is like a Unix directory tree Very convenient for objects like histograms, geometries, mag.field, calibrations Trees A generalization of ntuples to objects Designed for storing events split and no split modes query processor Chains: Collections of files containing Trees ROOT files are self-describing Interfaces with RDBMS also available Access to remote files (RFIO, DCACHE, GRID)

ROOT: Status & Developments41 ROOT I/O : An Example TFile f(“example.root”,”new”); TH1F h(“h”,”My histogram”,100,-3,3); h.FillRandom(“gaus”,5000); h.Write(); TFile f(“example.root”); TH1F *h = (TH1F*)f.Get(“h”): h->Draw(); f.Map(); Program Writing Program Reading / At:64 N=90 TFile / At:154 N=453 TH1F CX = / At:607 N=2364 StreamerInfo CX = / At:2971 N=96 KeysList / At:3067 N=56 FreeSegments / At:3123 N=1 END demoh.C demohr.C

ROOT: Status & Developments42 A Root file pippa.root with two levels of directories Objects in directory /pippa/DM/CJ eg: /pippa/DM/CJ/h15

ROOT: Status & Developments43 Self-describing files Dictionary for persistent classes written to the file when closing the file. ROOT files can be read by foreign readers (eg JavaRoot (Tony Johnson) Support for Backward and Forward compatibility Files created in 2003 must be readable in 2015 Classes (data objects) for all objects in a file can be regenerated via TFile::MakeProject Root >TFile f(“demo.root”); Root > f.MakeProject(“dir”,”*”,”new++”);

ROOT: Status & Developments44 Automatic Schema Evolution

ROOT: Status & Developments45 Auto Schema Evolution (2)

ROOT: Status & Developments46 File types & Access in 4.02/xx Local File X.xml RFIOChirp CastorDcache Local File X.root http rootd/xrootd Oracle SapDb PgSQL MySQL TFile TKey/TTree TStreamerInfo user TSQLServer TSQLRow TSQLResult TTreeSQL

ROOT: Status & Developments47 Memory Tree Each Node is a branch in the Tree tr T.Fill() T.GetEntry(6) T Memory

ROOT: Status & Developments48 Tree example Event (write) void demoe(int nevents) { //load shared lib with the Event class gSystem->Load("$ROOTSYS/test/libEvent"); //create a new ROOT file TFile f("demoe.root",”new"); //Create a ROOT Tree with one single top level branch int split = 99; //try also split=1 and split=0 int bufsize = 16000; Event *event = new Event; TTree T("T","Event demo tree"); T.Branch("event","Event",&event,bufsize,split); //Build Event in a loop and fill the Tree for (int i=0;i<nevents;i++) { event->Build(i); T.Fill(); } T.Print(); //Print Tree statistics T.Write(); //Write Tree header to the file } All the examples can be executed with CINT or the compiler root >.x demoe.C root >.x demoe.C++

ROOT: Status & Developments49 Tree example Event (read 1) void demoer() { //load shared lib with the Event class gSystem->Load("$ROOTSYS/test/libEvent"); //connect ROOT file TFile *f = new TFile("demoe.root"); //Read Tree header and set top branch address Event *event = 0; TTree *T = (TTree*)f->Get("T"); T->SetBranchAddress("event",&event); //Loop on events and fill an histogram TH1F *h = new TH1F("hntrack","Number of tracks",100,580,620); int nevents = (int)T->GetEntries(); for (int i=0;i<nevents;i++) { T->GetEntry(i); h->Fill(event->GetNtrack()); } h->Draw(); } Rebuild the full event in memory

ROOT: Status & Developments50 8 Branches of T 8 leaves of branch Electrons A double-click to histogram the leaf

ROOT: Status & Developments51 The Tree Viewer & Analyzer A very powerful class supporting complex cuts, event lists, 1-d,2-d, 3-d views parallelism

ROOT: Status & Developments52 Tree Friends tr Public read Public read User Write Entry # 8

ROOT: Status & Developments53 Tree Friends Root > TFile f1(“tree1.root”); Root > tree.AddFriend(“tree2”,“tree2.root”) Root > tree.AddFriend(“tree3”,“tree3.root”); Root > tree.Draw(“x:a”,”k<c”); Root > tree.Draw(“x:tree2.x”,”sqrt(p)<b”); x Processing time independent of the number of friends unlike table joins in RDBMS Collaboration-wide public read Analysis group protected user private

ROOT: Status & Developments54 PROOF and GRID(s) interface

ROOT: Status & Developments55 Batch/Interactive models Batch Production Simulation reconstruction Interactiv e Chaotic analysis Interactiv e batch model Need experiment framework +widely available tools Need only widely available tools

ROOT: Status & Developments56 GRID: Interactive Analysis Case 1 Data transfer to user’s laptop Optional Run/File catalog Optional GRID software Optional run/File Catalog Remote file server eg rootd Trees Analysis scripts are interpreted or compiled on the local machine

ROOT: Status & Developments57 GRID: Interactive Analysis Case 2 Remote data processing Optional Run/File catalog Optional GRID software Optional run/File Catalog Remote data analyzer eg proofd Trees Commands, scripts histograms Analysis scripts are interpreted or compiled on the remote machine

ROOT: Status & Developments58 GRID: Interactive Analysis Case 3 Remote data processing Run/File catalog Full GRID software Run/File Catalog Remote data analyzer eg proofd Trees Commands, scripts Histograms,trees Trees slave Analysis scripts are interpreted or compiled on the remote master(s)

ROOT: Status & Developments59 Parallel ROOT Facility The PROOF system allows: Parallel analysis of trees in a set of files Parallel analysis of objects in a set of files Parallel execution of scripts on clusters of heterogeneous machines Its design goals are: Transparency, scalability, adaptability Prototype developed in 1997 as proof of concept, full version nearing completion now

ROOT: Status & Developments60 Parallel Script Execution root Remote PROOF Cluster proof TNetFile TFile Local PC $ root ana.C stdout/obj node1 node2 node3 node4 $ root root [0].x ana.C $ root root [0].x ana.C root [1] gROOT->Proof(“remote”) $ root root [0] tree->Process(“ana.C”) root [1] gROOT->Proof(“remote”) root [2] chain->Process(“ana.C”) ana.C proof proof = slave server proof proof = master server #proof.conf slave node1 slave node2 slave node3 slave node4 *.root TFile

ROOT: Status & Developments61 Google: a good model Make it simple Simple interface Available everywhere Hidden Parallelism Distributed DB Don’t know a priori the data location Fast

ROOT: Status & Developments62 I did not talk about

ROOT: Status & Developments63 Currently in ROOT 4.02 SQL interface (MySQL, Oracle, postgres,sapdb) Threads Networking Mass Storage interfaces (Dcache,Castor,Chirp..) Mathlibs, Physics vectors Linear Algebra The geometry package Virtual MonteCarlo Interfaces to event generators The GUI Builder Neural Network Image processing PyRoot (The ROOT/Python interface) Carrot: ROOT web interface

ROOT: Status & Developments64 Current developments (4.03/4.04) (in June release)

ROOT: Status & Developments65 Consolidation Robustness Zillions of tiny details User support

ROOT: Status & Developments66 OpenGL interface A lot of effort invested in the GL interface Real Time navigation in complex detectors with millions of physical volumes. Show only what makes sense Invisible, translucent, variable precision rendering Dynamic particles High quality output GUI

ROOT: Status & Developments67 Mathlib Consolidation of TMath (now a namespace) Improved algorithms Robust Fitters (from the R package) Outliers, cluster finders Interface with a subset of GSL Collaboration with SEAL/Math

ROOT: Status & Developments68 Image processing Collaboration with Sasha Vasko for the development of libAfterImage Fast graphics (also in batch): an alternative to X11. Will be used by Carrot for execution of remote scripts and fast interactive graphics on the web. Could be used for fast image manipulation (better than large scatter plots)

ROOT: Status & Developments69 xrootd Joint development with BaBar Robust and efficient distributed file server with load balancing. Could be integrated with Castor. Integration with PROOF Requires consolidation of authentication and security. Implementation of read-ahead for ROOT Trees.

ROOT: Status & Developments70 PROOF Add support for all features currently available in ROOT Trees (friends, event lists, selectors) Special (but growing case) of one client, one server with no slaves Stateless connections with multiple PROOF objects in the same ROOT session (long and short queries). View/Import results of batch queries Save ROOT/PROOF session with resume capabilities. Interfaces with GRID services.

ROOT: Status & Developments71 GUI Continuous improvements. More & more objects edited via the GUI. GUI objects persistent. Currently the GUI can be saved as a C++ script. We would like to save it as a canvas.root too. More features in the GUI Builder (+ doc)

ROOT: Status & Developments72 SQL Interface RDBMS access via a set of abstract base classes TSQLServer, TSQLResult and TSQLRow Concrete implementations for MySQL and Oracle exist TMySQLServer, TMySQLResult and TMySQLRow TOracleServer, TOracleResult and TOracleRow

ROOT: Status & Developments73 New TTreeSQL class TTreeSQL tree(const char *db,const char* uid,…) tree.Print(), Browse, Scan, etc tree.Draw(“var1:var2”,”varx <0”) TTree query style converted to SQL Connect to an existing db TTreeSQL tree(“mysql://localhost/test”,”nobody”,”new”); Event *event = new Event; tree.Branch(“top”,”Event”,&event); tree.Fill(); tree.AutoSave(); A TSQLRow is filled and sent to the server Columns created using the normal split algorithm. Blobs created below split. Create the data base on server

ROOT: Status & Developments74 CINT Masa Goto is now releasing a new version of CINT with better C++ support, in particular for namespaces and scoping rules. Only the byte code option supported

ROOT: Status & Developments75 ROOT and the LCG projects Applications manager Architects forum Applications area meeting Simulation project PI project SEAL project POOL project SPI project decisions strategy consultation ROOT User - provider LCG & ROOT in 2002,2003,2004 But substantial changes this month….

ROOT: Status & Developments76 PH/SFT group reorganisation VMC G4 GRID catalogs collections storage dict pgm POOL ROOT SEAL SIMU Merged dict and pgm mathlib move to use New SFT generators

ROOT: Status & Developments77 Final remarks 1965: CERNLIB 1975: HBOOK and other packages 1985: start of PAW project 1995: start of ROOT project 2005: ROOT & LCG common developments 2015: ??

ROOT: Status & Developments78 ROOT:  20XX Strong support from CERN/FNAL System will be consolidated, continuously developed. Better and better UI and GUIs, including web. More and more support for parallel distributed computing. What about C++ ?