Persistent Object References in ROOT1 Persistent Object References in ROOT I/O Status & Proposal CMS-ROOT meeting CERN- November 27 Ren é Brun ftp://root.cern.ch/root/refs.ppt

CMS 27 Nov Rene BrunPersistent Object References in ROOT2 Plan of talk Persistent C++ pointers Using TRef & TRefArray Action on Demand Combining TRef & Action on Demand Proposal for changes & extensions

CMS 27 Nov Rene BrunPersistent Object References in ROOT3 Normal Streaming mode References using C++ pointers A TBuffer b; A.Streamer(b) Only one copy of each object in the graph saved to buffer

CMS 27 Nov Rene BrunPersistent Object References in ROOT4 Normal Streaming mode References using C++ pointers TBuffer b1; A.Streamer(b1) TBuffer b2; B.Streamer(b2) B A Objects in red are in b1 and b2 C++ pointer

CMS 27 Nov Rene BrunPersistent Object References in ROOT5 Normal Streaming mode References using TRef pointers TBuffer b1; A.Streamer(b1) TBuffer b2; B.Streamer(b2) B Objects in blue are only in b1 C++ pointer TRef A Bz z Set pointer to z with: TRef Bz = z; Get pointer to z with: z = Bz.GetObject()

CMS 27 Nov Rene BrunPersistent Object References in ROOT6 Setting a TRef pointer Assuming obj = pointer to a TObject* TRef ref = obj; TRef is itself a TObject Its fUniqueID is set to obj - gSystem The obj kIsReferenced bit is set (fBits of obj) Get pointer obj with obj = ref.GetObject() returns fUniqueID + gSystem Class TRef : public TObject { TProcessID *fPID; //!pointer to process id Class TObject { unsigned int fBits; unsigned int fUniqueID;

CMS 27 Nov Rene BrunPersistent Object References in ROOT7 Writing TRefs to a buffer A TRef is written by TRef::Streamer Writes uid(8 bytes) + pid(4 bytes) 12 bytes in non compressed mode 2.4 bytes in compressed mode 1 (default) uid = object unique identifier default uid = pointer - gSystem (see proposal) pid = Process identifier Each process has a unique pid (TProcessID) A file contains the pids of all processes that have written objects to it.

CMS 27 Nov Rene BrunPersistent Object References in ROOT8 Writing Referenced objects A referenced object is written by obj->Streamer This Streamer at some point calls its TObject::Streamer In TObject::Streamer, if the kIsReferenced bit is set in fBits, the following additional info is also written: uid (8 bytes) = obj - gSystem pid (4 bytes) = TProcessID of current process A Referenced object may be written multiple times in the same file as the TRef or in other files

CMS 27 Nov Rene BrunPersistent Object References in ROOT9 Reading Referenced objects A referenced object is read by obj->Streamer This Streamer at some point calls its TObject::Streamer In TObject::Streamer, if the kIsReferenced bit is set in fBits, the following additional info is also read: uid (8 bytes) = obj - gSystem pid (4 bytes) = TProcessID of current process the fUniqueID is set to uid The pair (uid,obj) is added to the TExMap of the TProcessID corresponding to pid When obj is deleted, its pair (uid,obj) is also removed from the TProcessID TExMap.

CMS 27 Nov Rene BrunPersistent Object References in ROOT10 Reading TRefs from a buffer A TRef object is read by TRef::Streamer The pair uid,pid is read the fUniqueID of TRef is set to uid The transient pointer fPID is set to the TProcessID corresponding to pid via a direct access table in the gROOT object The bit 1 of fBits is set

CMS 27 Nov Rene BrunPersistent Object References in ROOT11 Using a TRef To get a pointer to the referenced object, do: Myclass *obj = (Myclass*)ref.GetObject() GetObject fBits[1] =0Returns obj = fUniqueID + gSystem =1 NO Set fBits[1]=0 return obj YESexecid = fBits[8/8] execid = 0 YES return null NO Execute TExec with execid Obj = 0 YES fPID = 0 or fPID>GetObjectWithUniqueID() = 0

CMS 27 Nov Rene BrunPersistent Object References in ROOT12 TRef example: Event.h class Event : public TObject { private: char fType[20]; //event type char *fEventName; //run+event number in character format int fNtrack; //Number of tracks int fNseg; //Number of track segments int fNvertex; int fMeasures[10]; float fMatrix[4][4]; float *fClosestDistance; //[fNvertex] EventHeader fEvtHdr; TClonesArray *fTracks; //->array with all tracks TRefArray *fHighPt; //array of High Pt tracks only TRefArray *fMuons; //array of Muon tracks only TRef fLastTrack; //reference pointer to last track TRef fWebHistogram; //EXEC:GetWebHistogram TH1F *fH; //-> public:... TH1F *GetHistogram() const {return fH;} TH1F *GetWebHistogram(Bool_t reload=kFALSE) const { return (TH1F*)fWebHistogram.GetObject(reload);}

CMS 27 Nov Rene BrunPersistent Object References in ROOT13 TRef & Action on Demand When the keyword “EXEC:” is found in the comments of the data member as in: TRef fWebHistogram; //EXEC:GetWebHistogram The information in the comment field is kept in the dictionary. Execid is saved in TStreamerElement When the TRef object is read, the execid is stored in the fBits on one byte (from TStreamerElement). When TRef::GetObject is called, TObjArray *lexecs = gROOT->GetListOfExecs(); TExec *exec = (TExec*)lexecs[execid]; exec->Exec(); fWebHistogram.GetObject() executes the action GetWebHistogram Action on Demand is Persistent

CMS 27 Nov Rene BrunPersistent Object References in ROOT14 What a TExec can do TExec is a CORE ROOT class that can be used to execute: a call to a compiled or interpreted function example: Exec:LoadHits() an interpreted script example: Exec:GetWebHistogram If GetWebHistogram is not a function (compiled or interpreted), then TExec::Exec will try to execute the script GetWebHistogram.C void GetWebHistogram(){ // example of script called from an Action on Demand when a TRef object // is dereferenced. See Event.h, member fWebHistogram const char *URL = " printf("GetWebHistogram from URL: %s\n",URL); TFile *f= TFile::Open(URL); f->cd("DM/CJ"); TH1 *h6 = (TH1*)gDirectory->Get("h6"); h6->SetDirectory(0); delete f; gROOT->SetSelectedPrimitive(h6); }

CMS 27 Nov Rene BrunPersistent Object References in ROOT15 A Working Example { gSystem.Load("libEvent"); TFile f("Event.root"); Event *event=0; T.SetBranchAddress("event",&event); T.GetEntry(45); event->GetWebHistogram()->Draw(); } void GetWebHistogram(){ // example of script called from an Action on Demand when a TRef object // is dereferenced. See Event.h, member fWebHistogram const char *URL = " printf("GetWebHistogram from URL: %s\n",URL); TFile *f= TFile::Open(URL); f->cd("DM/CJ"); TH1 *h6 = (TH1*)gDirectory->Get("h6"); h6->SetDirectory(0); delete f; gROOT->SetSelectedPrimitive(h6); } Action.C GetWebHistogram.C Root >.x Action.C

CMS 27 Nov Rene BrunPersistent Object References in ROOT16 Status with Persistent pointers C++ persistent pointers In all versions of ROOT since day1 TRef, TRefArray implemented in dev version 3.02/00 TRef with Action on Demand implemented in 3.02/05 (this week) TRef simplification & extensions see proposal

CMS 27 Nov Rene BrunPersistent Object References in ROOT17 Simplification Instead of storing obj-gSystem as unique id for a given process id, I propose to store the object number within a process. This is a trivial change to the current scheme in dev. Many advantages: simpler to manage no TExMap, but a simple TObjArray with direct access for each TProcessID One could imagine a very simple object catalog in the RDBMS Disadvantages; Require a table of pointers in the process writing the objects This problem could be solved by an Wipe_Event function

CMS 27 Nov Rene BrunPersistent Object References in ROOT18 Setting a TRef pointer Assuming obj = pointer to a TObject* TRef ref = obj; TRef is itself a TObject If the obj::kIsReferenced bit is not yet set, the obj::fUniqueID is set to the CurrentNumber+1 and obj::kIsReferenced is set to 1. Its fUniqueID is set to obj::fUniqueID CurrentNumber is managed by TProcessID fObjs[CurrentNumber] is set to obj in fPID Class TRef : public TObject { TProcessID *fPID; //!pointer to process id Class TObject { unsigned int fBits; unsigned int fUniqueID;

CMS 27 Nov Rene BrunPersistent Object References in ROOT19 Writing TRefs to a buffer A TRef is written by TRef::Streamer Writes uid(4 instead of 8 bytes) + pid(4 bytes) uid = object unique identifier uid = ref::fUniqueID = obj::fUniqueID = current object nr pid = Process identifier Each process has a unique pid (TProcessID) A file contains the pids of all processes that have written objects to it.

CMS 27 Nov Rene BrunPersistent Object References in ROOT20 Writing Referenced objects A referenced object is written by obj->Streamer This Streamer at some point calls its TObject::Streamer In TObject::Streamer, if the kIsReferenced bit is set in fBits, the following additional info is also written: uid (8 bytes) = obj - gSystem already in obj::fUniqueID pid (4 bytes) = TProcessID of current process A Referenced object may be written multiple times in the same file as the TRef or in other files

CMS 27 Nov Rene BrunPersistent Object References in ROOT21 Reading Referenced objects A referenced object is read by obj->Streamer This Streamer at some point calls its TObject::Streamer In TObject::Streamer, if the kIsReferenced bit is set in fBits, the following additional info is also read: uid (8 bytes) = obj - gSystem pid (4 bytes) = TProcessID of current process In TProcessID::fObjs fObjs[fUniqueID] = obj When obj is deleted, fObjs[fUniqueID] = 0;

CMS 27 Nov Rene BrunPersistent Object References in ROOT22 Reading TRefs from a buffer A TRef object is read by TRef::Streamer The pair uid,pid is read the fUniqueID of TRef is set to uid The transient pointer fPID is set to the TProcessID corresponding to pid via a direct access table in the gROOT object The bit 1 of fBits is set