1. Distributing Data around the BaBar collaboration’s Objectivity Federations Emanuele Leonardi - I.N.F.N. Roma Simon Patton - L.B.N.L. on behalf of the BaBar Data Distribution group CHEP2000 Padova, February 7-11, 2000

2. 9/2/2000E.Leonardi, S.Patton - CHEP20002 Objectivity Federations (FD) zOn-site yDAQ: IR2Boot yPrompt Reco: ProdBoot yAnalysis: AnalBoot yReprocessing: RecoBoot yMC Production: SP2ProdBoot yMC Analysis: SP2AnalBoot z Off-site yBaBar Regional Centers (CCIN2P3, INFN, RAL) copy (part of) AnalBoot and SP2AnalBoot yMC Production Centers write data to SP2ProdBoot How do we transfer data among these FD’s? How do we ensure consistency in the data?

3. 9/2/2000E.Leonardi, S.Patton - CHEP20003 Data Transfer zA simple file transfer is not possible yFD must be “told” about the presence of new data yFD structure must be updated zAd hoc tools (BdbDistTools) were developed yBased on BaBar database structure and organization yAssume total correspondence between FD’s DB Id’s (see later) yBy-pass standard Objectivity DB administration tools yOnly bulk data transfer for the moment

4. 9/2/2000E.Leonardi, S.Patton - CHEP20004 Data Export zSelect data formats to export (AOD, ESD, …) zCheck for modified/new DB files zExport these files (by-pass oocopydb: too slow) yLock full DB yCopy file to stage-out area yUnlock DB zGet DB ID’s corresponding to exported files zCreate TDF (Transfer Description File) with all info needed to re-attach the DB yDB file name, DB Id, Size,...

5. 9/2/2000E.Leonardi, S.Patton - CHEP20005 Data Import zCopy all exported files (network or tape) to stage-in area zUse TDF to identify each file zCopy files to correct position in FD structure yDepends on local FD lay-out zAttach files to FD yooattach makes a consistency check but it is VERY SLOW, so... ycreate empty DB file with correct name and position yooattach the empty file (VERY FAST) ycopy the real DB over the empty one

6. 9/2/2000E.Leonardi, S.Patton - CHEP20006 DBID Allocation zA “FD family” is a group of FD’s sharing part of the data yIR2Boot, ProdBoot, RecoBoot, AnalBoot, “off-site”-AnalBoot ySp2ProdBoot, SP2AnalBoot, “off-site”-SP2 FD’s zTo avoid overwriting of shared DB’s, each FD contains: yA “Global DB” area xall DB’s which may be shared with other FD’s in the same family yA “Local DB” area xall DB’s which are unique to that FD. zEach FD has write access to the Local area and to a preallocated portion of the Global area

7. 9/2/2000E.Leonardi, S.Patton - CHEP20007 DB Id’s allocation schema DB Id’s Local DB Global DB ProdBoot AnalBoot INFN-AnalBoot Reco Data Part of Reco Data Production Skims User Skims Off-line Processing Off-site User Processing ProdBoot Area AnalBoot Area INFN Area

8. 9/2/2000E.Leonardi, S.Patton - CHEP20008 Conclusions zUse of multiple Objectivity FD’s in different sites requires ad hoc tools to move data around. zObjectivity Administration tools are usually too slow: BaBar developed its own set of utilities to handle data distribution. These tools assume a complete correspondence between FD’s structures. zA careful management of DB Id’s is mandatory in order to avoid DB overwriting during data distribution.