1. The COMPASS event store in 2002
CHEP03 - March
The COMPASS event store in 2002
Presented by Vinicio Duic
Massimo Lamanna
IT Division CERN - CH
On leave from INFN Trieste - Italy
Vinicio Duic
INFN Trieste - Italy

2. Summary COMPASS experiment Off-Line System Overview Data Model
Adopted Data Storage Strategies
2002 Data Taking
2002 Data Storage & Processing
Conclusions
CHEP03 - March
M. Lamanna & V. Duic

3. Fixed-target experiment at CERN SPS
COMPASS Experiment (Common Muon-Proton Apparatus for Structure & Spectroscopy)
Fixed-target experiment at CERN SPS
Broad physics programme (m & p beams) Different apparatus set-ups High intensities and trigger rates Large accumulated data sets
1010 events/y ´ 30 kB/event = 300 TB/y
High data-acquisition rate (35 MB/s)
Design figures
CHEP03 - March
M. Lamanna & V. Duic

4. Centralised Data Storage
The off-line system: choices
Storage
High data flux from on-line
Huge amount of data (300 TB/y, many years of data taking)
Continuous access from collaboration institutes
Centralised Data Storage
(transfer from exp. hall to tape drives -9540a in 2002, 9540b in in the computing centre through dedicated optic fibre)
Processing
Long period data taking Þ quasi on-line reconstruction
Complex reconstruction Þ ~20k SI2000 (i.p.40k req’d)
Wide physics programme Þ flexibility
Design figures
FARM of PC’s
(need parallel reconstruction to get high performance/cost)
CHEP03 - March
M. Lamanna & V. Duic

5. The off-line system: choices
Reconstruction
Reconstruction complexity
OOP
(C++)
Flexibility
Code handling
CORAL (COmpass Reconstruction and AnaLysis)
base components
abstract interface to event-reconstruction components
abstract interfaces to external packages
in particular to access the data (DB, online files, …)
components for reconstruction
interchangeable, to suit reconstruction to different needs
CHEP03 - March
M. Lamanna & V. Duic

6. Data Model
Collected data are shaped into objects (C++), stored hierarchically by Objectivity/DB
Run DB
DST DBs
Disc Resident (metadata)
Held on tert. MS
Event DB
RAW DBs
CHEP03 - March
M. Lamanna & V. Duic

7. Data Storage: CDR Design speed: 35 MB/s
Data-Servers (Up to 20 X 0.5 TB)
Thanks B. Gobbo INFN Trieste - It
On-line PC’s (3.5 TB)
CASTOR
Execution Hosts
(400)
IN COMPASS has been CASTOR (heavy ) user No 1
CHEP03 - March
M. Lamanna & V. Duic

8. Data taking 2002 4.5 TB! Design rate (35 MB/s) Collected sample
Data (TB)
Design rate
(35 MB/s)
Time (days)
Collected sample
(270 TB)
In ~100 days
of DAQ
Data (TB)
CHEP03 - March
M. Lamanna & V. Duic

9. At-peak: Network Activityb a c
Traffic to tape a c b
(Same time windows)
CHEP03 - March
M. Lamanna & V. Duic

10. At-peak: DB Input Activity
Traffic to tape b
(Same time windows)
CHEP03 - March
M. Lamanna & V. Duic

11. Objectivity/DB pro’s & con’s
Logical/physical layer separation independent from HSM details
Clean integration w/t HSM
Client driven
Nice performances
High concurrency in production (> 400 clients)
Weak on client abort
Oocleanup sometimes tricky
Poor flexibility for read locks (100 clients)
LockServer, AMS (network server): 1 box ´ process
CHEP03 - March
M. Lamanna & V. Duic

12. Migration to Oracle
CERN Contract with Objectivity terminated; they have chosen Oracle as new DBMS
PAST (Objectivity/DB 6.1.3)
In the ODBMS:
Metadata & Associations
Raw Data
Reconstructed Events
Conditions
In plain files:
DB overhead
~30% of Raw Data size
(turns into ~6% on tape)
FUTURE (ORACLE 9i)
In the ORDBMS:
Relations (metadata)
In plain files:
Raw Data
Reconstructed Events
Conditions
DB overhead:
~0.3% of Raw Data size
CHEP03 - March
M. Lamanna & V. Duic

13. Oracle vs Objectivity COMPASS only has to get » 35 MB/s
CHEP03 - March
M. Lamanna & V. Duic

14. 2002 Data Production DST on 2001 data (103 GB) with OBJY
DST on 2002 data with OBJY
Big; i/o size:
80 TB RAWs processed (multiple processing)
5.7 TB DSTs produced
DST on 2002 data with ORACLE
Just started (yet 1.3 TB DSTs produced)
Incremental changes on DST formats + CORAL tuning (algorithm, calibrations,…)
CHEP03 - March
M. Lamanna & V. Duic

15. Oracle Production Input: Output: 17 TB RAWs 1.3 TB DSTs
(8% of total)
Feb. - Mar. 2003
CHEP03 - March
M. Lamanna & V. Duic

16. Batch CPU Load at CERN COMPASS 320 280 240 200 160 120 80 40 127 131
2002
2003
DAQ*
Prod.
Prod.
320
280
240
200
160
120 80 40
127
131
135
139
143
147
151
203
207
211
215
219
223
227
231
235
239
243
247
251
303
307
311
COMPASS
CPU Time (NCU khrs)
Objectivity/DB
Oracle
* CDR does not use the batch CPU
Week
CHEP03 - March
M. Lamanna & V. Duic

17. COMPASS experience Usage of very different technologies
Objectivity/DB
Oracle
Continuity in data access
Only new I/O plug-in (CORAL good design!)
Huge data migration (see M. Nowak’s talk)
Very large data rate and data sizes
Comparable to one LHC experiment
Heavy duty usage of CASTOR in production
CHEP03 - March
M. Lamanna & V. Duic