LHCb File-Metadata: Bookkeeping Carmine Cioffi Department of Physics, Oxford University UK Metadata Workshop Oxford, 04 July 2006

UK Metadata Workshop Oxford 4 July 2006 LHCb Metadata 2 Outline File-Metadata. The metadata’s Logical Data Model. LHCb Services. DB schema. Some Numbers. Conclusions.

UK Metadata Workshop Oxford 4 July 2006 LHCb Metadata 3 File-Metadata LHCb store only data that describes and gives information about data-files (File- Metadata) The File-Metadata are divided in two groups: –Job provenance: Information about how a data-file was created –Bookkeeping: All the information about Files and Jobs.

UK Metadata Workshop Oxford 4 July 2006 LHCb Metadata 4 The Metadata’s Logical Data Model

UK Metadata Workshop Oxford 4 July 2006 LHCb Metadata 5 The Metadata’s Logical Data Model The main subjects of the Logical Data Model are the Jobs and the Files. Files and Jobs have their distinct set of metadata. LHCb Physicist usually see in the file related metadata a subset of information called Quality. The relation between jobs and files are of type input or output.

UK Metadata Workshop Oxford 4 July 2006 LHCb Metadata 6 LHCb services File Catalog service: –Provides information about associations between LFN and PFN. Servlet service : –the service allows the selection of datasets based on their history (job provenance) by the web browser. XML-RPC service: –access to and modification of the metadata. –allow GANGA to access Bookkeeping data.

UK Metadata Workshop Oxford 4 July 2006 LHCb Metadata 7 LHCb services architecture Oracle DB JDBC Driver BK Service BookkeepingSvc BookkeepingQuery Jython Server XML-RPC Tomcat Servlet GANGA application Web Browser lbnts3 Write Read/Write Read

UK Metadata Workshop Oxford 4 July 2006 LHCb Metadata 8 LHCb services architecture XML-RPC Bookkeeping ODBC File Catalog Pool.xml Gaudi.xml Pool.xml Gaudi.xml Pool.xml Gaudi.xml Pool.xml Gaudi.xml

UK Metadata Workshop Oxford 4 July 2006 LHCb Metadata 9 DB Schema The two schemas: –Warehouse Schema. –View Schema. Warehouse schema used to store bookkeeping information. View schema used to store Job Provenance information. Views are specialized to provide the best performance to the service that access them.

UK Metadata Workshop Oxford 4 July 2006 LHCb Metadata 10 ER DB Schema

UK Metadata Workshop Oxford 4 July 2006 LHCb Metadata 11 DB Schema Warehouse DB schema:

UK Metadata Workshop Oxford 4 July 2006 LHCb Metadata 12 DB Schema View DB schema:

UK Metadata Workshop Oxford 4 July 2006 LHCb Metadata 13 Updating the Views Warehouse DB View If there are few changes in the Warehouse DB the views are just updated otherwise are regenerated. This process is done periodically (every night ) or on demand. SQL script

UK Metadata Workshop Oxford 4 July 2006 LHCb Metadata 14 Some Numbers LHCb is using ORACLE 10g technology for its DB –4 nodes cluster with RedHat Enterprise Server 3 –running Oracle Real Application Cluster –connected via fiber channel switches to a Storage Area Network –composed by storage arrays with 16 SATA disks of 300GB each –each node is 2-xeon 2.8GHz with 4GB memory The DB contains ~45GB of data –Shared between real data and indexing tables –~6.5M jobs rows –~18M files rows –~200M rows in parameters.

UK Metadata Workshop Oxford 4 July 2006 LHCb Metadata 15 Conclusions LHCb deal only with File-Metadata. File-Metadata are divided in two logical sets: –Job Provenance. –Bookkeeping DB characterized by two schemas: –Warehouse schema. –View schemas. View Schema specialized for specific use.