Elizabeth Gallas Oxford University FILE: RunLumDB / 11-Dec-06 / Page 1 Growing a Luminosity Database for Atlas Elizabeth Gallas Oxford University.

Slides:



Advertisements
Similar presentations
The Latest news … and Future of ATLAS Databases Elizabeth Gallas - Oxford ATLAS Software & Computing Workshop CERN November 29 to December 3, 2010.
Advertisements

Conditions Metadata for TAGs Elizabeth Gallas, (Ryan Buckingham, Jeff Tseng) - Oxford ATLAS Software & Computing Workshop CERN – April 19-23, 2010.
Multiple Tiers in Action
Star (Traditional) Database Tasks & MySQL 1. Database Types & Operation Issues 2. Server & Database deployments 3. Tools with MySQL 4. Data definition.
CERN - IT Department CH-1211 Genève 23 Switzerland t Monitoring the ATLAS Distributed Data Management System Ricardo Rocha (CERN) on behalf.
May 14, 2001E. Gallas/Trigger Database1 Status of the Trigger Database Elizabeth Gallas, Rich Wellner, Vicky White Fermilab - Computing Division See my.
Shuei MEG review meeting, 2 July MEG Software Status MEG Software Group Framework Large Prototype software updates Database ROME Monte Carlo.
ATLAS : File and Dataset Metadata Collection and Use S Albrand 1, J Fulachier 1, E J Gallas 2, F Lambert 1 1. Introduction The ATLAS dataset search catalogs.
ATLAS Data Periods in COMA Elizabeth Gallas - Oxford ATLAS Software and Computing Week CERN – April 4-8, 2011.
Jeff LandgrafSTAR Trigger Workshop: Oct 21, 2002 Trigger Accounting for 2002.
1 Alice DAQ Configuration DB
LHC: ATLAS Experiment meeting “Conditions” data challenge Elizabeth Gallas - Oxford - August 29, 2009 XLDB3.
Bookkeeping Tutorial. Bookkeeping & Monitoring Tutorial2 Bookkeeping content  Contains records of all “jobs” and all “files” that are created by production.
Databases E. Leonardi, P. Valente. Conditions DB Conditions=Dynamic parameters non-event time-varying Conditions database (CondDB) General definition:
July 10, 2006ElizabethGallas1 Luminosity Database Elizabeth Gallas Fermilab Computing Division / D0 Computing and Analysis Group D0 Database ‘Taking Stock’
Intro – Part 2 Introduction to Database Management: Ch 1 & 2.
November SC06 Tampa F.Fanzago CRAB a user-friendly tool for CMS distributed analysis Federica Fanzago INFN-PADOVA for CRAB team.
G.Corti, P.Robbe LHCb Software Week - 19 June 2009 FSR in Gauss: Generator’s statistics - What type of object is going in the FSR ? - How are the objects.
JANA and Raw Data David Lawrence, JLab Oct. 5, 2012.
EXAM 1 NEXT TUESDAY…. EXAMPLE QUESTIONS 1.Why is the notion of a “state” important in relational database technology? What does it refer to? 2.What do.
Elizabeth Gallas August 9, 2005 CD Support for D0 Database Projects 1 Elizabeth Gallas Fermilab Computing Division Fermilab CD Grid and Data Management.
DØ Data Handling & Access The DØ Meta-Data Browser Pushpa Bhat Fermilab June 4, 2001.
Accessing Relational Databases from the World Wide Web by Tam Nguyen & V. Srinivasan Presented by Megan Thomas and Randi Thomas CS294-7 February 11, 1999.
Conditions Metadata for TAGs Elizabeth Gallas, (Ryan Buckingham, Jeff Tseng) - Oxford ATLAS Software & Computing Workshop CERN – April 19-23, 2010.
Artemis School On Calibration and Performance of ATLAS Detectors Jörg Stelzer / David Berge.
Jan 28, 2005Elizabeth_Gallas D0_Trigger_Database 1 D0 Trigger Database Status Elizabeth Gallas Fermilab CD/D0CA Margherita Vittone, Vijay Murthi, and Steve.
V.Sirotenko, C/Al/Db Meeting, 4/25/01 Run_Conf Database Status:run_conf database runs on d0onprd being updated by runGrabber Also exists on development.
Marcelo R.N. Mendes. What is FINCoS? A set of tools for data generation, load submission, and performance measurement of CEP systems; Main Characteristics:
Michele de Gruttola 2008 Report: Online to Offline tool for non event data data transferring using database.
CERN - IT Department CH-1211 Genève 23 Switzerland t High Availability Databases based on Oracle 10g RAC on Linux WLCG Tier2 Tutorials, CERN,
Bookkeeping Tutorial. 2 Bookkeeping content  Contains records of all “jobs” and all “files” that are produced by production jobs  Job:  In fact technically.
The ATLAS TAGs Database - Experiences and further developments Elisabeth Vinek, CERN & University of Vienna on behalf of the TAGs developers group.
May 21, 2004ElizabethGallas -- D0 Luminosity Db1 D0 Luminosity Database Status Elizabeth Gallas Fermilab Computing Division / D0 Computing and Analysis.
Analysis Tools at D0 PPDG Analysis Grid Computing Project, CS 11 Caltech Meeting Lee Lueking Femilab Computing Division December 19, 2002.
TAG Metadata Browser (a conceptual implementation in development) Elizabeth Gallas - Oxford - March 4, 2009 ATLAS TAG Developers meeting.
November 1, 2004 ElizabethGallas -- D0 Luminosity Db 1 D0 Luminosity Database: Checklist for Production Elizabeth Gallas Fermilab Computing Division /
Jan 7, 2002E. Gallas/Trigger Db1 Trigger Database and Trigger Configurations and Trigger Issues Elizabeth Gallas, Jeremy Simmons (Fermilab - Computing.
User Guide to Trigger/Prescale information in COMA runBrowserReport Elizabeth Gallas - Oxford December, 2010.
Status of tests in the LCG 3D database testbed Eva Dafonte Pérez LCG Database Deployment and Persistency Workshop.
Conditions Metadata for TAGs Elizabeth Gallas, (Ryan Buckingham, Jeff Tseng) - Oxford ATLAS Software & Computing Workshop CERN – April 19-23, 2010.
Event Management. EMU Graham Heyes April Overview Background Requirements Solution Status.
TAG and iELSSI Progress Elisabeth Vinek, CERN & University of Vienna on behalf of the TAG developers group.
The Database Project a starting work by Arnauld Albert, Cristiano Bozza.
Joe Foster 1 Two questions about datasets: –How do you find datasets with the processes, cuts, conditions you need for your analysis? –How do.
1 Simulations Purpose of the simulation codes Structure of the E866 simulation codes Issues of simulations for E906 Jen-Chieh Peng University of Illinois.
CPSC-310 Database Systems
Backups for Azure SQL Databases and SQL Server instances running on Azure Virtual Machines Session on backup to Azure feature (manual and managed) in SQL.
Jacek Otwinowski (Data Preparation Group)
An Oracle-based Event Index for ATLAS
KID - KLOE Integrated Dataflow
Database Replication and Monitoring
SuperB and its computing requirements
CMS High Level Trigger Configuration Management
BOSS: the CMS interface for job summission, monitoring and bookkeeping
Elizabeth Gallas - Oxford ADC Weekly September 13, 2011
Database operations in CMS
WLCG: TDR for HL-LHC Ian Bird LHCC Referees’ meting CERN, 9th May 2017.
Existing Perl/Oracle Pipeline
NCAR-Developed Tools Bill Anderson and Marc Genty
POOL persistency framework for LHC
Design a Relational Database Identify Database Purpose
Overload of frontier lpad by MC Overlay
ALICE Computing Upgrade Predrag Buncic
TriggerDB copy in TriggerTool
New developments on the LHCb Bookkeeping
Team #TBD Member1, Member2, Member3, Member4 [Alphabetical]
ATLAS DC2 & Continuous production
Event Storage GAUDI - Data access/storage Framework related issues
Status and plans for bookkeeping system and production tools
Team #TBD Member1, Member2, Member3, Member4 [Alphabetical]
Presentation transcript:

Elizabeth Gallas Oxford University FILE: RunLumDB / 11-Dec-06 / Page 1 Growing a Luminosity Database for Atlas Elizabeth Gallas Oxford University December 2006

Elizabeth Gallas Oxford University FILE: RunLumDB / 11-Dec-06 / Page 2 Outline The 'Streaming Test 2006' Luminosity Database Schema (Tables, Views...) Input Simulation Data For the Streaming Test Evolution of the Streaming Test, placeholders Simple Query -- Output Cross Sections Input Real (Online) Data ConditionsDB Luminosity Database Distribution (Tier 0, 1, 2...)

Elizabeth Gallas Oxford University FILE: RunLumDB / 11-Dec-06 / Page 3 A Luminosity Database for Atlas This DB is ONE component of the “Streaming Test 2006” Purpose: Study online physics data streaming models Streaming Test Complete ATLAS simulation chain Generate MC Detector Simulation Trigger Simulation Data Reconstruction Data Analysis Cross Section calculation by Run/LBN/Trigger using a Database This DB reflects many features we need in the ATLAS Luminosity Database

Elizabeth Gallas Oxford University FILE: RunLumDB / 11-Dec-06 / Page 4 Start with something simple SQLite for the 'Test' – lightweight database engine Many advantages: Public domain, SQL based, portable, single file up to 2TB, no administrative overhead, command line access, many APIs (C/C++, python, TCL...) May be used at Tier 2 sites for ConditionsDB data... Plan is Use this SQLite LumDB for the Test Analyze relational structure for Conditions DB storage With a view that it might be re-extracted to SQLite at Tier 2...

Elizabeth Gallas Oxford University FILE: RunLumDB / 11-Dec-06 / Page 5 Schema – 2 Relational Tables Run_LBNs TableRun_LBN_Triggers Table ● Run_Number ● LBN ● Start_Time ● End_Time ● Duration (in seconds) ● Delivered_Lum (in nb-1) ● Live_Fraction ● Quality ● Run_Number ● LBN ● Trigger_Name ● L1_Accepts ● L2_Accepts ● L3_Accepts ● L1_Prescale ● L2_Prescale ● L3_Prescale

Elizabeth Gallas Oxford University FILE: RunLumDB / 11-Dec-06 / Page 6 Schema – a View ● LBN ● Trigger_Name ● Run_Number ● TRIG_LUM = ● DELIVERED_LUM * LIVE_FRACTION / PRESCALE ● TRIG_CROSS_SECTION = ● L3_ACCEPTS * PRESCALE / DELIVERED_LUM / LIVE_FRACTION V_RLT – Calculated luminosity quantities by trigger.

Elizabeth Gallas Oxford University FILE: RunLumDB / 11-Dec-06 / Page 7 Loading RunLumDB for the Streaming Test Columns populated from Ayana's log files: in RUN_LBNS.RUN_NUMBER, LBN in RUN_LBN_TRIGGERS.TRIGGER_NAME, L3_ACCEPTS. Other database columns Start, End_times, prescales, luminosity... populated with 'fake' data based on logical assumptions about what we expect the data to look like, will evolve with the Test. Placeholders for Deadtime, Prescales, Level 1,2 Accepts, Data Quality, Duration

Elizabeth Gallas Oxford University FILE: RunLumDB / 11-Dec-06 / Page 8 Online monitoring and offline analysis need Run,LBN,Trigger, Trigger Group, BCID dependent: Luminosity from detector scalars (normalized) Live fraction (1 – d) from Level 1 LBN duration, start and end times from Run Control (RC) Prescales (at all trigger levels) from Trigger configuration Number of accepted events (at each trigger level) From “Event Loss Monitoring”

Elizabeth Gallas Oxford University FILE: RunLumDB / 11-Dec-06 / Page 9

Elizabeth Gallas Oxford University FILE: RunLumDB / 11-Dec-06 / Page 10 Conclusions