Central DQM Shift Tutorial Online/Offline. Overview of the CMS DAQ and useful terminology 2 Detector signals are collected through individual data acquisition.

Slides:



Advertisements
Similar presentations
Calculating Quality Reporting Service
Advertisements

Web Shift Booking System
E-Portfolio July2014 Managing Multi-source Feedback.
Clara Gaspar on behalf of the LHCb Collaboration, “Physics at the LHC and Beyond”, Quy Nhon, Vietnam, August 2014 Challenges and lessons learnt LHCb Operations.
1 Vendor Reverse Auction - Event User Guide. 2 Minimum System Requirements Internet connection - Modem, ISDN, DSL, T1. Your connection speed determines.
Service Manager Service Desk Overview
Student Organization Training 2008 Why put it on TAP? More convenient than the old Blue Form system 24 Access from any browser location Who may use TAP?
Spouse Tuition Assistance Program Spouse Briefing STAP Development Team.
Introduction to Online Data Collection (OLDC) Community Based Abstinence Education September, 2009.
Logging In Go to web site:
Center Accreditation Online System
2 eGrants Coaching Unit 3 create/edit Service Opportunity Listings Grantee Recruiter search for listings, register, & apply to serve Applicants view.
Welcome to the Brookdale Community College Online Employment System Applicant Tutorial.
Welcome to the Ivy Tech Community College Online Employment System Applicant Tutorial.
1 Welcome to the Colgate University Online Employment System Applicant Tutorial.
Software How-To Using the HelpDesk Ticket System You can use the Helpdesk to request help from the technology department of your local district. This will.
1 Vendor RFI - Event User Guide. 2 Minimum System Requirements Internet connection - Modem, ISDN, DSL, T1. Your connection speed determines your access.
Welcome to the Sinclair Community College Online Employment Applicant Tutorial.
Welcome to And the Rice University Online Job Application Tutorial.
Welcome to the Southeastern Louisiana University’s Online Employment Site Applicant Tutorial!
Data Quality Monitoring of the CMS Tracker
Welcome to the University of West Florida Online Employment System Applicant Tutorial.
UNCLASSIFIED User Guide Applicant. UNCLASSIFIED Table of Contents What is the SAFETY Act? Applicant Guide Help Desk.
GFP in the IUID Registry – A Basic Look Walt Clark, CPPM Raytheon IIS.
A Guide to the BIZNET Online Filing System STATE OF CONNECTICUT DEPARTMENT OF CHILDREN & FAMILIES (DCF) DEPARTMENT OF DEVELOPMENTAL SERVICES (DDS) DEPARTMENT.
System for Administration, Training, and Educational Resources for NASA SATERN Overview for Learners May 2006.
Central DQM Shift Tutorial Online/Offline. Overview of the CMS DAQ and useful terminology 2 Detector signals are collected through individual data acquisition.
Central DQM Shift Tutorial Online/Offline. Overview of the CMS DAQ and useful terminology 2 Detector signals are collected through individual data acquisition.
5. Data Manager 1. Introduction 2. Data Manager Duties 3. Quality Checking 4. Problem Reporting 5. Data Monitoring 6. Histogram Presenter 7. Trend Presenter.
Offline Tracker DQM Shift Tutorial. 29/19/20152 Tracker Shifts Overview Online Shifts at P5 (3/day for 24 hours coverage) – One Pixel shifter and one.
Lead Management Tool Partner User Guide March 15, 2013
Web Based Monitoring DT online shifter tutorial Jesús Puerta-Pelayo CIEMAT Muon_Barrel_Workshop_07/July/10.
Welcome to the Manage Scoping module of the “MIP Release 3 Study Workflow Training” course! This module guides you through the process of managing the.
System for Administration, Training, and Educational Resources for NASA SATERN Overview for Users December 2009.
Offline shifter training tutorial L. Betev February 19, 2009.
How to Request Materials Tutorial Order, Borrow, Renew made easy (expected running time: ~7 minutes) Oregon State Library.
OFFLINE TRIGGER MONITORING TDAQ Training 5 th November 2010 Ricardo Gonçalo On behalf of the Trigger Offline Monitoring Experts team.
A.Golunov, “Remote operational center for CMS in JINR ”, XXIII International Symposium on Nuclear Electronics and Computing, BULGARIA, VARNA, September,
CMS pixel data quality monitoring Petra Merkel, Purdue University For the CMS Pixel DQM Group Vertex 2008, Sweden.
Early Childhood Outcomes Indicator 7 Data Collection Application Review.
FP6 IT System 1 ELECTRONIC PROPOSAL SUBMISSION SYSTEM.
1 AARP Tax-Aide Prospective Volunteer Recruitment System Leadership Reports and VMIS Functionality 11/4/09.
Human Resources 1 G-Top Global Workflow Employee View September 2014.
Office of Housing Choice Voucher Program Voucher Management System – VMS Version Released October 2011.
 Why?  * Department of Education (not LHU) awards PA certification to qualified applicants  * Students must apply for certification individually 
Vendor Bid System (VBS) Seminar. Agenda Vendor Bid System Overview Step-by-Step Advertisement Posting Editing Active Advertisements Recommended Practices.
Pixel DQM Status R.Casagrande, P.Merkel, J.Zablocki (Purdue University) D.Duggan, D.Hidas, K.Rose (Rutgers University) L.Wehrli (ETH Zuerich) A.York (University.
DQM for the RPC subdetector M. Maggi and P. Paolucci.
Online Consumers produce histograms (from a limited sample of events) which provide information about the status of the different sub-detectors. The DQM.
CSC Shifter Training Course – Global Running Fred Borcherding Reach from CSCOperations Twiki page or directly:
Commissioning and run coordination CMS week Commissioning plenary 28-February 2007 DQM and monitoring workshop MandateGoals.
Document Module Features Streamlines the control, routing and revision process for critical documents and records Controls documents in any format (Excel,
Page ADP PearsonAccess Proctor Training. Page Agenda Test Overview Testing Components Proctor Roles and Responsibilities Overview Administering the Test.
2016 TELPAS Online Testing & Data Collection. Disclaimer  These slides have been prepared by the Student Assessment Division of the Texas Education Agency.
Central DQM Shift Tutorial Online/Offline
Project Management: Messages
Central Online DQM Shift Tutorial March 2017, CMS DQM group
Shift instructions August 16, 2017 Antoni Aduszkiewicz
Offline shifter training tutorial
Offline shifter training tutorial
Data Quality Monitoring of the CMS Silicon Strip Tracker Detector
Central DQM Shift Tutorial Online/Offline
CMS Pixel Data Quality Monitoring
DQM for the RPC subdetector
CSC Shift Training and Operation + B904 Status
Presentation transcript:

Central DQM Shift Tutorial Online/Offline

Overview of the CMS DAQ and useful terminology 2 Detector signals are collected through individual data acquisition systems (cables and boards) that end up at the FEDs: the first element of Global Data Acquisition system (DAQ) FED ( detector FrontEnd boards): multiple FEDs per detector collect event fragments that are sent to the online event processing farm Builder Units: Computing farm that collects event fragments from all FEDs and merge them to produce full event information Filter Units: Computing farm where the High Level Trigger (HLT) is run to filter interesting events Storage Manager: application that saves to local disks events selected by the HLT Detector signals are collected through individual data acquisition systems (cables and boards) that end up at the FEDs: the first element of Global Data Acquisition system (DAQ) FED ( detector FrontEnd boards): multiple FEDs per detector collect event fragments that are sent to the online event processing farm Builder Units: Computing farm that collects event fragments from all FEDs and merge them to produce full event information Filter Units: Computing farm where the High Level Trigger (HLT) is run to filter interesting events Storage Manager: application that saves to local disks events selected by the HLT Storage Manager

Online DQM Online DQM: suite of CMSSW applications that run either on all events in the Filter Farm or on a selection of events served by the Storage manager Since Dec2009 Online DQM consume DCS information in addition to Event data Online DQM Infrastructure is completed by the DQM data transfer system up to the DQM servers where the histograms and other DQM data are uploaded and visible to the shifters and the CMS community Scope of Online DQM Shifts: Identify problems with detector performance or data integrity during the run 3 Storage Manager

Offline Data Processing and Offline DQM Online environment T0, CAF Prompt Reconstruction at T0 and CAF is performed from within one hour up to 48 hours after data is transferred from P5 to T0 and CAF (CERN) T1 Subsequent iterations of re-reconstruction at the T1’s follow periodically the Prompt Reco with improved Alignment and Calibration constants, bug fixes. Offline DQM is part of the Offline data processing that, in addition to detector data analyses, includes higher level reconstruction objects, aka Physics Objects (POG’s) Scope of Offline DQM Shifts: Produce the data certification for various reconstruction iterations  USED FOR CMS OFFICIAL GOOD RUN LISTS!!! 4

DQM Shift Tools DQM GUI: Graphical User Interface, for histogram viewing DQM Run Registry: web interface to the Database that holds run information. Used by shifters to register interesting runs (Online shifters) and to collect quality information Elog: for end of shifts and problem reports TWiki pages: for shift instructions 5

Online Shifts at P5 (3/day for 24 hours coverage) 23:00-7:00 | 7:00-15:00 | 15:00-23:00 On the first day of global running, shifts start at 9:00 (for this shift you can use the regular 8:15 shuttle) For latest run plan updates (shift cancelations), inscribe to: Offline Shifts run at Control Rooms away from P5 (4/day) 1:00 – 7:00 at Fermilab 7:00 – 13:00 at CERN-CMS Centre (Meyrin site) 13:00 – 19:00 at DESY 19:00 – 1:00 at Fermilab In order to certify reprocessing data, Offline DQM shifts can also be scheduled outside of global data taking periods DQM Shifts - Overview 6

7 Access to Control Rooms on CERN Site Safety requirements for P5: –Online CMS level 4C safety class must be passed. –Each shifter should have appropriate access rights: through EDH: request access to « CMS CR » for P5 request access to « CMS CEN » for CMS Center at Meyrin Regular shuttle service which runs 7 days per week

8 In preparation for the shift Read the DQM shift instructions, even if you have been on shift before: Check the instructions at the beginning of each shift to find out if something changed since your last shift.ng of each shift to find out if something changed since your las

9 In preparation for the shift For newcomers: Schedule your first shift in the daytime so assistance will be readily available if needed Attend the shift tutorial on Monday (possibly the latest one before your shift or the one before that) Attend a trainee shift between the tutorial and your first shift

10 Task 0: Applications (1) Make sure DQM applications and GUI are running during data taking: –Online: check update of histograms during runs –Offline: check arrival of histograms from Tier-0/CAF processing (2) Check correct updating of Run Registry (3) In case of problems in DQM Tools (persisting longer than 15 mins), call: (4) During Online shifts stay in close contact with the Shift Leader (P5) DQM on-call Expert :

Task 1: Histogram Inspection Follow online/offline shift instructions Run-by-run procedure: –(1) Enter significant runs in the Run Registry –Online Shifter has the task to make the decision on which are the significant runs to analyze and registers them in the Run Registry (confirm with the Shift Leader!) –Offline Shifter: analyzes the runs that have been previously registered by the Online shifter –(2) Shift histogram Inspection Look at the Summary, the Reports, and the Shift Workspace in the DQM GUI Make an effort to look at all the plots one by one. If you spot a problem or have a question regarding a specific plot Check the sub-system instruction and take action(s) accordingly If not explained in the instruction, discuss with P5 shift leader and if needed inform him to call Detector On Call (DOC) In offline, inform sub-system DQM experts, as noted in the instruction page by mail or phone Make Elog entry (Type : “Problem Report”) 11

GUI – Summary Workspace 12

13 GUI – Summary Workspace

14 GUI – Reports Workspace

15 GUI – Shift Workspace

15 GUI – Shift Workspace I nspect Error plots (Online GUI -> Work Space -> Shift -> Errors They show plots that indicate errors. Please read sub-system short term instruction about the evaluation

16 DQM Shift Instructions

16 DQM Shift Instructions Check that the HV information on the DQM GUI summary ( "Info" histogram) and the Run Registry ( "LumiSec" table) are consistent.DQM Sometimes "LS" in the Runs table shows 0. If this happens: make an ELOG entry "Problem report" indicating the run number ensure that the DQM expert on-call is aware of the problem. ex put the information manually into the general comments section of the Run Registry. format: Example: LS 0 = CASTOR, Strips, RPC, Pixel, and DT with HV OFF. All others with HV ON. Correlation between Data Cert and HV conditions: If HV remains OFF throughout the entire run then mark the subsystem as BAD If HV varies do not mark the subsystem as BAD based on that, but proceed following the subsystem instructions as usual.

17 Task 2: Run Registry Hover over the quality flag to see the comment

18 Run Registry Run Registry Collect automated run info and information filled by the DQM shifters: Data quality and Comments. Threre are three view in the RunRegistry browsable through the buttons on top of the page: -RunInfo -Table -LumiSec Threre are three view in the RunRegistry browsable through the buttons on top of the page: -RunInfo -Table -LumiSec

The RunInfo table contains information that is automatically entered All Runs are entered in this table, from short start/stop runs to stable long runs This table is used by the ONLINE DQM SHIFTER to select Significant runs: –Register only runs that have more than 10,000 events and/or have been running for more than 10 minutes (if you are in doubt, ask the shift leader). –Follow the run-by-run workflow inspecting the shift histograms. –Ask the shift leader for confirmation on the info reported and then move th enrty to « SIGNOFF » - Run Registry: RunInfo View 19

Runs that are registered by the Online DQM Shifters are included and visible in the DB table of the “Table” view This is the view used by both Online and Offline DQM Shifters to log in the information, i.e. comments and by –subsystems quality flags - Run Registry: Table View, Online Shifter’s use 20 Online Shifters: Click on the dataset “/Global/Online/ALL” and then on “Edit”, to open the RunRegistry editing page. Analyze subsystems one by one through the related histograms in the GUI and Assign run type + set the quality flags in the editing page of the run registry.

21 Register Runs in Run Registry During the run: Click run number itself->View details->Edit Based on the shift histogram instructions, set the online subsystem flags (GOOD/BAD) and enter comments If a subsystem is BAD, inform the shift leader and the subsystem expert, and enter comment Also, enter the relevant luminosity range for “physics” (click “Beam Status” towards the bottom of the page).  You can get this information either from the shift leader of the shift elog. Try to provide complete information by adding info like Beam 1 or 2, etc in the “comments” field. Online

22 After the run: Enter the ‘stop reason’ (in the stop reason field, NOT under comments) The certification results must be confirmed by the shift leader, before the status of a given run is moved to « SIGNOFF » from main “Runs info” view-> click on dataset name -> move to SIGNOFF Once the run is in SIGNOFF state, it cannot be modified by the Online shifter. Register Runs in Run Registry

Online DQM Shifter: Run Classification 23 Note that Runs are classified though the Group name in the RunRegistry: Assigning correct Group name is of vital importance as it will affect Offline determination of Runs to be used for different analyses Assigning correct Group name is of vital importance as it will affect Offline determination of Runs to be used for different analyses Current rules follow, make sure you check updated rules at start of your shift In the Online DQM Shift Instructions: 1. Select "Collisions10" if the run is taken for physics analysis purposes and contains at least one lumi section with two stable beams (colliding or non-colliding). 2.Select "Cosmics10" if the run is taken for analysis purposes and there is no beam activity throughout the run, i.e. stable "no beam" conditions 3.Select "Commissioning10" for all other runs, i.e. those taken for tests or specific detector studies only, i.e. not meant for general offline physics analysis. -> Ask the shift leader if in doubt ! Text

Run Registry: Table View, Offline Shifter’s use 24 Offline DQM Shifter: Select which runs to analyze: oldest run where the “Global/Online/ALL” dataset is in SIGNOFF status ADD the Offlien dataset to analyze (as per instructions) and proceed with subsystem evaluatio nbased on relevant histograms in the GUI, assign quality flags Move the dataset entry to SIGNOFF when all subsystems are analyzed

Run Registry – Datasets 25 P5 Meyrin

28 ELOG Log in with your AFS account –Click on "Elog" and choose Subsystems -> "Event Display and DQM" Problem Report ( 1 entry per problem ) –For each problem arising during your shift make a "Problem Report" entry – Please use “Elog” to report problems! Shift Summary ( 1 entry per shift ) –At the end of each shift, write a short "Shift Summary" N.B. (!!!): –Use Types "Problem Report" or "Shift Summary" only, do NOT create new –Do not enter run-by-run information in the Elog that should be in the Run Registry. –Make sure all run-by-run information is in the Run Registry, not in the Elog

29 Shift Hand-over Make sure to arrive 5-10 minutes early for shift hand-over. Upon your arrival in the control room, the previous shifter will be there –Get from her/him the information about the current status of the data taking and what happened during the previous shift. –The shift person will show you where the tools are running, which you will be using (DQM GUI, CMS Online page, Run Registry). –If anything with your tasks is not clear, ask at that moment! –At the end of your shift, wait for the next shift person to arrive and provide the same support.

30 Links Shift instructions: – – – – DQM Online GUI: – – follow certificate instructions at at Run Registry page: – presently only working from CERN (need tunnel from outside CERN).tly only working from CERN (need Elog: –

Final Suggestions From now to your shift: - Get familiar with the DQM TWiki pages (general structure & dedicated On/Offline): Shortly before each shift you MUST read : - Short term instructions - Event Filter/DQM Announcements HN ( - Online/Offline Shift Histograms by Subsystem <---- especially Online shifters - Read the Elog of the shift before yours to be aware of the recent activity (please read both Shift Leader Elog and Event Display and DQM Elog)Shift Leader Elog DQM Elog) 31

Backup 31

26 Run Registry – Message Board Communication between shift persons and DQM operators: Opening: Close all browser windows in the workstation Open run registry (RR) Login as yourself Select  Tools  Message Board Use phone or RR message board. Do not use ELOG or (!) Stick to using English

27 Run Registry – Message Board Automatic messagesChat windowLogged users