Risultati del run di integrazione M4 E. Pasqualucci INFN Roma
Scopo dei run di integrazione Integrazione di Detector DAQ, DCS ed online monitoring Trigger di primo livello Trigger di alto livello Infrastruttura Registrazione dei dati ed analisi “prompt” Data Quality Comportamento del sistema “globale” Run di cosmici Run ad alto rate Organizzazione dei run Programmazione Shift e uso della control room Run coordinator/ Shift leader
Integration weeks schedule Dates Systems Integration Detector configuration Operations Cosmic run Training ACR M1: 11-19/12 2006 DAQ R/O Barrel Lar & Tile CTP Barrel calorimeters Achieve combined run 2 days Tile cosmic trigger N/A Initial setup: 5 desks Central DCS M2 28/2 to 13/3 2007 DAQ/EB DAQ V. 1.7 Muon barrel (S. 13) Monitoring/DQ Barrel Muon Combined runs Mixed runs 2 x weekd ends Tile cosmic trigger + RPC cosmic trigger Periodic cosmic runs after M2 After M2 week Increase to 7 desks M3 4/6 to 18/6 2007 Barrel SCT Barrel TRT Muon EC (MDT, TGC) Offline Barrel and End Cap calorimeters Barrel muon (5&6) EC muon MDT Barrel SCT, TRT EC muon TGC 1st week focus on operations, checklist management, coordination between desks 1 week Tile + Muon cosmic trigger (side A) 4/6 to 11/6 Towards final layout: 13 desks M4 23/8 to 3/9 2 day setup 2 week ends Level-1 Calo HLT DAQ 1.8 Offline 13 Barrel & EC calos Barrel & EC muon SCT R/O Level-1 Mu, Calo ATLAS-like operations Use of DQ assessment Try also calorimeter trigger Whole week Final setup M5 22/10 to 5/11 ID EC (TRT) Pixel (probably R/O only) SCT quadrant Converge to ATLAS detector 1 week + 1 week M6 November/December As for M5 ATLAS-like Run during ECT magnet test Global cosmic run ECT Magnets on only A V I L B E F O R U S E June 16, 2018
Atlas in M1 (Dic 2006) Tile + LArg + CTP DAQ read-out ROS/RCD-like event building Online monitoring Tile with GNAM Very long initialization procedures
Atlas in M2 Muon Barrel added RPC trigger Sector 13 A, 9 MDT chambers + RPC RPC trigger ~20 Hz First checklists for improved startup procedure Shifter operation started for calorimeters Data Collection added Final Event Builder and HLT infrastructure Athena monitoring at EF level Added for calorimeters First connection Online Monitoring to Data Quality desk
Atlas in M3 First integration exercise for L1 calo DAQ, DCS, no trigger SCT added (only DAQ, no detector) TRT added (part of the detector) MDT endcap added (sectors 10,11,12) RPC trigger via final distribution 50-60 Hz, downscaled due to Castor limitations TGC added (sector 9, TGC 1) Final ROD read-out Commissioning Trigger Module used 4 Hz/1 sector ~75 % of Tile HTL added with dummy algorithm Complete slice of TDAQ Improved system stability Typical run preparation time from 3++ h in M2 to < 1 h Improved checklist system
Event with Tile trigger
Atlas in M4: calorimeters Tile Cosmic trigger since May 2007 Tile > 75 % Cosmic trigger almost complete LArg Barrel: C side complete Large part of A side EndCap: A side Missing A01 C side not used in M4 Electronics refurbishment on-going
Atlas in M4: inner detector SCT 4 “noise module” Test of DAQ (1 ROD) and monitoring TRT
Atlas in M4: Muon Spectrometer MDT MDT: 224 chambers, 72 ROD, 3 partitions Barrel A/C (sectors 3,4,5,6), Endcap C (entire BW) RPC Complete sector 5 Emulated ROD Stable online monitoring with GNAM Trigger given to CTP TGC Sectors 9 (TGC 1, 2, 3) Sectors 10, 11 (TGC 1) HPT on sector 9 Problem with readout, solved after M4 Extensive use of Oracle configuration DB
DCS sub-detector integration System FSM Alarms CIC EXTERNAL , Cryo LAR (EMBA, EMBC, EMECA, EMECC, HECA, HECC) ROD FECLV HECLV monitoring only HV monitoring only* TIL (LBA, LBC, EBA, EBC) LV MDT (BA, BC, EA, EC) LV/HV TGC (C) IDE EVCOOL SCT ENV PS Extended/New systems System FSM Alarms TGC TEMP monitoring only RPC MDT GAS TRT TIL (LBA, LBC, EBA, EBC) HV
Trigger Level 1 HLT Random and pulsed Tile + RPC + TGC Latency measured HLT Baseline Level 2 algorithms TrigL2CosmicMuon, TrigTRTSegFinder and TrigT2CaloEgamma Baseline Event Filter algorithms TrigEFIDCosmic and CaloRec MDT end-cap TDC spectrum
HLT/DAQ M4 highlights Hardware the same as that deployed for M3, but bigger… 7 more Online operations nodes HLT nodes increased by factor 2, i.e. 124 nodes One LVL2 rack 30*8 =240 L2PUs Three EF racks 90*8=720 PTs 6 SFO machines (4 used) 13
Rate tests and data recording 4 final SFO machines writing events 200 Hz (event size ~ 2 MB), 400 MB/s With SCT, TRT, LArg, Tile, RPC, MDT Staged data written to Castor 2 Available throughput ~140 MB/s 50 KHz achieved with LArg only
Online monitoring GNAM for low-level monitoring Tile, MDT, RPC TGC under development Online monitoring at level 2 Athena @ EF for Larg, TRT, SCT Online event display at EF level Both Atlantis and VP1
Data Quality DQ Jobs and Tools OHP few (typically ~4) most significant histograms per sub-detector/system to be frequently checked by-eye, no more than O(100) histograms in total (expect ~60) DQMF online (on DQM Display, see next slide) automatic histogram checking, number of histograms can be larger than sub-set for DQ shifter (expect ~1000) Offline histogram check via web browser http://cern.ch/atlasdqm histograms for DQ shifter to be checked, again O(100) or less, ~same as online DQMF offline to be used first time in M4 access via web browser Atlantis 2D event display, client/server system VP1 (3D event display) running and display on same monitoring PC, export X-display t o ACR DQ desk
Counting room(s) and organization Atlas Counting Room completed Some satellite counting room in use Many people around… Most of them learning operation Run organized with Shift leader DAQ expert DQ expert Sub-system shifters
Atlas Control Room
Events with RPC and TGC trigger
Cosmic event with VP1
M4 problems Only in “combined mode” Run stability to be improved Some Even Counter Reset signals lost (Tile) Lost triggers Holes in trigger sequence (Tile, TRT, …) De-synchronization (MDT) Run stability to be improved Many runs end with a busy This problems moves from one detector to another Connected with lost triggers Initialization and re-start very slow Stop/start sequence ECR management during run to be improved HLT configuration too slow
Integration weeks schedule Dates Systems Integration Detector configuration Operations Cosmic run Training ACR M1: 11-19/12 2006 DAQ R/O Barrel Lar & Tile CTP Barrel calorimeters Achieve combined run 2 days Tile cosmic trigger N/A Initial setup: 5 desks Central DCS M2 28/2 to 13/3 2007 DAQ/EB DAQ V. 1.7 Muon barrel (S. 13) Monitoring/DQ Barrel Muon Combined runs Mixed runs 2 x weekd ends Tile cosmic trigger + RPC cosmic trigger Periodic cosmic runs after M2 After M2 week Increase to 7 desks M3 4/6 to 18/6 2007 Barrel SCT Barrel TRT Muon EC (MDT, TGC) Offline Barrel and End Cap calorimeters Barrel muon (5&6) EC muon MDT Barrel SCT, TRT EC muon TGC 1st week focus on operations, checklist management, coordination between desks 1 week Tile + Muon cosmic trigger (side A) 4/6 to 11/6 Towards final layout: 13 desks M4 23/8 to 3/9 2 day setup 2 week ends Level-1 Calo HLT DAQ 1.8 Offline 13 Barrel & EC calos Barrel & EC muon SCT R/O Level-1 Mu, Calo ATLAS-like operations Use of DQ assessment Try also calorimeter trigger Whole week Final setup M5 16/10 to 23/10 23/10 to 1/11 ? ID EC (TRT) Pixel (probably R/O only) SCT quadrant Converge to ATLAS detector M6 November/December As for M5 ATLAS-like Run during ECT magnet test Global cosmic run ECT Magnets on only A V I L B E F O R U S E June 16, 2018
Next milestone runs M5 (22 Oct – 5 Nov) M6 (Dec ?) Add Pixel (R/O only), SCT detector quadrant ? More HLT algorithms and calibration stream More detector parts Build on M4; better operations Improve on DQ/monitoring Problem fixing Improve stability (week 1) and get stable runs (week 2) M6 (Dec ?) M5 + magnetic field Only overnight
M5 preparation Muon runs (MDT, RPC, TGC) Calorimeter runs Dedicated to debug and stabilize the system Running the spectrometer as a whole 1-7 Oct. and 3-9 Dec. RPC ROD in M5? Calorimeter runs Week-ends are dedicated to combined runs Technical run To test the entire DAQ/HLT system Next one 24-30 Sept. Organization ongoing
Conclusions Very good achievements through Mx runs Good exercise Converging from “sub-detectors” to “Atlas” Including off-line prompt analysis in DQA Start integrating communities… Good exercise To discover problems And solve them during next milestone preparation To get smooth operation of the detector To achieve optimal run organization