1. Risultati del run di integrazione M4
E. Pasqualucci
INFN Roma

2. Scopo dei run di integrazione
Integrazione di
Detector
DAQ, DCS ed online monitoring
Trigger di primo livello
Trigger di alto livello
Infrastruttura
Registrazione dei dati ed analisi “prompt”
Data Quality
Comportamento del sistema “globale”
Run di cosmici
Run ad alto rate
Organizzazione dei run
Programmazione
Shift e uso della control room
Run coordinator/ Shift leader

3. Integration weeks schedule
Dates
Systems Integration
Detector configuration
Operations
Cosmic run
Training
ACR
M1:
11-19/
DAQ R/O
Barrel Lar & Tile
CTP
Barrel calorimeters
Achieve combined run
2 days
Tile cosmic trigger
N/A
Initial setup: 5 desks
Central DCS M2
28/2 to 13/3 2007
DAQ/EB
DAQ V. 1.7
Muon barrel (S. 13)
Monitoring/DQ
Barrel Muon
Combined runs
Mixed runs
2 x weekd ends
Tile cosmic trigger + RPC cosmic trigger
Periodic cosmic runs after M2
After M2 week
Increase to 7 desks M3
4/6 to 18/6 2007
Barrel SCT
Barrel TRT
Muon EC (MDT, TGC)
Offline
Barrel and End Cap calorimeters
Barrel muon (5&6)
EC muon MDT
Barrel SCT, TRT
EC muon TGC
1st week focus on operations, checklist management, coordination between desks
1 week
Tile + Muon cosmic trigger (side A)
4/6 to 11/6
Towards final layout: 13 desks M4
23/8 to 3/9
2 day setup
2 week ends
Level-1 Calo
HLT
DAQ 1.8
Offline 13
Barrel & EC calos
Barrel & EC muon
SCT R/O
Level-1 Mu, Calo
ATLAS-like operations
Use of DQ assessment
Try also calorimeter trigger
Whole week
Final setup M5
22/10 to 5/11
ID EC (TRT)
Pixel (probably R/O only)
SCT quadrant
Converge to ATLAS detector
1 week + 1 week M6
November/December
As for M5
ATLAS-like
Run during ECT magnet test
Global cosmic run
ECT Magnets on only A V I L B E F O R U S E
June 16, 2018

4. Atlas in M1 (Dic 2006) Tile + LArg + CTP DAQ read-out
ROS/RCD-like event building
Online monitoring
Tile with GNAM
Very long initialization procedures

5. Atlas in M2 Muon Barrel added RPC trigger
Sector 13 A, 9 MDT chambers + RPC
RPC trigger
~20 Hz
First checklists for improved startup procedure
Shifter operation started for calorimeters
Data Collection added
Final Event Builder and HLT infrastructure
Athena monitoring at EF level
Added for calorimeters
First connection Online Monitoring to Data Quality desk

6. Atlas in M3 First integration exercise for L1 calo
DAQ, DCS, no trigger
SCT added (only DAQ, no detector)
TRT added (part of the detector)
MDT endcap added (sectors 10,11,12)
RPC trigger via final distribution
50-60 Hz, downscaled due to Castor limitations
TGC added (sector 9, TGC 1)
Final ROD read-out
Commissioning Trigger Module used
4 Hz/1 sector
~75 % of Tile
HTL added with dummy algorithm
Complete slice of TDAQ
Improved system stability
Typical run preparation time from 3++ h in M2 to < 1 h
Improved checklist system

7. Event with Tile trigger

8. Atlas in M4: calorimeters
Tile Cosmic trigger since May 2007
Tile > 75 %
Cosmic trigger almost complete
LArg
Barrel:
C side complete
Large part of A side
EndCap:
A side
Missing A01
C side not used in M4
Electronics refurbishment on-going

9. Atlas in M4: inner detector
SCT
4 “noise module”
Test of DAQ (1 ROD) and monitoring
TRT

10. Atlas in M4: Muon Spectrometer
MDT
MDT: 224 chambers, 72 ROD, 3 partitions
Barrel A/C (sectors 3,4,5,6), Endcap C (entire BW)
RPC
Complete sector 5
Emulated ROD
Stable online monitoring with GNAM
Trigger given to CTP
TGC
Sectors 9 (TGC 1, 2, 3)
Sectors 10, 11 (TGC 1)
HPT on sector 9
Problem with readout, solved after M4
Extensive use of Oracle configuration DB

11. DCS sub-detector integration
System
FSM
Alarms
CIC 
EXTERNAL
, Cryo 
LAR (EMBA, EMBC, EMECA, EMECC, HECA, HECC)
ROD
FECLV
HECLV
monitoring only HV
monitoring only*
TIL (LBA, LBC, EBA, EBC) LV
MDT (BA, BC, EA, EC)
LV/HV
TGC (C)
IDE
EVCOOL
SCT
ENV PS 
Extended/New systems
System
FSM
Alarms
TGC
TEMP 
monitoring only
RPC  
MDT
GAS
TRT
TIL (LBA, LBC, EBA, EBC) HV

12. Trigger Level 1 HLT Random and pulsed Tile + RPC + TGC
Latency measured
HLT
Baseline Level 2 algorithms
TrigL2CosmicMuon, TrigTRTSegFinder and TrigT2CaloEgamma
Baseline Event Filter algorithms
TrigEFIDCosmic and CaloRec
MDT end-cap
TDC spectrum

13. HLT/DAQ M4 highlights
Hardware the same as that deployed for M3, but bigger…
7 more Online operations nodes
HLT nodes increased by factor 2, i.e. 124 nodes
One LVL2 rack 30*8 =240 L2PUs
Three EF racks 90*8=720 PTs
6 SFO machines (4 used) 13

14. Rate tests and data recording
4 final SFO machines writing events
200 Hz (event size ~ 2 MB), 400 MB/s
With SCT, TRT, LArg, Tile, RPC, MDT
Staged data written to Castor 2
Available throughput ~140 MB/s
50 KHz achieved with LArg only

15. Online monitoring GNAM for low-level monitoring
Tile, MDT, RPC
TGC under development
Online monitoring at level 2
EF for Larg, TRT, SCT
Online event display at EF level
Both Atlantis and VP1

16. Data Quality DQ Jobs and Tools OHP
few (typically ~4) most significant histograms per sub-detector/system to be frequently checked by-eye, no more than O(100) histograms in total (expect ~60)
DQMF online (on DQM Display, see next slide)
automatic histogram checking, number of histograms can be larger than sub-set for DQ shifter (expect ~1000)
Offline histogram check via web browser
histograms for DQ shifter to be checked, again O(100) or less, ~same as online
DQMF offline
to be used first time in M4 access via web browser
Atlantis
2D event display, client/server system
VP1 (3D event display)
running and display on same monitoring PC, export X-display t o ACR DQ desk

18. Counting room(s) and organization
Atlas Counting Room completed
Some satellite counting room in use
Many people around…
Most of them learning operation
Run organized with
Shift leader
DAQ expert
DQ expert
Sub-system shifters

19. Atlas Control Room

20. Events with RPC and TGC trigger

21. Cosmic event with VP1

22. M4 problems Only in “combined mode” Run stability to be improved
Some Even Counter Reset signals lost (Tile)
Lost triggers
Holes in trigger sequence (Tile, TRT, …)
De-synchronization (MDT)
Run stability to be improved
Many runs end with a busy
This problems moves from one detector to another
Connected with lost triggers
Initialization and re-start very slow
Stop/start sequence
ECR management during run to be improved
HLT configuration too slow

23. Integration weeks schedule
Dates
Systems Integration
Detector configuration
Operations
Cosmic run
Training
ACR
M1:
11-19/
DAQ R/O
Barrel Lar & Tile
CTP
Barrel calorimeters
Achieve combined run
2 days
Tile cosmic trigger
N/A
Initial setup: 5 desks
Central DCS M2
28/2 to 13/3 2007
DAQ/EB
DAQ V. 1.7
Muon barrel (S. 13)
Monitoring/DQ
Barrel Muon
Combined runs
Mixed runs
2 x weekd ends
Tile cosmic trigger + RPC cosmic trigger
Periodic cosmic runs after M2
After M2 week
Increase to 7 desks M3
4/6 to 18/6 2007
Barrel SCT
Barrel TRT
Muon EC (MDT, TGC)
Offline
Barrel and End Cap calorimeters
Barrel muon (5&6)
EC muon MDT
Barrel SCT, TRT
EC muon TGC
1st week focus on operations, checklist management, coordination between desks
1 week
Tile + Muon cosmic trigger (side A)
4/6 to 11/6
Towards final layout: 13 desks M4
23/8 to 3/9
2 day setup
2 week ends
Level-1 Calo
HLT
DAQ 1.8
Offline 13
Barrel & EC calos
Barrel & EC muon
SCT R/O
Level-1 Mu, Calo
ATLAS-like operations
Use of DQ assessment
Try also calorimeter trigger
Whole week
Final setup M5
16/10 to 23/10
23/10 to 1/11 ?
ID EC (TRT)
Pixel (probably R/O only)
SCT quadrant
Converge to ATLAS detector M6
November/December
As for M5
ATLAS-like
Run during ECT magnet test
Global cosmic run
ECT Magnets on only A V I L B E F O R U S E
June 16, 2018

24. Next milestone runs M5 (22 Oct – 5 Nov) M6 (Dec ?)
Add Pixel (R/O only), SCT detector quadrant ?
More HLT algorithms and calibration stream
More detector parts
Build on M4; better operations
Improve on DQ/monitoring
Problem fixing
Improve stability (week 1) and get stable runs (week 2)
M6 (Dec ?)
M5 + magnetic field
Only overnight

25. M5 preparation Muon runs (MDT, RPC, TGC) Calorimeter runs
Dedicated to debug and stabilize the system
Running the spectrometer as a whole
1-7 Oct. and 3-9 Dec.
RPC ROD in M5?
Calorimeter runs
Week-ends are dedicated to combined runs
Technical run
To test the entire DAQ/HLT system
Next one Sept.
Organization ongoing

26. Conclusions Very good achievements through Mx runs Good exercise
Converging from “sub-detectors” to “Atlas”
Including off-line prompt analysis in DQA
Start integrating communities…
Good exercise
To discover problems
And solve them during next milestone preparation
To get smooth operation of the detector
To achieve optimal run organization