DQM Services: Status and Plans DQM Workshop (CMS week) – 28 February 2007 Christos Leonidopoulos CERN PH for the EvF/DQM group

DQM Services – 28 Feb 2007Christos Leonidopoulos 2 What is DQM? DQM is: –A monitoring project originally created for the Event Filter Farm –A set of monitoring tools that can be used either on- or off-line: Tree-like structures w/ histograms, profiles, scalars, strings Quality tests that produce warnings, errors, alarms Visualization tools Transfer of monitoring information to remote nodes –A wrapper around ROOT objects that offers the above functionality DQM is not: –A validation suite (just like ROOT is not a physics analysis) –A set of out-of-the-box templates that are ready to use

DQM Services – 28 Feb 2007Christos Leonidopoulos 3 What can DQM do for me? Main reasons for using DQM: –Transfer of monitoring information over network Creation/updates of histograms/profiles/etc transparent to user Suitable for distributed systems –Library with quality tests, alarm generation/storage/distribution Often updated w/ algorithm donations from users –Bonus: Extended usability by CMS helps w/ bug discovering/fixing (EvF, OnSel, all subdetector groups, beam-tests, MTCC, …) Ongoing development over last 2 years: Emilio Meschi (original design, run-control, mentoring) CL (core functionality, rules & alarms library, tech support) Dimitrios Tsirigkas (Web interface), Giulio Eulisse, Andrea Carboni (Qt-GUI) Ilaria Segoni (specialized clients, coordination with detector groups) Kurt Biery (DQM data & Storage Manager)

DQM Services – 28 Feb 2007Christos Leonidopoulos 4 DQM: On-line mode On-line operating mode: Monitoring producers server monitoring consumers Useful when: Need to access info from multiple monitoring producers Need clear separation of production (sources) and processing (client) of monitoring information (e.g. Filter Farm) Monitoring producers Monitoring consumers See Emilios slides

DQM Services – 28 Feb 2007Christos Leonidopoulos 5 DQM: On-line mode #2 Clients Super Clients CERN Network Non-CERN Network What if you want something simpler? See Emilios slides

DQM Services – 28 Feb 2007Christos Leonidopoulos 6 DQM: Off-line (standalone) mode Merge source & client; eliminate collector Useful when one wants to: First access/produce monitoring information Then process it In the same program! Equivalent to off-line or validation mode Equivalent to a standalone ROOT macro Monitoring producer Monitoring consumer Single program/node

DQM Services – 28 Feb 2007Christos Leonidopoulos 7 DQM from a users perspective Client DQM Monitoring information Configuration Reference objects Historic plots Etc… Comparison-to-reference Collation of similar objects Database Tools Alarm System ok Create monitoring information Process: Visualize, analyze, determine quality, store (à la ROOT or kumac macro) Possible today, but not managed by DQM

DQM Services – 28 Feb 2007Christos Leonidopoulos 8 2 Core Features #1: The Basics Support for all the usual stuff: static and dynamic sets of objects 1,2,3-D histograms, 1,2-D profiles, integers, floats, strings (ROOT objects behind the scenes) Support for tree-like directory structures 1 3

DQM Services – 28 Feb 2007Christos Leonidopoulos 9 Core Features #2: Quality Tests Library with rules for assigning quality value to tests Comparison to reference (χ 2, Kolmogorov tests) Contents within range ([x min, x max ], ([y min, y max ]) Exact match Mean of (e.g. gaussian) distribution near expected value Check for dead channels Check for noisy channels Mean & RMS of bins within allowed range Quality tests run automatically by DQM Results attached to Monitor Elements Can be saved in ROOT file for off-line use Alarm library Warnings & error messages propagating to all clients downstream

DQM Services – 28 Feb 2007Christos Leonidopoulos 10 1 Quality Tests: Examples 23

DQM Services – 28 Feb 2007Christos Leonidopoulos 11 Core Features #3: Output/Input Support for ROOT-files Create and save ROOT-files w/ (any subset of ) monitoring structure on the fly (including results of quality tests) Can also read back file (playback), create MonitorElements, attach quality test results (1_2_0 feature)

DQM Services – 28 Feb 2007Christos Leonidopoulos 12 Web interface v2.0

DQM Services – 28 Feb 2007Christos Leonidopoulos 13 Customized SiStrip Web interface

DQM Services – 28 Feb 2007Christos Leonidopoulos 14 DQM Iguana Qt-GUI #1

DQM Services – 28 Feb 2007Christos Leonidopoulos 15 Cosmic test data for calorimeter detector (reading from file) DQM Iguana Qt-GUI #2

DQM Services – 28 Feb 2007Christos Leonidopoulos 16 Recent highlights Added the ability to group MEs from different folders/nodes: Tags Similar to google mails labels Improved interface for accessing MEs Can open ROOT files in playback mode Fit functions/results can be transferred to remote nodes Improvements in code stability

DQM Services – 28 Feb 2007Christos Leonidopoulos 17 Historic Plots & DQM

DQM Services – 28 Feb 2007Christos Leonidopoulos 18 Open items & to-do lists (savannah) DQM support requests: DQM bugs: DQM tasks:

DQM Services – 28 Feb 2007Christos Leonidopoulos 19 DQM documentation Release notes for latest developments Instructions for getting started Archive with presentations from DQM group: DQM status for subdetector groups Twiki page with discussions, notes, manuals, links