1. CO HW Monitoring Architecture
FESA3 LOGS & DIAGNOSTICS
WG Weekly meeting, 30/06/1016, F.Locci

2. FESA3 METRICS
C2MON server
FESA3 provide metrics of the Core component through CLIC-CMX agent mechanism
MONITORED DATA
Components: CMW_SERVER, CONCUR_LAYERS, EVENT_SOURCE, LOGICAL_EVENTS, NOTIFICATION_QS, ON_DEMAND_QS, RT_ACTIONS, SERVER_ACTIONS
Unidirectional (no command, setting)
Other CO-SRC libraries provide Metrics in the same way, like: Timing GMT, COHAL, etc.
Please make the Metrics browser (CMX) enabled by default !!
JMS
CLIC DAQ
JMS
JMS/STOMP broker
STOMP
CLIC agent
CMX library
FESA3
Metrics
SHM
CMX library
FESA3 core

3. CMW-LOG server JMS/STOMP broker
FESA3 LOGS & DIAGNOSTICs
DIAGNOSTICS
CMW Log GUI
FESA3 Navigator
FESA3 uses the CMW-logger to trace the FESA server activities
Both USER (specific) and FRAMEWORK topics can be logged from the FESA server
Unidirectional (no command, setting)
Framework LOGS:
ERROR, WARNING, INFO: common for operation and support (runtime)
DEBUG, TRACE: limited use for experts (development, test)
JMS
CMW-LOG server JMS/STOMP broker
STOMP/
UDP
Framework DIAGNOSTICS
PROCESS CONFIG & STATUS: memory alloc., resources usage, counters, etc.
Redundant with metrics (FESA 2.10 legacy).
TRACKING & PROFILING: execution flow, perform. measures, etc.
Logs cfg. file
CMW-log library
FESA3 core

4. CGMON: FAN tray monitoring via FESA3 generic classes
Required from CO-IN installation team to periodically query via SNMP the crate status information.
SPECIFIC REQUIREMENT From OP (to TE-ABT) for critical LHC HW with redundant powers: beam/no-beam from SIS? (no interlock from DIAMON)
HARDWARE ELMA crate, WIENER crate, XLI GPS receiver, WR crate (NEW)
MONITORED DATA
Common: Crate ON, Powers mod., Current, FanSpeed, Temp., Voltage, …
Specific: XLI GPS, CTB
PROJECT STATUS
Open & flexible solution for centralized monitoring (polling)
Relies on standard protocol (e.g.: SNMP)
About 40 devices so far
KIBANA Dashboard not up-to-date

5. Discussions?
Logs and monitoring tools provide numerous and various services, like: Infrastructure monitoring, services & applications logging, diagnostic, alarms, metrics, statistics, etc.:
We need to clarify use and purpose of each
Where are the overlaps?
Single access-points, GUI?
Should we provide self-service solutions (for whom?) … how to control the volume and quality?
Should we better provide centralized solutions?
Pre-processing at the source level, or massive processing/filtering of raw data at server level (DAQ?, C2MON?)
We still use many house-made solutions …. (e.g.: system parameters files parsing!)
Are there modern and standard technologies that could do the JOB?
Where to use OTS solutions: Low-level, middleware, GUI … everywhere?
Risk with OTS solutions (GUI in particular)?: non-stable, too much generic  take time to learn, not fully adapted to the needs, …?
Shouldn’t we be more conservative with monitoring and diagnostic tools than any other Control software?
We need some statistics/survey about logs and monitoring?
Categorize the logs (Infrastructure monitor., Alarms, Debug, Diagnostics, …)
Volume, performance impact, usage (%), …
Who use it, which tools, to do what, how effectively?