CMS-ECAL DCS Prototype Trigger/DAQ project  DCS CMS DCS Status report ECAL DCS G. Dissertori / ETHZ JCOP Meeting, Dec 2, 2004 ECAL - DCS 10.11.2004

Outline Overview of ECAL DCS, short description of PVSS Monitoring Precision Temperature Monitoring (PTM) Humidity Measurement (HM) ECAL Safety System (ESS) PVSS Monitoring Experience gained with a distributed system during a testbeam campaign (Oct-Nov 04, H4 area, Prevessin) ECAL - DCS 10.11.2004

CMS ECAL 61’200 Barrel Crystals 14’648 Endcap Crystals ECAL - DCS 10.11.2004

Supermodule SM10 Supermodule 1700 Crystals ECAL - DCS 10.11.2004

Overview PVSS Offline, DAQ prec. temp. sensor ESS temp. sensor + WLD Monitoring ESS temp. sensor + WLD Monitoring Safety, PLC interlock Laser system cooling humidity sensor Data Bases ECAL module Offline, DAQ crystal PCBs for VFE and FE PVSS LV, HV cooling, faults ECAL - DCS 10.11.2004

Precision Temperature Monitoring - PTM S. Zelepoukine, IHEP Protvino and ETHZ ECAL - DCS 10.11.2004

PTM Requirements: very high precision temp. monitoring : check stability of water cooling system, to stay within 0.02 - 0.05 deg C. no hardwired feedback to cooling relative precision : 0.01 deg C one sensor/module on thermal screen and grid main water IN, main water OUT ECAL - DCS 10.11.2004

PTM : Readout - Layout Network access Counting room UXC55 balcony: PC / PVSS Counting room CANbus cables DC Power supplies Monitoring application: PVSS based; data storage/archiving, visualization/trending, warnings/alarms. Network access Galvanic isolation ELMB Cur.src UXC55 balcony: 20x2 STP cables Readout electronics: ELMB based; ELMB reads voltages and converts to digital data, then sends data via CANbus. DC-DC converters ECAL ~ 500 sensors (NTC thermistor Betatherm; 100K @ 25C; indiv.calibrated; better than 0.01C rel.accuracy) ECAL - DCS 10.11.2004

PTM @ H4 ECAL - DCS 10.11.2004

Cooling Stability Zoom ECAL - DCS 10.11.2004

Zoom : Grid M4 0.01C M3, M2 M1 Thermal screen M4, M3 M1 0.01C Water - IN 0.01C Cooling stability = 0.01C (incl. day/night oscillations) ECAL - DCS 10.11.2004

Humidity Monitoring - HM S. Zelepoukine, IHEP Protvino and ETHZ ECAL - DCS 10.11.2004

Requirements For HM we want to Sensors Readout Measure humidity level inside modules, precision about 5% Sensors One per module UPS600 from Ohmic Instruments resistive type, cheap, radiation hard Transducers also from Ohmic Instr. Readout Same principles as for PTM : ELMB + Canbus to PC ECAL - DCS 10.11.2004

M1, M3, M4 M2 (conn.broken) RH – inside SM10 RH -- ambient RH cable shield grounded to SM10 M1, M3, M4 M2 (conn.broken) RH – inside SM10 RH -- ambient Amb1/2 (SM10) Amb_3 (DCS rack) water out Water temperatures water in Ambient temperatures ECAL - DCS 10.11.2004

ECAL Safety System - ESS See also CMS NOTE 2004-013 Belgrade group, ETHZ ECAL - DCS 10.11.2004

Schematic layout Three interconnected system layers: Temperature conversion and channel multiplexing - ESS FE Layer, Data acquisition, data processing and interlock generating - ESS PLC Layer, System monitoring and system control - ESS Soft Layer; RS485 ? RS485 ECAL - DCS 10.11.2004

ESS : Sensors Temperature sensors : NTC 470 Ohms thermistors (EPCOS) positioned in pairs at each measurement point ( “Twin” sensors ). Water Leak Detection: TraceTek ECAL - DCS 10.11.2004

ESS : PLC Siemens S7- 400 Family PLC System for ESS: ECAL - DCS 10.11.2004

ECAL - DCS 10.11.2004

M1 M2 M3 M4 28.5 30.6 28.2 25.7 ESS Panel: Temperatures inside the electronics compartment M1 M2 M3 M4 28.5 30.6 28.2 25.7 M2 M1, M3 M4 ECAL - DCS 10.11.2004

Experience up to now Excellent performance of ESS No problem during SM10 running Interlock tests at the startup were successful Afterwards, no interlock ECAL - DCS 10.11.2004

Other Hardware High Voltage Low Voltage Laser monitoring CAEN SY1527 Low Voltage CAEN EASY + CAEN SY2527 Laser monitoring Just received (via DIM interface) basic status parameters Cooling system Complete prototype of cooling system at H4 testbeam area, communication with cooling PLC via OPC ECAL - DCS 10.11.2004

PVSS Monitoring and Control S. Zelepoukine, P. Milenovic, R. Ofierzynski, F. Beaudette, A. Lister, R. Gomez-Reino, F. Glege ECAL - DCS 10.11.2004

PVSS @H4 Have implemented the monitoring of the following subsystems: Temperatures from ESS Water Leakage Detection System (via ESS) Temperatures from PTM Humidity (HM) Cooling System (temperatures, flow, pressures) Status of Laser System (via DIM interface) HV and LV Not implemented/used Link with Run Control was there, but only partly used (reading of status) Temperatures from DCUs Storage to ORACLE DB ECAL - DCS 10.11.2004

PVSS @H4 : Prototype for ECAL First complete set-up of PVSS DCS system includes: PTM - HM - ESS - LV - HV - Cooling - Laser + Supervisory system Every application implements a Finite State Machine Based on PVSS II, v.3, and the JCOP framework (v2.0.9) Distributed system (Windows XP) on 3 PCs : 1 : Supervisor, Laser, Cooling 2 : HV, LV 3 : PTM, HM, ESS ECAL - DCS 10.11.2004

ECAL FSM Structure CMS_ECAL Channel06 Channel07 Channel08 Channel09 ECAL_LV ECAL_HV ECAL_ESS ECAL_LASER ECAL_HM ECAL_PTM ECAL_COOLING ECAL_BARREL- ECAL_ENDCAP+ ECAL_ENDCAP- ECAL_BARREL+ ECAL_SM01 ECAL_SM18 ECAL_SM10 ECAL_COOLING_SM10 ECAL_PTM/HM_SM10 ECAL_ESS_SM10 ECAL_HV_SM10 ECAL_LV_SM10 ECAL_LV_SM10M1 ECAL_LV_SM10M2 ECAL_LV_SM10M3 ECAL_LV_SM10M4 ECAL_HV_SM10M1 ECAL_HV_SM10M2 ECAL_HV_M10M3 ECAL_HV_SM10M4 Channel06 Channel07 Channel08 Channel09 Channel11 Channel12 Channel13 Channel14 Channel15 Channel16 Channel17 Channel18 ECAL - DCS 10.11.2004

ECAL FSM Structure pcethdcs1 pcethdcs2 pcethdcs3 CMS_ECAL ECAL_ESS ECAL_HM ECAL_PTM ECAL_PTM/HM_SM10 ECAL_ESS_SM10 pcethdcs3 CMS_ECAL_PTM_HM CMS_ECAL_ESS ECAL_LV ECAL_HV ECAL_HV_SM10 ECAL_LV_SM10 pcethdcs2 CMS_ECAL_HV CMS_ECAL_LV ECAL_BARREL+ ECAL_LASER pcethdcs1 ECAL_SM10 CMS_ECAL_COOLING ECAL_COOLING ECAL_COOLING_SM10 ECAL - DCS 10.11.2004

Supervisory System : 1 ECAL - DCS 10.11.2004

Supervisory System : 2 ECAL - DCS 10.11.2004

Supervisory System : 3 ECAL - DCS 10.11.2004

HV Application ECAL - DCS 10.11.2004

LV Application ECAL - DCS 10.11.2004

Cooling Application Note: Communication With cooling System via OPC This application only monitors user can not operate the cooling system from here ! ECAL - DCS 10.11.2004

ESS Application Comm. with PLC via OPC ECAL - DCS 10.11.2004

PVSS @H4 : Data Logging For the moment, all data go to the PVSS archive (proprietary) Not easy to get data out (apart trending, which is ok) not foreseen as final solution (!) Done because of pressure to get the FSM at H4 etc running Faced some problem of heavy writing to archive because of difficulty to create alarms out of logical connections of states On the long term : ORACLE A new expert is joining the efforts (Rick Egeland, Minnesota) He will try to help us setting up the commun. with ORACLE and to develop scripts to easily retrieve data Detailed discussions on ConditionsDB and ConfigurationDB just started ECAL - DCS 10.11.2004

Major Issues During development : Already preparing for the final ECAL application Some debugging of Framework tools, in particular FSM and DIM, was necessary. Thanks to C. Gaspar for her feedback. Suffered from missing documentation of Framework Note : The continuous and friendly help by S. Schmeling was absolutely essential !! Thanks!! HV - LV: Major problems with the connection to CAEN crates See later ECAL - DCS 10.11.2004

Experience with distr. PVSS system The different parts of the tree should be created as installable applications and not as complete projects Reduce the overload No PVSS port conflicts Easy to move or recover an application PVSS as a service Naming agreements On the system names On the component/applications names On the folder names On the FSM node names and FSM type names On the library names ECAL - DCS 10.11.2004

Experience with distr. PVSS system System names in all the dp references of the tree components (panels, libs…) should be a dollar/reference parameter that would depend on where the tree part is installed Network in the testbeam area can create problems…. VNC is the best friend for (shared and non-screen-blocking) remote connections It is very difficult to overestimate the needed resources in terms of man power … d;-) ECAL - DCS 10.11.2004

Experience with CAEN crates HV : A1520PE PC in 32 and CAEN 1527 in 867 (cmsechv05) never worked consistently longest period with stable connection about 20 hours average time otherwise: 30 mins both firmware versions (1.14, 2.00.2) were tried PC in 32 and CAEN 1527 (1.14) in 27 (cmsechv00) worked better longest period with stable connection about 5 days average time otherwise: 2-3 days testbeam setup (CAEN 1527 (2.00.2) cmsechv04[->05->03], PC in H4) 1. connected to central infrastructure hub -> no connection, also PCs did     not work on this initially 2. connected to LANtronix switch -> worked in the beginning and then     showed intermittent connection 3. connected to hub (10Mbit old type) in CR -> since then it works without     problem ECAL - DCS 10.11.2004

Experience with CAEN crates LV: (A3100 EASY) PC in 867 and CAEN 1527 in 867 average time: 30 mins (July) end July: little activity otherwise in 867 -> worked well PC in 867 and CAEN 2527 in 157 average stable time: 1 day PC in H4 and CAEN 2527 in H4 (lvecal2) 1. connected to local network (BNC) in lower area -> worked fine for one week suddenly stopped 2. connected with UTP cable to LANtronix switch -> worked shortly but then failed completely 3. connected to central infrastructure hub -> worked shortly but then failed completely 4. connected with UTP cable to LANtronix switch -> worked shortly but then failed completely 5. connected with UTP cable to hub (10Mbit old type) -> since then intermittent connection October 25th: OPC server stopped, LV crate rebooted (hard and power cycle), OPC server restarted -> no change When LV crate not reachable via OPC, also ping not possible (from linux pc), after PC rebooting, ping worked again for 5min, but then OPC and ping showed again intermittent behaviour. October 29th: upgrade of OPC server (pcethdcs2) to 2.8 -> no change ECAL - DCS 10.11.2004

Conclusions Hardware Software PTM / HM / ESS / Cooling: perform according to specs HV + LV (CAEN crates) : problems with network connections, now investigated by experts (from JCOP and CAEN) Software Major efforts gone into development of DCS prototype Mostly achieved our goals Note : A major effort in terms of man power was needed (here : approx. 1y FTE). This should be a warning for similar projects….. ECAL - DCS 10.11.2004