Pixel Systemtest Workshop, , Kerstin Lantzsch Joachim Schultes University of Wuppertal DCS status and experience in the system test

Pixel Systemtest Workshop, , Kerstin Lantzsch Outline 1. short description of system HWHW SWSW system overviewsystem overview 2. General Experience + Special Issues 3. conclusion

Pixel Systemtest Workshop, , Kerstin Lantzsch BBM DCS-PCs CAN-Open protocol TCP/IP Environm. SC-OLink BOC Optoboard Door Cover T Data VPin VISet Regulator Station Wiener Module LV-PP4 T HVVDDVDDA Iseg HV-PP4 T Interlock System Distance from interaction point Sensors The DCS Hardware CAN-Open protocol VVDC BBIM

Pixel Systemtest Workshop, , Kerstin Lantzsch HW HV: 2 Iseg crates (1 channel/Detector-module, 1 iseg module/sector) 16 modules EHS16 modules EHS 8 modules EHS + EHQ8 modules EHS + EHQLV: WIENER ( 2 channels/sector) 4 crates, 12 channels each4 crates, 12 channels each PP4 (½ ELMB/sector) 4 crates à 3 ELMBs à 4 Wiener channels4 crates à 3 ELMBs à 4 Wiener channels Regulators (1 board/sector) 2 crates with 12 Boards each (+ 1 „old“ crate, 2 Boards)2 crates with 12 Boards each (+ 1 „old“ crate, 2 Boards)Optolink: SC-OLink (1 complex channel/sector) 2 crates à 4 ELMBs, 4 complex channels per ELMB (Viset, Vpin, Vvdc)2 crates à 4 ELMBs, 4 complex channels per ELMB (Viset, Vpin, Vvdc)

Pixel Systemtest Workshop, , Kerstin Lantzsch HW, II Environment:BBM 1 crate à 3 ELMBs (2NTC, 1HS)1 crate à 3 ELMBs (2NTC, 1HS)Interlock: BBIM (½ ELMB plug/sector T_mod + 1/2 ELMB plug/6 T_opto (+ T_PP2)) 2 crates à 4 ELMBs à 4 plugs2 crates à 4 ELMBs à 4 plugs BOC-I-BOX (5 racks -> up to 2 crates -> 12 BOCs -> 1-4 sectors) 1 crate à 1 ELMB1 crate à 1 ELMBPP1-BOX LU 2 LU à 4 ELMBs (3 Tmod, 1 FPGA (TOpto, TPP2, IPP1, IBOC, IDSS))2 LU à 4 ELMBs (3 Tmod, 1 FPGA (TOpto, TPP2, IPP1, IBOC, IDSS))IDB-x 1 IDB-SC for 5 SC-OLink crates (5 * 4 * 4 = 80 ELMB channels)1 IDB-SC for 5 SC-OLink crates (5 * 4 * 4 = 80 ELMB channels) 1 IDB-LV for 6 Wiener-crates (6 * 12 = 72 ELMB channels)1 IDB-LV for 6 Wiener-crates (6 * 12 = 72 ELMB channels) 1 IDB-HV for 5 iseg slots (5*2*8 = 80 ELMB Channels)1 IDB-HV for 5 iseg slots (5*2*8 = 80 ELMB Channels)

Pixel Systemtest Workshop, , Kerstin Lantzsch SW FITs: functional order SIT: geographical order FSM: hierarchical DDC: DAQ-DCS communication FSM commands + set voltages + Temps

Pixel Systemtest Workshop, , Kerstin Lantzsch overview of systems in SR1, I dcs2dcs1 dcs3 _DriverConnection Peak (iseg) Iseg OPCServer PVSS: OPCClient (num 3) (-data dcs3 –event dcs3) _DriverConnection KVASER (ELMBs) CANopen OPCServer PVSS: OPCClient (num 2) (-data dcs3 –event dcs3) WIENER OPCServer PVSS: OPCClient (num 14) (-data dcs3 –event dcs3) UserInterface (SR1EnvironmentCooling_Disc.pnl) network drive S: -> D:\PVSS_Project_Update(Subprojects) \\dcs1.pixel.cern.ch\PVSS_Project_Update _FW [21] Data-, Event- Manager User Interface Control Manager -num 11 (FSM), 20 (Cooling) Subprojects on S:\... dist -> [3], [40] Sysname: ATLPIX_FSM_dcs1 network drive S: -> \\dcs1.pixel.cern.ch\PVSS_Project_Update _FW_DDC [33] Data, Event, UI DIM Ctrl 12 (DDC Handling) Subprojects on S:\... dist -> [3], [21] Sysname: ATLPIX_DDC_dcs3 ATLASPixDCS [3] Data, Event, UI Archiving (RDB) Ctrl 10 (ELMB Watchdog) Subprojects on S:\... dist -> [40] Sysname: ATLPIX_3

Pixel Systemtest Workshop, , Kerstin Lantzsch overview of systems in SR1, II control3 _FSM [26] Data-, Event- Manager User Interface (DEN) dist -> [3], [21] Subprojects on S:\... Sysname: ATLPIX_FSM_CONTROL3 dcs4 _CAN-ELMB_FW [4] KVASER (ELMBs) CANopen OPCServer PVSS: OPCClient (num 7) Data-, Event- Manager User Interface (DEN) Control Manager dist -> [3] Sysname: ATLPIX_4 ServiceTest ServiceTest [99] Data-, Event- Manager ServiceTest-MainPanel (GE) dist -> [3] Subprojects local Sysname: ATLPIX_ServiceTest _RemoteUI Remote User Interface ATLASPixDCS (dcs3): SR1Menu.pnl nezwork drive S: -> \\dcs1.pixel.cern.ch\PVSS_Project_Update

Pixel Systemtest Workshop, , Kerstin Lantzsch PP2 many problems with PP2 crates for endcap new hardware slower, was not taken into account in PVSS (FIT and FSM), commands were sent while Controller was still busy no reliable execution of commands -> introduction of very conservative delaysnew hardware slower, was not taken into account in PVSS (FIT and FSM), commands were sent while Controller was still busy no reliable execution of commands -> introduction of very conservative delays no ADC reading (no feedback about Vset) -> missing -15V supply for controller, need PS monitored by PVSSno ADC reading (no feedback about Vset) -> missing -15V supply for controller, need PS monitored by PVSS problems in ELMB/FPGA code -> already solved by firmware upgradeproblems in ELMB/FPGA code -> already solved by firmware upgrade move trimmer to inhibit/uninhibit channel,move trimmer to inhibit/uninhibit channel, GU when moving trimmer of Board1, Ch1,GU when moving trimmer of Board1, Ch1, sending inhibit for one channel caused another channel to be inhibitedsending inhibit for one channel caused another channel to be inhibited Inhibiting/Uninhibiting of Channels 1 when reading T1/T2 after powering up PP2Inhibiting/Uninhibiting of Channels 1 when reading T1/T2 after powering up PP2 „Reset“ (i.e. set all Trimmers to Min) does not work properly in SR1 -> solved with new firmware version probably was not recognized more clearly, because most of the time PP2 in SR1 was operated in the way that trimmersteps were not calculated directly, but trimmer was set via moving to minimum„Reset“ (i.e. set all Trimmers to Min) does not work properly in SR1 -> solved with new firmware version probably was not recognized more clearly, because most of the time PP2 in SR1 was operated in the way that trimmersteps were not calculated directly, but trimmer was set via moving to minimum no Imon, because new crates were already defined, and monitoring with PDO3 was not debugged previously with real HW -> changing to PDO3 now done for PP2_S21, PP2_S22no Imon, because new crates were already defined, and monitoring with PDO3 was not debugged previously with real HW -> changing to PDO3 now done for PP2_S21, PP2_S22

Pixel Systemtest Workshop, , Kerstin Lantzsch PP2 New Communication Mechanism: Acknowledgment between each node: FPGA ELMB PVSS Commands are acknowledged (no chance of loosing command without noticing between FPGA and PVSS)Commands are acknowledged (no chance of loosing command without noticing between FPGA and PVSS) only as much delay as necessaryonly as much delay as necessary 1000 ms -> 10 ms (Inhibit)1000 ms -> 10 ms (Inhibit) 3000 ms -> 200 ms (100 Trimmer Steps)3000 ms -> 200 ms (100 Trimmer Steps)

Pixel Systemtest Workshop, , Kerstin Lantzsch PP2 Also: Watchdog between ELMB and FPGAWatchdog between ELMB and FPGA FPGA can reset ELMBFPGA can reset ELMB FPGA failure is recognized by ELMBFPGA failure is recognized by ELMB Reduction of number of commands sent by PVSS to ELMBReduction of number of commands sent by PVSS to ELMB Readout will be stopped/started for commands by ELMBReadout will be stopped/started for commands by ELMB PDO will be sent to signal routine mode (still makes sense to stop routine before long sequence of commands)PDO will be sent to signal routine mode (still makes sense to stop routine before long sequence of commands) New CommandsNew Commands Firmware version of FPGAFirmware version of FPGA Firmware version of ELMBFirmware version of ELMB multiple commands reasonable? e.g. „inhibit_Board“? „inhibit_Modules“...? can be implemented on a short timescale in ELMB codemultiple commands reasonable? e.g. „inhibit_Board“? „inhibit_Modules“...? can be implemented on a short timescale in ELMB code Number of ADC bits for Monitoring values will be increased (8 bit now)Number of ADC bits for Monitoring values will be increased (8 bit now) faster reading of monitoring values possiblefaster reading of monitoring values possible

Pixel Systemtest Workshop, , Kerstin Lantzsch iseg some iseg channels were not switching off, when using „SR1DirectLink“ (while switching off one disk at a time and switching on worked) FSM: order of switching on introduced delays (-> PP2) before switching on iseg- channels -> not so many commands were sent simultaneouslyFSM: order of switching on introduced delays (-> PP2) before switching on iseg- channels -> not so many commands were sent simultaneously introducing delay between commands for modules also helped (but not 100%)introducing delay between commands for modules also helped (but not 100%) problem was the way PVSS accessed the opc items: 1 item for each iseg module with one bit for each channel1 item for each iseg module with one bit for each channel this item is „split“ by OPC client and „distributed“ to the DPEs (trans.type bitstring, DPE bool)this item is „split“ by OPC client and „distributed“ to the DPEs (trans.type bitstring, DPE bool) old values for first bits caused channels to be turned on againold values for first bits caused channels to be turned on again using now opc item for iseg channel instead of module, and „write only“ trans.type bool for DPE booltrans.type bool for DPE bool no more problems seen till now (switching off 4 times, first week of january)no more problems seen till now (switching off 4 times, first week of january)

Pixel Systemtest Workshop, , Kerstin Lantzsch FSM many problems seen in system test were problems of underlying layer (PP2, iseg) missing functionality: order for switching on/off should be flexibleorder for switching on/off should be flexible also for example switching on without HV should be possiblealso for example switching on without HV should be possible command for reading in of (default) values (->SIT)command for reading in of (default) values (->SIT) take into account HV for state for modularity 1 (done for 6/7)take into account HV for state for modularity 1 (done for 6/7) overall status of FSM („idle“, „active“) instead of only on ROgroup leveloverall status of FSM („idle“, „active“) instead of only on ROgroup level SW-interlock implemented but not yet testedSW-interlock implemented but not yet tested

Pixel Systemtest Workshop, , Kerstin Lantzsch SIT provides mapping for FSMprovides mapping for FSM improved xml parsing (mapping and configuration)improved xml parsing (mapping and configuration) reading in of configuration filesreading in of configuration files

Pixel Systemtest Workshop, , Kerstin Lantzsch DDC supports all FSM commands supports all FSM commands SWITCH_ONSWITCH_ON SWITCH_OFFSWITCH_OFF RECOVERRECOVER RESETRESET ENABLEENABLE DISABLEDISABLE Setting of Voltages Setting of Voltages SET_VDDASET_VDDA SET_VDDDSET_VDDD SET_HVSET_HV SET_VISetSET_VISet SET_VPinSET_VPin SET_VVDCSET_VVDC Publishing of values: Publishing of values: Set Voltages, measured Voltage, measured CurrentSet Voltages, measured Voltage, measured Current T_Module, T_OptoT_Module, T_Opto improved monitoring of necessary processes in work (dim, dist, ctl)improved monitoring of necessary processes in work (dim, dist, ctl) boc monitoring in work (DT)boc monitoring in work (DT) message transfer (MT) todomessage transfer (MT) todo

Pixel Systemtest Workshop, , Kerstin Lantzsch Archiving, Trending PVSS access to Oracle database? trending very painful for large amount of DPEs or longer timespan (trying to access 24 T_Opto-DPEs of ~2 weeks before almost impossible with PVSS trending, but also problems with recent trends) -> PVSS dataviewer

PVSS Data Viewer What's new : Improved data export Possibility to filter values depending on their PVSS status word Improved data extracting (faster and can now display deleted elements) To do : Un-/Aliasing of DPEs Additional information on requests Different display options ( Olivier Pisano

Pixel Systemtest Workshop, , Kerstin Lantzsch Temperature reading (BBIM/BBM) BBMs were defined as standard ELMB Temperatures had to be converted from Voltages manuallyTemperatures had to be converted from Voltages manually new ELMB-type „BBM_NTC“new ELMB-type „BBM_NTC“ Problem with Temp-conversion in OPC-server (BBIM, BBM) if disconnected (Vref = VNTC) value cannot be evaluated by OPCserver no check possibleif disconnected (Vref = VNTC) value cannot be evaluated by OPCserver no check possible if value ok again, does not get updated immediately inside PVSS wait sometimes minutes for up-to-date value (looks like combination of „division by zero“ in OPC-Server and PVSS-smoothing) GQ gets actual value, but not practicableif value ok again, does not get updated immediately inside PVSS wait sometimes minutes for up-to-date value (looks like combination of „division by zero“ in OPC-Server and PVSS-smoothing) GQ gets actual value, but not practicable no smoothing, Vref = const 2.5V seems to improve this: update with next SYNCno smoothing, Vref = const 2.5V seems to improve this: update with next SYNC archiving?archiving? amount of data without smoothingamount of data without smoothing change of value propagated to archive if smoothing applied there?change of value propagated to archive if smoothing applied there?

Pixel Systemtest Workshop, , Kerstin Lantzsch conclusion iseg problem seems solvediseg problem seems solved PP2: many updates in the next few days scheduledPP2: many updates in the next few days scheduled FSM: much additional functionality desiredFSM: much additional functionality desired