DCS Instructions K. Grogg, M. Weinberg, M. Grothe 1/24/20161

DCS Monitoring  Check that monitoring is working  Data is being recorded  Bugs in the code  Check that the system is in a good state  Top level state machine  Histograms for each RMC  Enter information in elog  What was checked and results  Details on any errors, alarms, warnings 1/24/20162

Starting  The following slides have step by step instructions for what to look at and what to expect, along with screen shots  Be sure to read Twiki carefully, it has more information than these slides  Start by  Tunnelling  ssh -Y lxplus.cern.ch -L 60001:pcwiscms05.cern.ch:3389  Set up putty and pvss (See Twiki for full set up instructions!)  This only needs to be done once if you save you settings 1/24/20163

Starting PVSS 1/24/20164 Click here to start

Finite state machine  Are the FSM states of unmasked RMCs either standby or ok?  Any known errors should be masked  RMC10 might be off or in error, usually ok  If need to mask, take control (click lock icon), and change check mark to X  Does the alarm/alert overview panel report any warnings and faults?  Does the alarm/alert history show any new alarm/alert entries?  Need to take control (lock icon) to see  Record any new or fixed alarms  Be sure you have released control when done! 1/24/20165

RMC 10 is masked Alarm Overview Take Control 1/24/ Finite State Machine

After taking control Do not leave it like this. Be sure to release control! Mask/Unmask Alarm History Release Control 1/24/20167

RMC GUI  Double click on each RMC to get panel  Click “Send Unprivileged Command”  Is information about the crate sent every minute?  If not, note that there is a monitoring problem, contact an expert  Look at histograms – (time histories) (right click on top of histogram to see actual values)  Is the system healthy?  Are the temperatures, voltages, currents, etc stable?  Look for spikes or slowing rising/falling values  Is the Alarm Status 0, and Online status 1?  Make sure numbers in histograms match those in the “Detailed System Status”  Checks that the values are recorded in the database 1/24/20168

RMC GUI  Double click on each RMC to get panel  Click “Send Unprivileged Command”  Is information about the crate sent every minute?  If not, note that there is a monitoring problem, contact an expert  Look at histograms – (time histories) (right click on top of histogram to see actual values)  Is the system healthy?  Are the temperatures, voltages, currents, etc stable?  Look for spikes or slowing rising/falling values  Is the Alarm Status 0, and Online status 1?  Make sure numbers in histograms match those in the “Detailed System Status”  Checks that the values are recorded in the database 1/24/20169

Getting to RMC GUI Double click to open RMC GUI 1/24/201610

RMC GUI  Double click on each RMC to get panel  Click “Send Unprivileged Command”  Is information about the crate sent every minute?  If not, note that there is a monitoring problem, contact an expert  Look at histograms – (time histories) (right click on top of histogram to see actual values)  Is the system healthy?  Are the temperatures, voltages, currents, etc stable?  Look for spikes or slowing rising/falling values  Is the Alarm Status 0, and Online status 1?  Make sure numbers in histograms match those in the “Detailed System Status”  Checks that the values are recorded in the database 1/24/201611

RMC GUI Panel Check the circled buttons 1/24/201612

RMC GUI  Double click on each RMC to get panel  Click “Send Unprivileged Command”  Is information about the crate sent every minute?  If not, note that there is a monitoring problem, contact an expert  Look at histograms – (time histories) (right click on top of histogram to see actual values)  Is the system healthy?  Are the temperatures, voltages, currents, etc stable?  Look for spikes or slowing rising/falling values  Is the Alarm Status 0, and Online status 1?  Make sure numbers in histograms match those in the “Detailed System Status”  Checks that the values are recorded in the database 1/24/201613

Histograms 1/24/ Check all histograms for any changes/problems

RMC GUI  Double click on each RMC to get panel  Click “Send Unprivileged Command”  Is information about the crate sent every minute?  If not, note that there is a monitoring problem, contact an expert  Look at histograms – (time histories) (right click on top of histogram to see actual values)  Is the system healthy?  Are the temperatures, voltages, currents, etc stable?  Look for spikes or slowing rising/falling values  Is the Alarm Status 0, and Online status 1?  Make sure numbers in histograms match those in the “Detailed System Status”  Checks that the values are recorded in the database 1/24/201615

Temperatures 1/24/ Right click here to get this toggle option and select

Temperatures 1/24/ Axes have been adjusted (dragged) to show all crate temps Temperature should be roughly around these values: Temp A ~ 27 ± 3 Temp B ~ 25 ± 3 Temp C ~ 30 ± 3 Temp D ~ 32 ± 3 Temperature should be roughly around these values: Temp A ~ 27 ± 3 Temp B ~ 25 ± 3 Temp C ~ 30 ± 3 Temp D ~ 32 ± 3

Voltage and Analog Temp 1/24/ Axes have been adjusted to display all voltages For most RMCs +5V is usually V is usually V is usually For most RMCs +5V is usually V is usually V is usually Analog Temp should be in the low 20s For RMC 3 +5V is usually V is usually V is usually -8.7 For RMC 3 +5V is usually V is usually V is usually -8.7

Supply Current 1/24/ Current should be between A, except for RMC 5 which is usually A

Alarm Status 1/24/ Should be zero (0), otherwise there is an alarm!

Online Status 1/24/ Should be one (1) unless there is a known reason otherwise

RMC GUI  Double click on each RMC to get panel  Click “Send Unprivileged Command”  Is information about the crate sent every minute?  If not, note that there is a monitoring problem, contact an expert  Look at histograms – (time histories) (right click on top of histogram to see actual values)  Is the system healthy?  Are the temperatures, voltages, currents, etc stable?  Look for spikes or slowing rising/falling values  Is the Alarm Status 0, and Online status 1?  Make sure numbers in histograms match those in the “Detailed System Status”  Checks that the values are recorded in the database 1/24/201622

RMC GUI Panel Check the circled buttons 1/24/ The values here should match those in the histograms

Closing an RMC GUI 1/24/ Use this button to close, the X doesn’t work

What to expect  Temperatures should be fairly stable  Temp A ~ 27 ± 3  Temp B ~ 25 ± 3  Temp C ~ 30 ± 3  Temp D ~ 32 ± 3  Look for spikes or slowly rising values  Voltages should be very stable  For most RMCs:  +5V is usually 5.06  +12V is usually 12.2  -12V is usually  Analog Temp is in low 20s  Supply current should be around 50  Between A, except for RMC 5 which is usually A  Alarm Status should be zero  Note any changes and when they occurred  Online/Offline should be one  Note any changes and when they occurred  For RMC 3:  +5V is usually 5.7  +12V is usually 14.3  -12V is usually /24/201625

In case of problems  Check “Detailed Alarm Status” and “Detailed System Status”  Know where alarms are coming from  Diagnose the problem  Call an expert  In case of Alarm, check “Most Recent Fault Record” to see error info  Check time stamp  If rack power is switched off, some information may not be updated  Do not Refresh All!  Do not click POWER ON, POWER OFF, CLEAR ALARM, or Further Expert Action!  Be sure to enter everything you find in the elog 1/24/201626