1. Soft-IOC Based Alarm Handler Pam Gurd July 14, 2006

2. Why? Wanted to be able to incorporate alarm summaries in edm screens And to call edm screens from alarm screens.

3. Where? Standard soft IOC Alarm_softIOC/Devel…/commonApp/ –src contains the template for the logging sequences –Db contains the database templates. –srcOpi holds the common alarm screens and the templates to be used to build specific alarm screens. –Alarm_softIOC/Devel…/alarmTopApp/srcOpi contains specific menus (for example, for IOC heartbeat screens).

4. Where? continued iocBoot/ics-ioc-linux-alrm soft IOC for common stuff (now alarmModes, later summaries of everything.) Alarm_softIOC/Devel…/xml_handlers contains the scripts that follow. Alarm_softIOC/Devel…/alarmCfg contains the alarm configuration.xml files. Presently defined: –ICS_IOC_HBt.xml – IOC heartbeats (non-RF) –ICS_IOC_HBt_Soft.xml – Soft IOC heartbeats –ICS_RF_IOC_HBt.xml – RF IOC heartbeats –PPS_ICS_Chmk_list.txt – chipmunks –ICS_RF_Util.xml – RFQ LLRF IOC1 utility status –CF_CU_config.xml – example xml produced from a CF.alhConfig file –CHL.xml – CHL alarms

5. What? Set of scripts to create soft IOCs from alarm configuration files. –make_all_alarm_apps.sh – calls make_alarm_app for each configuration file –make_alarm_app.sh – makes an alarm app ( App) and an iocBoot directory for a soft IOC ( - -ioc-linux ) using the standard makeBaseApp templates. SNS-name-centric-ness has been reduced somewhat. (eg, you can define the “App” and “ioc” names you want.)

6. What’s in the application? Scripts define a soft IOC from an alarm configuration file –make_db_files.pl – makes all the EPICS database files: summaries, disable bits, timestamps, alarm logs, etc. –make_screen_substitutions.pl – makes a substitutions file to be used to create the alarm edm screen. –make_alarm_seqs.pl – makes the EPICS image file including the sequence to handle the alarm log –make_arreqs.pl – makes the archive req files (includes, mostly, the alarm log.) –make_ioc.pl – makes the autosave.req file and adds the databases, sequences and the autosave.req to the soft IOC startup command script.

7. Other scripts Utilities for scripts to create soft IOCs from alarm configuration files. –get_alarm_app.pl – translates S, SS and N to sSsN for app name (eg, S=PPS, SS=ICS, N=_Chmk translates to ppsICS_ChmkApp) (We’ll call that sSsNApp later on.) –get_ioc_name.pl – translates S, SS and N to - -ioc-alrm, converting “_” to “-” for app name (eg, S=PPS, SS=ICS, N=_Chmk translates to pps-ics-ioc-linux-alrm-chmk)

8. Configuration File Top-level Summary definition Mid-level Summary definitions (up to 144 for each top-level summary) PV lists for each (up to 144 to each mid- level summary) + all specifications needed to build all the goodies described previously

9. Configuration File Elements XML files parsed using XML:Simple. An contains 1 or more and/or and may contain a. A contains 1 or more and/or and may contain a. A contains 1 or more and may contain a. An contains 1 or more and may contain a. A specifies a channel to watch and may contain a. A specifies a file to show the device or system status and the parameters to use in calling it.

10. Configuration File: attributes sns_top_s – usual SNS system sns_top_ss – subsystem sns_top_n – device instance Or top_pv – to make this less SNS-specific eg These are used for the App name, the soft IOC name, and the PV names for the top- level summary and alarm log.

11. Configuration File: attributes Up to 144 ’s for each sns_s – usual SNS system sns_ss – subsystem sns_n – device instance Or name mode – EPICS PV to define machine mode. mode_desc – Short string to describe machine mode. eg <alarm sns_s=“PPS” sns_ss=“Lin” sns_n=“_ChmkA” mode=“Tgt_ICS:Summary:MachMode” mode_desc=“Tgt”>

12. Configuration File: attributes (up to 144 ’s for each mid-level summary) name – EPICS PV name Or sns_hbt_name and sns_hbt_sig for IOC heartbeats desc – text description of alarm eg

13. Configuration File: attributes file – edm screen filename for details parameters – macro definitions for calling the detail screen type – “edm_file” or “edm_menu” (allows bringing up vxStats/utility/timing screens, for example. Defaults to “edm_file”. eg <display file=“pps_frntend_phase1_3” parameters=“S=Cryo,SS=ICS,N=57,DI=''” type=“edm_menu”>

14. Not complete There are other attributes, but this is long enough already.

15. WARNING Don’t edit the stuff “Under the hood” and then go back and run the script again. Your changes will be deleted.

16. How to Run the Script In $IOCTOP/Alarm_softIOC/ make alarm_apps Performs this command: commonApp/src/make_all_alarm_apps.sh creates an app for each *xml file in alarmCfgs. Or commonApp/src/make_alarm_app.sh / creates one app. Don’t forget the warning!

17. Under the Hood – EPICS Database Files (sSsNApp Db directory) alarmModes.db – Defines machine modes for mid-level summaries (loaded by the soft IOC ics-ioc-linux-alrm.) _ _Alarm.substitutions – defines records using the following VDCT templates: –alarmLatch.template –alarmLog.template –sumLatch.template –topSumLatch.template _ _ Alarm _top.db – records that are directly written by the script - -ioc-linux-alrm.substitutions – standard linuxStats soft IOC status.

18. Under the Hood – Top-Level Summary PVs _ :Summary :StsCurr – current highest severity of included PVs (masked as specified in disable bits) _ :Summary :StsLtch – summary of all lower-level latches _ :Summary :StsNotOK– actual current highest severity of included PVs, not masked by disable bits _ :Summary :ClrCmd – command to clear latches, including all lower-level latches _ :Summary :Ltch – highest alarm severity since last latch clear command _ :Summary :LtchCh – 1 when the latch is becoming set _ :Summary :LtchTS – timestamp the last time the Latch value changed. _ :Summary :DisSumSts – summary of all lower level disables

19. Under the Hood – Logging PVs (for each Top-Level Summary) _ :Summary :AlarmLog – Description of last PV of which the alarm severity increased (masked by disable bits) _ :Summary :AlarmLogTime – the time that the alarm severity increased _ :Summary : AlarmLogSevr – the new alarm severity Similarly, _ :Summary :NotOKLog logs the unmasked alarm PV, and _ :Summary_ioc:AlarmLtchLog logs the latch state Each Log PV is rolled into a set of PVs with a number from 00 to 19 appended to the PV name so the last 20 alarms can be viewed on the Alarm Log screen.

20. Under the Hood – Mid-Level Summary PVs _ :Summary :StsCurr – current highest severity of included PVs (masked as specified in disable bits) _ :Summary :StsLtch – summary of all lower-level latches _ :Summary :StsNotOK– actual current highest severity of included PVs, not masked by disable bits _ :Summary :LtchTS – timestamp the last time the Latch value changed. _ :Summary :Ltch – highest alarm severity since last latch clear command. _ :Summary :ClrCmd – command to clear latches, including all lower-level latches _ :Summary :Dis – disable at this level _ :Summary :DisSumSts – summary of all lower level disables and the disable at this level _ :Summary :StsDis – summary of all lower level disables _ :Summary :ModeSts – state of mode status PV _ :Summary :LtchCh – 1 when the latch severity increases.

21. Under the Hood – for each PV AlarmSts – current severity, masked by disable bits as specified. AlarmLtch – highest alarm severity since last latch clear command AlarmNotOK – actual current severity, not masked by disable bits AlarmDis – disable alarm for this PV AlarmCnt – counts how many times in a row the PV severity has been greater than the latched severity value. Used to delay latch. AlarmLtchCh – 1 when the latch changes AlarmLtchTS – timestamp the last time the Latch value changed AlarmStsCalc – using the “OUT” of this calcout allows unconnected PVs to be masked. AlarmOKCalc – similarly to the above, but for unmasked value. (?) This might look like a lot, but PVs in soft IOCs are pretty cheap. Only 1 CA connection to each real PV.

22. Under the Hood – Alarm Log Sequence (sSsNApp/src directory) The sequence _ _alarmLogSeq.st is created to make alarm logs for edm display. –The PV descriptions go there. –Called 3 times, to make 3 logs corresponding to current status after disables are applied, latched status and unmasked status. Plus Include.dbd and Makefile in support.

23. Under the Hood – Archive reqs (sSsNApp/srcCfg/arCfg directory) _ _Alarm _Summary.arReq to specify archiving the alarm logs.

24. edm Screens – Overall Summary This one isn’t getting generated automatically yet.

25. edm Screens – Top-Level Summary (in sSsNApp/srcOpi directory) Click for details (Alarm details or edm screen from system) Click to change disable status This circle includes current status, latch status, disable status and unmasked status. You can copy it to your own screen.

26. edm Screens – Mid-Level Summary Click for details (edm screen from system) Click to change disable status Here’s another circle you could copy to your own screen. Click for alarm configuration details

27. edm Screens – Alarm Circle Legend

31. Alarm Logs

32. edm Screens – Alarm Log

33. edm Screens – Alarm Configuration (needs work)

34. iocBoot Directory ics-ioc-linux-alrm soft IOC for common stuff (now alarmModes, later summaries of everything.) iocBoot/ - -ioc-linux-alrm directory –“_” in are changed to “-”. –eg pps-ics-ioc-linux-alrm-chmk –contains - -ioc-linux-alrm.req (autosave req file for alarm parameters) and - -ioc-linux- alrm _st.cmd (startup command script.) IOC name can also be specified by using the ioc=“ ” attribute.

35. Future Developments Configuration screen needs work, and need different screens for different record types. Create top-level summaries of everything. Automatically generate a top-level screen. –Big-font screen for good visibility –Clear alarms when okay More examples to find other needs and shortcomings (and, hopefully, because they’d be useful.) Add enable/disable commands to logs. Clear messages when okay.

36. More Future Developments Interface with phone dialer & alarm annunciator Maybe specify using a directive. Alarm priority levels. Link with Oracle database for configuration. Provide a way for Ops to do configuration and boot soft IOCs. Document heartbeat and operating status definitions. Define a “flatline” alarm. Integrate and document.alhConfig conversion.