Soft-IOC Based Alarm Handler

Pam Gurd

May 25, 2006

Why?

• Wanted to be able to incorporate alarm summaries in edm screens

• And to call edm screens from alarm screens.

Where?

• Standard soft IOC• Alarm_softIOC/Devel…/commonApp/

contains everything needed to produce the individual apps.

• src contains the scripts that follow.• Db contains the database templates.• srcOpi holds the common alarm screens

and the templates to be used to build specific alarm screens.

Where? continued

• iocBoot/ics-ioc-linux-alrm soft IOC for common stuff (now alarmModes, later summaries of everything.)

• Alarm_softIOC/Devel…/alarmLists contains the alarm configuration files.

• Presently defined:– ICS_Ctl_IOC_list.txt – IOC list– PPS_ICS_Chmk_list.txt – chipmunk list

What?

• Set of scripts to create soft IOCs from alarm configuration files.– make_all_alarm_apps.sh – calls

make_alarm_app for each configuration file– make_alarm_app.sh – makes an alarm app

(<s><ss><n>App) and an iocBoot directory for a soft IOC (<s>-<ss>-ioc-linux<n>) using the standard makeBaseApp templates.

• Notice that it’s very SNS-name-centric

What’s in the application?

• Scripts define a soft IOC from an alarm configuration file– make_db_files.pl – makes all the EPICS database files:

summaries, disable bits, timestamps, alarm logs, etc.– make_screen_substitutions.pl – makes a substitutions file to be

used to create the alarm edm screen.– make_alarm_seqs.pl – makes the EPICS image file including the

sequence to handle the alarm log– make_arreqs.pl – makes the archive req files (includes, mostly,

the alarm log.)– make_ioc.pl – makes the autosave .req file and adds the

databases, sequences and the autosave .req to the soft IOC startup command script.

Other scripts

• Utilities for scripts to create soft IOCs from alarm configuration files.– get_alarm_app.pl – translates S, SS and N to

sSsN for app name (eg, S=PPS, SS=ICS, N=_Chmk translates to ppsICS_ChmkApp)

• (We’ll call that sSsNApp later on.)

– get_ioc_name.pl – translates S, SS and N to <s>-<ss>-ioc-alrm<n>, converting “_” to “-” for app name (eg, S=PPS, SS=ICS, N=_Chmk translates to pps-ics-ioc-linux-alrm-chmk)

Configuration File

• Top-level Summary definition

• Mid-level Summary definitions (up to 12 for each top-level summary)

• PV lists for each (up to 12 to each mid-level summary)

• + all specifications needed to build all the goodies described previously

Configuration File Format – General Notes

• Everything is defined using <keyword>=<value>• Keyword examples: TOP_S, TOP_SS, TOP_N,

S, SS, N, PV, EDL, EDL_MACROS• Generally, white space is ignored.• Order matters. For example, TOP_S, TOP_SS,

and TOP_N have to be defined before the S, SS and N of the first mid-level summary to be included.

• Some strings have to be enclosed in quotes (eg. EDL_MACROS=“"M=1“)

• Comment lines starting with “#” are ignored.

Configuration File: Top-level Summary definition

• TOP_S – usual SNS system– eg TOP_S=PPS

• TOP_SS - subsystem – eg TOP_SS=ICS

• TOP_N – device instance – eg TOP_N=_Chmk

• These are used for the App name, the soft IOC name, and the PV names for the top-level summary and alarm log.

• Must appear before the first mid-level summary keyword.

Configuration File: Mid-level Summary definition

• Up to 12 mid-level summaries for each top-level summary• S – usual SNS system, eg S=PPS• SS – subsystem, eg SS=Lin• N – device instance, eg N=_ChmkA• EDL – edm screen filename for details, eg

EDL=pps_frntend_phase1_3• MODE – PV to define machine mode. Eg,

MODE=Tgt_ICS:Summary:MachMode• MODE_DESC – Short string to describe machine mode; eg,

MODE_DESC=Tgt• These are used for the PV names and configuration parameters for

the mid-level summary.• All keywords for the mid-level summary must appear before the next

S keyword.

Configuration File: PV parameters

• (up to 12 PVs for each mid-level summary)

• PV – EPICS PV name– eg PV=ICS:IOC_linux1:HBtNLOK_FE1

• EDL and EDL_MACROS as for mid-level summaries

• DESC – text description of alarm– eg DESC="Front End IOC 1"

WARNING

Don’t edit the stuff “Under the hood” and then go back and run the script again. Your changes will be deleted.

How to Run the Script

• In $IOCTOP/Alarm_softIOC/<vers>make alarm_apps

Performs this command:commonApp/src/make_all_alarm_apps.sh

creates an app for each *list.txt file in alarmLists.

Or commonApp/src/make_alarm_app.sh alarmLists/<your file>

creates one app.

Don’t forget the warning!

Under the Hood – EPICS Database Files (sSsNApp Db directory)

• alarmModes.db – Defines machine modes for mid-level summaries (loaded by the soft IOC ics-ioc-linux-alrm.)

• <S>_<SS>_Alarm<N>.substitutions – defines records using the following VDCT templates:– alarmLatch.template– alarmLog.template– sumLatch.template– topSumLatch.template

• <S>_<SS>_ Alarm<N>_top.db – records that are directly written by the script

• <s>-<ss>-ioc-linux-alrm<n>.substitutions – standard linuxStats soft IOC status.

Under the Hood – Top-Level Summary PVs

• <s>_<ss>:Summary<n>:StsCurr – current highest severity of included PVs (masked as specified in disable bits)

• <s>_<ss>:Summary<n>:StsLtch – summary of all lower-level latches

• <s>_<ss>:Summary<n>:StsNotOK– actual current highest severity of included PVs, not masked by disable bits

• <s>_<ss>:Summary<n>:ClrCmd – command to clear latches, including all lower-level latches

• <s>_<ss>:Summary<n>:Ltch – highest alarm severity since last latch clear command

• <s>_<ss>:Summary<n>:LtchCh – 1 when the latch is becoming set• <s>_<ss>:Summary<n>:LtchTS – timestamp the last time the Latch

value changed.• <s>_<ss>:Summary<n>:DisSumSts – summary of all lower level

disables

Under the Hood – Logging PVs(for each Top-Level Summary)

• <s>_<ss>:Summary<n>:AlarmLog – Description of last PV of which the alarm severity increased (masked by disable bits)

• <s>_<ss>:Summary<n>:AlarmLogTime – the time that the alarm severity increased

• <s>_<ss>:Summary<n>: AlarmLogSevr – the new alarm severity

• Similarly, <s>_<ss>:Summary<n>:NotOKLog logs the unmasked alarm PV, and <s>_<ss>:Summary_ioc:AlarmLtchLog logs the latch state

• Each Log PV is rolled into a set of PVs with a number from 00 to 19 appended to the PV name so the last 20 alarms can be viewed on the Alarm Log screen.

Under the Hood – Mid-Level Summary PVs

• <s>_<ss>:Summary<n>:StsCurr – current highest severity of included PVs (masked as specified in disable bits)

• <s>_<ss>:Summary<n>:StsLtch – summary of all lower-level latches• <s>_<ss>:Summary<n>:StsNotOK– actual current highest severity of

included PVs, not masked by disable bits• <s>_<ss>:Summary<n>:LtchTS – timestamp the last time the Latch value

changed.• <s>_<ss>:Summary<n>:Ltch – highest alarm severity since last latch clear

command.• <s>_<ss>:Summary<n>:ClrCmd – command to clear latches, including all

lower-level latches• <s>_<ss>:Summary<n>:Dis – disable at this level• <s>_<ss>:Summary<n>:DisSumSts – summary of all lower level disables

and the disable at this level • <s>_<ss>:Summary<n>:StsDis – summary of all lower level disables • <s>_<ss>:Summary<n>:ModeSts – state of mode status PV• <s>_<ss>:Summary<n>:LtchCh – 1 when the latch severity increases.

Under the Hood – for each PV• <pv>AlarmSts – current severity, masked by disable bits as specified.• <pv>AlarmLtch – highest alarm severity since last latch clear command• <pv>AlarmNotOK – actual current severity, not masked by disable bits• <pv>AlarmDis – disable alarm for this PV• <pv>AlarmCnt – counts how many times in a row the PV severity has been

greater than the latched severity value. Used to delay latch.• <pv>AlarmLtchCh – 1 when the latch changes• <pv>AlarmLtchTS – timestamp the last time the Latch value changed• <pv>AlarmStsCalc – using the “OUT” of this calcout allows unconnected

PVs to be masked. • <pv>AlarmOKCalc – similarly to the above, but for unmasked value. (?)

• This might look like a lot, but PVs in soft IOCs are pretty cheap.• Only 1 CA connection to each real PV.

Under the Hood – Alarm Log Sequence (sSsNApp/src directory)

• The sequence <S>_<SS><N>_alarmLogSeq.st is created to make alarm logs for edm display.– The PV descriptions go there.– Called 3 times, to make 3 logs corresponding

to current status after disables are applied, latched status and unmasked status.

• Plus <s><Ss><N>Include.dbd and Makefile in support.

Under the Hood – Archive reqs (sSsNApp/srcCfg/arCfg directory)

• <S>_<SS>_Alarm<N>_Summary.arReq to specify archiving the alarm logs.

edm Screens – Overall Summary

• This one isn’t getting generated automatically yet.

edm Screens – Top-Level Summary (in sSsNApp/srcOpi directory)

Click for details (Alarm details or edm screen from system)

Click to change disable status

This circle includes current status, latch status, disable status and unmasked status. You can copy it to your own screen.

edm Screens – Mid-Level Summary

Click for details (Alarm details or edm screen from system)

Click to change disable status

Here’s another circle you could copy to your own screen.

Click for alarm configuration details

edm Screens – Alarm Circle Legend

edm Screens – Alarm Log

edm Screens – Alarm Configuration (needs work)

iocBoot Directory

• ics-ioc-linux-alrm soft IOC for common stuff (now alarmModes, later summaries of everything.)

• iocBoot/<s>-<ss>-ioc-linux-alrm<n> directory – “_” in <n> are changed to “-”.– eg pps-ics-ioc-linux-alrm-chmk– contains <s>-<ss>-ioc-linux-alrm<n>.req (autosave

req file for alarm parameters) and <s>-<ss>-ioc-linux-alrm<n>_st.cmd (startup command script.)

Future Developments

• Configuration screen needs work, and need different screens for different record types.

• Create top-level summaries of everything.• Automatically generate a top-level screen.• For Cryo, need to add latch delay to configuration.• More examples to find other needs and shortcomings

(and, hopefully, because they’d be useful.)• Allow inclusion of PVs in more than one summary.• Remove 12 PV limit.• Link with Oracle database for configuration.• Provide a way for Ops to do configuration and boot soft

IOCs.