Page 1 of 12

Howto: BMC RBA Adapter for BEM Author: Markus Dillmann (markus_dillmann@bmc.com) Version: 1.2, 2009-02-17

Monitor Adapter (getting Events from BEM)

This section describes how to setup BEM and RBA to receive and handle events from BEM. Please refer to the RBA BEM Application Adapter Users guide for more detail regarding the BEM adapter.

Initial setup

This is the sample configuration from the adapter documentation:

<config> <port>1859</port> <gateway-name>RBAGW</gateway-name> <encryption-key>mc</encryption-key> </config>

Basically you are free on what port the monitor adapter listens (as long as it is not used by another application). Gateway-name can also be chosen at will as long as you follow the BEM naming rules .1 The encryption-key is default “mc” across the BEM product. It is strongly recommended that this value be change to something else, which can be up to 63 characters in length. After you setup up the monitor, you need to inform the BEM environment about it. To do so, add the following info to the mcell.dir located in $MCELL_HOME/etc. Note: If you have multiple BEM/SIM cells on different machines which will send events to RBA you have to add the information to all mcell.dir-files. If all cells are on the same box, you only need to add that once. From the docs:

# Line to add in mcell.dir gateway.rba RBAGW mc localhost:1859

“RBAGW” is the name you chose in the adapter config. “1859 is the value you provided in the port element of the adapter config. If the RBA-adapter doesn’t run on the local box specify the ip-adress or the FQDN instead of localhost. The encryption key (“mc” in this example) must match the one in the adapter config. “gateway.rba” : “gateway” is a fixed text but “rba” is the name that is used to identify the RBA to use. It is perfectly legal to have multiple RBA-entries here like:

1 sequence of characters excluding blanks and " ' ; : , = ( ) [ ] { }

Page 2 of 12

gateway.rba-customer1 RBAGW mc localhost:1860 gateway.rba-dev RBAGW devkey localhost:1861

After making changes to mcell.dir restart BEM. If restart is not possible, use the BEM CLI “mcontrol –n <cellname> reload dir” to force the reload of the directory file. Each of the “gateway instances” you configured (normally just one named gateway.rba) needs a configuration file which controls the format how the information is sent to the recipient. Create a new file named “gateway.rba” with the following content in the “etc” directory of the Impact Manager:

cond.new=propagate cond.mod=always

slots.new=[$ALL] slots.mod=[mc_ueid, $MODS] init.new=$CLASS;\n init.mod=modify $GHANDLE;\n body=\t$NAME=$VALUE;\n term=END\n

If you named your gateway differently in mcell.dir (e.g. gateway.rba-customer1) create the file with that name but the content shown above.

Setting up a BEM policy to automate handling of events by RBA

The fastest/easiest way to automate the handling of events with RBA is to create a Notification Policy. A Policy is divided in two parts: the selector, which defines the conditions an event must satisfy, and the policy itself. First create a selector:

Page 3 of 12

Switch to the “Administration” tab (arrow) and select/expand the cell you want to create

the selector. In the toolbar click the leftmost-icon : . Then enter a name for the selector (required) and specify the event-class for the field “Base Event Class”. Via the “add” button you can add additional conditions that events must match (e.g. severity, specific hosts the event came from). In the example above the selector triggers on all events of class “PROCESS_HANG” (this is not a standard BEM-class but a custom one). You can choose any class (e.g. the root-class EVENT or PATROL_EV) and further qualify which events you want to process with additional conditions. This limits the number of events sent over to RBA. On the RBA monitor-adapter you also specify “rules” (conditions) which will then trigger the RBA-workflow. You have to judge how much you filter on BEM and RBA. Now create the policy which propagates the events to our RBA monitor adapter:

Page 4 of 12

First select the Propagation Policy on the left and then hit “Add Event Policy ” at the top. (Note: if you do not first select the policy type but hit add first, a dialog shows up asking to choose the type of policy to create. Simply choose “Propagation Policy” here. After hitting add, BEM asks for the selector which should be used with that new policy:

Page 5 of 12

Choose the Selector you created in the first place. In the now following policy-details choose the “rba” for “Propagate to One Of” and enter a name for the policy. “rba” will be the name you chose in mcell.dir for “gateway.xxx” – see page 1.

Page 6 of 12

Now you’re done. Once saved, all new events satisfying the selector will be sent to RBA.

Send an event over to RBA manually

You way want to send events over to RBA at a later time instead of when they are initially received and processed (with the policy). This allows BEM-rules to control the forward to RBA at a later time based on other events or conditions that occur after the original event. E.g. you can have a timer-rule which sends the event to RBA for handling if not acknowledged by an operator within 10 minutes. Also, you can use this to have a context action in IX (Impact Explorer) to allow the operator to handle an event by RBA on demand. This setup requires several steps:

1. Create an additional slot (aka attribute) which will control the forward 2. Create a rule which reacts on this slot 3. Create an IX menu action

To create a slot, you must first create a BEM-class (e.g. named “rba.baroc”) file with the following content, put it in “.load”, compile and restart the cell. (See KB-Reference guide for more info on that process):

Page 7 of 12

MC_EV_CLASS : EVENT DEFINES { handle_rba : STRING; }; END

Create the following rule (e.g. named rba.mrl) in the BEM-rules directory, activate it by putting it in “.load” for the rules, compile and restart:

execute handle_rba_interactive: EVENT ($EV) when $EV.handle_rba=='true' { send_to ('rba',$EV); } END

Now add the following to the file “mc_actions.mrl” in the BEM-bin dir. Also needs compilation:

action 'Handle Event with RBA': { ['Service Administrators'] } : EVENT($E) { set_list_slotvalues([$E],[handle_rba],[true]); } END

You can name the action anything you like. Also Role-based access can be given (see KB-ref for details). Note: you can combine all compiles, by doing all changes first and then one single compile run. After successful compilation and restart the context-sensitive action is available:

Page 8 of 12

Any change to the value “true” of slot “handle_rba” e.g. by a timer-rule will trigger the send to RBA.

Notes on updated BEM events

The above methods use intentionally two different ways of transporting the info to RBA. The propagation (which is used with the policy) maintains a link between the event in BEM and the event as sent to RBA. This means that modifications to the event that occur later in the event’s lifecycle are automatically sent over to RBA as event updates. For example if the event is closed (by an operator) after it’s a few days old, this close-information is sent to RBA and – depending on how you set up the rules in RBA – may trigger workflow again. The interactive solution used the send_to mechanism – using this the events are not coupled so later updates to the event are not forwarded to RBA. You could change the interactive solution to use the propagation-mechanism too (by using an propagation-rule instead of an execute-rule), but not the other way round: the propagation policy can’t be changed to “send_to” without causing severe side-effects. A new event looks like this:

Page 9 of 12

<event> <cell-name>bem</cell-name> <class-name>PROCESS_HANG</class-name> <mc-arrival-time><![CDATA[1217251315]]></mc-arrival-time> <msg><![CDATA[Hung process detected!]]></msg> <severity><![CDATA[CRITICAL]]></severity> <mc-date-modification><![CDATA[1217251315]]></mc-date-modification> <date><![CDATA[20080728082155.000000-360]]></date> <mc-object><![CDATA[cpuhog.sh]]></mc-object> <date-reception><![CDATA[1217251315]]></date-reception> <mc-host><![CDATA[fra-mdil-sun-01.bmc.com]]></mc-host> <mc-local-reception-time><![CDATA[1217251315]]></mc-local-reception-time> <mc-host-address><![CDATA[192.168.116.129]]></mc-host-address> <mc-ueid><![CDATA[mc.bem.88dc7f3.0]]></mc-ueid> <mc-location><![CDATA[bmc.com]]></mc-location> <mc-history><![CDATA[[bem:2648]]]></mc-history> </event>

whereas an update looks like this: <event> <status><![CDATA[CLOSED]]></status> <mc-ueid><![CDATA[mc.bem.88dc7f3.0]]></mc-ueid> </event>

An update-event only contains the updated slots (may be more than one) and the unique-id (mc-ueid) which uniquely identifies the event inside BEM. If you want to trigger only on new events, you can set up the rule in RBA to check for the existence (and perhaps the value) of the slot/attribute class-name. As class-names can’t change in BEM, you will never receive an update-event containing this. An example rule:

Page 10 of 12

An alternative approach to distinguish new and update-events is to add a tag in the gateway-configuration like this:

slots.new=[$ALL] slots.mod=[mc_ueid, $MODS] init.new=$CLASS;\nBEMEventType=NEW;\n init.mod=modify $GHANDLE;\nBEMEventType=MOD;\n body=\t$NAME=$VALUE;\n term=END\n

This instructs BEM to send a new “pseudo-slot” with each event to RBA which contains the value “NEW” for new events and “MOD” for modified ones. Events in RBA will look like this:

<event> <cell-name>bem</cell-name> <class-name>PROCESS_HANG</class-name> <mc-arrival-time><![CDATA[1217251315]]></mc-arrival-time> <BEMEventType><![CDATA[NEW]]></BEMEventType> <msg><![CDATA[Hung process detected!]]></msg> <severity><![CDATA[CRITICAL]]></severity> <mc-date-modification><![CDATA[1217251315]]></mc-date-modification> <date><![CDATA[20080728082155.000000-360]]></date> <mc-object><![CDATA[cpuhog.sh]]></mc-object> <date-reception><![CDATA[1217251315]]></date-reception> <mc-host><![CDATA[fra-mdil-sun-01.bmc.com]]></mc-host>

Page 11 of 12

<mc-local-reception-time><![CDATA[1217251315]]></mc-local-reception-time> <mc-host-address><![CDATA[192.168.116.129]]></mc-host-address> <mc-ueid><![CDATA[mc.bem.88dc7f3.0]]></mc-ueid> <mc-location><![CDATA[bmc.com]]></mc-location> <mc-history><![CDATA[[bem:2648]]]></mc-history> </event>

<event> <BEMEventType><![CDATA[MOD]]></BEMEventType> <status><![CDATA[CLOSED]]></status> <mc-ueid><![CDATA[mc.bem.88dc7f3.0]]></mc-ueid> </event>

So you can easily set up the RBA-rules to trigger the workflow you want. If you want to react properly on modifications (e.g. when a close of an event is received and you need to access other values from the event besides the “status”-slot) you may need an additional workflow-step which reads the complete event from the BEM based on the mc-ueid and pass this on.

Troubleshooting

If your workflow doesn’t get started on RBA, you first should check if BEM correctly sent over the event. If you used the propagation-method, BEM keeps track of that and helps you diagnose problems. Select the event in question and click on the “Internals” tab. Look at the field “Propagations”:

Page 12 of 12

If this contains the name of your gateway followed by a number, then the event was received by the RBA adapter and acknowledged. In this case check RBA’s process log – perhaps the filters there are not correct. If you see something like “PEND:” followed by a number in the Propagations-slot, this means that the RBA-adapter could not be connected. Perhaps the adapter/peer isn’t started or you have networking issues (e.g. no connectivity or wrong encryption-key ). Fix that and BEM will retry the propagations automatically each 10min by default. If you are in a hurry, you can tell BEM to force-retry propagations immediately via BEM-CLI “mcontrol –n <cellname> prop” If you see nothing in “Propagations” at all or at least you don’t see your “rba”, this means that your propagation-policy did not trigger. This is mostly likely caused by the event not having the exact conditions you specified in the selector. You could test your selector with e.g. an notification-policy (to send you an email) to verify that the selector works.