CA Workload Automation ESP Edition Wobtrigs and File

CA Workload Automation ESP Edition Wobtrigs and File_Triggers:The Good the Bad and the Ugly

Don Powell – Sr. Support Engineer, CA Technologies

April 26, 2016

2 © 2016 CA. ALL RIGHTS RESERVED.

Agenda

WOBTRIGS/FILE_TRIGGERS – THE BASICS

WOBDEFS AND FILE_TRIGGERS DEFINED

QUESTIONS

MONITORING AND RESOLVING ERRORS

MISC. - REMOTE FILE SYSTEMS, HA SCENARIOS AND OTHER FUN THINGS

WOBTRIGS VS FILE_TRIGGERS AT THE APPL LEVEL

1

2

3

4

5

6

Wobtrigs/File_Triggers – The basics


Wobtrigs and File_Triggers

Wobtrigs

Defined at the event level and are active all the time.

File_Triggers

Defined at the application level ( APPL level )

It is used once and then completed unless the CONTINUOUS keyword is used.


The FILE_TRIGGER statement – Used in both events and applications

FILE_TRIGGER Agent(AGENTU_01) FileName('/opt/data/itz') CREATE

What am I watching for? The simple case…

– What server is the file going to be created on?

– What is the name of the file?

– CREATE, UPDATE, DELETE, EXIST, NOTEXIST, EXPAND, SHRINK

Update requires the file to exist when the Wobtrig/file_trigger is defined.

More complex

– Continuous – Used in an application. It will trigger an alert each time the condition is met

May want to delete and retrigger this daily.

– Recursive – Search all subdirectories

Careful – Especially with large file systems

Careful – With “*” generic searches


How does it do it?

There is no master catalog or SMF records like the mainframe…. It is a simple poll

every 30 seconds (by default) for the condition in the FILE_TRIGGER.

– It is possible to change the polling interval but I would not recommend it.

How does it know not to trigger while the file is still being updated?

– The agent is waiting for a stable file. It will check the file size and then wait the polling

period to recheck. If there have been no changes in the file size, then the file_trigger is

satisfied.

The NOCHANGE parameter can be used on the FILENAME statement to control how

long a file must remain unchanged before the trigger happens.

In rare cases a file that is being updated will stop. FTP and some other tools are smart

enough to understand that the file transfer had an issue and resend.

Wobtrigs and File_Triggers defined


Wobtrigs – Example

Wobtrigs were created to provide an Event Sensor for the unpredictable distributed data events without the need for defining scheduling criteria.

WOBTRIG FILE_TRIGGER AGENT(agent_name) -FileName('/webdata/batch/gis/receive/NAPS*_REQ_*.*') CREATE -

NoChange(1)

After the EVENT is saved and the agent acknowledges that the file trigger is set, it will look like the one below.

WOBTRIG FILE_TRIGGER AGENT(agent_name) -FileName('/webdata/batch/gis/receive/NAPS*_REQ_*.*') CREATE -

NoChange(1) State(Monitored) Status('Monitored for CREATE')


Wobtrigs – Notes

Wobtrigs can be a possible choice if it is unknown what days or how many times per day the file will be created/modified. It is a simple case to implement

Since people are NOT monitoring the state of events, if an event goes into “Scan Failed” it can go undetected unless precautions are taken.

– MAILBOX: Set up an a MAILBOX at the event level so that a group will be notified if an event fails.

– Scripts. People have created REXX routines to monitor for specific states. An example can be found out on the community at https://communities.ca.com/thread/101793452

If there are two nodes that are set up in a failover scenario and the agent(s) are monitoring a single file share it is possible that the event will trigger two times when the file arrives. More on this in the section on HA.

https://communities.ca.com/thread/101793452


Wobtrigs: (cont ‘d)

Suspending or holding an event does not stop the file trigger.

– If the event is suspended, any triggers that come in will be discarded.

– If the event is held, any file triggers that come in will be held based on the OVERDUE count specified in the event’s SCHEDULE statement.

Defining an event that waits for multiple files before triggering using the “MULTIPLE” keyword may give you an unexpected outcome…


File_Triggers in an application – Example

FILE_TRIGGER FILE.001

RUN DAILY

AGENT AGENTU_01

FILENAME '/opt/data/*.tmp' CREATE CONTINUOUS ALERT(FTG1)

ENDJOB


Application level File_Trigger – Notes

For the simple case where a file arrives a small number of times per day, at set times, this is absolutely the best method.

Each file that should show up can have a late time and trigger due out notifications if it hasn’t arrived.

The File_Trigger job is visible to the Monitoring group. It goes late or failed just like any other job they have to monitor.

Using Continuous. Each time the File_Trigger conditions are met it will trigger an Alert. It may be best to trigger this daily and have it complete just before the next day triggers.

Scan Failed jobs will turn red and trigger alerts for the failed process. Just like any other failed job.


Using Variables

ESPWTAGENT – Contains the name of the agent that the file trigger was detected on. Using Continuous. It may be best to trigger this daily and have it complete just before the next day triggers.

ESPWTFILE – Contains the name of the file that file trigger was satisfied for.

File_Triggers using “continuous” can use “Monitor Variables”. These contain information like job name For the simple case where a file arrives a small number of times per day, at set times, this is the best method.

Monitoring and resolving errors


The current state of affairs….

STATE(Failed|In Preparation|Monitored)

Indicates the state of the monitoring request. You do not code STATE; it is displayed in the WOBTRIG or CSF

Failed: The monitoring request failed.

– Example - Status('Scan Failed') LStatus('The /opt/data does not exist or not a directory')

In Preparation: The monitoring request is being prepared.

Monitored: The agent is monitoring.


Scan Failed issue

A Wobtrig/File_Trigger goes into scan failed if it cannot get to the path to look for the file.

In some cases a brief network glitch or a short period where a remote file system is offline can cause a “Scan Failed” error. Since people are NOT monitoring the state of events, “Scan Failed” events can go undetected unless precautions are taken.

MAILBOX: Set up an a MAILBOX at the event level so that a group will be notified if an event fails.

Scripts: People have created REXX routines to monitor for specific states. An example can be found out on the community at https://communities.ca.com/thread/101793452

https://communities.ca.com/thread/101793452


Scan Failed – Resolving

Another workaround is to use the parameters below to stop the File_Trigger from going into “Scan Failed” if there is a brief outage/issue. The scan.retries parameter will let the agent retry x number of times before actually failing.

filemonplugin.scan.retries=5 filemonplugin.scan.retries.sleep=60000

The example above will have the agent retry for 5 minutes before failing. It is not recommended to extend this period for too long.

Retries - default =1 Retries.sleep - default = 10000 or 10 seconds


Scan Failed - Resolving

EVENT LEVEL - To fix an event that is in “Scan Failed” remove everything after the word “CREATE/UPDATE…” and save out of the event in ISPF. Upload the modified event using Workstation. The event will resend and verify the new File_Trigger.

/*************************************************/ /* DEFINED BY POWDO03 AT 00.54 ON MON 25APR2016 */ /* LAST RUN AT 00.57 ON MON 25APR2016 */ /*************************************************/ EVENT ID(POWDO03.DPFILEYY) MAILBOX(POWDO03) SYSTEM(X114DPM) REPLACEINVOKE 'POWDO03.TEST.APPL(DPFILEYY)' WOBTRIG FILE_TRIGGER Agent(AGENTU_01) FileName('/opt/data/z8.txt') -

CREATE State(Failed) Status('Scan Failed') -LStatus('The /opt/data does not exist or not a directory')

ENDDEF

APPLICATION Level - To resolve a “Scan Failed at the Application level restart the job by putting an “R” beside it in CSF.

Misc. - Remote file systems, HA Scenarios and other fun things


Remote file systems

Many times the file isn’t local on the server.

– The path is a UNC path like \\location1\path1\path2\filename.abc

The agent running as Local System account or root has full authority to everything on the server but it doesn’t have access to anything outside the box

It is not possible to grant Local System account access to anything outside the box.

Running the job as a USER does not grant the authority to access the file share.


Remote file systems: ( cont’d)

2 Options Running the agent as a user –

– Go to the agent Service >> Right click on Properties >> Log On tab. Change the user to a service account with authority to the file share.

It is possible to make the location look local. – Mounting the drive. – Mapping the drive. Making the Mapping persistent is the key. – “Add a Network Location”


Agents that are set up with High Availability(HA)

When two agents are set up to fail back and forth using an alias and they are watching a remote file system for an update:

Use application level non-continuous file monitoring.

Application level File_Triggers can only see the update one time for that application. generation. Both agents will send back to the master that they both saw the File_Trigger condition met but they will both complete the same wob in the correct application.gen.

Event level file triggers can be passed out to both agents. When the file comes in both will send an update to the master and two triggers will happen. – May not see the bad behavior until after a failover happens.


Using Regular Expressions

It is possible to use a regular expression to look for a file with a name that may vary from time to time.

To enable regular expressions, add filemon.filename.r7regex=true to the agentparm.txt and bounce agent.

The job below will look for files that start with a lower case “t” and then have 3 characters then have any suffix.

This is very useful for files with dates

FILE_TRIGGER FILE.005 RUN DAILY AGENT AGENTU_01 FILENAME '/opt/data/t???.*' UPDATE

ENDJOB

Wobtrigs VS File_Triggers


Wobdef vs File trigger in appl

Event Wobdef Appl definition

Wobtrigs are invisible to the Monitoring staff

It is treated as any other job by Monitoring team.

Unable to know if file does not arrive on time.

Able to set late times if file has not arrived by a specific time.

Can watch for update or create using “CREATE”

“CREATE” will fail if file does not exist when job runs

Can watch for update or create using “CREATE”

Need to use continuous keyword but needs an alert and separate event


Wobdef vs File_Trigger in the appl

Event Wobdef Appl definition

If there are two nodes that are set up in a failover scenario and the agent(s) are monitoring a single file share it is possible

Waiting for multiple files can cause problems if only one file arrives.

Use multiple jobs for multiple files.

Questions

Sr. Support Engineer

[email protected]

Don Powell

@cainc

Slideshare.net/CAinc

Linkedin.com/company/ca-technologies

CA.com

Documents

CA Workload Automation ESP Edition Wobtrigs and File