Pre-Con Ed: Wobtrigs and File triggers with CA Workload Automation ESP Edition

World®’16

Wobtrigs andFile_Triggers withCAWorkloadAutomationESPEditionJohnRooney,Sr.PrincipalConsultant,CATechnologies

MFX24E

MAINFRAMEANDWORKLOADAUTOMATION

2 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD

©2016CA.Allrightsreserved.Alltrademarksreferencedhereinbelongtotheirrespectivecompanies.

Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.

ForInformationalPurposesOnlyTermsofthisPresentation


Abstract

ThissessionwillreviewtheCAWorkloadAutomationESPEdition(CAWLAESP)

EventlevelWobtrig commandthatwasintroducedinCAWLAESP11.3.Itisused

totriggeraneventbaseduponadistributedfilebeingcreated.Alsointhissession,

youwilllearnabout

– ApplicationlevelFileTriggerworkloadobjectalongwithallofitscapabilities– Bestpracticesastowhichmethodwouldbethemostappropriatetousefor

yourparticularcircumstance– RecommendationsonparameterstouselikeMAILBOXtoprovidea

notificationonEventexceptionconditionslike“ScanFailed”.

JohnP.RooneyCATechnologiesSr.PrincipalConsultant


Agenda

WOBTRIGS/FILE_TRIGGERS – THEBASICS

WOBDEFS ANDFILE_TRIGGERS DEFINED

QUESTIONS

MONITORINGANDRESOLVINGERRORS

MISC.- REMOTEFILESYSTEMS,HASCENARIOSANDOTHERFUNTHINGS

WOBTRIGS VSFILE_TRIGGERS ATTHEAPPLLEVEL

1

2

3

4

5

6


Wobtrigs/File_Triggers – Thebasics


Wobtrigs – Example§ Wobtrigs werecreatedtoprovideanEventSensorfortheunpredictable

distributeddataeventswithouttheneedfordefiningschedulingcriteria.

§ WOBTRIGFILE_TRIGGERAGENT(agent_name)-FileName('/webdata/batch/gis/receive/NAPS*_REQ_*.*')CREATE-NoChange(1)

§ AftertheEVENTissavedandtheagentacknowledgesthatthefiletriggerisset,itwilllookliketheonebelow.

WOBTRIGFILE_TRIGGERAGENT(agent_name)-FileName('/webdata/batch/gis/receive/NAPS*_REQ_*.*')CREATE-NoChange(1)State(Monitored)Status('MonitoredforCREATE')


Wobtrigs – Notes§ Wobtrigs canbeapossiblechoiceifitisunknownwhatdaysorhowmanytimes

perdaythefilewillbecreated/modified.Itisasimplecasetoimplement

§ SincepeopleareNOTmonitoringthestateofevents,ifaneventgoesinto“ScanFailed”itcangoundetectedunlessprecautionsaretaken.– MAILBOX:SetupanaMAILBOXattheeventlevelsothatagroupwillbe

notifiedifaneventfails.– Scripts.PeoplehavecreatedREXXroutinestomonitorforspecificstates.An

examplecanbefoundoutonthecommunityathttps://communities.ca.com/thread/101793452

§ Iftherearetwonodesthataresetupinafailoverscenarioandtheagent(s)aremonitoringasinglefileshareitispossiblethattheeventwilltriggertwotimeswhenthefilearrives.MoreonthisinthesectiononHA.


Wobtrigs:(cont ‘d)§ Suspendingorholdinganeventdoesnotstopthefiletrigger.

– Iftheeventissuspended,anytriggersthatcomeinwillbediscarded.– Iftheeventisheld,anyfiletriggersthatcomeinwillbeheldbasedonthe

OVERDUEcountspecifiedintheevent’sSCHEDULEstatement.

§ Defininganeventthatwaitsformultiplefilesbeforetriggeringusingthe“MULTIPLE”keywordmaygiveyouanunexpectedoutcome…


File_Triggers inanapplication– Example

FILE_TRIGGERFILE.001RUNDAILYAGENTAGENTU_01

FILENAME'/opt/data/*.tmp'CREATECONTINUOUSALERT(FTG1)ENDJOB


ApplicationlevelFile_Trigger – Notes§ Forthesimplecasewhereafilearrivesasmallnumberoftimesperday,atset

times,thisisabsolutelythebestmethod.

§ Eachfilethatshouldshowupcanhavealatetimeandtriggerdueoutnotificationsifithasn’tarrived.

§ TheFile_Trigger jobisvisibletotheMonitoringgroup.Itgoeslateorfailedjustlikeanyotherjobtheyhavetomonitor.

§ UsingContinuous.EachtimetheFile_Trigger conditionsaremetitwilltriggeranAlert.Itmaybebesttotriggerthisdailyandhaveitcompletejustbeforethenextdaytriggers.

§ ScanFailedjobswillturnredandtriggeralertsforthefailedprocess.Justlikeanyotherfailedjob.


UsingVariables§ ESPWTAGENT– Containsthenameoftheagentthatthefiletriggerwasdetected

on.UsingContinuous.Itmaybebesttotriggerthisdailyandhaveitcompletejustbeforethenextdaytriggers.

§ ESPWTFILE– Containsthenameofthefilethatfiletriggerwassatisfiedfor.

§ File_Triggers using“continuous”canuse“MonitorVariables”.ThesecontaininformationlikejobnameForthesimplecasewhereafilearrivesasmallnumberoftimesperday,atsettimes,thisisthebestmethod.


Monitoringandresolvingerrors


Thecurrentstateofaffairs….STATE(Failed|In Preparation|Monitored)

§ Indicatesthestateofthemonitoringrequest.YoudonotcodeSTATE;itisdisplayedintheWOBTRIGorCSF

§ Failed:Themonitoringrequestfailed.– Example- Status('ScanFailed')LStatus('The/opt/datadoesnotexistornota

directory')

§ InPreparation:Themonitoringrequestisbeingprepared.

§ Monitored:Theagentismonitoring.


ScanFailedissue§ AWobtrig/File_Trigger goesintoscanfailedifitcannotgettothepathtolookfor

thefile.

§ Insomecasesabriefnetworkglitchorashortperiodwherearemotefilesystemisofflinecancausea“ScanFailed”error.SincepeopleareNOTmonitoringthestateofevents,“ScanFailed”eventscangoundetectedunlessprecautionsaretaken.

§ MAILBOX:SetupanaMAILBOXattheeventlevelsothatagroupwillbenotifiedifaneventfails.

§ Scripts:PeoplehavecreatedREXXroutinestomonitorforspecificstates.Anexamplecanbefoundoutonthecommunityathttps://communities.ca.com/thread/101793452


ScanFailed– Resolving§ AnotherworkaroundistousetheparametersbelowtostoptheFile_Trigger from

goinginto“ScanFailed”ifthereisabriefoutage/issue.Thescan.retries parameterwilllettheagentretryxnumberoftimesbeforeactuallyfailing.

§ filemonplugin.scan.retries=5§ filemonplugin.scan.retries.sleep=60000

§ Theexampleabovewillhavetheagentretryfor5minutesbeforefailing.Itisnotrecommendedtoextendthisperiodfortoolong.

§ Retries- default=1§ Retries.sleep - default=10000or10seconds


ScanFailed- ResolvingEVENTLEVEL- Tofixaneventthatisin“ScanFailed”removeeverythingaftertheword“CREATE/UPDATE…”andsaveoutoftheeventinISPF.UploadthemodifiedeventusingWorkstation.TheeventwillresendandverifythenewFile_Trigger.

/*************************************************//*DEFINEDBYPOWDO03AT00.54ONMON25APR2016*//*LASTRUNAT00.57ONMON25APR2016*//*************************************************/EVENTID(POWDO03.DPFILEYY)MAILBOX(POWDO03)SYSTEM(X114DPM)REPLACEINVOKE'POWDO03.TEST.APPL(DPFILEYY)'WOBTRIGFILE_TRIGGERAgent(AGENTU_01)FileName('/opt/data/z8.txt')-

CREATEState(Failed)Status('ScanFailed')-LStatus('The/opt/datadoesnotexistornotadirectory')

ENDDEF

APPLICATIONLevel- Toresolvea“ScanFailedattheApplicationlevelrestartthejobbyputtingan“R”besideitinCSF.


Misc.- Remotefilesystems,HAScenariosandotherfunthings


RemotefilesystemsManytimesthefileisn’tlocalontheserver.

– ThepathisaUNCpathlike\\location1\path1\path2\filename.abc

§ TheagentrunningasLocalSystemaccountorroothasfullauthoritytoeverythingontheserverbutitdoesn’thaveaccesstoanythingoutsidethebox

§ ItisnotpossibletograntLocalSystemaccountaccesstoanythingoutsidethebox.

§ RunningthejobasaUSERdoesnotgranttheauthoritytoaccessthefileshare.


Remotefilesystems:(cont’d)§ 2Options§ Runningtheagentasauser–

– GototheagentService>>RightclickonProperties>>LogOntab.Changetheusertoaserviceaccountwithauthoritytothefileshare.

§ Itispossibletomakethelocationlooklocal.– Mountingthedrive.– Mappingthedrive.MakingtheMappingpersistentisthekey.– “AddaNetworkLocation”


AgentsthataresetupwithHighAvailability(HA)

Whentwoagentsaresetuptofailbackandforthusinganaliasandtheyarewatchingaremotefilesystemforanupdate:§ Useapplicationlevelnon-continuousfilemonitoring.§ ApplicationlevelFile_Triggers canonlyseetheupdateonetimeforthatapplication.

generation.BothagentswillsendbacktothemasterthattheybothsawtheFile_Trigger conditionmetbuttheywillbothcompletethesamewob inthecorrectapplication.gen.

§ Eventlevelfiletriggerscanbepassedouttobothagents.Whenthefilecomesinbothwillsendanupdatetothemasterandtwotriggerswillhappen.– Maynotseethebadbehavioruntilafterafailoverhappens.


UsingRegularExpressions

Itispossibletousearegularexpressiontolookforafilewithanamethatmayvaryfromtimetotime.§ Toenableregularexpressions,addfilemon.filename.r7regex=truetothe

agentparm.txtandbounceagent.§ Thejobbelowwilllookforfilesthatstartwithalowercase“t”andthenhave3

charactersthenhaveanysuffix.§ Thisisveryusefulforfileswithdates

FILE_TRIGGERFILE.005RUNDAILYAGENTAGENTU_01FILENAME'/opt/data/t???.*'UPDATE

ENDJOB


WobtrigsVSFile_Triggers


Wobdef vsFiletriggerinapplEventWobdef Appl definitionWobtrigs areinvisibletotheMonitoringstaff

Itistreated asanyotherjobbyMonitoringteam.

Unable toknowiffiledoesnotarriveontime.

Abletosetlatetimes iffilehasnotarrivedbyaspecifictime.

Canwatchforupdateorcreateusing“CREATE”

“CREATE”willfailiffiledoesnotexistwhenjobruns

Canwatchforupdateorcreateusing“CREATE”

Need tousecontinuouskeywordbutneedsanalertandseparateevent


Wobdef vsFile_Trigger intheappl

EventWobdef Appl definitionIftherearetwonodesthataresetupinafailoverscenarioandtheagent(s)aremonitoringasinglefileshareitispossible

Waitingformultiplefilescancauseproblemsifonlyonefilearrives.

Usemultiplejobsformultiplefiles.


RecommendedSessionsSESSION# TITLE DATE/TIME

MFX33EBestPracticesforsettingupCAWorkloadAutomationESPApplicationProcedures

11/15/2016at1:00pm

JasmineB

MFX129SVisionandRoadmap:CAWorkloadAutomationESPEdition

11/16/2016at12:45pm

JasmineC

MFX180SHowtoHaveaVoiceinProductEnhancementsthroughCACommunitiesIdeationandValidation

11/17/2016at3:00pm

JasmineB

MFX136SDeliveringCAWorkloadAutomationfromaSaaSplatform

11/17/2016at4:30pm

JasmineB


MustSeeDemos

CAWLAAE&DEMFandWorkloadAutomation

SeamlessApplicationAutomationAgents&AdvancedIntegrationsMFandWorkloadAutomation

CAConversionService

CAWorkloadAutomationMFandWorkloadAutomation

Innovations:Mainframe:WorkloadAutomationCAWLACA7&ESPMFandWorkloadAutomation

Innovations:CrossEnterpriseWorkloadAutomation


Questions?

@CAWORLD#CAWORLD ©2016CA.AllRIGHTSRESERVED.28 @CAWORLD#CAWORLD

MainframeandWorkloadAutomation

FormoreinformationonMainframeandWorkloadAutomation,pleasevisit:http://cainc.to/9GQ2JI


Thankyou.

Stayconnectedatcommunities.ca.com

Technology

Pre-Con Ed: Wobtrigs and File triggers with CA Workload Automation ESP Edition