Onboard Automation with EEM

OnboardAutomationwithEmbeddedEventManager

ShailaSharminSeniorEngineer,Core&IPNetworkBanglalion CommunicationsLTD

Index

• EmbeddedEventManager(EEM)Overview• UnderstandingEEMEventDetectors• EEMPolicieswithsamplescripts• EEMDemos

Introduction:EEM(EmbeddedEventmanager)EEMisasoftwarecomponentofciscoIOS,XR,andNX-OSthatallowsyoutorunascriptorasetofcommandsuponaneventandmakeslifeeasierforadministratorsbytrackingandclassifyingeventsthattakeplaceonarouterandprovidingnotificationoptionsforthoseevents..

Therearetwoindependentpieces:AppletsandScripting->AppletsareacollectionofCLIcommands->ScriptsareactionscodedupinTCL(interpreterlanguage)

EEMCoreEventDetectors&Components

EEMdetectorscanbe1)Syslog2)CLIevents3)Counter4)Timers5)SNMP6)IPSLAandNetflows events.7)None:simply"eventmanagerrun"command.

EEMComponents:EEMserverEEMpublisher(detector)EEMsubscriber(policy)

DeterminingtheVersionofEEMCISCOACCESSROUTERS- CurrentModels

CISCOACCESSROUTERS- OldModels

DeterminingtheVersionofEEMCISCOSERVICEAGGREGATION/COREROUTERS

CISCOCATALYSTSWITCHES

CreatinganEEMapplet

TherearethreestepstocreatingthisEEMapplet.1:CreatetheappletandgiveitanameR6(config)#eventmanagerappletIntf_Management

2:TelltheappletwhattolookoutforR6(config-applet)#eventsyslogpattern"%LINK-5-CHANGED:InterfaceLoopback0,changedstatetoadministrativelydown“

3:Whatdoyouwanttheroutertodowhenitseeswhatyouhavedefined instep2– simple!R6(config-applet)#action1.0clicommand“enable”R6(config-applet)#action2.0clicommand“conf t”R6(config-applet)#action3.0clicommand"interfaceloopback0"R6(config-applet)#action4.0clicommand“noshut”R6(config-applet)#action5.0clicommand"end"R6(config-applet)#action6.0clicommand"who"R6(config-applet)#action7.0mailserver"58.97.254.49"to"s.sharmin@banglalionwimax.com"from "s.sharmin@banglalionwimax.com"subject"ISP1_Interface_loopback0SHUTDOWN" body"Currentusers$_cli_result"R6(config-applet)#end

SampleEEMapplet

EEMEventDetector– SyslogED

FunctionalityTriggersEventonMatchesforSyslogMessagesBasedonRegularExpression

Exampleeventsyslogpattern"%LINK-5-CHANGED:InterfaceLoopback0,changedstatetoadministrativelydown“

UseCaseTroubleshooting,AutomaticFaultDetectionandAlert

SyslogEDExampleSyslogmessagesarethemessagesthatshowupbydefaultonconsole.Thisexampleshowsthesyslogeventdetector.Configuration:

SMTPServerisreachable,Loopback0isup.

SyslogEDExamplecontinue..WhentheLoopback0 interfacehasbeenshutdown,thebelowappletautomaticallyrunstoturnontheinterfaceandsendthealerttospecificemailaddressincludingloggedinuserinformation.

Todisplay theEmbedded EventManagereventsthathavebeentriggeredinR1,usethefollowing command:

Samplemailreceivedbynetworkadministrator

EEMEventDetector– CLIED

FunctionalityTriggersSynchronousorAsynchronousEventsWhenCertainCLIIsExecuted.AllowCustomCLICreation(EEM3.0).

Exampleeventclipattern"reload"syncnoskipyesoccurs1

UseCaseConfigManagement,Security,FeatureCustomization

CLIED EDExampleItcantakeactionbasedoncommandsthatareusedontheCLIConfiguration:

Insteadoflookingforapatterninsyslog,thistimewe’rewaitingforapatternenteredontotheCLI.breakdown:eventclipattern:DefinestheeventcriteriatoinitializetheEEMapplet.sync: SpecifiesifthepolicyshouldbeexecutedsynchronouslybeforetheCLIcommandsexecutesskip: IndicatesiftheCLIcommandsshouldbeexecutedoccurs:IndicatesthenumberofoccurrencesbeforetheEEMappletistriggers.

Whenweattempttoreloadtherouter,theresultsareasexpected

EEMEventDetector– InterfaceEDFunctionalityTriggersEventWhenInterfaceCountersCrossThreshold.22CountersSupported,Includinginput_error,interface_reset,transmit_rate,etc.

Exampleeventtagif_1interfacenameFa0/0parameterinput_errors_crc entry-opge entry-val 10entry-typeincrementpoll-interval60

UseCaseRealTimeAlertandRecoveryofInterfaceError

InterfaceEDExampleMonitorCRCerrorsonmultipleWANinterfacesandnotifytheoperator(viae-mail) whenaninterfacehasmorethantwoerrorsperminute.Configuration:

eventmanagerappletmultiple_ifeventtagif_1interfacenameFa0/0parameterinput_errors_crc entry-opge entry-val 2entry-typeincrementpoll-interval60eventtagif_2interfacenameGi01/0parameterinput_errors_crc entry-opge entry-val 2entry-typeincrementpoll-interval60triggercorrelateeventif_1oreventif_2action1.0syslogmsg "CRCfailureleasedline$_interface_name"action2.0mailserver"58.97.254.49"to"s.sharmin@banglalionwimax.com"from"s.sharmin@banglalionwimax.com"subject"CRCproblemson

$_info_routername interface$_interface_name"body"CRCfailureshaveexceededthethreshold“

ToviewtheregisteredpoliciesonrouterR1,youcanusethefollowingcommand:

EEMEventDetector– TimerED

FunctionalityTriggersEventsonWatchdog,CountDown,cron andAbsoluteTimer

Exampleeventtimercron cron-entry"019**0-7"eventtimerwatchdogtime300

UseCaseSystemMonitoringviaPeriodicAction,PeriodicDataCollectionandReporting

TimerEDExampleThisappletisputintoCron,whichwillbetriggeredattheexactspecifictime.Itiscomposedof5valuesseparatedbyaspace.Minuteshoursdaymonth{dayofweek(0-6,0isSunday)}Configuration:

EEMEventDetector– SNMPEDFunctionalityTriggersEventBasedonSNMPOIDValueCrossingPredefinedThreshold

Exampleeventsnmp oid " 1.3.6.1.4.1.9.9.109.1.1.1.1.5"get-typeexactentry-opge entry-val50exit-opleexit-val 5poll-interval5

UseCaseSystemStatsMonitoringandAlerting,e.g.CPUandMemoryUtilization

SNMPEDExampleFollowingEEMscriptrunthecommandwhentheCPUgoesaboveacertainvalue.Configuration:eventmanagerapplethighcpueventsnmp oid " 1.3.6.1.4.1.9.9.109.1.1.1.1.5"get-typeexactentry-opge entry-val 50exit-opleexit-val 5poll-interval5action1.0clicommand"enable"action2.0clicommand"showproc cpu sorted"action3.0mailserver"58.97.254.49"to"s.sharmin@banglalionwimax.com" from"s.sharmin@banglalionwimax.com" subject "HighCPUAlert"body"$_cli_result"End

ThiswillpollthefivesecondCPUutilizationoftherouteprocessoreveryfiveseconds.Iftheutilizationisatorabove50%,theeventwillfire.TheeventwillnotfireagainuntiltheCPUdropsbelow5%,thengoesbackto50%.Thedefinitionsofvariablesare:highcpu - nameoftheeventmanagerapplet/script1.3.6.1.4.1.9.9.109.1.1.1.1.5 /cpmCPUTotal5min- Objectidentifier(OID)forpollingthetotalCPUutilizationoftherouteprocessor(RP)entry-val 50 - CPUutilizationthattriggersthescriptpoll-interval0.5- Frequency(every0.5seconds)thescriptmonitorstheCPU

EEMEventDetector– IPSLAEDFunctionalityTriggerEventsWhenIPSLATestResultsCrossCertainThreshold.IntegratedwithAutoIPSLAGrouptoMonitorLargeNumberofIPSLAOperationResults

Exampleeventmanagerappletwatch-jittereventipsla operation-id1reaction-type jitterAvgaction001cli command"enable"action002if$_ipsla_measured_threshold_value >$_ipsla_threshold_risingaction003cli command"config t"action004cli command"iproute10.10.20.0255.255.255.0192.168.15.1"action005cli command"end“

UseCaseLinkFailureDetection,DiagnosticsandRecovery

Thedefinitionsofvariablesare:operation-id - SpecifiestheIPSLAsoperationID.

operation-id-value- Numberintherangefrom1to2147483647.

reaction-type- SpecifiesthereactiontobetakenforthespecifiedIPSLAsoperation.

jitterAvg JitterAverageinboth thedirections

EEMTCL-BasedPolicyExample

EEMscriptsarewrittenusingTCL.TCL(ToolControlLanguage)isascriptinglanguageused byCiscofortestingandautomatingofvariousfunctionsintheIOS.Inthisexample,smallTCLscriptconfiguredtocheckreachabilityoffewIPfromtheCorerouter.

EEMDemos

TheProblem: AnEnterprisenetworkconnectedwithtwoISP.WhileloadsharingtrafficwithbothISP,ifonelinktoISPfailsthentrafficshouldshifttoanotherISP.ButNATtranslationsarenotclearingaftertheprimarylinkfails.Whentheprimarylinkrecovers,trafficstillgoingovertheback-up link.

TheSolution:UsingIPSLAandEEMapplettofailoverthetrafficusingNAT.

1. DualISP:NATProblem

Topology

EEMActioneventmanagerappletlink-ISP-1-Downeventsyslogpattern"1ip sla 1reachability Up->Down"action1.0cli command"enable"action1.1cli command"configureterminal"action1.2cli command"noip nat insidesourcelist101interfaceGigabitEthernet1/0overload"action1.3cli command"noip nat insidesourcelist102interfaceGigabitEthernet2/0overload"action1.4cli command"ip nat insidesourcelist100interfaceGigabitEthernet2/0overload"eventmanagerappletlink-ISP-1-UPeventsyslogpattern"1ip sla 1reachability Down->Up"action1.0cli command"enable"action1.1cli command"configureterminal"action1.2cli command"noip nat insidesourcelist100interfaceGigabitEthernet2/0overload"action1.3cli command"ip nat insidesourcelist102interfaceGigabitEthernet2/0overload"action1.4cli command"ip nat insidesourcelist101interfaceGigabitEthernet1/0overload"eventmanagerappletlink-ISP-2-Downeventsyslogpattern"2ip sla 2reachability Up->Down"action1.0cli command"enable"action1.1cli command"configureterminal"action1.2cli command"noip nat insidesourcelist101interfaceGigabitEthernet1/0overload"action1.3cli command"noip nat insidesourcelist102interfaceGigabitEthernet2/0overload"action1.4cli command"ip nat insidesourcelist100interfaceGigabitEthernet1/0overload"eventmanagerappletlink-ISP-2-UPeventsyslogpattern"2ip sla 2reachability Down->Up"action1.0cli command"enable"action1.1cli command"configureterminal"action1.2cli command"noip nat insidesourcelist100interfaceGigabitEthernet1/0overload"action1.3cli command"ip nat insidesourcelist102interfaceGigabitEthernet2/0overload"action1.4cli command"ip nat insidesourcelist101interfaceGigabitEthernet1/0overload"!

Resources

• Support forums for this technology are GREAT• “Living” document at https://supportforums.cisco.com/docs/DOC-12757 Contains helpful tips

and tricks to get the most out of EEM . • For reading material and further resources for this session, visit www.pearson-

books.com/CLMilan2014. • https://networklessons.com/network-management/cisco-ios-embedded-event-manager/. • http://www.techtutsonline.com/cisco-ios-embedded-event-manager/• http://www.cisco.com/c/en/us/td/docs/ios/netmgmt/configuration/guide/12_2sx/nm_12_2sx_bo

ok/nm_eem_overview.html for basic info• http://www.cisco.com/c/en/us/td/docs/ios/netmgmt/configuration/guide/12_2sx/nm_12_2sx_bo

ok/nm_eem_policy_cli.html for Policies Using the Cisco IOS CLI• http://www.cisco.com/c/en/us/td/docs/ios/netmgmt/configuration/guide/12_2sx/nm_12_2sx_bo

ok/nm_eem_policy_tcl.html for Policies Using Tcl

Questions?