SNMP Monitor Adapter Software - CorreLog.com Monitor Adapter, Page - 5 Section 1: Introduction This manual provides a detailed description of the CorreLog SNMP Monitor software. This

CorreLog®

SNMP Monitor Adapter Software Users Manual

http://www.CorreLog.com mailto:[email protected]

SNMP Monitor Adapter, Page - 2

CorreLog, SNMP Monitor Users Manual Copyright © 2008 - 2018, CorreLog, Inc. All rights reserved. No part of this manual shall be reproduced without written permission from the publisher. No patent liability is assumed with respect to the use of the information contained herein. Although every precaution has been taken in the preparation of this book, the publisher and author assume no responsibilities for errors or omissions. Nor is any liability assumed for damages resulting from the use of this information contained herein.


Table of Contents Section 1: Introduction

………….. 5

Section 2: Software Installation

………….. 11

Section 3: Software Operation

………….. 15

Section 4: Application Notes

………….. 25

Additional Help And Information ………….. 33



Section 1: Introduction This manual provides a detailed description of the CorreLog SNMP Monitor software. This is an optional set of files and executables added to the CorreLog Server order to expand the role of the CorreLog to include monitoring of SNMP MIB objects. The manual provides information on specific features and capabilities of this special software, including installation procedures, operating theory, application notes, and certain features not documented elsewhere. The SNMP Monitor software consists of several components. A background process continuously polls MIB objects, and compares them to thresholds. The user can configure the objects to poll, and can specify the syslog message that is sent when thresholds are violated. Additionally, the user can inspect the value of SNMP objects collected on the system. These components are described in detail within this document. This manual is intended for CorreLog users who will operate the system, as well as system administrators responsible for installing the software components. This information will also be of interest to program developers and administrators who want to extend the range of the CorreLog system's role within an enterprise to include SNMP monitoring.


Overview Of Operation The SNMP Monitor Adapter software extends the CorreLog system to permit polling and monitoring of system values using standard SNMPv1 and v2 protocol. This allows CorreLog to actively monitor system dynamics and states in a way that may not be easily achievable through other methods. The CorreLog SNMP background process continuously polls certain selected devices for values, compares these values against user-defined thresholds, and then sends Syslog messages (of user specified severity and content) to the main CorreLog server. This gives CorreLog more awareness of the network and enterprise state. The CorreLog SNMP background process is configured and monitored using a tightly coupled integration with the main CorreLog web interface. The user configures "address groups" that are polled for specific values, and defines the message and severity that is sent to CorreLog when a threshold is tripped. This permits such awareness as CPU usage, network traffic, TCP connections, as well as more arcane configuration items such as printer states, host resources, disks, and peripheral devices.

SNMP Basics SNMP is an application layer protocol that is supported by many network devices, including Windows servers, UNIX workstations, routers, and switches. These devices execute an "SNMP Agent" program, which listens for UDP data requests. These requests identify specific values that may (or may not) be supported by the managed device including (but not limited to) system information, TCP information, process information, and interface information. If the platform supports the specified SNMP request, a value is returned to the requestor. SNMP can be a complex subject, with many different aspects and considerations. However, the basics of SNMP (used by CorreLog) are quite simple. Each SNMP request contains the following information:

IP Address. Each SNMP request is directed to a single managed device, which must be SNMP capable. On Windows platforms, the "SNMP.exe" program (SNMP Service) should be running. On UNIX platforms, this process is commonly called the "snmpd" program.

Read Community. Each SNMP request must contain a "password" for the agent called a "Read Community" name. This is a short name that is configured both at the agent and the requesting program. For an SNMP request to be accepted, the requestor must know this read community value.


Request Type. There are two basic ways of requesting information. The "Get" request will get a single value from the system, and the value is specified by an object identifier (discussed below.) The "Get Next" request operates similarly, but will get the NEXT object identifier and value, given an initial object identifier. The "Get Next" request can be used to traverse (i.e. walk) through all values of the SNMP MIB, and is commonly used to ascertain the values available at the agent for the requestor.

Object Identifier (OID) Each SNMP request identifies the particular value to be fetched using an object identifier, commonly referred to as the MIB OID (Management Information Base Object Identifier.) This is simply a unique identifier that has been standardized upon by industry and various specifications. Object identifiers are expressed in dot notation, such as "1.3.6.1.2.1.1.1.0" (which is the name for the System Description of an agent.)

CorreLog SNMP MIB Objects and Values The MIB OID (MIB object identifier) values are essential to using SNMP. Each OID is standard across all programs and processes. A standard list of common OIDs exists, and individual vendors can augment this list by registering their unique enterprise numbers with IANA. Generally, each OID has a common name, a type, and a description. The OID names and descriptions are typically documented in a specific notation called ASN.1, often referred to as a "MIB Definition" file, or simply a "MIB file". The basic types of each OID value are as follows:

Text String. A MIB object can correspond to a particular text string, typically fewer than 255 characters in length. Examples of this MIB object type include the system description, system contact, interface descriptions, and other textual values.

Counter. A MIB object can correspond to an integer number that continuously increments, and which rolls over when its maximum value is reached. Examples of this MIB object type include IP packet counters, interface counters, and the system "Up Time" counter.

Gauge. A MIB object can correspond to an integer number that can either increase or decrease. Examples of this MIB object type include interface count, CPU load, number of TCP connections, and interface speeds.


Enum Value. A MIB object can correspond to an integer number that takes on discrete values. Examples of this MIB object type include interface status, administrative status, and connection state.

Octet String. A MIB object that cannot be type in any other way can be considered as a string of eight bit bytes (or octets). This permits any data to be represented.

Vendors can define their own types, deriving these from the above types. For example, one special type of "Counter" value is a "Time Ticks" value, which is simply a counter that represents the elapsed time of a system, where the least significant digit is 1/100th of a second. Likewise, the user can define an "Octet String" to be a "MAC Address" type object consisting of precisely six bytes. This aspect of SNMP is sometimes confusing to new SNMP users, but it is actually quite simple to classify the universe of MIB object types into one of the five basic types described above.

SNMP Monitor System Software Components The CorreLog SNMP software comes as a single downloadable package in self-extracting WinZip format. This package is installed at the CorreLog server, and contains the following specific components.

CO-SNMP.exe Program. This is the polling agent that is responsible for gathering SNMP information on the system. The process is configured to run on CorreLog system startup (via the "System > Schedule" screen, as documented below.)

SNMP Configuration Screen. This is a support screen, available under the "Messages > Adapters > SNMP" tab of the CorreLog web interface as part of the Windows component installation. This screen allows the user to configure the MIB objects to be polled, the thresholds for MIB objects, and the Syslog message that is sent when thresholds are violated.

GETSNMP Support Utility. This is a command line utility installed in the "system" directory of CorreLog as part of the Windows component installation. The can be used to interactively fetch MIB objects by their OID value, useful for testing and for system development.

System Block Diagram The CorreLog SNMP Monitor process consists of a single background process, which executes at the CorreLog server. This process reads configuration data that has been configured by the operator, and continuously polls a list of devices.


(The devices are specified by IP address, by an IP address wildcard, or by a standard CorreLog Address Group specification.) As the list of managed devices is polled for values, each value is compared to an operator configured threshold. When the threshold is exceeded, the SNMP Poller process issues a Syslog message to the Main CorreLog server. The actual message (and its severity) is configured by the operator, and appears in correlation threads and tickets like any other received message.

As indicated in the above diagram, the CO-SNMP.exe process (installed and configured as described in the next chapters) continuously polls a list of managed devices. These devices run native SNMP agents, and can be Windows platforms, UNIX servers, Routers, Switches, and other network equipment. The process is completely controlled and monitored by configuration data that is configured by the operator using the "Messages > Adapters > SNMP" screen of the Main CorreLog Server web interface.

How To Use This Manual The next section of this manual (Section 2) provides the essential information needed to install the CorreLog SNMP Monitor software. Note that the only required components of the system are the CO-SNMP.exe program and the SNMP configuration screen, documented herein. Other information on the CorreLog server can be found in the "User Manual", including operation and application notes that will be of assistance in processing the SNMP messages generated by the CO-SNMP.exe program, and received by the CorreLog Syslog receiver process.



Section 2: Software Installation The CorreLog SNMP Monitor software is usually delivered as a self-extracting WinZip file. The installation requires a few simple manual installation steps, and no automatic installation is provided or required. The basic installation steps are as follows:

1. The user obtains the CorreLog SNMP Monitor software, in self-extracting WinZip format.

2. The user stops the CorreLog Server "Framework Service", and verifies via

the task manager that all CorreLog background processes have stopped. 3. The user executes the self-extracting WinZip file. This unzips the SNMP

software into the CorreLog Windows Distribution, including all configuration data and executables, and modifies the CorreLog program to start the CO-ping.exe program on system startup

4. The user restarts CorreLog, and configures address groups and other

items via the "Messages > Adapters > SNMP screen.

5. The user configures other parts of the CorreLog system, such as Threads, Alerts, and Ticket users, to correlate and process the syslog messages that are generated by the SNMP Monitor software.

Administrative logins are required in order to perform the software installation. The detailed steps needed to perform the installation are provided in the sections that follow.


Installation Requirements

Existing CorreLog Server Installation. Prior to installing the SNMP Monitor software, the CorreLog Server system must be installed on a Windows platform, as discussed in the CorreLog User Reference Manual.

Disk Space Requirements. The SNMP Monitor software requires no significant disk space beyond the normal footprint of the CorreLog server. There is generally no extra disk space load due to this software.

CPU Requirements. The SNMP Monitor software requires very little extra CPU requirements. A single process is started the CorreLog Windows platform, which consumes minimal CPU resources.

Service Ports. The SNMP Monitor software Agent requires access to the standard SNMP port of 161. Firewalls should be adjusted to accommodate this service port and communication channel between SNMP agents and the CorreLog server.

To insure proper installation of the program, the user should close all windows, and temporarily disable any port blocking or Virus Scan software on the system. The existing CorreLog server process should be stopped prior to the installation. Reboot, after installation, is not required.

Windows Installation Procedure The specific steps needed to install the software are as follows:

1. Login to the CorreLog Server Windows platform using an "Administrator" type login.

2. Stop the CorreLog Server processes via the Windows Service Manager,

or via the "Start and Stop Services" utility found in the Windows Start menu. Verify with the Windows "Task Manager" that all CorreLog processes are stopped.

3. Obtain and execute the "co-n-n-n-snmp.exe" package, extracting files to

the directory location where CorreLog is installed (by default the location "C:\CorreLog").

4. Restart the CorreLog Server processes via the Windows Service

Manager, or via the "Start and Stop Services" utility.

5. Verify with the Windows "Task Manager" that the "CO-snmp.exe" process is now running on the system.


SNMP Software, Device Group Configuration Once the CO-snmp.exe program has been installed and is running on the system, the user can configure the list of devices that are polled by the agent. The user accomplishes this activity via the "Messages > Adapters > SNMP" tab of the web browser interface. (The "Adapters" tab is automatically added to your system, if it does not already exist.) Note that, by default, the CO-snmp.exe program does not poll devices. The address group of "0.0.0.0" (which is the default poll address for all items) disables the polling process. The user must configure a device IP address (or list of addresses), which is polled by the CO-snmp.exe program. The user clicks on the "Edit" button to edit an existing monitor. The user can provide an IP address or device group as follows.

1. The user can specify a static IP address, such as "10.1.1.1". In this case, the monitor will poll the single device.

2. The user can specify an IP address with wildcards, such as 10.1.1.*. In

this case, the monitor will poll all devices in the "Devices" tab of CorreLog that match the specified wildcard.

3. The user can specify an address group such as @@my_servers@@ that

describes one or more devices. These device groups provide the most maintainable way of polling the devices. The device groups are configured in the "Correlation > Config > Address Groups" tab of the program.

Note that when using wildcards as the IP address, the devices are polled only if the wildcard matches one or more entries in the "Messages > Devices" tab of the program. The user can add device entries with the "Add New" button, found on this screen. Setting the IP address value to "0.0.0.0" effectively disables any polling for a specified SNMP Monitor. This provides a way of disabling the polling associated with a particular MIB object without deleting the SNMP Monitor from the system.

SNMP Software, Read Community Configuration Each SNMP Monitor is associated with a "Read Community", which must be known to the administrator. This community value serves as a password to the agent, and no communication with the agent is possible unless this value is known. (The value is typically configured at the device where the agent resides, using a variety of techniques.)


The SNMP Monitor software permits a read community to be specified using the following techniques.

1. If the user enters the keyword "Default" as the read community for the SNMP Monitor, the program first looks in the "Device Information" screen for each device, and then uses the default read community configured on the "System > Parms" screen.

2. If the user enters a value other than "Default" as the read community for

the SNMP Monitor, the program will use that community string value as the read community for all devices polled by that monitor, ignoring any other read community information on the system.

MIB Object Configuration The SNMP Monitor comes pre-configured with several useful MIB object values that check for common security problems. These may be sufficient for many enterprises. However, for specialized monitoring (such as Cisco monitoring or Printer monitoring) it may be useful to configure other MIB objects that are polled by the system. SNMP MIB objects correspond to specific values that can be fetched from a managed platform using SNMP requests. Each MIB object consists of an object identifier (OID) expressed in dot-notation (such as 1.3.6.1.2.1.1.1.0) as well as a value type (either textual or numeric.) If the object value is numeric, it can be either a "counter", or a "gauge", or an "enum" type value.

For numeric values, a "delta" type threshold should normally be used for "counter" type objects; an "abs" type threshold should be used for "gauge" type objects, and an "equal" type threshold should be used for "enum" type objects. Any MIB object may also have a "change" or "timeout" type threshold, to detect changes or timeout conditions. A list of common MIB objects is available from the "MIB Help" hyperlink, found on the "AddNew" and "Edit" screens. These MIB objects can be appended with a ".0" extension (or some other extension if the MIB object is an SNMP table object) to fetch values from a managed device. Note that not all SNMP capable devices will support all of the MIB objects found at this link. The user should consult RFC documents on the web, or contact CorreLog support for specific information and guidance.


Section 3: Software Operation The CorreLog SNMP Monitor program requires very limited operating notes. Once the program is installed, it makes use of reasonable default values. The operator only needs to set the device groups and read community values in order to immediately start using the programs. The user may add SNMP Monitors, in addition to the pre-configured monitors that come with the system. This typically requires a moderate understanding of SNMP, and may require investigation of the specific SNMP capabilities of the enterprise, such as what SNMP agents are available, how they are configured, and what types of monitoring will provide the most visibility without loading CorreLog down with useless message information. CorreLog provides a "MIB Browser" utility (documented in this section) as a special utility program, which can facilitate the setup of SNMP monitors by exploring what SNMP MIB objects are actually supported by managed devices on the system. This section provides a description of these optional software elements, their usage, and other considerations, including screenshots and explanation of monitor configuration values.


SNMP Monitor Screen As part of the Windows installation, a new tab is created in the "Message > Adapters" section of the CorreLog web interface, which permits the user to configure various parameters associated with the SNMP Monitor program. This screen is available only to CorreLog administrators. The screen is depicted below.

The above screen is a standard CorreLog parameter "AddNew" editor screen. The user can click the "SaveNew" button to save a new monitor value. Once the monitor value is created, the user can further modify the entry by clicking on the "Edit" button on the parent screen for the new entry. The SNMP Monitor screen provides the following parameters, which are read by the CO-snmp.exe program


SNMP Monitor Title. This is a short title that prefixes any message sent by the monitor background process as part of the Syslog message. This value also appears on the top-level SNMP Monitor page, used to quickly identify the nature and purpose of the monitor function, usually the same as the MIB object that is being polled.

IP Addr / Group. This value identifies the particular device or list of devices polled by the background process. This value can be a single IP address, an IP address wildcard, or an "Address Group" defined in the "Correlation > Config > Address Groups" screen. If an IP address wildcard is specified, the addresses listed in the "Devices" tab are polled if they match the wildcard value. The special value of "0.0.0.0" disables polling on the system.

Read Community. This is the SNMP read community used by the monitor value. If a value other than "Default" is specified, then this is the value used as the read community when polling all devices in the specified group. If the value is the keyword "Default", then the read community configured for each device is used, or the default read community configured on the "System > Parms" screen is used if no specific read community is configured for a specific device.

Timeout / Retries. These settings provide control over the timeout and retry values of the polling process The timeout is typically one second, and retry value is typically under 3. Specifying a high value for timeout and retry may seriously degrade the poll time for the specified SNMP Monitor. These values should be adjusted carefully.

SNMP Method. This setting permits the user to specify the method of getting SNMP values, either "Get" (the default) or "GetNext", or "External". These methods are described in more detail within a later section.

MIB Object OID. This is the specific SNMP identifier that is fetched from the SNMP object using a "Get" or "GetNext" request. This value is expressed in standard SNMP dot-notation, prefixed by "1.3.6.1", corresponding to a MIB object supported on the system. For assistance on common MIB objects, the operator can click the "MIB Object Help" hyperlink at this field. The user can test a MIB object value, read community, and address by clicking the "Test" button of the system.

Monitor Type. This is the type of comparison to be made on the results of the MIB object poll. This value usually depends upon the type of MIB object specified above. Counter values are typically "Delta" type monitors (where the current value is subtracted from the previous value, and scaled to provide counts per minute.) Gauge and Integer values are typically "Abs" (absolute) type monitors (where the value is directly compared to a


threshold without scaling.) Textual objects are typically "Change" type monitors, where any change to the value is considered an event. The special "Timeout" value is used to generate an event should the SNMP agent timeout. These monitor types are described in more detail within a later section.

Monitor Threshold. This is the numeric threshold for the MIB object. If the monitor type described above is "Timeout" or "Change", then this threshold value is ignored. Otherwise, the threshold represents counts per minute for "Delta" type monitors, and an absolute value for "Abs" type monitors.

Alert Message Severity. This is the severity of the message that is generated by the CO-snmp.exe program when a threshold is violated. This is a standard syslog severity ranging from "debug" to "emergency".

Alert Message. This is the actual content of the message that is sent by the CO-snmp.exe process when a threshold is violated. The message will consist of the "Monitor Title" configured above, and the content of this message. The value should be descriptive of the particular event, and may include corrective action or remediation steps. The value can be selected to include keywords that cause the message to be recorded in certain CorreLog threads, or to match triggers and actions.

SNMP Methods The user can select from three different methods, "Get", "GetNext" or "External". The default method is an SNMP "Get". The various methods are described in detail below.

SNMP Get. This method fetches the exact value of the SNMP agent precisely specified by the MIB object OID. It is mainly useful for getting "scalar" type MIB objects. (Scalar type MIB objects typically end in a ".0". These MIB objects usually contain summary object values for a managed system, such as number of interfaces, number of TCP segments processed, etc.) For the SNMP "Get" request to return a value, the MIB OID must precisely express an object supported by the agent. If the object is not supported by the agent, then a value of "*Error*" is returned, and no notification or syslog message is generated.

SNMP Get Next. This method fetches the value of the SNMP agent specified by the next MIB object supported by the agent. This is a standard SNMP function, and is mainly useful for fetching the first value of an SNMP table column. The SNMP "Get Next" request will typically return a legitimate value. However, the precise OID returned by the agent may be different than that which is expected. Hence, this method should be


used with care. Although the method works reliably in many different situations, a naïve user may find that the OID value being returned by the Get Next method is not actually the value sought or desired.

External Method. This is a special method that can be used to extend the range of operations performed by the CorreLog SNMP Monitor software. When the method type is "External", then a program should be specified in the MIB object OID field rather than an OID. The program can be a ".bat" or a ".exe" program, and the program MUST reside in the "./snmp" directory of the system. The first non-blank line of the program's standard output is taken as the SNMP value, and is compared to the specified threshold. This allows the user to perform complex operations, such as counting up the number of rows in an SNMP table, or seeking a specific SNMP value within a table. (See later section for more information.)

Monitor Types The "Monitor Type" setting describes the type of measurement that is applied to the value obtained by the "SNMP Method". Each monitor type has specific characteristics and uses, as described below.

Change. This monitor type causes a syslog message of the specified severity and content to be sent to CorreLog if the value resulting from the SNMP Method changes. This is the only type of monitor that can be applied to textual values (such as system descriptions, interface descriptions, or octet string values.) This is useful for detecting changes to system configuration values. An alert is raised each time that a change occurs in the specified MIB object.

Delta-GT. This monitor type causes a syslog message of the specified severity and content to be sent to CorreLog if the current numeric value, minus the previous numeric value, divided by the elapsed time, exceeds the specified threshold. This type of monitor is used almost exclusively with Counter type SNMP objects, and is useful for detecting when a system rate (in "counts per minute") is exceeded.

Delta-LT. This monitor type is similar to the Delta-GT type, except a syslog message of the specified severity and content is sent to CorreLog if the counts per second are below the specified threshold. In particular, this can be used to determine whether a system is rebooted (by monitoring the system up time for a negative change.) This value is used less often than the Delta-GT, but may be of interest to determine whether a gauge value is falling.

Abs-GT. This monitor type causes a syslog message of the specified severity and content to be sent to CorreLog if the absolute value of the


result is above a threshold. This type of monitor is used almost exclusively with Gauge type SNMP objects, such as TCP connection counts, interface counts, disk space values, process counts, and / or percent CPU utilization.

Abs-LT. This monitor type causes a syslog message of the specified severity and content to be sent to CorreLog if the absolute value of the result is below a threshold. As with the Abs-GT monitor type, this type of monitor is used almost exclusively with Gauge type SNMP objects.

Equal. This monitor type causes a syslog message of the specified severity and content to be sent to CorreLog if the value of the MIB object is identical to the threshold. This type of monitor is typically used with Enumerated integer values such as the interface status of devices, to detect when a particular value or condition is satisfied.

Not-Equal. This monitor type causes a syslog message of the specified severity and content to be sent to CorreLog if the value of the MIB object is not equal to the threshold. This complements the Equal type of monitor, permitting the user to determine when an enumerated value is not at its expected value.

Bit-Test. This is a highly specialized monitor type that causes a syslog message of the specified severity to be sent to CorreLog with one or more bits of an integer value match the threshold. For example, if the value 0x01 indicates that a device is idle, and the value 0x02 indicates that a device is faulted, then setting the threshold to integer 3 will cause a syslog message to be sent when the device is both idle and faulted. The threshold is combined to the value with a bitwise "and" operator, and compared to the threshold again. In other words, if the value is "V" and the threshold is "T", then the syslog message is sent if and only if "(V & T) = T". The value of V and T are expressed by CorreLog (and its operator) as integer representations of binary numbers. For example, specifying a threshold of 26 tests whether all the bits 2, 4, and 5 are set in the return value.

Timeout. This is a special monitor that causes a syslog message of the specified severity and content to be sent to CorreLog if the value of the MIB object cannot be fetched. This type of monitor can be used to determine (via an SNMP request) whether the device is online. If a timeout condition exists, then either the read community of the device is incorrect, or the agent is unable to respond. (The agent may not respond because the node is not booted, or the CPU is too busy to service the agent, or the agent is not network-reachable by CorreLog.)


Additional Notes on External Methods The "External" type of SNMP method is typically used to perform a complex operation, such as getting the value of an OID from one location of the SNMP MIB, and fetching a new value based on the result. For example, to count the number of processes on a system, the user may have to walk an entire table column using SNMP "GetNext" requests until an OID outside of the table is found. Or, the user may need to scan the interface descriptions in the interface table to find the index necessary to get the speed or bandwidth of that interface. The user can write a simple batch file, Perl script, or other program to perform the custom method. This file is placed in the "./snmp" directory of the CorreLog system, and the name of the script is specified (without the full pathname) in the MIB Object OID field of the screen. This causes the script to be executed for each device in the device group, performing a simple or complex operation to get a resulting value. To return a value to CorreLog, the external program simply writes one line or one value to standard output. CorreLog will read the first line of standard output (discarding all other lines) and uses this as the value to compare to the threshold. When the external program is executed, data is passed to the program using environmental variables, as follows:

SN_ADDRESS. This is the address of a single device, one of the devices specified in the address group or IP address match pattern. It will be an IP address specified in standard dot notation.

SN_READCOM. This is the read community that is appropriate for the specified IP address. It may be the default read community for the system, the read community of the group, or the specific read community for the device (configured in the device information screen.)

SN_TIMEOUT. This is the integer timeout value configured in CorreLog for the SNMP Monitor. This value is not always necessary or useful for an external program, but is provided for completeness.

SN_RETRY. This is the integer retry value configured in CorreLog for the SNMP Monitor. This value is not always necessary or useful for an external program, but is provided for completeness.

The external program will typically read the environmental values, and fetch one or more SNMP values from the specified address, using the specified read community. (The timeout and retry values can be used or ignored by the program.) When the external program derives a final result, this value is printed to standard output, where it is read by CorreLog.


Note that the CorreLog SNMP Monitor blocks further execution until the program is finished. Hence, it is important for the user to perform the operations as quickly as possible and return control back to the SNMP Monitor CO-snmp.exe process. If the user program hangs, this will block further operation of the CO-snmp.exe process.

SNMP MIB Browser CorreLog includes a "MIB Browser" screen, which can be used to browse the MIB of a managed device. The operator can click the "SNMP MIB Browser " link on the "Add New" screen, the "Edit Screen", or the "Test" screen of the CorreLog SNMP adapter, which causes a new screen to be launched as depicted below:

A MIB Browser is a traditional application of most SNMP managers, and CorreLog provides a comprehensive browsing utility useful for configuration and debug of the SNMP system. A MIB Browser function allows the user to view all the MIB objects and values of a managed device in tabular form, showing which


MIB objects are supported by the device, and what these object values currently are. The user must specify an IP address and a Read Community at the top of the screen. (If the user clicks down on the "Edit" screen, or the "Test" screen, these values are automatically supplied based upon the settings of these screens.) The special "Default" read community directs the program to fetch the read community from the managed device, or from the default read community specified on the "Messages > Config > Parms" screen of CorreLog. Once the device information has been specified, the user can set the "Start OID" value to view the MIB object names and values at a certain location. This is the starting point for the listing. The starting point can be specified manually (by entering an OID and clicking apply.) The starting point can also be set via the "JUMP" menu, which contains several predefined starting points. To traverse the MIB object list, the user clicks the "Next" button (which shows the next page). The "Skip" button is similar to the "Next" button, except this button advances the listing to the next MIB object group. The "Reset" button changes the "Start OID" value back to the first object in the list. Note that the CorreLog MIB browser automatically provides the name of the MIB object in the first column of the listing. The user can click the hyperlinked name to see the help file starting with the specified name. Within the help file, the user can click a different hyperlinked name to advance the "Start OID" to some other branch of the system (if supported by the device.) This provides an alternate and powerful method of traversing the list of SNMP MIB objects.

Section Summary, Additional Notes

1. The user adds SNMP monitor screens via the "AddNew" button on the

top-level "Messages > Adapters > SNMP" screen, where each monitor describes a MIB object to be polled from a specific list of devices, as well as a threshold and Syslog message.

2. The list of polled devices can be specified as an IP address, an address

wildcard, or an "Address Group".

3. The read community used for each device can be configured on the monitor screen, or within the "Device Information" screen of each device. If no read community is specified, the value configured in the "Messages > Config > Parms" screen is used (by default the community "public".)

4. Various threshold tests are supported. The particular type of threshold test

generally depends upon the type of MIB object being polled. Counter type MIB objects usually have a "Delta" type threshold, Gauge type objects


usually have an "Abs" type threshold, and textual objects usually have a "Change" type threshold.

5. Various methods of getting SNMP data are supported, including "Get",

"Get-Next", and "External" methods. The "Get-Next" method is especially useful for setting thresholds on MIB objects within tables.

6. If an SNMP operation is complex, the user can select an "External" type

method, which gets SNMP data from an external program residing in the "snmp" folder of CorreLog. The external program performs some operation such as counting table rows, and then outputs a value to standard output, where it is handled exactly like the "Get" and "Get-Next" type methods.

7. A "MIB Browser" utility is provided, which permits the user to browse the

list of MIB objects supported by a particular device. Note that not all devices support all MIB objects.

8. The MIB Browser is launched from the "AddNew", "Edit", and "Test"

screens of CorreLog via the "SNMP MIB Browser" hyperlink at the top of these screens.

9. The MIB Browser lists all the MIB objects beginning with a "Start OID",

where this OID is manually entered by the user, or set via a JUMP menu, or changed via the "Next", "Skip", or "Reset" values.

10. The MIB Browser lists object names, OIDs, and values. The user can click

down on any MIB object name to view help on the MIB object, or to jump to arbitrary locations within the MIB of a managed device.


Section 4: Application Notes The SNMP Monitor software allows the user to correlate message information, sent by devices in the form of Syslog messages or SNMP traps, with other information polled from the network. This provides an extra capability to gather information in a consistent way, which may not necessarily be available or self-reported by devices. For example, the SNMP Monitor software can easily detect when devices are rebooted, or are offline. Additionally, the SNMP Monitor software can determine such things as whether a port-scan is in progress (which is not necessarily reported or detected by managed devices.) Finally, the SNMP Monitor software provides persistent "state" information, such as whether a disk is filled up, or an interface is disabled. This section provides additional application notes that describe techniques for integrating the SNMP Monitor messages with other "non-solicited" messages received by the system, as well as methods of effectively configuring and managing the SNMP software. This section will be of interest to both operators and administrators. Note that the SNMP Monitor software comes with various out-of-the-box capabilities that may be sufficient for most enterprises. The operator may choose to simply enable these functions (by providing valid address groups for each monitor.) Or, the operator may wish to define different groups and monitors as described in this section.


SNMP Monitors The SNMP Monitor software operates on a series of monitor groups. These groups are displayed on the entry screen of the "Messages > Adapters > SNMP" tab. Each group is an arbitrary partition consisting of a particular metric (OID), and a group of devices that this information will be acquired from. There can be multiple SNMP Monitors on the system, with overlap between the polled objects and devices.. The SNMP Monitor title, displayed on the entry screen, is hyperlinked to the list of polled devices and the most recently polled values for each device. The user can click on the SNMP Monitor title to view all the devices being polled and the value as of the last poll. This provides an easy way to assess the nature of the monitor, including whether the threshold for the monitor is set inappropriately. The titles of each group are arbitrary, and typically represent the type of SNMP object that is being polled for the group There can be a maximum limit of 2000 SNMP Monitors, each polling a maximum of 10,000 devices. In practice, the number of SNMP Monitors will be much less. The larger the number of SNMP Monitors and polled devices, the slower the polling process will be (since SNMP objects and devices are polled sequentially.) The system will never poll faster than once every 60 seconds. The actual time to poll may be much larger, especially if there are many SNMP Monitors, each with large numbers of polled devices. The actual time to finish a single poll cycle is displayed in the lower left of the screen

Monitor Status Bar At the bottom of the SNMP Monitor screen, beneath the list of SNMP Monitors, are a series of metrics that indicate the progress and state of the CO-snmp.exe background process. These metrics are updated at the end of each poll cycle, and provide the following information:

Poll Duration. This is the time in seconds needed to poll all monitors on the system one time. The time is calculated at the end of each poll cycle, and will indicate the general load on the system. If the time is less than 60 seconds, then the CO-snmp.exe program will wait until at least 60 seconds have elapsed before resuming polling. (See additional notes below.)

Number Of Objects Polled. This is the total number of objects polled during the last cycle. It represents the total number of SNMP requests that have been issued by the program during the last poll cycle. This number will be equal to the number of SNMP Monitors multiplied by the total number of devices for each monitor. The value will be under 10,000.


Number Of Poll Timeouts. This is the total number of poll timeouts during the last cycle, indicating that an object could not be fetched. This typically indicates that one or more devices are either offline, or the read community of the device has changed or is misconfigured at CorreLog. If this value is high, the operator should address the issue by returning the device to an online state, removing the device from the SNMP Monitor, or changing the read community of the managed device. The particular devices that have timed out can be viewed by clicking on the SNMP Monitor title hyperlink.

Number Of Poll Errors. This is the total number of MIB object errors during the last cycle, indicating that the remote SNMP agent does not support the specified object. This typically indicates that the OID is misconfigured at CorreLog, or that the managed device should not be part of the group. In either case, the operator should address the problem by clicking on the SNMP Monitor title hyperlink and either removing the device or fixing the MIB OID setting.

Number Of Poll Cycles. This is the total number of poll cycles since the system started. This value increments each time a complete poll cycle finishes. This value, when divided by the system up time of the CorreLog server, will indicate the average time to poll all SNMP Monitor devices and objects.

Number of Messages Sent. This is the total number of Syslog messages that have been issued by the SNMP polling process to the CorreLog server since the system started, useful for assessing how busy the polling monitor is. The number should precisely correspond to the total number of messages in the "Messages" tab (related to the SNMP poller.)

Poll Duration The "Poll Duration" found in the lower left of the SNMP Monitor screen (and first mentioned above) provides special utility in determining the polling performance of the CO-snmp.exe program. This value indicates the total time to SNMP poll all the devices and MIB objects per cycle, taking into consideration network latencies and delays, as well as timeout and retries. This value can become fairly large, and is useful for determining performance. For example, if the value is 300 seconds, then the fastest any error condition will be detected is once every five minutes. If the value is 3600 seconds, the fastest any error condition will be detected is once each hour. To reduce this value and increase the polling screen, the user can eliminate SNMP Monitors that are not useful, or reduce the number of devices polled by each monitor to a minimum.


It is a common mistake to load the SNMP Monitor with many different MIB objects and devices, especially devices that do not support SNMP. For example, setting an address group to be "*.*.*.*" (i.e. all devices) can have a deleterious effect on the program's performance. This may result in a high value for the "Poll Duration", reducing the effectiveness of the program to rapidly detect network conditions. In some (but not all) environments, this may not be a desirable configuration for the software. Additionally, the "Poll Duration" setting is useful for setting the "Alert Interval", when opening tickets on the system. When configuring Correlation Threads and Alerts the "Alert Interval" should be greater than the "Poll Duration" setting to prevent multiple tickets from being opened due to a single network condition. This special consideration is discussed in a later section.

Working With Address Groups Each SNMP Monitor polls a list of one or more devices. A device can (and typically does) exist in multiple SNMP Monitors, and is polled for multiple values. The list of devices is configured for each SNMP Monitor using one of the following techniques:

Single IP Address. The user can specify a single IP address to be polled. In this case, the SNMP Monitor polls no other devices. The user can configure multiple SNMP Monitors, each polling the same or different MIB object, and each containing the same or different IP address.

IP Address Wildcard. The user can specify an IP address wildcard to be polled, in the form, such as 10.*.1.*, or "10.5.1.*", which will cause a range of IP addresses to be polled. This is especially useful if networks and devices follow some convention (such as routers ending in a ".100" IP address.) The special case of "*.*.*.*" matches all devices in the "Messages > Devices" tab of the program, and should be specified with caution because it can dramatically increase the poll duration, reducing the responsiveness of the polling process.

Address Group. The user can specify a CorreLog Address Group, in the form "@@name@@", configured in the "Correlation > Config > Address Groups" tab of the program. This permits the user to specify multiple IP addresses, IP address wildcards, exclude addresses, and exclude IP address wildcards. More information on Address Groups and their usage is provided in the CorreLog User Manual.

When using an IP address wildcard or address group, each address must exist in the "Messages > Devices" tab of the program. If the device IP address is not listed on the system, the user may manually add a new device using the "AddNew" button on the "Devices" screen. When specifying an IP address


wildcard, only those addresses listed in the "Devices" screen are actually polled. Specifying a wildcard that does not match any address, while not an explicit error, has no effect on polling, and causes no devices to be polled.. The special IP address "0.0.0.0" disables polling for the SNMP Monitor. This is the default address for each monitor on the system. Therefore, to use any of the "out-of-the-box" monitors, the administrator must first configure a valid IP address, IP address wildcard, or Address Group for the monitor to actually begin polling. Finally, note that the most maintainable way of configuring devices is via the "Address Group" function. This permits the user to update, add, or delete from an address group to change the polling behavior of an SNMP Monitor, without having to edit the SNMP Monitor. This is an effective technique for managing the list of polled devices. Rather than using a single IP address, it may be better practice to define an address group that represents that single IP address.

Sending Syslog Messages A separate message (of content and severity as defined by the user) is sent each time an SNMP Monitor threshold violation is detected, during every poll cycle. While the alert condition exists, the system sends additional messages, at a rate no faster than once each minute, or at the "Poll Duration" value found in the lower left of the SNMP Monitor screen. It should be well noted by the reader that, if the "Poll Duration" is 120 seconds, a new message will be sent by the CO-snmp.exe process (and received in the CorreLog "Messages" tab) every two minutes while the threshold violation exists. This means that a chronic problem can generate multiple repetitive messages scattered throughout the event logs. At first glance, it may seem to a new user that it would be better for the SNMP Monitor to send a single message only (rather than multiple messages each time the condition is detected). However, experience with CorreLog quickly demonstrates that the behavior of identifying an alert condition with multiple Syslog messages (rather than a single message) provides considerable more safety and flexibility to the user, and leveraging the analytical power of the program. Specifically, CorreLog uses the "Threads", "Alerts" and "Tickets" system functions to reduce the number messages to a single alert condition and actionable ticket. The CorreLog alert facility detects the continuous stream of messages, sets the alert, and prohibits any further tickets or alerts from being generated while the messages are being sent. This means (for example) that a user is notified a single time when a problem is first detected, and not necessarily each time a message is received indicating that the problem still persists.


Experience demonstrates that this behavior is both desirable and well handled by the Correlation functions of the program. Because the SNMP Monitor never sends the same message more than once every 60 seconds, there is no danger that CorreLog will be overburdened by these SNMP Monitor messages. Instead, this provides a clear indication of a chronic or unaddressed problem on the system, and a clear indication when that problem is finally resolved (causing the messages to stop, and any associated alert to clear.)

Creating Threads, Tickets, and Alerts Because the messages sent by the SNMP Monitor are fully under the control of the operator, it is easy to create threads, tickets and alerts that will correlate and reduce the monitor's message into actionable data. The basic method for correlating the SNMP Monitor messages is no different that the techniques discussed elsewhere. The basic steps are provided below.

1. The operator creates a thread to tabulate the messages sent by the monitor using the "Correlation > Threads > Add New" screen. This screen is used to collect all the messages of a particular type (such as all messages with "Router" or "SNMP" in their title, possibly further qualified by a particular address group, severity, or time of day.)

2. The operator creates an Alert for the thread counter using the "Alerts >

Counters > Add New" screen. This alert will send a Syslog message back to the main list of messages when one or more messages are received during an interval of time. As is always the case, when an alert is triggered, a single message is sent back to CorreLog, and a single ticket is opened while the alert is set. (See additional notes below.)

3. The operator optionally identifies an "Assignee" for the alert via the "Alerts

> Counters > Add New" screen. This causes a ticket to be opened on the system, and assigned to a particular user or a ticket group. The user can assign a ticket to any existing user, or ticket group.

4. The operator optionally adds a "Ticket Action" to the system, which sends

e-mail (or performs some other action) when a new ticket is opened on the system, providing a real-time indication that a threshold of the SNMP Monitor software has been violated. This message will typically contain the descriptive text entered by the operator when the alert was created, which may be slightly (or totally) different than the originating SNMP Monitor message.

As a special note, if only one ticket is to be opened on the system per SNMP threshold violation (as will often be the case), then the "Alert Interval", configured on the "Alerts > Counters" screen, should be higher than the "Poll Interval"


displayed at the lower left of the "Messages > Adapters > SNMP" screen. Additionally, the "Auto-Learn" function for the alert should probably be disabled to prevent this interval from changing automatically. Failure to understand or implement this consideration may result in multiple tickets being opened for the same system threshold violation, which will not be desirable, especially if one of the ticket actions is to send e-mail or provide other intrusive notifications to the ticket assignee.

Section Summary, Additional Notes

1. The CO-snmp.exe program polls each MIB object no faster than once per minute. While the threshold is in violation, the CO-snmp.exe program will repeatedly issue messages indicating the violation. These messages will cease when the violation is cleared.

2. The user can determine the poll time and poll values for the CO-snmp.exe

program by drilling down into the SNMP Monitor name hyperlink, which shows the current values for all devices during the last poll cycle.

3. If a poll fails, the value of "*Timeout*" will appear in the list of current

values (accessed by clicking the SNMP Monitor name hyperlink on the top level screen.) Unless the user has configured a "Timeout" type monitor, these timeouts are silently ignored.

4. If the poll receives an error from an agent, the value for this poll is the text

string "*Error*", which indicates that the agent is online but the MIB object (or other parameter) is not valid. These errors are silently ignored. The user should verify that the MIB object OID is configured correctly, and should verify that the agent actually supports that MIB object. (Not all MIB objects are supported by all agents.)

5. Caution should be taken to avoid specifying devices in the poll lists that

have bad read communities, or no SNMP capability. This can substantially degrade the performance of the polling (especially if the timeout and retry value is high for the monitor.) Additionally, this may cause authentication traps to be issued by the SNMP agent, which may be received by some program other than CorreLog.

6. If the read community of a device is incorrect, the user will receive a

"*Timeout*" indication as the value for the poll. This may also cause an "SNMP authentication" type trap to be issued to CorreLog. This may be the only indication that the read community is incorrect (because CorreLog will silently ignore Timeout conditions unless a "Timeout" type monitor has been configured.


7. Caution should be taken when specifying an address group of "*.*.*.*", which will cause all the devices in the "Devices" tab to be polled. This may result in multiple timeouts and errors that will degrade the performance of the polling agent and increase the "Poll Interval" (described below.) This may also cause multiple "SNMP authentication" type traps to be issued (described previously.)

8. The "Poll Interval" metric, available at the bottom-left of the SNMP Monitor

screen, indicates the time (in seconds) needed to poll all values during a single cycle. This value, if over 60 seconds, indicates the typical duration between poll cycles, and the rate at which the SNMP Monitor will send Syslog messages when a threshold is violated.

9. When configuring a CorreLog alert, the "Alert Interval" should be greater

than the "Poll Interval" value to prevent multiple tickets from being opened for a single incident. Additionally the "Auto-Learn" function for the alert should typically be disabled.


For Additional Help And Information… Detailed specifications regarding the CorreLog Server, add-on components, and resources are available from our corporate website. Test software may be downloaded for immediate evaluation. Additionally, CorreLog is pleased to support proof-of-concepts, and provide technology proposals and demonstrations on request. CorreLog, Inc., a privately held corporation, has produced software and framework components used successfully by hundreds of government and private operations worldwide. We deliver security information and event management (SIEM) software, combined with deep correlation functions, and advanced security solutions. CorreLog markets its solutions directly and through partners. We are committed to advancing and redefining the state-of-art of system management, using open and standards-based protocols and methods. Visit our website today for more information.

CorreLog, Inc. http://www.CorreLog.com mailto:[email protected]

Documents

SNMP Monitor Adapter Software - CorreLog.com Monitor Adapter, Page - 5 Section 1: Introduction This manual provides a detailed description of the CorreLog SNMP Monitor software. This