Embed Size (px)
Citation preview
Introduction To OpenNMS
OpenNMS is the world's
first enterprise-grade
network management
application platform
developed under the
open source model.
“enterprise-grade”
Over 60,000 Devices on a Single Instance (Swisscom)
1.2 Million Data Points Every Five Minutes (New Edge)
120,000 syslog message per minute (SRNS)
320,000 Interfaces per Device (Wind)
3000 Remote Monitors (Papa Johns)
“network managementapplication platform”
The Architecture of OpenNMS has been
designed to allow for easy integration of
other tools, both proprietary and open.
“open source model”
OpenNMS is published under the GPL
and all components are licensed under
an OSI-qualified free software license.
The Four Main Areas ofOpenNMS
Provisioning: Both Automated Discovery andDirected Discovery.
Event and Notification Management: Generate,receive, reduce and correlate various networkalerts and feed them to a robust notificationsystem.
Service Assurance: Is a particular network servicereachable and available?
Performance Data Collection: Gather numericdata from across the network for display,trending and thresholding.
The Architecture
OpenNMS Versions
Stable (Production) Versions Have an Even Number:
– 1.8
– 1.10
– 1.12
Unstable (Development) Versions Have an Odd Number:
– 1.11
– 1.13
OpenNMS Installation:Windows
http://www.opennms.org/wiki/Installation:Windows
Install the JDK
• Download the Java 6 (1.6), Java 5 (1.5) orhigher Java SE JDK from java.sun.com andinstall it. The JDK, not just a JRE, is required bythe web UI, since JSP pages are dynamicallycompiled. You need the version labeled "JavaSE" (for "standard edition"), not EE, ME, or FX.
Install PostgreSQL
• Download the Installer
• Download the one-click PostgreSQL for Windows installer.
• http://www.postgresql.org/download/windows
Run the PostgreSQL Installer
• Run the installer. For the most part, thedefaults should be just fine. You should notneed to run the Stack Builder for OpenNMS,although if you intend to use PostgreSQL forother things, it lets you install replication,web, and ODBC tools.
Initialize the Database
• Create a Database in PostgreSQL_Root\data– If for some reason you don't have a default
database initialized from the installer, you cancreate it yourself:
– open a command prompt (Start -> Run -> cmd)and change to the bin directory of yourPostgreSQL install (by default, C:\ProgramFiles\PostgreSQL\X.X\bin)
– Initialize the database with the followingcommand:• initdb -E UTF-8 -U postgres ..\data
Install OpenNMS• If you did not start PostgreSQL already, start it by going
to the "PostgreSQL X.X" menu in the Start bar, and click"Start service".
• Then, all you need to do is download the lateststandalone-'opennms-installer-X.X.X.zip' from theopennms section on the OpenNMS download page.Once it is downloaded to your hard drive, you shouldbe able to just double-click setup32.exe or setup64.exeit in Explorer, and it will start installation. Note that thesetupXX.exe you run should match your JVM's arch, soif you're running a 32-bit JVM on 64-bit Windows,install using setup32.exe.
• Follow the instructions and you should have a complete OpenNMS installation!
Run OpenNMS
• OpenNMS can be run from the command-line, usingopennms. bat in your %OPENNMS_HOME%\bindirectory. Assuming you installed OpenNMS toC:\Program Files\OpenNMS, you would open acommand-prompt, and cd to C:\ProgramFiles\OpenNMS\bin. Then run:
– opennms.bat start
• ...and OpenNMS should start. Open your browser and point it at http://localhost:8980/opennms and log in as "admin" with the password "admin".
Logging In
Default username/password is admin/admin
Click on Admin
Admin Menu
Add A User
Go to:
• Configure Users, Groups and On-Call Roles
Choose:
• Configure Users
Set the Username and Password
Add User Details
Click Finish!
Now, Add to Admin Group
Duty Schedules
• Users can have duty schedules
–No notices when not on duty
–Multiple schedules can exist
• Groups can have duty schedules
–Outstanding notices sent when back onduty
–Overridden by users
• On-Call roles always get notices
OpenNMS File Locations
/opennms/bin: OpenNMS binary files
/opennms/etc: Configuration files
/opennms/jetty-webapps: Web server
/opennms/lib: Compiled libraries
/opennms/share: Reports and RRD data (symlink to /var/opennms)
/opennms/logs: Log Files
(symlink to /var/log/opennms)
Edit magic-users.properties• #######################################################
####################• ## R O L E S• #######################################################
####################• # A comma-separated list of role keys. A role.{KEY}.name and• # role.{KEY}.users property must be set for each key in this
property.• roles=rtc, admin, rouser, dashboard, provision, remoting, rest• # This role allows a user to make RTC data posts.• role.rtc.name=OpenNMS RTC Daemon• role.rtc.users=rtc• role.rtc.notInDefaultGroup=true• # This role allows users access to configuration and• # administrative web pages.• role.admin.name=OpenNMS Administrator• role.admin.users=admin,tarus
Login as New User
Configure Automatic Discovery
Click Home > Nodelist
Events
Introduction
http://www.opennms.org/wiki/Tutorial_Notifications
Events
• OpenNMS is event driven
• The key process is called “eventd”
• Listens on port 5817 for XML
• Daemon config: eventd-configuration.xml
• Events config: eventconf.xml
• Events identified by a “uei”: unique eventidentifier
• Use send-event.pl to send events
Unique Event Identifier
• A new interface is discovered:
uei.opennms.org/internal/discovery/newSuspect
• A service is down:
uei.opennms.org/nodes/nodeLostService
• All services on an interface are down:
uei.opennms.org/nodes/interfaceDown
• All interfaces on a node are down:
uei.opennms.org/nodes/nodeDown
eventd-configuration.xml
<EventdConfiguration
TCPAddress="*"
TCPPort="5817"
UDPAddress="*"
UDPPort="5817"
receivers="5"
getNextEventID="SELECT nextval('eventsNxtId')"
socketSoTimeoutRequired="yes"
socketSoTimeoutPeriod="3000">
</EventdConfiguration>
eventconf.xml
This file contains three things:
• Global security settings
• OpenNMS event definitions
• Include files for non-OpenNMS event definitions
Try and open the eventconf.xml file from opennms/etc folder
Security Settings
<global><security>
<doNotOverride>logmsg</doNotOverride><doNotOverride>operaction</doNotOverride><doNotOverride>autoaction</doNotOverride><doNotOverride>tticket</doNotOverride><doNotOverride>script</doNotOverride>
</security></global>
An OpenNMS Event<uei>uei.opennms.org/internal/discovery/newSuspect</uei>
<event-label>OpenNMS-defined internal event: discovery
newSuspect</event-label>
<descr>
<p>Interface %interface% has been discovered and is
being queued for a services scan.</p>
</descr>
<logmsg dest='logndisplay'>
A new interface (%interface%) has been discovered and is
being queued for a services scan.
</logmsg>
<severity>Warning</severity>
</event>
Event Include Files
<event-file>events/Translator.default.events.xml</event-file>
<event-file>events/Rancid.events.xml</event-file>
<event-file>events/3Com.events.xml</event-file>
<event-file>events/AdaptecRaid.events.xml</event-file>
<event-file>events/ADIC-v2.events.xml</event-file>
<event-file>events/Adtran.events.xml</event-file>
<event-file>events/Adtran.Atlas.events.xml</event-file>
<event-file>events/Aedilis.events.xml</event-file>
<event-file>events/AirDefense.events.xml</event-file>
<event-file>events/AIX.events.xml</event-file>
<event-file>events/AKCP.events.xml</event-file>
<event-file>events/AlcatelLucent.SMSBrick.events.xml</event-file>
<event-file>events/Allot.NetXplorer.events.xml</event-file>
<event-file>events/Allot.SM.events.xml</event-file>
<event-file>events/Alteon.events.xml</event-file>
<event-file>events/Altiga.events.xml</event-file>
<event-file>events/APC.events.xml</event-file>
<event-file>events/APC.Best.events.xml</event-file>
<event-file>events/APC.Exide.events.xml</event-file>
Let’s open C:\Program Files\OpenNMS\etcView event-file from C:\Program Files\OpenNMS\etc\events
Let’s try to send some event
Restarting eventd
• Simply restart OpenNMS
• Use a reload event:send-event.pl uei.opennms.org/internal/eventsConfigChange
Remember that there is no error checking
Event Severities
Create an Event
• Send a uei.opennms.org/class/happiness event
• Note how it appears in the GUI
• Create a Class.events.xml file
• Add it to eventconf.xml
• Reload the event configuration
• Send the event again
• Note how it appears in the GUI
<events>
<event>
<uei>uei.opennms.org/class/happiness</uei>
<event-label>OpenNMS defined event: The OpenNMS Class is so
happy</event-label>
<descr>
<p>This event is sent when the OpenNMS Class is happy.</p>
<ul>
<li>Dance, Everybody Dance!</li>
<li>Life is Good!</li>
<li>This is Fun!</li>
</ul>
</descr>
<logmsg dest='logndisplay'>
<p>OpenNMS Class is Happy! </p>
</logmsg>
<severity>Normal</severity>
</event>
</events> Save Class.events.xml in: C:\Program Files\OpenNMS\etc\events
Add the new Class.events.xml file to eventconf.xml
<event-file>events/Class.events.xml</event-file>
<event-file>events/Translator.default.events.xml</event-file>
<event-file>events/Rancid.events.xml</event-file>
<event-file>events/3Com.events.xml</event-file>
<event-file>events/AdaptecRaid.events.xml</event-file>
<event-file>events/ADIC-v2.events.xml</event-file>
<event-file>events/Adtran.events.xml</event-file>
<event-file>events/Adtran.Atlas.events.xml</event-file>
Save that file then send the reload event:
• Note that the reload was successful:
• Resend the “happiness” event:
Notifications
Introduction
http://www.opennms.org/wiki/Tutorial_Notifications
Notices
Notices bring events to others attention
• Events can trigger notices
– The event happens
– A notice is triggered
– It “walks a path”
– Those along the path get notified
• Escalations can insure notices get attention.
Built-in Notifications
• A number of notifications are built in
• Turn on notices
• Add a node and see the notifications, as
“nodeAdded” is a default notice.
Create a User
• You can do so by clicking the "Admin" link in themenu bar, going to "Configure Users, Groups andOn-Call Roles", and then clicking "ConfigureUsers".
• Then, click the "Add New User" link, and enter ausername and password in the form, then click"OK". Next, fill out the relevant user information.
• Finally, click "Finish" down at the bottom of theform.
Destination Paths
• To receive a notification, your user or a groupthat your user is in needs to be a part of adestination path. A destination path helpsOpenNMS determine who is eligible to receivea particular notification, depending on username, groups, duty schedules, and otherrules.
Create a Destination Path
• Go to the Admin page, click "ConfigureNotifications" in the "Operations" section,then "Configure Destination Paths," then clickthe "New Path" button.
• Give it a name, like "Tutorial", then click"Edit". Add your new user by selecting it, thenclick "Next >>>":
Admin → Notifications → Destination Paths
The “Email-Admin” Path
• On the next page, leave the defaults ("javaEmail" and"on") and click "Next >>>" again.
• Finally, click "Finish" and you should see your newtutorial path in the "Existing Paths" section.
Different Types of Targets
Group Intervals
Notification Command
Event Notifications
• The next thing to do is create an eventnotification, which ties any OpenNMS eventtogether with a destination path.
• Go to the Admin page, click "ConfigureNotifications" in the "Operations" section,then "Configure Event Notifications."
• You can create a new event notification byclicking the "Add New Event Notification“, editor delete.
Creating an Event Notification
• "Add New Event Notification" button at thetop of the Event Notifications page.
• look for "OpenNMS-defined internal event: anauthentication failure has occurred in WebUI":
Then click "Next >>>".you can build a rule for matching a set of IP addressesand/or services. In this case, we're just matching aninternal OpenNMS event. click "Skip resultsvalidation >>>".
Choose the Triggering Event
Apply Any Filters
you can build a rule for matching a set of IP addresses and/or services.
Fill in the following values:
• Name: authenticationFailed• Choose A Path: Tutorial• Text Message: The OpenNMS Web UI had a failed
login attempt, by user '%parm[user]%', from IP address %parm[ip]% (Exception message: %parm[exceptionMessage]%)
• Short Message: Authentication failed by user %parm[user]% (Notice #%noticeid%)
• Email Subject: Authentication Failed (Notice #%noticeid%)
The Notice
• You should now see your notification event inthe list, but it will be disabled. Set the radiobutton to "On" to enable it, and you should beready to send a notification:
Hint: UEI Filter
• You can use Regex UEI filter in notifications touse one notification for several events.
• Unfortunately you can't use the WebUI for thisconfiguration. You need to edit yournotification in /etc/opennms/notifications.xml.
Examples:
<uei>~uei.opennms.org/custom/event.*</uei><uei>~uei.opennms.org/.*/event</uei> <uei>~uei.opennms.org/custom/event[0-9]+</uei><uei>~uei.opennms.org/custom/event\d+</uei>
Configure Mail
Unless you have an SMTP server running on yourOpenNMS host, you will need to configure mailbefore notifications will work. Mail configurationfor notifications is in the
– "$OPENNMS_HOME/etc/javamail-configuration.properties" file.
# This is the e-mail address that OpenNMS puts in the "From" field:
• org.opennms.core.utils.fromAddress=FROM-EMAIL-ADDRESS
• org.opennms.core.utils.mailHost=MAIL-SERVER-IP
if your mail server requires authentication forsending, see the section under "Theseproperties define the Mail authentication."and edit them as appropriate for yourenvironment.
• The last bit is to pass the mail server information to OpenNMSBacking up file /opt/opennms/etc/javamail-configuration.properties, and change the contents to followingorg.opennms.core.utils.useJMTA=falseorg.opennms.core.utils.transport=smtporg.opennms.core.utils.mailHost=smtp.gmail.comorg.opennms.core.utils.smtpport=587org.opennms.core.utils.smtpssl.enable=falseorg.opennms.core.utils.authenticate=trueorg.opennms.core.utils.authenticateUser=hello@gmail.comorg.opennms.core.utils.authenticatePassword=mypasswordorg.opennms.core.utils.starttls.enable=trueorg.opennms.core.utils.messageContentType=text/htmlorg.opennms.core.utils.charset=UTF-8
Above configuration allows you to send email to your gmailaccount, which might be good if you want use OpenNMS to monitor your home server.
Try the email configuration with your internet account
• if you are using OpenNMS in your company, you need to make it to talk to your own mail server instead, you can use following configurationorg.opennms.core.utils.transport=smtporg.opennms.core.utils.useJMTA=falseorg.opennms.core.utils.mailHost=mailhost.doloveyou.comorg.opennms.core.utils.smtpport=25org.opennms.core.utils.authenticate=trueorg.opennms.core.utils.authenticateUser=leaonow@doloveyou.comorg.opennms.core.utils.authenticatePassword=passwordorg.opennms.core.utils.fromAddress=opennms@doloveyou.comorg.opennms.core.utils.charset=UTF-8
• some server will complain if you do notspecify org.opennms.core.utils.fromAddress
• now save the file, and restart OpenNMS, thengenerate some event and you should be ableto receive a notification email.
• if not, you can check the log file for notification which is /opennms_home/logs/daemon/notifd.log
Enable Notifications
• By default, OpenNMS ships with notifications disabled. In the main OpenNMS Admin page, there is a radio selector labeled, "Notification Status." Change it to "On" and click update.
Now let's trigger your notification. Log out and Log in with wrong password.
You should receive an email telling you about the
authentication failure. If you don't, take a look at
"$OPENNMS_HOME/logs/daemon/notifd.log"
and see if you get any error messages.
Alarm
Introduction
Alarms
• OpenNMS gives you the ability to indicatewhich events are important and they becomealarms. Also, with this ability, you can nowreduce these important events to one row inthe alarms table with the reduction-keyelement of the the <alarm-data> tag inthe event configuration files. This allows youto decide the granularity of the reduction asyou will see in the sample images below.
Alarms with new style sheet and sorted by severity
This is much nicer and generally more useful than the raw events.
Alarms sorted by count
This shows the number of events that were reduced to a single alarm row.
Un-reduced event list
Click on a count in the alarm listing and and jump to the list of un-reduced events for that alarm.
Create an Alarm from an EventSimply add <alarm-data> to the event definition
<event>
<uei>uei.opennms.org/default/event</uei>
<event-label>OpenNMS-defined default event: event</event-label>
<descr>
<p>An event with no matching configuration
was received from interface %interface%. This event
included the following parameters:
%parm[all]%</p>
</descr>
<logmsg dest='logndisplay'>
An event with no matching configuration was received from interface
%interface%.
</logmsg>
<severity>Indeterminate</severity>
<alarm-data
reduction-key="%uei%:%dpname%:%nodeid%:%interface%"
alarm-type="3" />
</event>
Capability Scanning
Introduction
http://www.opennms.org/wiki/Tutorial_Capability_Scanning
Detecting Services on Devices
• In order to detect services on devices, theProvisiond daemon executes detectors to detectthe presence of configured services. Once aservice is detected, it is used by other daemonsto monitor availability, or to collect data about it.
• These services are configured in the defaultforeign-source definition, which is pre-configuredwith a number of common services, like HTTP,SMTP, and so on. All Provisiond needs to getstarted is an IP address that OpenNMS suspectsmay provide one or more of these services.
New Suspect Events
• Provisiond listens for "new suspect" events, which tell OpenNMS, "I suspect the IP address at <x> has services." These newSuspect events can come from a number of sources: – an event sent from the command-line, using the send-
event.pl script
– an event sent from a 3rd-party tool, talking on the OpenNMS event listener TCP port
– the Discovery daemon, which scans IP ranges for valid IP addresses (using ICMP "ping")
– the "Add Interface" page on the OpenNMS administrator web UI
Scanning an Address
• When Provisiond receives a newSuspect event, itexecutes the configured detectors to determinethe presence of each service on the IP addressprovided by the event. It then creates a node andinterface representing the IP address, and addsany services that it detects to that interface.
• After the scan is complete, the node, interfaces,and services are written to the database, andthen OpenNMS events are sent to indicate thatthese new nodes, interfaces, and services exist.
Adding a Custom Service
• To add a custom service to Provisiond, log into the web UI as an admin user. Go to Admin /Manage Provisioning Requisitions and clickthe Edit Default Foreign Source Definitionbutton and add a detector for your customservice. (Gold, Silver, Bronze package)
All Provisiond detectors take the port, timeout, and retry parameters, but manycan take additional parameters as well which allow you to tune a particularplugin's behavior. The available parameters for the chosen detector areenumerated in a drop-down list as you build the detector definition
• Some example detectors are HttpDetector fordetecting web servers, TcpDetector fordetecting an arbitrary open TCP port. To add acustom service is make a new detector entryin the default foreign-source definition, with aunique service name defined.
Let’s Try:-title=Adding the OpenNMS Web Servercontents=As an example, Make OpenNMS detect its web UI. The OpenNMS web UI is just an HTTP server, listening on port 8980, so we'll use the HttpDetector to detect it. Edit the default foreign-source definition and add the following to the end of the list of configured detectors: (don’t forget to synchronize)
By default, Provisiond will re-scan all of your devices every 24 hours. If you wish to force itto schedule a rescan for your device immediately, it will do so when it receives the"forceRescan" event. Click “Node List”, select the added node then click “rescan”.
Discovery_Configuration_How-To
http://www.opennms.org/wiki/Discovery_Configuration_How-To
Discovery Configuration Details
The global discovery attributes are: threads packets-per-second initial-sleep-time restart-sleep-time timeout retries specific include-range exclude-range include-url
• threads: This is the number of threads that willbe used for discovery. By default this is set to 1.
• packets-per-second: This is the number of ICMPpackets that will be generated each second. Thedefault is 1.
• Relationship between the packets-per-secondand the number of threads. If a network average latency = 500ms, if packets-per-second = 2:– Then double the speed of NewSuspect messages were
created. If = 1 thread, packets-per-second = 3 would have little effect
– The single thread would be processing as many packets as it could as fast as it could.
• initial-sleep-time: This is in milliseconds,before the discovery process will commenceafter OpenNMS is started (by default 5minutes). This delay is put in place to allowthe product to fully start before generatingnew events.
• restart-sleep-time: Once the discoveryprocess has completed, in milliseconds, beforeit will start again. By default, the process willrepeat 24 hours after the last discovery runhas completed.
• timeout: this is the amount of time, inmilliseconds, that the discovery process willwait for a response from a given IP addressbefore deciding that there is nothing there.This can be overridden later in the file.
• retries: this is the number of attempts thatwill be made to query a given IP addressbefore deciding that there is nothing there.This can be overridden later in the file.
• Once the defaults are in place (defaultsmeaning the global values that will be used ifthey are not overridden in the tags below),the only thing left to tell the discovery processis which IP addresses to try. This is controlledby four different tags:
– Specific
– include-range
– exclude-range
– include-url
specific
• specify a IP address to be discovered. Multiplespecific tags can be used.<specific>ip-address</specific>Where ip-address is theaddress you want discovered. Note the lack ofspaces between the tags.
include-range
• Specify a range of IP addresses to bediscovered. Multiple include-range tags can beused.
<include-range>
<begin>start-ip-address</begin>
<end>end-ip-address</end>
</include-range>
• Where start-ip-address is the beginning of arange to be scanned and end-ip-address is theend of that range.
exclude-range• Specify a range of IP address to be excluded from
discovery.<exclude-range> <begin>start-ip-address</begin><end>end-ip-address</end></exclude-range>
• Where start-ip-address is the beginning of a range tobe excluded and end-ip-address is the end of thatrange.
• Note that the exclude-range tag will only overrideaddresses in an include-range. It will not overridespecific IP addresses or addresses included in a file.
• There is no "specific" version of the exclude tag - if youwant to exclude a specific IP address use an exclude-range where the beginning and ending IP addresses arethe same.
include-url
• Specify a file containing IP addresses to be included in discovery.<include-url>file:filename</include-url>
• Where filename is the full path to a text filelisting IP addresses, one to a line. Commentscan be imbedded in this file. Any line thatbegins with a "#" character will be ignored, aswill the remainder of any line that includes aspace followed by "#".
• All tags are optional and unbounded (you canhave as many as you wish).
• Let's try having OpenNMS ping all the addressesclose to your management system.
• First determine the IP address of yourmanagement system. (We'll use 10.0.1.17 in thisexample.) We will configure Discovery with theclass C range of your address (For this example,10.0.1.1 to 10.0.1.254).
• Go to the admin page in the OpenNMS webUI and then click "Configure Discovery", underthe "Operations" section. In the DiscoveryConfiguration GUI click on Add New in theInclude Ranges section
• This will bring up a window like the one below.Fill it in with the begin and end address of yourmanagements systems network and click 'Add'.After this return the main DiscoveryConfiguration GUI and click Save & RestartDiscovery.
• Assuming you have no other include rangesconfigured on this system it should takeopennms about 5 minutes to ping all of thosedevices. After that time go look at the NodeList and see the devices it has discovered.
Polling_Configuration_How-To
http://www.opennms.org/wiki/Polling_Configuration_How-To
• There are two major ways that OpenNMS gathers data about the network. The first is through polling.Processes called monitors connect to a network resource and perform a simple test to see if the resourceis responding correctly. If not, events are generated. The second is through data collection using collectors.Example, SNMP data-collector
OpenNMS Data Gathering
pollingData
collection
Monitor processes
Collector processes
Network resources
Polling/Monitoring
• The basic idea behind the poller starts with grouping network devices into packages. Each package willconsist of various services and how they are to be polled (i.e. frequency). In addition, should an outage bedetected, each package can have its own downtime model which controls how the poller will dynamicallyadjust its polling on services that are down. Finally, each package has an outage calendar that schedulestimes when the poller is not to poll (i.e. scheduled downtime).
poller
Monitor processes
Network devices
Packages:1.Services2.frequency
Downtime model:1. Dynamically adjust2. Outage calendar
control
Polling/Monitoring
Polling/Monitoring
The OpenNMS pollerd subsystem is responsible for pollingits defined list of services each monitoring interval (bydefault, every 5 minutes). If any service is down, pollerd willsend an event notification to other OpenNMS subsystemsto handle as desired (such as the notification system, whichcould then send an e-mail alert).
Pollerd
(every 5 minutes)
SYS#3DOWN
DOWN EVENT
SYS#1-UP
SYS#2-UP
Notification
OpenNMS
• Outages are an output of the poller, as are the nodeLostService / nodeRegainedService and equivalentpaired events for interface- and node-level outages. When the poller sees an outage, it broadcasts a"down" event and opens an outage. Persisting the outage to the DB is Pollerd's job; persisting the event isEventd's job. Internally, the event bus is implemented as a FIFO queue, so events cannot "leap-frog" eachother while they're on the bus.
pollerd
nodeRegainedSer
vice
nodeLostService
Network devices
Packages:Downtime
model:
outage
outage
DB Eventd
“do
wn
”
FIFO queue
Poller-configuration
• Pollerd's main configuration file is /etc/poller-configuration.xml. – open the file
• Note that pollerd requires its monitoredservices to already have been initiallyprovisioned in order to start monitoringthem. Provisiond handles this serviceprovisioning using either requisitions ordetectors
Requisitions –manually define nodes & services
Detectors –Auto discovery the nodes and services
The Poller Configuration File Header
• <poller-configuration threads="30" serviceUnresponsiveEnabled="false" pathOutageEnabled="false"> <node-outage status="on" pollAllIfNoCriticalServiceDefined="true"> <critical-service name="ICMP"/> </node-outage>
http://www.opennms.org/documentation/java-xsddocs-stable/poller-configuration.html
• poller-configuration threads :- Determines the maximum number of threads that will beused for polling, and can be adjusted up or down depending on the size of your network andthe power of your server.
poller#2
Monitor processes Network devices size
Packages:1.Services2.frequency
Downtime model:1. Dynamically adjust2. Outage calendar
control
poller#1 poller#30
Server power
Threads
poller-configuration threads
• serviceUnresponsiveEnabled :- A poll consists of a connection to a particular port on a remote interface,and then a test to see if the service on that port returns an expected response. If the response is notreceived within the timeout, the service is considered down. In some networks, however, short,intermittent failures are common. This will result in what is known as a "30 second outage". Due to thedefault downtime model, a failed service will be polled again in 30 seconds. Problem: a user attempting toaccess that resource would also have experienced a timeout. So the option was added to denote a failureas when the port connection fails and not the response. In this case, an unresponsive service does notgenerate an outage, but only a "service unresponsive" event. To enable this behavior, set this value to"true".
pollerConnection:1.Service response ?2.Port connection ?
Packages:ServiceFreq=30sec
Downtime model:
If 1==timeout then unresponsive service eventElse if 2==fail thenoutage event
If 1==30 sec outage And Freq Poll==30 sec thenoutage event
serviceUnresponsiveEnabled =“False” serviceUnresponsiveEnabled =“True”
serviceUnresponsiveEnabled
• node-outage:- The basic event that is generated when a poll fails is called "NodeLostService". If > 1service is lost, multiple NodeLostService events will be generated. If all the services on an interface aredown, instead of a NodeLostService event, an "InterfaceDown" event will be generated. If all theinterfaces on a node are down, the node itself can be considered down, and this section of theconfiguration file controls the poller behavior should that occur. If a "NodeDown" event occurs and node-outage status="on" then all of the InterfaceDown and NodeLostService events will be suppressed and onlya NodeDown event will be generated. Instead of attempting to poll all the services on the down node, thepoller will attempt to poll only the critical-service, by default ICMP. Once the critical service returns, thepoller will then resume polling the other services. If the critical service is not available on a node, thepollAllIfNoCriticalServiceDefined parameter controls the behavior. If set to "true" then all services will bepolled. If set to "false" then the first service in the package that exists on the node will be polled untilservice is restored, and then polling will resume for all services.
# a node consists of interfacesIf poll == node service fail then NodeLostService eventIf poll == node services fail ( >1) then multiple NodeLostService eventIf poll == interface all services fail then InterfaceDown eventIf poll == all interfaces in a node down the NodeDown event
if node-outage status="on" suppressed InterfaceDown and NodeLostService eventspoll only critical-service (default ICMP) until services return
If poll == critical services return then resume polling other servicesIf poll == critical services not available And
if pollAllIfNoCriticalServiceDefined =“true” then poll all servicesif pollAllIfNoCriticalServiceDefined =“false” then
poll first services exists in package until services return
node-outage status
pathOutageEnabled
• To be Updated soon
http://www.opennms.org/wiki/Path_Outage_How-To
pollerd
Network devices
Downtime model:
Packages1:Http, SNMP,DNSICMP, JMX,SMTPFreq: 1 minute
Packages2:Http, SNMP,DNSICMP, Freq: 5 minute
Packages3:Http, ICMP, Freq: 15 minute
Downtime model:
Downtime model:
A poller package consists of a name, a group of interfaces to poll, and the services to be polled on those interfaces. Multiplepackages can be configured, and an interface can exist in more than one package (although the value of that is questionable). This gives great flexibility to how the service levels will be determined for a given device. In addition to a list of services, eachpackage can have a "downtime" model and an "outage calendar”
Poller Packages
The definition of a package starts with a package tag:• <package name="example1"> This is followed by a list of
tags that define what interfaces will be included in thepackage. There are five of these tags:
filter• IPADDR IPLIKE *.*.*.* Each package must have a filter tag
that performs the initial test to see if an interface should beincluded in a package. Filters operate on interfaces (notnodes). Only one filter statement can exist per package.
specific• <specific>192.168.1.59</specific> .This specifies a
particular IP address to include in a package. include-range• <include-range begin="192.168.0.1"
end="192.168.0.254"/> .This specifies a particular range of IP addresses to include in a package.
exclude-range• <exclude-range begin="192.168.0.100"
end="192.168.0.104"/> This specifies a particularrange of IP addresses to exclude in a package. This willoverride an include-range tag.
include-url• <include-url>file:/opt/OpenNMS/etc/include</include-url>
This tag will point to a file that consists of a list of IPaddresses, one to a line, that will be included in thepackage. Comments can be imbedded in this file. Any linethat begins with a "#" character will be ignored, as will theremainder of any line that includes a space followed by "#".
• All of the above tags, except for filter, are optional andunbounded.
Poller Services
Once the IP addresses to include in a packageare defined, the services to be polled are listed.For example:
• <service name="DNS" interval="300000" user-defined="false" status="on"> <parameter key="retry" value="3"/> <parameter key="timeout" value="5000"/> <parameter key="port" value="53"/> <parameter key="lookup" value="localhost"/> </service>
There must be at least one service defined per package.
The common parameters for the poller service are as follows:
• retry– The number of attempts that will be made to
connect to the service. Default is 3
• timeout– The amount of time, in milliseconds, that
OpenNMS will wait for a response from the service. Default is
port lookup
• Note that the service configuration parameters in thepoller can be different from the detector configurationin the foreign-source definition, but the service namesin these two places must match exactly. You may wanta longer timeout during service detection.
• In this example, a DNS request will be made to look up"localhost". This should return an error (as localhost isusually not listed in a DNS) but if that error is returned,DNS is functioning properly and the test passes.
• Microsoft's implementation of DNS, however,sometimes has problems with this, so you may want toput a real host for the lookup value (and in thedetector entry in the foreign-source definition as well).
Scheduled Outages
• In order to keep servers operating properly, it isoften necessary to bring them down forscheduled maintenance. Instead of having thesemaintenance outages reflected as a true serviceoutage, they can be included in a ScheduledOutage and then referenced by the pollerpackage using the outage-calendar tag. This tagcontains the name of a valid outage in the poll-outages.xml file.
• The outage-calendar tag is optional andunbounded (i.e. you can reference more than oneoutage).
• Since version 1.5.91 you can configurescheduled outages from the GUI, go to Admin-> Scheduled Outages. Note that scheduledoutages may be edited fully from the web UI;it's no longer necessary to edit the pollerconfiguration manually to associate ascheduled outage with a poller package,although it's still possible.
• Before version 1.5.91, there were three typesof scheduled outages: weekly, monthly andspecific. Since 1.5.91 there is also thepossibility to configure daily scheduledoutages.
• If you have the problem that nodes arereported to be down although they are withina daily outage which goes past midnight, try todefine two timespans within the outage, oneuntil midnight and the other one starting aftermidnight, e.g. instead of outage 22:00:00-01:00:00 define 22:00:00-23:59:59 and00:00:00-01:00:00.
Examples from the poll-outages.xml file:
<outage name="global" type="weekly"> <time day="sunday" begins="12:30:00" ends="12:45:00"/> <time day="sunday" begins="13:30:00" ends="14:45:00"/> <time day="monday" begins="13:30:00" ends="14:45:00"/> <time day="tuesday" begins="13:00:00" ends="14:45:00"/> <interface address="192.168.0.1"/> <interface address="192.168.0.36"/> <interface address="192.168.0.38"/> </outage>
This defines an outage calendar called "global" that is run everyweek. It specifies four outage times: Sunday starting at 12:30 pmand lasting 15 minutes, Sunday starting at 1:30 pm and lasting anhour and fifteen minutes, the same outage on Monday, and oneon Tuesday from 1:00 pm to 2:45 pm. Three interfaces will beaffected.
<outage name="hub maintenance" type="monthly"> <time day="1" begins="23:30:00" ends="23:45:00"/> <time day="15" begins="21:30:00" ends="21:45:00"/> <time day="15" begins="23:30:00" ends="23:45:00"/> <interface address="192.168.100.254"/> <interface address="192.168.101.254"/> <interface address="192.168.102.254"/> <interface address="192.168.103.254"/> <interface address="192.168.104.254"/> <interface address="192.168.105.254"/> <interface address="192.168.106.254"/> <interface address="192.168.107.254"/> </outage>This outage calendar is called "hub maintenance" that is runevery month. On the first of the month the outage begins at11:30 pm and lasts 15 minutes. The same outage occurs on the15th of the month in addition to another outage from 9:30 pmto 9:45 pm.
• <outage name="proxy server tuning" type="specific"> <time begins="10-Nov-2001 17:30:00" ends="11-Nov-2001 08:00:00"/> <interface address="192.168.0.1"/> </outage>
This outage named "proxy server tuning" began on November 10th,2001 at 5:30 pm and lasted until 8:00 am the next day. This affectedone interface. You can have more than one "time" entry per specificoutage.If a particular outage calendar is included in a poller package, thenpolling will not occur during this time. This does not mean that theservice will be considered "up" during this time. If the maintenanceis started a minute too soon and an outage is detected, then no pollwill be made to restore the service until after the outage window hasclosed.
Downtime Models
• The goal of the poller is to verify service levels,and everyone involved would like to see thosebe as high as possible.
• By default, the poller will poll every fiveminutes. If that polling rate was static, thenthe shortest an outage could be would be fiveminutes: one poll to note the outage and thenext to note it was restored.
• In these days of service levels in the "99.99%"range, a five minute outage can bedevastating.
• To help combat this, OpenNMS uses adaptivepolling. Once an outage is detected, polling istemporarily increased to try and detect, as soonas possible, when the service is restored.
• <downtime interval="30000" begin="0" end="300000"/> <!-- 30s, 0, 5m --> <downtime interval="300000" begin="300000" end="43200000"/> <!-- 5m, 5m, 12h --> <downtime interval="600000" begin="43200000" end="432000000"/> <!-- 10m, 12h, 5d --> <downtime begin="432000000" delete="true"/> <!-- anything after 5 days delete -->
What this downtime model will do is thefollowing:
• from the moment the outage begins (time 0)until five minutes later (time 300,000 ms), thepoller will poll every 30 seconds (30,000 ms).
• After five minutes, it is assumed that anyservice level that would be greatly affected bya five minute outage has been broken, so fromfive minutes (300,000 ms) into the outageuntil the first 12 hours of the outage(43,200,000 ms) polling resumes its fiveminute (300,000 ms) interval.
• If the outage is older than 12 hours, it must notbe important and/or it is difficult to fix, so fromwhen the outage is 12 hours old until it is 5 days(432,000,000 ms) old, the interval is reduced topoll once every ten minutes (600,000 ms).
• If a service has been down for longer than fivedays, it is deleted (well, marked as "forcedunmanaged") and no longer polled. Note that thisis optional, you can continue to poll a downservice for as long as you would like. For the lastdowntime interval in the model, just leave the"end" time off in order to extend pollingindefinitely.
Poller Monitors
• For each service in a poller package, theremust be a corresponding monitor. In thedetector configuration in the foreign-sourcedefinition, the detector class was included inthe detector entry itself, but since there is thepotential for a particular service to exist manytimes in the poller configuration file, this bit ofbookkeeping was put, once, at the end of thefile.
<monitor service="DominoIIOP" class-name="org.opennms.netmgt.poller.DominoIIOPMonitor"/> <monitor service="ICMP" class-name="org.opennms.netmgt.poller.IcmpMonitor"/> <monitor service="Citrix" class-name="org.opennms.netmgt.poller.CitrixMonitor"/> <monitor service="LDAP" class-name="org.opennms.netmgt.poller.LdapMonitor"/> <monitor service="HTTP" class-name="org.opennms.netmgt.poller.HttpMonitor"/> <monitor service="HTTP-8080" class-name="org.opennms.netmgt.poller.HttpMonitor"/> <monitor service="HTTP-8000" class-name="org.opennms.netmgt.poller.HttpMonitor"/> <monitor service="HTTPS" class-name="org.opennms.netmgt.poller.HttpsMonitor"/> <monitor service="SMTP" class-name="org.opennms.netmgt.poller.SmtpMonitor"/> <monitor service="DHCP" class-name="org.opennms.netmgt.poller.DhcpMonitor"/> <monitor service="DNS" class-name="org.opennms.netmgt.poller.DnsMonitor" /> <monitor service="FTP" class-name="org.opennms.netmgt.poller.FtpMonitor"/> <monitor service="SNMP" class-name="org.opennms.netmgt.poller.SnmpMonitor"/> <monitor service="Oracle" class-name="org.opennms.netmgt.poller.TcpMonitor"/> <monitor service="Postgres" class-name="org.opennms.netmgt.poller.TcpMonitor"/> <monitor service="MySQL" class-name="org.opennms.netmgt.poller.TcpMonitor"/> <monitor service="Sybase" class-name="org.opennms.netmgt.poller.TcpMonitor"/> <monitor service="Informix" class-name="org.opennms.netmgt.poller.TcpMonitor"/> <monitor service="SQLServer" class-name="org.opennms.netmgt.poller.TcpMonitor"/> <monitor service="SSH" class-name="org.opennms.netmgt.poller.TcpMonitor"/> <monitor service="IMAP" class-name="org.opennms.netmgt.poller.ImapMonitor"/> <monitor service="POP3" class-name="org.opennms.netmgt.poller.Pop3Monitor"/>
You should not need to modify this section unless you manually add your own pollers.
SNMP Data Collection Configuration How-To
http://www.opennms.org/wiki/Data_Collection_Configuration_How-To
• OpenNMS focuses on the services networkresources provide: web pages, databaseaccess, DNS, DHCP, etc. (although informationon network elements is also available).
• There are two major ways that OpenNMSgathers data about the network.
– The first is through polling. Processescalled monitors connect to a network resourceand perform a simple test to see if the resource isresponding correctly. If not, events are generated.
– The second is through data collectionusing collectors.
Currently data can be collected by :
• SNMP,
• NSClient (the Nagios Agent),
• JMX,
• HTTP
There are several things that have to happen in orderfor this to work. For all data collection methods:
• Provisiond During the scanning process, Provisionddiscovers whether the various collectable servicesexist on the discovered node. More specifically forSNMP collection, Provisiond must be able to accessSNMP information on that interface and to formsome basic mappings, such as IP Address to ifIndex.
• collectd-configuration.xml Just as in the poller-configuration.xml file, interfaces are mapped topackages for collection in this file. If data collection isrequired on an interface, it needs to exist in apackage in this file. The default configuration issuitable for most initial purposes.
For SNMP data collection, the following files mustbe configured correctly:
• snmp-config.xml For each interface, a validcommunity string must exist in this file.
• datacollection-config.xml Each package inthe collectd configuration file points to an snmp-collection definition in this file. Each snmp-collection defines what information to collect viaSNMP. The default configuration is fairly completefor basic purposes, and will probably not requiremuch changing initially.
snmp-config.xml
• The parameters used to connect with SNMPagents are defined in the snmp-config.xml file.Here is an example:
<snmp-config retry="3" timeout="800" read-community="public" write-community="private"> <definition version="v2c"> <specific>192.168.0.5</specific> </definition> <definition retry="4" timeout="2000">
<range begin="192.168.1.1" end="192.168.1.254"/> <range begin="192.168.3.1" end="192.168.3.254"/>
</definition> <definition read-community="bubba" write-community="zeke">
<range begin="192.168.2.1" end="192.168.2.254"/> </definition> <definition port=“1161">
<specific>192.168.5.50</specific> </definition>
</snmp-config>
The common attributes for the snmp-config tagare as follows:
• retry
– The number of attempts that will be made to connect to the SNMP agent. Default is 1
• timeout
– The amount of time, in milliseconds, that OpenNMS will wait for a response from the agent. Default is 3000
• read-community
– The default "read" community string for SNMP queries. If not specified, defaults to "public"
• write-community
– The default "write" community string forSNMP queries. Note that this is for futuredevelopment - OpenNMS does not performSNMP "sets" at the moment.
• port
– This overrides the default port of 161.
• version
–Here you can force either SNMP version 1by specifying "v1", version 2c with "v2c", orversion 3 with "v3". Default is "v1"
For SNMPv3 authentication and collection(only available when using SNMP4J):
• security-name- A security name for SNMP v3 authentication
• auth-passphrase- The passphrase to use for SNMP v3
authentication
• auth-protocol- The authentication protocol for SNMP v3.
Either "MD5" or "SHA". Default is MD5
• privacy-passphrase- A privacy pass phrase used to encrypt the
contents of SNMP v3 packets
• privacy-protocol
– The privacy protocol used to encrypt thecontents of SNMP v3 packets. Either "DES","AES","AES192" or "AES256". Default is DES.
• engine-id
– The engine id of the target agentcontext-name The name of the context to obtaindata from on the target agent.
• context-engine-id
– The context engine id of the target entity onthe agent.
• enterprise-id
– An enterprise id for SNMP v3 collection
More rarely used attributes in the snmp-config tag are:
• proxy-host
– A proxy host to use to communicate with thespecified agent(s)
• max-vars-per-pdu
– Number of variables per SNMP request. Default is10
• max-request-size
– If using SNMP4J as the SNMP library, themaximum size of outgoing SNMP requests.Defaults to 65535, must be at least 484
• As explained in the Discovery How-To, thecapabilities check process starts with anewSuspect event (generated either manually orthrough the discovery process). This NewSuspectevent is received by the provisioning daemon(Provisiond).
• The Provisiond process is responsible for scanningIP addresses for particular services. Each servicethat can be detected on a discovered node isdefined in the default foreign-source definition.Upon receipt of a newSuspect event, Provisiondbegins to test each configured service detector tosee if it exists on that device.
• When testing SNMP, Provisiond makes an attempt to receive the System Object ID (systemOID) for the device using the community string and port defined in snmp-config.xml.
• If the sysObjectID is successfully retrieved, Provisiond gathers additional SNMP attributes from the system group, the ipAddressTable (if present), ipAddrTable (if ipAddressTable is not present), ifTable, and ifXTable.
• If the ipAddressTable (or ipAddrTable) or ifTable are unavailable, the scan aborts (but the SNMP system data may show up on the node page
• Second, all of the sub-target IP addresses in the ipAddressTable or ipAddrTable have all the configured service detectors run against them.
• Third, every IP address in the ipAddressTableor ipAddrTable that supports SNMP is tested to see if it maps to a valid ifIndex in the ifTable. Each one that does is marked as a secondary SNMP interface and is a contender for becoming the primary SNMP interface.
• Finally, all secondary SNMP interfaces aretested to see if they match a valid package inthe collectd-configuration file. If more thanone valid IP address meets all three criteria(supports SNMP, has a valid ifIndex and isincluded in a collection package), then thelowest-numbered IP address is marked asprimary. All SNMP data collection isperformed via the primary SNMP interface.
• When the Provisiond node scan and servicedetectors are completed, events aregenerated, including nodeGainedServiceevents.
collectd-configuration.xml
• Data collection is handled via the collectdprocess. collectd listens forNodeGainedService events for the SNMP"service". When this happens, it checks to seeif the primary SNMP interface for that nodeexists in a collection package (which it shouldby definition). If so, the SNMP collector isinstantiated for that IP address.
• Let's look at the collectd-configuration.xml file:
<collectd-configuration threads="5"> <package name="example1">
<filter>IPADDR IPLIKE *.*.*.*</filter> <specific>0.0.0.0</specific> <include-range begin="192.168.0.1" end="192.168.0.254"/> <include-url>file:/opt/OpenNMS/etc/include</include-url> <service name="SNMP" interval="300000" user-defined="false“status="on">
<parameter key="collection" value="default"/> <parameter key="port" value="161"/> <parameter key="retry" value="3"/><parameter key="timeout" value="3000"/>
</service> <outage-calendar>zzz from poll-outages.xml zzz</outage-calendar>
</package> <collector service="SNMP" class-name="org.opennms.netmgt.collectd.SnmpCollector"/>
</collectd-configuration>
• The threads attribute limits the number ofthreads that will be used by the datacollection process. You can increase ordecrease this value based upon your networkand the size of your server.
• Just like pollers have poller packages,collectors have collection packages. Eachpackage determines how often the device willbe polled for SNMP data, and through thecollection key, what will be polled and how itwill be stored. The example1 package is thedefault included out of the box.
Event Configuration How-To
http://www.opennms.org/wiki/Event_Configuration_How-To
OpenNMS has three main functional areas:
• Determining Availability of NetworkServices - discovery
• Gathering Performance Data – collectd
• Event Management and Notifications –trapd, pollerd
“Monitor Availability of tcp port 80 in OpenNMS”
1. capsd-configuration.xml
• Detects the service
2. pollerd-configuration.xml
• Monitor the service and define a downtime policy
capsd-configuration.xml
• <protocol-plugin protocol = “HTTP-80”class-name =“org.opennms.net.mgt.plugins.TcpPlugin” scan=“on”><property key=“port” value=“80”/><property key=“banner” value=“*”/><property key=“timeout” value=“1000”/><property key=“retry” value=“1”/></protocol-plugin>
### Older version OpenNMS
pollerd-configuration.xml• <package name=“http-tcp”>
<filter>!(IPADDR IPLIKE *.*.*.252-254/filter><rrd step=“300”> polling freq 5 min
<rra>RRA:Average:0.5:1:2016 </rra> 1.<rra>RRA:Average:0.5:12:1488 </rra> 2.<rra>RRA:Average:0.5:288:366</rra> 3.<rra>RRA:Max:0.5:288:366 </rra> 4.<rra>RRA:Min:0.5:288:366 </rra> 5.
</rrd>
### Older version OpenNMS
By default is “example”
RRA:Cf:xff:steps:rows
RRD Configuration
http://www.opennms.org/wiki/Data_Collection_Configuration_How-To#RRD_Configuration
Cf: consolidation function = AVERAGE, MAX, MIN, or LAST.
xff: This factor determines how many of the samples can be UNKNOWN for theconsolidated sample is considered UNKNOWN. By default this is set to 0.5 or 50%.
steps: if the step size is 300 seconds (5 minutes) and the number of steps is 12,then the RRA is 12 x 5 minutes = 60 minutes = 1 hour long, and it will stored theconsolidated value for that hour.
rows: The rows field determine the number of values that will be stored inthe RRA.
RRD Configuration
• RRA:AVERAGE:0.5:1:8928
– AVERAGE value collected over 1 step and store up to8928.
– If, for any step, more than 50% of the values areUNKNOWN, then the average value will beUNKNOWN.
– Default step size is 300 seconds (5 min, same withdefault poller and collectd configuration), So there willbe one value per step (AVERAGE=MAX=MIN=LAST).
– 5 minutes/sample12 sample/hour288 samples/24hours. 31 Days x 288 samples = 8928
http://www.opennms.org/wiki/Data_Collection_Configuration_How-To#RRD_Configuration
