(RMLL 2010) Nagios® and large environments monitoring2010.rmll.info/IMG/pdf/RMLL2010-AdminSys-SupervisionGrandsEnviro… · R and large environments monitoring Gab es Jean RMLL 2010,

The good part The less good part The solution

(RMLL 2010) Nagios R© and largeenvironments monitoring

Gabes Jean

RMLL 2010, Bordeaux, July 7

1 / 49(RMLL 2010) Nagios R© and large environments monitoring

N


<SolutionLinux> Nagios R© is the industry standard in systemmonitoring </SolutionLinux>


N


In real life

Nagios R© is a daemon that schedule checks and reacts to theresults like send an email or try to correct the problem.(And that’s all!)


N


Simple Nagios R© daemon


N


Green IT slide


N


The good part

Size problems

For 500 servers and 7000 services ⇒ 60K lines

”A good computer scientist is lazy”

sysadmins are (very) good computer scientists

need to factorize the configuration


N


The good part

Size problems






N


The good part

Size problems






N


The good part

Simple inheritance

Got templates that hosts and services can inherit from.


N


The good part

Simple inheritance example

Template

name web_tpl

register 0

check_command check_http

contact_groups AdmWeb

Defined service

use web_tpl

service_description Http

host_name web_1

Service in memory

service_description Http

host_name web_1

check_command check_http

contact_groups AdmWeb


N


The good part

Not enough

What about crossed criterions?

GNU/Linux - Windows ⇒ check commandfile server - web server ⇒ contactsprod - qualif ⇒ notification period

define 23 = 8 templates ?

change 1 parameter ⇒ 4 modifications !


N


The good part

Multiple inheritance

An element can inheritated from numerous templates

First find value is taken


N


The good part

Multiple inheritance

Defined host

define host{

host_name web_1

address 209.85.227.104

use Linux,Web,Prod

}

Host in memory

define host {

host_name web_1

address 209.85.227.104

check_command check_tcp!22

contacts AdminWeb1

notification_period 24x7

}


N


The good part

OGM inside (implicit inheritance)

Why re-defined a service property already set in the host?

Void service parameter will be filled with host value

Less service duplications

Available for notification part only :(


N


The good part

OGM inside (implicit inheritance)

Why re-defined a service property already set in the host?

Void service parameter will be filled with host value

Less service duplications

Available for notification part only :(


N


The good part

Implicit inheritance

Hosts defined

define host{

host_name web_1

contacts AdmWeb

}

define host{

host_name file_1

contacts AdmFile

}

Defined service

define service{

host_name web_1,file_1


#contacts none

}


N


The good part

Implicit inheritance

Services in memory

define service{

host_name web_1


contacts AdmWeb

}

define service{

host_name file_1


contacts AdmFile

}


N


The good part

Service on host groups

Apply services on host groups instead on hosts one by one

Less errors

Far less configuration lines!


N


The good part

Number of lines at the end

All theses inheritances can be used togethers

From 60K lines to 7K

Lazy admin is happy :)


N


The less good part

Distributed environnements

When one Nagios R© is not enough (more than 10k checks in 5minutes)

Add more daemons :)

But some difficulities about configuration and data management

More hacks than real solutions


N


The less good part

Distributed environnements

When one Nagios R© is not enough (more than 10k checks in 5minutes)

Add more daemons :)

But some difficulities about configuration and data management

More hacks than real solutions


N


The less good part

Manual management


N


The less good part

DNX : distributed checks


N


The less good part

NDO : all data in database

Warning : NDO is no more maintained


N


The less good part

Centreon : more easy to manage


N


The less good part

High Availability

Murphy’s law is the only stable thing in the world...

The monitoring solution must be as available as the best monitoredelement


N


The less good part

Active/active


N


The less good part

Active/passive


N


The less good part

And with distributed...

Manual management : even harder :)

DNX : do not change a lot

NDO : Quite hard (must drop connexions)

Centreon : not managed (from now)


N


The less good part

A not so happy world

Technical difficulties to manage huge environnements

Code hard to hack

Serious concurrents (Zenoss, Zabbix,...)

Some problems with the Nagios R© copyright and the community

Project from Open Source to Open Core ?

Already fork (Icinga), but no real changes


N


The less good part



Code hard to hack






N


The less good part



Code hard to hack






N


The less good part



Code hard to hack






N


The solution

Easy as in marketing cloud slides


N


The solution

How does it work?

No more monolitic : Unix Way!

Arbiter : read, cut and dispatch configuration

Schedulers : schedule.. and that’s all!

Pollers : get checks from schedulers and launch thems

Reactionner : launch notification and event handlers

Broker : grab all data and export thems


N


The solution

How does it work?








N


The solution

How does it work?








N


The solution

How does it work?








N


The solution

How does it work?








N


The solution

How does it work?








N


The solution

A little illustration?


N


The solution



N


The solution



N


The solution



N


The solution



N


The solution

What if a node die?


N


The solution

What if a node die?


N


The solution

Example of mixed architecture

GNU/Linux vs Windows


N


The solution

And for DMZ

LAN vs DMZ


N


The solution

Multisites? Tempest in the clouds!

Asia poller take a check for a US host from an Eu scheduler...

Poller of a customer A take job of a scheduler of the customer B

Multisites? Ok if still one admin/data place

Realms


N


The solution





Realms


N


The solution





Realms


N


The solution

What are Realms?

See it like a ressource pool where to put host or host groups

Way to cut by organizations

Can have a hierarchy

Still one Arbiter


N


The solution

What are Realms?




Still one Arbiter


N


The solution

What are Realms?




Still one Arbiter


N


The solution

What are Realms?




Still one Arbiter


N


The solution

Distrinct realms


N


The solution

Global realm


N


The solution

Imagine...


N


The solution

One answer


N


The solution

Project with a strange history

Start as a proof of concept for Nagios R©

Huge perf! More than 150K checks in 5 minutes

Multiplatform (everywhere Python runs!!)

Too far to stop

THE proposal(s)

THE .. no... (Python was too much?)


N


The solution





Too far to stop

THE proposal(s)



N


The solution





Too far to stop

THE proposal(s)



N


The solution





Too far to stop

THE proposal(s)



N


The solution





Too far to stop

THE proposal(s)



N


The solution





Too far to stop

THE proposal(s)



N


The solution

Module export?

NDO : MySQL/Oracle for Centreon

Merlin : for Ninja

Status.dat : if you still use the old CGI

LiveStatus : for Thruk

Log : all logs in one file

CouchDB, sqlite : for... nothing in fact :)


N


The solution

How to consider it?

Not a fork

New monitoring tool compatible with Nagios R© configuration andplugins


N


The solution

How to test it?

(easy) install from source

(lazy) get the demo VM (with Ninja and Thrunk interface)

http://www.shinken-monitoring.org

Thanks to Monitoring-fr.org for the VM hosting :)


N


The solution

How to test it?






N


The solution

How to test it?






N


The solution

Monitoring-fr, was Nagios-fr

Remember the R© ?

Hint : do not joke with Nagios R© trademark

New start for the community (OpenNMS, Zabbix,...)

Come see our demos :)


N


The solution


Remember the R© ?





N


The solution


Remember the R© ?





N


The solution


Remember the R© ?





N


The solution

Questions

(Big thanks to my wife for the diagrams)

Questions?

Examples :

Why Python?Why the Affero GPL license?Why this name Shinken?Vi or Emacs?


N

Documents

(RMLL 2010) Nagios® and large environments monitoring2010.rmll.info/IMG/pdf/RMLL2010-AdminSys-SupervisionGrandsEnviro… · R and large environments monitoring Gab es Jean RMLL 2010,