18
The Art and Zen of Managing Nagios with Puppet Michael Merideth VictorOps

Puppet Camp Denver 2015: Nagios Management With Puppet

Embed Size (px)

Citation preview

The  Art  and  Zen  of  Managing  Nagios  with  PuppetMichael  Merideth  -­‐  VictorOps

Why  Manage  Nagios  with  Puppet?

• Mind  the  gaps  

• Keep  the  config  readable  

• Good  intro  to  key  Puppet  features

Key  Puppet  Features

• Exported  Resources  

• Hiera  

• Inline  Templates  

• Facter  

• Nagios  Resource  Types  

• Puppet  Forge

Key  Puppet  Features

Exported  Resources:  

• Resources  get  defined  on  one  

host,  and  “collected”  on  another  

• Requires  the  use  of  PuppetDB  

• Saves  having  to  use  shared  

storage  or  insecure  file  transfers  

• Keeps  the  number  of  config  files  

low

   @@nagios_host  {  $::fqdn:          alias              =>  $nag_alias,          address          =>  $::ipaddress_eth1          use                  =>  "${::vo_env}-­‐host",          hostgroups    =>  $::my_hostgroups,          tag                  =>  "${::vo_env}-­‐host",          target            =>  "${cfg_tmpdir}/hosts.cfg",          owner              =>  'nagios',          group              =>  'nagios',          mode                =>  '0644',      }  

Key  Puppet  Features

Hiera:  

• Separate  code  from  data  

• Set  defaults  and  provide  

overrides  

• Encrypted  back-­‐ends  allow  for  

security,  even  if  your  source  

code  is  stolen

Key  Puppet  Features

Inline  Templates:  

• Not  just  for  file  content  

• Allows  the  use  of  native  Ruby  code  within  

Puppet  policy

inline_template(  '<%=  Hash[scope.to_hash.select{  |k,v|  k  =~  /nag_gr_/  &&  v  }].keys.join(",")  %>'  )

Key  Puppet  Features

Facter:  

• Variable  values  defined  at  run-­‐time,  on  the  client  

• Easily  extensible

Key  Puppet  Features

Nagios  Resource  Types:  

• Makes  creating  Nagios  configs  

simple  

• Enforces  correct  syntax  

• Not  suitable  for  every  config  in  

every  file

Key  Puppet  Features

Puppet  Forge:  • A  public  repository  of  Puppet  modules  

• Modules  can  be  libraries,  defined  resource  types,  or  classes  

• Quality  Score  and  Community  Rating  help  you  choose  safe,  well-­‐maintained  modules  

• You  should  still  review  third-­‐party  modules  carefully

Why  Not  Use  an  Existing  Module?

There  are  several  user-­‐submitted  Nagios  modules  in  the  Forge,  but  I  wanted  one  that  would:  

• Automatically  add  new  hosts  

• Provision  some  services  within  other  modules  

• Automatically  remove  decommissioned  hosts  

• Make  use  of  host  and  service  groups

Putting  It  All  Together

Hiera  and  Facter  define  the  variables  

• Sane  defaults  for  most  values  

• Environment-­‐specific  overrides  (dev,  stage,  prod)  

• Site-­‐specific  overrides  for  different  datacenters

Putting  It  All  Together

Monitored  hosts  build  their  own  config  

• Hostgroup  membership  is  determined  in  the  client  context  

• Other  facts  integrate  into  the  host  definition  

• NRPE  config  built  from  a  common  template  

• Client  puppet  run  exports  resources

Putting  It  All  Together

Some  service  definitions  get  embedded  in  the  manifest  for  that  service  

• A  change  to  a  service  may  mean  a  change  to  how  

you  monitor  it  

• Manage  that  all  in  one  place  

• profile::vo_webserver  provides  an  example

Putting  It  All  Together

Some  service  definitions  are  done  with  templates  

• Not  everything  needs  to  get  built  dynamically  

• Some  services  are  monitored  on  all  hosts  

• Less  dynamic  config  means  shorter  catalog  compilation  

• Templates  can  be  easier  to  debug

Ugly  Hack  Alert

Dynamic  files  built  every  run,  installed  if  there’s  a  diff  

• Allows  keeping  backup  directories  with  previous  versions  of  the  

configs  

• Allows  automatic  removal  of  decommissioned  hosts  

• Prevents  excessive  Nagios  restarts

Vagrant  • AWESOME  way  to  test  Puppet  code  as  

you  develop  

Github  repository  • Contains  Vagrantfile  and  complete  

Puppet  policy  

• github.com/victorops/puppet-­‐nagios

The  Demo  Environment  

Demo  Time!Running low on slides, so it must be…

The  Demo  Environment  

github.com/victorops/puppet-­‐nagios  • Check  it  out!  

• Contribute!  

• Or  fork  it!  

• Help  work  towards  a  Puppet  Forge  module:  

• Parameterize  the  class  

• More  types  of  config  resources  

• Cross  platform  support  

• Documentation