Puppet Data Mining

Puppet Data MiningReport processors, stored configs and more

PuppetCamp 22nd March 2012

gareth rushgrove | morethanseven.net http://www.flickr.com/photos/map408/2412123378

http://www.flickr.com/photos/map408/2412123378

http://www.flickr.com/photos/map408/2412123378

Gareth Rushgrove @garethr

gareth rushgrove | morethanseven.net

Blog at morethanseven.net


Curate devopsweekly.com


Work at UK Government Digital Service

Text


Serious Government Business


http://www.flickr.com/photos/iancarroll/5027441664gareth rushgrove | morethanseven.net

Puppet data

- Why do we want it?- How do we get our hands on it- What can we build with it

http://www.flickr.com/photos/iancarroll/5027441664



Puppet data





Puppet data




Why


Single source of truth

http://www.flickr.com/photos/joeshlabotnik/4132307070/

http://www.flickr.com/photos/joeshlabotnik/4132307070/sizes/l/in/photostream/

http://www.flickr.com/photos/joeshlabotnik/4132307070/sizes/l/in/photostream/


Example: find a list of hostnames

- Spreadsheets- Wiki- Deployment scripts- Application code- SSH configs- More deployment scripts- Monitoring configuration

http://www.flickr.com/photos/nettsu/5276327339gareth rushgrove | morethanseven.net

Operational reports

http://www.flickr.com/photos/nettsu/5276327339



Example: hardware spreadsheet

Company hardware


Hooks for tooling

http://www.flickr.com/photos/nickstone333/3135318558




Example: Nagios integration


Example: Rundeck integration

How


Where to find the data

- Report processors- Stored configurations- Puppet internal APIs

http://www.flickr.com/photos/nathaninsandiego/3757033518gareth rushgrove | morethanseven.net

Report processors

http://www.flickr.com/photos/nathaninsandiego/3757033518

http://www.flickr.com/photos/nathaninsandiego/3757033518

[agent]report = truepluginsync = true [master]reports = storepluginsync = true


Configuration

[agent]report = truepluginsync = true [master]reports = store,http,log,samplepluginsync = true


Adding reports


Built in


Store

Store the YAML report on disk. Each host sends its report as a YAML dump and this just stores the file on disk, in the reportdir directory.


Log

Send all received logs to the local log destinations. Usually the log destination is syslog.


HTTP

Send report information via HTTP to the reporturl. Each host sends its report as a YAML dump and this sends this YAML to a client via HTTP POST.


Tagmail

This report sends specific log messages to specific email addresses based on the tags in the log messages.


RRDgraph

Graph all available data about hosts using the RRD library.


Simple report structure

.└── sample-report ├── lib │ └── puppet │ └── reports │ └── sample.rb └── manifests └── init.pp


Simple report processor

require 'puppet'require 'yaml'require 'logger'

LOG = Logger.new('/tmp/puppet.log')

Puppet::Reports.register_report(:sample) do def process message = "Puppet run for #{self.host} #{self.status}" Puppet.debug "Hello from sample report processor" LOG.info message endend


Register report





Do something with the data in self





Open source examples





IRC/Campfire


IRC/Campfire


Ganglia


Graphs!


Zendesk


James Turnbull will have got there first


Stored configuration database


Configuration

[master]storeconfigs = truedbadapter = mysqldbuser = puppetdbpassword = passworddbserver = localhostdbsocket = /var/run/mysql.sock


SQL




mysql> SHOW TABLES;+------------------+| Tables_in_puppet |+------------------+| fact_names || fact_values || hosts || inventory_facts || inventory_nodes || param_names || param_values || puppet_tags || resource_tags || resources || source_files |+------------------+11 rows in set (0.00 sec)


Lots of tables to explore

mysql> SELECT name,ip,environment,last_compile FROM hosts;+--------------------+-----------------+-------------+---------------------+| name | ip | environment | last_compile |+--------------------+-----------------+-------------+---------------------+| ubuntu.localdomain | 192.168.157.129 | production | 2012-03-04 21:19:41 |+--------------------+-----------------+-------------+---------------------+1 row in set (0.00 sec)


Easily get a list of hosts

mysql> SELECT filename,updated_at FROM source_files;+-----------------------------------------------------+---------------------+| filename | updated_at |+-----------------------------------------------------+---------------------+| NULL | 2011-10-11 10:34:36 || /etc/puppet/modules/users/manifests/init.pp | 2011-10-12 14:13:49 || /etc/puppet/modules/nagios/manifests/init.pp | 2011-10-12 14:13:49 || /etc/puppet/modules/ntp/manifests/init.pp | 2011-10-12 14:13:50 || /etc/puppet/modules/base_packages/manifests/init.pp | 2011-10-12 14:13:50 || /etc/puppet/modules/sudo/manifests/init.pp | 2011-10-12 14:13:50 || /etc/puppet/modules/apt/manifests/init.pp | 2011-10-12 14:13:50 || /etc/puppet/modules/logrotate/manifests/init.pp | 2011-10-12 14:13:51 || /etc/puppet/modules/ganglia/manifests/init.pp | 2011-10-12 14:13:52 || /etc/puppet/modules/motd/manifests/init.pp | 2011-10-12 14:13:57 || /etc/puppet/manifests/classes.pp | 2011-10-12 14:17:25 || /etc/puppet/modules/mysql/manifests/init.pp | 2011-10-12 14:17:25 || /etc/puppet/modules/apache2/manifests/init.pp | 2011-10-12 14:17:25 || /etc/puppet/modules/passenger/manifests/init.pp | 2011-10-12 14:17:26 |+-----------------------------------------------------+---------------------+


Other interesting data too


Active Record

http://www.flickr.com/photos/simplebitsdan/480696708



#!/usr/bin/env ruby

require 'puppet/rails'

Puppet[:config] = '/etc/puppet/puppet.conf'Puppet.parse_configconfig = Puppet.settings.instance_variable_get(:@values)master_config = config[:master]


Load Puppet master configuration

adapter = master_conf[:dbadapter]args = {:adapter => adapter, :log_level => master_conf[:rails_loglevel]}

args[:host] = master_conf[:dbserver]args[:username] = master_conf[:dbuser]args[:password] = master_conf[:dbpassword]args[:database] = master_conf[:dbname]args[:database] = "puppet" unless not args[:database].to_s.empty?args[:port] = master_conf[:dbport]socket = master_conf[:dbsocket]args[:socket] = socket unless socket.to_s.empty?


Map database connection settings

ActiveRecord::Base.establish_connection(args)

Puppet::Rails::Host.all.each { |h| puts "#{h.name}"}


Make an active record query


Example run

⚡ ./list.rbubuntu.localdomain


Code spelunking

#!/usr/bin/env ruby

require 'puppet'Puppet[:config] = "/etc/puppet/puppet.conf"Puppet.parse_configPuppet[:clientyamldir] = Puppet[:yamldir]Puppet::Node.indirection.terminus_class = :yaml


Load Puppet master configuration

nodes = Puppet::Node.indirection.search("*")nodes.each do |n| facts = Puppet::Node::Facts.indirection.find(n.name) tags = Puppet::Resource::Catalog.indirection.find(n.name).tags puts "#{n.name} - #{tags.join(', ')}"end


Make a query using the internal API


Needs root permissions

⚡ sudo ./puppettags.rbubuntu.localdomain - settings, default, node

What


Command line tools

⚡ ./puppetlast.rb+-----------+-----------------+| hostname | last puppet run |+-----------+-----------------+| hostname4 | 100 minutes || hostname1 | 25 minutes || hostname5 | 10 minutes || hostname2 | 15 minutes || hostname3 | 5 minutes |+-----------+-----------------+


Dashboards


web-puppet


JSON over HTTP


capistrano-puppet

require 'capistrano-puppet'

web_puppet = CapistranoPuppet::Server.new( 'http://username:password@localhost:9295')

role :web doweb_puppet.get_servers('webserver')

end


Get hosts from puppet

http://username:password@localhost:9295

http://username:password@localhost:9295


Visualisation


Your imagination

?

Links

- web-puppet - https://github.com/garethr/web-puppet- capistrano-puppet - https://github.com/garethr/capistrano-puppet- puppet-ganglia - https://github.com/jamtur01/puppet-ganglia- puppet-zendesk - https://github.com/jamtur01/puppet-zendesk- puppet-irc - https://github.com/jamtur01/puppet-irc- puppet-campfire - https://github.com/jamtur01/puppet-campfire


https://github.com/garethr/web-puppet






https://github.com/jamtur01/puppet-zendesk

https://github.com/jamtur01/puppet-zendesk

https://github.com/jamtur01/puppet-irc




http://www.flickr.com/photos/benterrett/6852348725/

One more thing




Questions?

gareth rushgrove | morethanseven.net http://flickr.com/photos/psd/102332391/

http://flickr.com/photos/psd/102332391/

http://flickr.com/photos/psd/102332391/

Technology

Puppet Data Mining