Upload
puppet-labs
View
64
Download
2
Tags:
Embed Size (px)
Citation preview
Puppet at thePuppet at thecenter of everythingcenter of everything
http://www.rankpop.com/you-need-to-start-structuring-your-blog-posts-asap/
Server Density Architecture
100+ servers – Ubuntu 12.04
50% Virtual
Nginx, Python, MongoDB
25TB data per month
Puppet Use Cases
Code deploys
Failover
Configuration
System Updates
http://www.arcomem.eu/wp-content/uploads/2011/03/usecases.png
http://www.flickr.com/photos/conskeptical/
Puppet Use CasesConfiguration
nginx::config::worker_rlimit_nofile: 30000nginx::config::worker_connections: 10240nginx::config::worker_processes: 4
storm::config_file: '/etc/storm/storm.yaml'storm::drpc::manage_service: falsestorm::drpc::enable: false
Forge modules - don't reinvent the wheel:https://forge.puppetlabs.com/jfryman/nginx
https://forge.puppetlabs.com/deric/storm
http://www.flickr.com/photos/conskeptical/
Puppet Use CasesConfiguration
include nginx
include storm
Puppet Use CasesCode deploys
Deploy
Puppet::Type.type(:package).provide(:honshuu, :parent => Puppet::Provider::Package) do desc "Server Density deployinator"
package { 'honshuualertsprocessing': ensure => present, provider => honshuu, require => File['/var/www/'], notify => Service['celerydalerts','apache2'],}
nginx 'on-the-fly' update
class serverdensitynginx{ class { 'nginx': } nginx::resource::upstream { 'socky_rack': ensure => present, members => split( $lbTargetHostsSocky, ',' ), }(...)
Puppet Use CasesSystem updates
Run
CANARIES=" \ hstage1.honshuu.dev \ hstage2.honshuu.dev \ hstagelb1.honshuu.dev \ exma1.sng.amz \ ma1.dal.sl \ sdcomweb1.wdc.sl \ queuesrab2.wdc.sl \"
Puppet Use CasesSystem updates
Run
for i in $CANARIES do mco rpc puppetral I $i create type=exec title="/bin/bash c 'aptget distupgrade y'"done
Reducing the code baseMongoDB
class mongoa{(…)191 lines of code+Templates for config files and init scripts
class mongoc{(…)179 lines of code+Template for config file
Reducing the code baseMongoDB
class mongod{(…)257 lines of code+Templates for config files, init scripts, pam and limits
class mongos{(…)180 lines of code+Template for config file and init script
Reducing the code baseMongoDB
Forge module + Hierahttps://forge.puppetlabs.com/dwerder/mongodb
class site::mongomodule { class { '::mongodb::globals': manage_package_repo => true, version => '2.4.12' } > class { '::mongodb::server': verbose => true }}
Extending Server Density
Server Density collects server data from an open source agent deployed on each device
The agent reads the device ID from a config file:
[Main]sd_url: http://mydomain.serverdensity.comagent_key: 8160b409de9d8a612ec8d964c5b26bf(...)
Copy the device ID (agent key)
Create a device on Server Density UI
Server Density v1:
[Main]sd_url: http://boxedice.serverdensity.comagent_key: <%= sdAgentKey %>(...)
Extending Server Density
'sdagent': ensure => installed,
file { 'config.cfg': path => '/etc/sdagent/config.cfg', ensure => file, require => Package['sdagent'], content => template('sdagent/config.erb'),}
Server Density v2 - implementation:
Manages the life cycle of Rackspace and Amazon cloud instances
A custom fact will return the cloud instance agent key
Extending Server Density
or use Server Density API to create a device on the fly and return the resulting agent key
https://github.com/serverdensity/puppet-serverdensity
Custom fact:
Facter.add(:agent_key, :timeout => 10) do if File::exist?('/etc/sdagentkey') result = Facter::Util::Resolution.exec("cat /etc/sdagentkey") elsif Facter.value('ec2_instance_id') uri = URI("http://ec2meta.serverdensity.com/latest/userdata") req = Net::HTTP::Get.new(uri.request_uri) res = Net::HTTP.start(uri.host, uri.port) {|http| http.request(req) } result = res.body.split(':').last if res.code == 200 end
setcode { result }end
Extending Server Density
Custom function:
(...) agent_key = lookupvar("agent_key")
# lookupvar returns undef if no value # test against nil just in case unless agent_key.nil? or agent_key == :undef notice ["Agent Key Provided via Facter: #{ agent_key }"] return agent_key End(...)
Extending Server Density
Custom function:
(...)base_url = "https://api.serverdensity.io"
filter = { 'type' => 'device', 'hostname' => hostname,}
filter_json = URI.escape(PSON.dump(filter))
uri = URI("#{base_url}/inventory/devices?filter=#{filter_json}&token=#{token}")req = Net::HTTP::Get.new(uri.request_uri)https = Net::HTTP.new(uri.host, uri.port)https.use_ssl = trueres = https.start { |cx| cx.request(req) }(...)
Extending Server Density
Custom function:
(...)device = PSON.parse(res.body)
if device['status'] == 2notice ["Device not found, creating a new one"]
uri = URI("#{ base_url }devices/add?account=#{ sd_url }")req = Net::HTTP::Post.new(uri.request_uri)req.basic_auth sd_username, sd_password
params = { 'name' => server_name, 'hostName' => hostname, 'notes' => 'Created automatically by puppetserverdensity',}(...)
Extending Server Density
Server Density v2:
Next: Publishing to the Forge
Extending Server Density
https://forge.puppetlabs.com/serverdensity/serverdensity_agent