Embed Size (px)
DESCRIPTION
Chef is awesome, but it’s also very easy to go overboard. In terms of testing and maintainability, sometimes its better to refactor your long recipe into an LWRP. As your infrastructure evolves, so should you cookbooks. But at some point your bound to have a cookbook 500+ lines of antiquated logic. How do you refactor such a large chunk of code that is critical to your infrastructure? How much logic should me moved into other cookbooks? How much logic should be extracted into LWRPs? How much logic should be moved out of Chef, into Ruby, and packaged as a gem?
Citation preview
Cookbook Refactoring
A
Cookbook Refactoring
... and extracting logic into Rubygems
A
We're Hiring!
We're Hiring!
Colorado
New Branding
We're Hiring!
UDO YOU SOMETIMES
FEEL LIKE
THIS
template '/etc/hosts' do owner 'root' group 'root' source 'etc/hosts'end
recipes/default.rb
# This file is managed by Chef for "<%= node['fqdn'] %>"# Do NOT modify this file by hand.
<%= node['ipaddress'] %> <%= node['fqdn'] %>127.0.0.1!localhost <%= node['fqdn'] %>255.255.255.255!broadcasthost::1 localhost fe80::1%lo0! localhost
templates/default/etc/hosts.erb
default['etc']['hosts'] = [] unless node['etc']['hosts']
attributes/default.rb
# This file is managed by Chef for "<%= node['fqdn'] %>"# Do NOT modify this file by hand.
<%= node['ipaddress'] %> <%= node['fqdn'] %>127.0.0.1!localhost <%= node['fqdn'] %>255.255.255.255!broadcasthost::1 localhost fe80::1%lo0! localhost
# Custom Entries<% node['etc']['hosts'].each do |h| -%><%= h['ip'] %> <%= h['host'] %><% end -%>
templates/default/etc/hosts.erb
include_attribute 'hostsfile'
default['etc']['hosts'] << { 'ip' => '1.2.3.4', 'host' => 'www.example.com'}
other_cookbook/attributes/default.rb
node.default['etc']['hosts'] << { 'ip' => '1.2.3.4', 'host' => 'www.example.com'}
other_cookbook/recipes/default.rb
default_attributes({ 'etc' => { 'hosts' => [ {'ip' => '1.2.3.4', 'host' => 'www.example.com'}, {'ip' => '4.5.6.7', 'host' => 'foo.example.com'} ] }})
roles/my_role.rb
{ "default_attributes": { "etc": { "hosts": [ {"ip": "1.2.3.4", "host": "www.example.com"}, {"ip": "4.5.6.7", "host": "foo.example.com"} ] } }}
environments/production.json
node.set['etc']['hosts'] = { ip: '7.8.9.0', host: 'bar.example.com'})
recipes/default.rb
arr = [1,2,3]
arr << 4 => [1,2,3,4]arr = 4 => 4
arr = [1,2,3]
arr << 4 => [1,2,3,4]arr = 4 => 4
Not an Array
TODO: Add infographics
# This file is managed by Chef for "www.myapp.com"# Do NOT modify this file by hand.
1.2.3.4 www.myapp.com127.0.0.1!localhost www.myapp.com255.255.255.255!broadcasthost::1 localhost fe80::1%lo0! localhost
# Custom Entries1.2.3.4 www.example.com4.5.6.7 foo.example.com7.8.9.0 bar.example.com
/etc/hosts
TODO: Add infographics
# This file is managed by Chef for "www.myapp.com"# Do NOT modify this file by hand.
1.2.3.4 www.myapp.com127.0.0.1!localhost www.myapp.com255.255.255.255!broadcasthost::1 localhost fe80::1%lo0! localhost
# Custom Entries7.8.9.0 bar.example.com
/etc/hosts
Post Mortem
<< =
<< =!=
Post Mortem
Action Items
7
Monkey patch Chef to raise an exception when redefining that
particular node attribute.
Monkey patch Chef to raise an exception when redefining that
particular node attribute.t
Create a special cookbook that uses a threshold value and raises an
exception if the size of the array doesn't "make sense".
Create a special cookbook that uses a threshold value and raises an
exception if the size of the array doesn't "make sense".t
Move all entries to a data bag
Move all entries to a data bag
u
Move all entries to a data bag66 Add tests
Data Bags
[ "1.2.3.4 example.com www.example.com", "4.5.6.7 foo.example.com", "7.8.9.0 bar.example.com"]
data_bags/etc_hosts.json
hosts = data_bag('etc_hosts')
template '/etc/hosts' do owner 'root' group 'root' source 'etc/hosts' variables( hosts: hosts )end
recipes/default.rb
# This file is managed by Chef for "<%= node['fqdn'] %>"# Do NOT modify this file by hand.
<%= node['ipaddress'] %> <%= node['fqdn'] %>127.0.0.1!localhost <%= node['fqdn'] %>255.255.255.255!broadcasthost::1 localhost fe80::1%lo0! localhost
# Custom Entries<%= @hosts.join("\n") %>
templates/default/etc/hosts.erb
Move all entries to a data bag56 Add tests
require 'chefspec'
spec/default_spec.rb
require 'chefspec'
describe 'hostsfile::default' do
end
spec/default_spec.rb
require 'chefspec'
describe 'hostsfile::default' do let(:hosts) { ['1.2.3.4 example.com', '4.5.6.7 bar.com'] }
before do Chef::Recipe.any_instance.stub(:data_bag).with('etc_hosts').and_return(hosts) end
end
spec/default_spec.rb
require 'chefspec'
describe 'hostsfile::default' do let(:hosts) { ['1.2.3.4 example.com', '4.5.6.7 bar.com'] }
before do Chef::Recipe.any_instance.stub(:data_bag).with('etc_hosts').and_return(hosts) end
let(:runner) { ChefSpec::ChefRunner.new.converge('hostsfile::default') }
end
spec/default_spec.rb
require 'chefspec'
describe 'hostsfile::default' do let(:hosts) { ['1.2.3.4 example.com', '4.5.6.7 bar.com'] }
before do Chef::Recipe.any_instance.stub(:data_bag).with('etc_hosts').and_return(hosts) end
let(:runner) { ChefSpec::ChefRunner.new.converge('hostsfile::default') }
it 'loads the data bag' do Chef::Recipe.any_instance.should_receive(:data_bag).with('etc_hosts') end
end
spec/default_spec.rb
require 'chefspec'
describe 'hostsfile::default' do let(:hosts) { ['1.2.3.4 example.com', '4.5.6.7 bar.com'] }
before do Chef::Recipe.any_instance.stub(:data_bag).with('etc_hosts').and_return(hosts) end
let(:runner) { ChefSpec::ChefRunner.new.converge('hostsfile::default') }
it 'loads the data bag' do Chef::Recipe.any_instance.should_receive(:data_bag).with('etc_hosts') end
it 'creates the /etc/hosts template' do expect(runner).to create_template('/etc/hosts').with_content(hosts.join("\n")) endend
spec/default_spec.rb
$ rspec cookbooks/hostsfile
Running all specs
$ rspec cookbooks/hostsfile
Running all specs
**
Finished in 0.0003 seconds2 examples, 0 failures
$ rspec cookbooks/hostsfile
Running all specs
**
Finished in 0.0003 seconds2 examples, 0 failures
Really Fucking Fast™
#winning
10,000 tests
28 seconds
#winning
⏳⏳
hosts = data_bag('etc_hosts')
hosts << search(:node, 'role:mongo_master').first.tap do |n| "#{n['ip_address']} #{n['fqdn']}"end
template '/etc/hosts' do owner 'root' group 'root' source 'etc/hosts' variables( hosts: hosts )end
recipes/default.rb
hosts = data_bag('etc_hosts')
hosts << search(:node, 'role:mongo_master').first.tap do |n| "#{n['ip_address']} #{n['fqdn']}"end
hosts << search(:node, 'role:mysql_master').first.tap do |n| "#{n['ip_address']} #{n['fqdn']}"end
hosts << search(:node, 'role:redis_master').first.tap do |n| "#{n['ip_address']} #{n['fqdn']}"end
template '/etc/hosts' do owner 'root' group 'root' source 'etc/hosts' variables( hosts: hosts )end
recipes/default.rb
LWRPs
# List of all actions supported by the provideractions :create, :create_if_missing, :update, :remove
# Make create the default actiondefault_action :create
# Required attributesattribute :ip_address, kind_of: String, name_attribute: true, required: trueattribute :hostname, kind_of: String
# Optional attributesattribute :aliases, kind_of: Arrayattribute :comment, kind_of: String
resources/entry.rb
action :create do ::Chef::Util::FileEdit.search_file_delete_line(entry) ::Chef::Util::FileEdit.insert_line_after_match(/\n/, entry)end
protected
def entry [new_resource.ip_address, new_resource.hostname, new_resource.aliases.join(' ')].compact.join(' ').squeeze(' ') end
providers/entry.rb
hostsfile_entry '1.2.3.4' do hostname 'example.com'end
providers/entry.rb
Chef::Util::FileEdit is slow
Re-writing the file on each run
Provider kept growning
Untested
RefactorA
Move to pure Ruby classes
Ditch Chef::Util::FileEdit and manage the entire file
Only implement Ruby classes in the Provider (logic-less Provider)
Test the Ruby code
Test that the Provider implements the proper Ruby classes
TODO: Add infographics
class Entry attr_accessor :ip_address, :hostname, :aliases, :comment
def initialize(options = {}) if options[:ip_address].nil? || options[:hostname].nil? raise ':ip_address and :hostname are both required options' end
@ip_address = options[:ip_address] @hostname = options[:hostname] @aliases = [options[:aliases]].flatten @comment = options[:comment] end
# ...end
libraries/entry.rb
TODO: Add infographics
class Manipulator def initialize contents = ::File.readlines(hostsfile_path) @entries = contents.collect do |line| Entry.parse(line) unless line.strip.nil? || line.strip.empty? end.compact end
def add(options = {}) @entries << Entry.new( ip_address: options[:ip_address], hostname: options[:hostname], aliases: options[:aliases], comment: options[:comment] ) endend
libraries/manipulator.rb
# Creates a new hosts file entry. If an entry already exists, it# will be overwritten by this one.action :create do hostsfile.add( ip_address: new_resource.ip_address, hostname: new_resource.hostname, aliases: new_resource.aliases, comment: new_resource.comment )
new_resource.updated_by_last_action(true) if hostsfile.saveend
providers/entry.rb
RSpec
TODO: Add infographics
describe Entry do describe '.initialize' do subject { Entry.new(ip_address: '2.3.4.5', hostname: 'www.example.com', aliases: ['foo', 'bar'], comment: 'This is a comment!', priority: 100) }
it 'raises an exception if :ip_address is missing' do expect { Entry.new(hostname: 'www.example.com') }.to raise_error(ArgumentError) end
it 'sets the ip_address' do expect(subject.ip_address).to eq('2.3.4.5') endend
spec/entry_spec.rb
Chef Spec
Chef Spec
TODO: Add infographics
describe 'hostsfile lwrp' do let(:manipulator) { double('manipulator') } before do Manipulator.stub(:new).and_return(manipulator) Manipulator.should_receive(:new).with(kind_of(Chef::Node)) .and_return(manipulator) manipulator.should_receive(:save!) end
let(:chef_run) { ChefSpec::ChefRunner.new( cookbook_path: $cookbook_paths, step_into: ['hostsfile_entry'] ) }
spec/default_spec.rb
TODO: Add infographics
context 'actions' do describe ':create' do it 'adds the entry' do manipulator.should_receive(:add).with({ ip_address: '2.3.4.5', hostname: 'www.example.com', aliases: nil, comment: nil, priority: nil })
chef_run.converge('fake::create') end end endend
Open It
Gem It
$ bundle gem hostsfile
$ bundle gem hostsfile create hostsfile/Gemfile create hostsfile/Rakefile create hostsfile/LICENSE.txt create hostsfile/README.md create hostsfile/.gitignore create hostsfile/hostsfile.gemspec create hostsfile/lib/hostsfile.rb create hostsfile/lib/hostsfile/version.rbInitializating git repo in ~Development/hostsfile
entry.rb
manipulator.rb
99
9
9?
chef_gem 'hostsfile'
recipes/default.rb
require 'hostsfile'
providers/entry.rb
In another cookbook...
# ...
depends 'hostsfile'
other_cookbook/metadata.rb
{ "run_list": [ "recipe[hostsfile]" ]}
www.myapp.com (Chef Node)
ThankYou
z
