Test-Driven Infrastructure with Chef

Test-Driven Infrastructure with Chef

Principles & Tools

@kaktusmimi

Today

① Testing Principles

② Chef Testing Tools ③ Continuous Integration

Write Test

Run Test (it fails)

Write Code

Test Passes

Refactor TDD

BDD

TDDverification of a unit of code

specification of how code should „behave“

TDD pit

fall

Acceptance Tests

IntegrationTests

UnitTests

Toda

y

Today



✔

ChefSpec

• Unit Testing Framework for Chef Cookbooks

• Runs locally without converging a (virtual) machine

$ gem install chefspec

!"" site-cookbooks !"" pt_jenkins #"" recipes $ !"" default.rb !"" spec !"" default_spec.rb

execute "Install Jenkins from Ports" do command "cd #{node['pt_jenkins']['jenkins_port_dir']} && make install clean BATCH=\"YES\"" not_if {File.exist?(node['pt_jenkins']['jenkins_war_file'])}enddirectory node['pt_jenkins']['jenkins_home'] do owner node['pt_jenkins']['jenkins_user'] group node['pt_jenkins']['jenkins_group'] mode '0766' action :createendnode.default['user']['git_ssh_wrapper'] = "/tmp/git_ssh_wrapper"file node['user']['git_ssh_wrapper'] do owner node['pt_jenkins']['jenkins_user'] group node['pt_jenkins']['jenkins_group'] mode 0777 content "/usr/bin/env ssh -A -o 'StrictHostKeyChecking=no' $1 $2" action :createendcookbook_file 'Copy private vagrant key' do path "/home/vagrant/.ssh/id_rsa" source 'vagrant_private_key' owner 'vagrant' group 'vagrant' backup false mode 0600 action :createendcookbook_file 'Copy SSH config' do path "/home/vagrant/.ssh/config" source 'ssh_config' owner 'vagrant' group 'vagrant' backup false mode 0600 action :createendservice node['pt_jenkins']['jenkins_service'] do supports :status => true, :restart => true, :reload => true action [ :enable, :start ] end

require 'chefspec'RSpec.configure do |config| config.cookbook_path = [‚cookbooks','site-cookbooks'] config.role_path = 'roles'enddescribe 'pt_jenkins::default' do let(:chef_run) do ChefSpec::SoloRunner.new do |node| node.set['jenkins']['plugins'] = {} node.set['user']['git_ssh_wrapper'] = '/tmp/git_ssh_wrapper' end.converge(described_recipe) end it 'installs Jenkins from Ports' do expect(chef_run).to run_execute('Install Jenkins from Ports') end it 'creates a ssh wrapper file' do expect(chef_run).to create_file('/tmp/git_ssh_wrapper') end it 'copies the ssh config' do expect(chef_run).to create_cookbook_file('Copy SSH config') end it 'copies the private Vagrant key' do expect(chef_run).to create_cookbook_file('Copy private vagrant key') end it 'starts the Jenkins service' do expect(chef_run).to start_service('jenkins') endend

$ time bundle exec rspec site-cookbooks/pt_jenkins

pt_jenkins::default installs Jenkins from Ports creates a ssh wrapper file copies the ssh config copies the private Vagrant key starts the Jenkins service

Finished in 0.34419 seconds4 examples, 0 failures

1.43s user 0.24s system 76% cpu 2.186 total

Pros & Cons ChefSpecPro Contra

Fast White Box Testing

Can be run without convergence Harder to implement

Easy Setup Some tricky configuration

Don’t know what system looks like at the end

ServerSpec

• Describe desired state

• Check whether convergence result matches expectations

• Platform independent

• Run it „from inside“ or „from outside“

$ gem install serverspec

$ serverspec-initSelect OS type:

1) UN*X 2) Windows

Select number: 1

Select a backend type:

1) SSH 2) Exec (local)

Select number: 1

Vagrant instance y/n: yAuto-configure Vagrant from Vagrantfile? y/n: y

$ rake -Trake spec:pttech # Run serverspec tests to pttech

#"" site-cookbooks $ !"" spec $ #"" pt $ $ !"" jenkins_spec.rb $ !"" spec_helper.rb !"" Rakefile

require 'serverspec'require 'net/ssh'require 'tempfile'set :backend, :sshif ENV['ASK_SUDO_PASSWORD'] begin require 'highline/import' rescue LoadError fail "highline is not available. Try installing it." end set :sudo_password, ask("Enter sudo password: ") { |q| q.echo = false } else set :sudo_password, ENV['SUDO_PASSWORD'] endhost = ENV['TARGET_HOST'] `vagrant up #{host}` config = Tempfile.new('', Dir.tmpdir)`vagrant ssh-config #{host} > #{config.path}` options = Net::SSH::Config.for(host, [config.path])options[:user] ||= Etc.getloginset :host, options[:host_name] || hostset :ssh_options, options

require 'rake'require 'rspec/core/rake_task'task :spec => 'spec:all'task :default => :specnamespace :spec do targets = [] Dir.glob('./spec/*').each do |dir| next unless File.directory?(dir) targets << File.basename(dir) end task :all => targets task :default => :all targets.each do |target| desc "Run serverspec tests on #{target}" RSpec::Core::RakeTask.new(target.to_sym) do |t| ENV['TARGET_HOST'] = target t.pattern = "spec/#{target}/*_spec.rb" end endend

$ rake

Server contains - jenkins.war at the expected location - a folder /usr/local/jenkins, owned by user jenkins - a folder /usr/local/jenkins, with mode 766

jenkins_config = JSON.parse(File.open("#{File.dirname(__FILE__)}/../../roles/jenkins.json").read)jenkins_config["override_attributes"]["jenkins"]["plugins"].each do | plugin_name, plugin_config | describe "Plugin: #{plugin_name}" do if plugin_config['enabled'] it "has expected .hpi file in plugins directory" do expect(file("/usr/local/jenkins/plugins/#{plugin_name}.hpi")).to be_file end end if plugin_config['pinned'] it "is marked as pinned and has the .hpi.pinned file in the plugins directory" do expect(file("/usr/local/jenkins/plugins/#{plugin_name}.hpi.pinned")).to be_file end it "is listed as pinned in the Jenkins plugin status" do expect(check_jenkins_plugin_pinned(plugin_name)).to be_truthy end else it "isn't marked as pinned and has no .hpi.pinned file in the plugins directory" do expect(file("/usr/local/jenkins/plugins/#{plugin_name}.hpi.pinned")).not_to be_file end end if plugin_config['version'] it "is version #{plugin_config['version']}" do expect(check_jenkins_plugin_version(plugin_name, plugin_config['version'])).to be_truthy end end endend

Pros & Cons ServerSpecPro Contra

Easy Setup & well documented No specific Feedback

Black Box Tests Requires running Machine

Small but mighty command set Slow

(Easily?) extendable Can only be run once per convergence Run

Test Kitchen

• Test and Infrastructure Management

• Interesting for Cookbook Development

• Quite some (configuration) overhead

• Good Tutorial

• Bad Documentation

---driver: name: vagrant network: - ["private_network", {ip: "10.20.30.41", netmask: "255.255.255.0"}]provisioner: name: chef_zero require_chef_omnibus: false client_rb: cookbook_path: ["/var/chef/cookbooks", "/var/chef/site-cookbooks"]platforms: - name: FreeBSD-10-PtTech-TestKitchen driver: name: vagrant box: nanobsd-hosting-amd64.box-20140917 box_url: http:/vagrantbox.es/some-box-to-use synced_folders: - [".", "/var/chef", "create: true, type: :nfs"]suites: - name: server run_list: - recipe[testkitchen] - role[jenkins] attributes:

Pros & Cons TestKitchenPro Contra

Handles complex Setups „Touches“ your Machine after Convergence

Multi-platform Testing for Cookbook Development

No ServerSpec via ssh implemented

„Standard“ in many Tutorials

Requires machine startup from Scratch

Many Plugins

Foodcritic• Linting for Chef Cookbooks

• Complex Rule Set far beyond Code Indentation

• Goals (from Foodcritic Website)

• To make it easier to flag problems in your Cookbooks

• Faster feedback.

• Automate checks for common problems

$ gem install foodcritic

$ foodcritic site-cookbooks/pt_jenkins -t FC001

# FC001 Use strings in preference to symbols to access node attributes

FC001: Use strings in preference to symbols to access node attributes: site-cookbooks/pt_jenkins/attributes/default.rb:55FC001: Use strings in preference to symbols to access node attributes: site-cookbooks/pt_jenkins/attributes/default.rb:56FC001: Use strings in preference to symbols to access node attributes: site-cookbooks/pt_jenkins/recipes/default.rb:21FC001: Use strings in preference to symbols to access node attributes: site-cookbooks/pt_jenkins/recipes/default.rb:97FC001: Use strings in preference to symbols to access node attributes: site-cookbooks/pt_jenkins/recipes/default.rb:98FC001: Use strings in preference to symbols to access node attributes: site-cookbooks/pt_jenkins/recipes/default.rb:109FC001: Use strings in preference to symbols to access node attributes: site-cookbooks/pt_jenkins/recipes/default.rb:141

Further Testing Tools

• BATS: Bash Automated Testing System

• Shell Scripts 😟

• Cucumber Chef

• Great idea, but prototype state only

• Last commit > 1 year ago 😟

Today



✔

✔

Deployment Stage

Knife Upload to Chef Server

Acceptance Stage

Serverspec Test

Commit Stage

Build Project

ChefSpec Tests

Foodcritic

triggers uploads

Today



✔

✔✔

http://serverspec.org/

https://github.com/sethvargo/chefspec

http://kitchen.ci/

http://acrmp.github.io/foodcritic/

https://sethvargo.com/unit-testing-chef-cookbooks/

http://leopard.in.ua/2013/12/01/chef-and-tdd/

http://serverspec.org/

https://github.com/sethvargo/chefspec

http://kitchen.ci/

http://acrmp.github.io/foodcritic/

https://sethvargo.com/unit-testing-chef-cookbooks/

http://leopard.in.ua/2013/12/01/chef-and-tdd/

Software

Test-Driven Infrastructure with Chef