Upload
peter-souter
View
167
Download
2
Tags:
Embed Size (px)
DESCRIPTION
It's easy enough to test the correctness of the infracode we write with unit-tests and parsers, but testing it does what it's supposed to do in the enviornment itself is a little more challenging. In this talk, I'm going to talk about some of the tools and approaches to use to test your configuration automation tool of choice.
Citation preview
TESTING SERVERS LIKE SOFTWARE
MEDEVELOPER TURNED ops guy
WORKING AT KAINOS, CONTRACTING ON GOVERNMENT PROJECTS
PREVIOUSLY ON THE IER PROJECT
NOW LIVE! HTTPS://WWW.GOV.UK/REGISTER-TO-VOTE
NOW ON THE DEFRA CAPD PROJECT
FOREWARNING▸ Ruby and Puppet Biased
▸ They're the tools I use the most!▸ But most tools mentioned are system and tool agnostic
SO YOU'VE MADE A CONFIGURATION CODE
CHANGE...
THE UNIT TESTS PASS...
IT'S BEEN CODE REVIEWED
SO YOU PUSH TO PRODUCTION!
IT DOESNT WORK...
WHAT'S THE MISSING STEP?
ACCEPTANCE TESTING:Serverspec
"Serverspec tests your servers' actual state through SSH access"
CHECK YOUR SERVER▸ Is $package installed?▸ Does file contain $foo?
▸ Does the firewall have the correct rules?▸ Is service running?
▸ etc...
RESOURCE TYPEScgroup, command, cron, default_gateway, file,
group, host, iis_app_pool, iis_website, interface, ipfilter, ipnat, iptables,
kernel_module, linux_kernel_parameter, lxc, mail_alias, package, php_config, port, ppa,
process, routing_table, selinux, service, user, windows_feature, windows_registry_key, yumrepo,
zfsfs
MOST ARE FAIRLY SELF
EXPLANATORY
return_stdout
describe command('cat /etc/resolv.conf') do it { should return_stdout /8\.8\.8\.8/ }end
content
describe file('/etc/httpd/conf/httpd.conf') do its(:content) { should match /ServerName www.example.jp/ }end
MY BREAD AND BUTTER
BE_RESOLVABLEdescribe host('serverspec.org') do it { should be_resolvable }end
describe host('serverspec.org') do it { should be_resolvable.by('hosts') }end
describe host('serverspec.org') do it { should be_resolvable.by('dns') }end
BE_REACHABLEdescribe host('target.example.jp') do # ping it { should be_reachable } # tcp port 22 it { should be_reachable.with( :port => 22 ) } # set protocol explicitly it { should be_reachable.with( :port => 22, :proto => 'tcp' ) } # udp port 53 it { should be_reachable.with( :port => 53, :proto => 'udp' ) } # timeout setting (default is 5 seconds) it { should be_reachable.with( :port => 22, :proto => 'tcp', :timeout => 1 ) }end
FULL SPECS FOR A WEB SERVERrequire 'spec_helper'
describe package('apache2') do it { should be_installed }end
describe service('apache2') do it { should be_enabled } it { should be_running }end
describe port(80) do it { should be_listening }end
EXAMPLE:HTTPS://GITHUB.COM/JVOORHIS/
VAGRANT-SERVERSPEC
LIVE DEMO TIME: SERVERSPEC!
OUR WORKFLOW:CHANGE IN PUPPET MADE =>TESTED IN VAGRANT INSTANCE
CODE REVIEWED, PASSES CI AND MERGEDPUSHED TO INTEGRATION ENVIRONMENT
SERVERSPEC TESTS RUN ON INTEGRATION ENVIRONMENTANY ISSUES: FIX OR IF BIG ENOUGH, REVERT
CONTINUES DOWN THE PIPELINE TO PRODUCTION
SERVERSPEC:PRETTY NEAT
BUT THERE'S ALSO LANGUAGE SPECIFIC TOOLS!
TEST KITCHEN
▸ Uses serverspec as it's core▸ It's basically helper wrappers to install chef
▸ And run the cookbooks given▸ Used for cookbook acceptance testing
EXAMPLE:HTTPS://GITHUB.COM/OPSCODE-
COOKBOOKS/APT/
LIVE DEMO TIME: TEST-KITCHEN!
BEAKER
▸ Again: serverspec as it's core▸ But specs customised with Puppet references
▸ Specific rspec grammer around running manifests etc.
EXAMPLE:HTTPS://GITHUB.COM/PETEMS/PUPPET-
SWAP_FILE
require 'spec_helper_acceptance'
describe 'swap_file class', :unless => UNSUPPORTED_PLATFORMS.include?(fact('osfamily')) do
context 'swap_file' do context 'ensure => present' do it 'should work with no errors' do pp = <<-EOS class { 'swap_file': } EOS
# Run it twice and test for idempotency expect(apply_manifest(pp).exit_code).to_not eq(1) expect(apply_manifest(pp).exit_code).to eq(0) end
context 'custom parameters' do it 'should work with no errors' do pp = <<-EOS class { 'swap_file': swapfile => '/tmp/swapfile', swapfilesize => '5 MB', } EOS
it 'should contain the given swapfile' do shell('/sbin/swapon -s | grep /tmp/swapfile', :acceptable_exit_codes => [0]) shell('/sbin/swapon -s | grep 5116', :acceptable_exit_codes => [0]) end end endend
I KNOW WHAT YOU MIGHT BE THINKING...
Wait, whats the difference between this and monitoring?
MONITORINGKEEP THE LIGHTS ON▸ "Fix me now!" issues
▸ Dynamic changes - Things crashing, hard drives are full
ACCEPTANCE"MY CHANGES DIDN'T BREAK ANYTHING"
▸ Smoke tests▸ Human readable
▸ One-off for edge-cases▸ Auditing
HOWEVER
USING SERVERSPEC AS MONITORING IS POSSIBLE!
AND HAS SOME PRETTY SWEET BENEFITSHTTP://WWW.SLIDESHARE.NET/M_RICHARDSON/SERVERSPEC-AND-SENSU-TESTING-AND-MONITORING-
COLLIDE
Q&A
LINKShttp://serverspec.org/
http://vincent.bernat.im/en/blog/2014-serverspec-test-infrastructure.html
https://github.com/serverspec/serverspec
http://www.debian-administration.org/article/703/A_brief_introduction_to_server-testing_with_serverspec
"ChefConf 2014: Gosuke Miyashita, "Serverspec: The Simplest Server Testing Tool Ever" - http://www.youtube.com/watch?
v=6GvlHImeloo*