Upload
julien-pivotto
View
2.763
Download
2
Embed Size (px)
DESCRIPTION
My talk at RMLL 2013
Citation preview
;
Testing your puppet codeTesting your puppet code
Julien PivottoJulien Pivotto
Libre Software Meeting 2013Libre Software Meeting 2013July 15, 2013July 15, 2013
;
IntroductionTesting tools
JenkinsConclusion
1 IntroductionAutomationVagrantPuppet in a large scalePuppet code
2 Testing toolsStyle and lintingCatalogsrspec-puppet
3 Jenkins
4 ConclusionHomeworkConclusion
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
AutomationVagrantPuppet in a large scalePuppet code
Julien PivottoJulien Pivotto• sysadmin @ inuitssysadmin @ inuits• open-source defender for 7+ yearsopen-source defender for 7+ years• devops believerdevops believer• @roidelapluie on twitter/github@roidelapluie on twitter/github
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
AutomationVagrantPuppet in a large scalePuppet code
Infrastructure as Code
• Keep your environments under SCMKeep your environments under SCM• Overview of complete environmentsOverview of complete environments• Reduce the deployment timeReduce the deployment time
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
AutomationVagrantPuppet in a large scalePuppet code
Keep all environments the same
http://www.flickr.com/photos/bobvietnam/4828291896/
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
AutomationVagrantPuppet in a large scalePuppet code
Packaging with FPM
• Ruby gemRuby gem• package a directory (and much more)package a directory (and much more)• Support .deb, .rpmSupport .deb, .rpm• Package the code with several prefixesPackage the code with several prefixes• /etc/puppet/environments/infradev/etc/puppet/environments/infradev• /etc/puppet/environments/uat/etc/puppet/environments/uat
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
AutomationVagrantPuppet in a large scalePuppet code
Vagrant
• Create virtual machinesCreate virtual machines• Provision themProvision them• Destroy & recreateDestroy & recreate
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
AutomationVagrantPuppet in a large scalePuppet code
Vagrant
• Chef, scripts, puppet, . . .Chef, scripts, puppet, . . .• Backend: Virtualbox, KVM, . . .Backend: Virtualbox, KVM, . . .• A lot of baseboxes availableA lot of baseboxes available• http://vagrantup.comhttp://vagrantup.com
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
AutomationVagrantPuppet in a large scalePuppet code
Vagrant
• Local testingLocal testing• The same environment as the targetThe same environment as the target
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
AutomationVagrantPuppet in a large scalePuppet code
Puppet environments
• Multiple environmentsMultiple environments• The same tree for all the environmentsThe same tree for all the environments• Pushing changes to UAT/prod on-demandPushing changes to UAT/prod on-demand• Small changes vs big releasesSmall changes vs big releases
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
AutomationVagrantPuppet in a large scalePuppet code
Hiera
• Storing the data in Hiera(-gpg)Storing the data in Hiera(-gpg)• Usernames, password, IP addressesUsernames, password, IP addresses• Hiera is made to be structuredHiera is made to be structured• Using one hiera repo for all the environmentsUsing one hiera repo for all the environments• Using Hiera in your manifests, not in your modulesUsing Hiera in your manifests, not in your modules
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
AutomationVagrantPuppet in a large scalePuppet code
Hiera tree
• %{environment}/%{hostname}%{environment}/%{hostname}• %{environment}/common%{environment}/common• infradev/www45.yamlinfradev/www45.yaml• infradev/common.yamlinfradev/common.yaml
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
AutomationVagrantPuppet in a large scalePuppet code
Keeping clean puppet modules
http://www.flickr.com/photos/aurelie_solenne/8340968061/
• Make them readableMake them readable• Make them reusable and sharableMake them reusable and sharable• Don’t puppetize everythingDon’t puppetize everything• User generated content is not puppetizedUser generated content is not puppetized
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
AutomationVagrantPuppet in a large scalePuppet code
Use the right structure for your modules
• Package, config, servicePackage, config, service
• module::package, module::config, module::servicemodule::package, module::config, module::service
• Parameterized classesParameterized classes
http://www.slideshare.net/PuppetLabs/modern-module-development-ken-barber-2012-edinburgh-
puppet-camp
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
AutomationVagrantPuppet in a large scalePuppet code
Distribution-agnostic puppet modules
• You don’t have to support all the distrosYou don’t have to support all the distros• Adding support for another distro should be easyAdding support for another distro should be easy
$config_dir = $configroot ? {undef => $::operatingsystem ? {
/Debian|Ubuntu/ => ’/etc/apache2’,/CentOS|RedHat/ => ’/etc/httpd’,default => ’/etc/httpd’,
},default => $configroot,
}
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
AutomationVagrantPuppet in a large scalePuppet code
Puppet function
• The fail function prevents catalog to be appliedThe fail function prevents catalog to be applied• The notify function prints a warningThe notify function prints a warning
if (!$leftsubnet) and (!$leftsubnets) {fail(’$leftsubnets and $leftsubnet both empty’)
}
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
Style and lintingCatalogsrspec-puppet
Puppet parser
• Included in puppetIncluded in puppet• Validating the syntaxValidating the syntax• puppet parser validate init.pppuppet parser validate init.pp• find . -name ’*.pp’ -exec puppet parser validate+ ;
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
Style and lintingCatalogsrspec-puppet
Puppet lint
http://www.flickr.com/photos/voyages-provence/8127668094/
• Follow the puppet style guideFollow the puppet style guide• Two-space soft tabTwo-space soft tab• align fat comma arrows (=>) within blocks of attributesalign fat comma arrows (=>) within blocks of attributes• http://docs.puppetlabs.com/guides/style_guide.htmlhttp://docs.puppetlabs.com/guides/style_guide.html
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
Style and lintingCatalogsrspec-puppet
Cucumber puppet
• Write scenariosWrite scenarios• Easy to read (full sentences)Easy to read (full sentences)• Use your manifestsUse your manifests• Need some tricks to work with Puppet 3Need some tricks to work with Puppet 3• DiscontinuedDiscontinued
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
Style and lintingCatalogsrspec-puppet
Cucumber example
CucumberFeature: General catalog policy
In order to ensure applicability of a host’s catalogAs a manifest developerI want all catalogs to obey some general rules
Scenario Outline: Compile and verify catalogGiven a node specified by "features/yaml/<hostname>."When I compile its catalogThen compilation should succeedAnd all resource dependencies should resolve
Examples:| hostname || localhost |
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
Style and lintingCatalogsrspec-puppet
rspec-puppet
• Check what is the behaviour of puppetCheck what is the behaviour of puppet• Separate tests per modulesSeparate tests per modules• Add context, facts, . . .Add context, facts, . . .• Test custom functions, hosts, manifests, . . .Test custom functions, hosts, manifests, . . .
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
Style and lintingCatalogsrspec-puppet
rspec-puppet
Start with rspec puppetgem install rspec-puppetgem install puppetcd my-modulerspec-puppet-init
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
Style and lintingCatalogsrspec-puppet
rspec-puppet
spec/defines/connection_spec.rbrequire ’spec_helper’describe ’openswan::connection’ do
describe ’should require rightsubnet or rightsubnets’ dolet(:title) { ’foobar’ }let (:params) { {
:ike => ’aes256-sha1;modp1024’,:esp => ’aes256-sha1;modp1024’,:leftsubnet => ’8.8.5.5’,:right => ’84.54.105.5’,:left => ’68.65.98.6’,:foreignip => ’45.25.5.5’,:localtestip => ’82.8.8.8’, } }it do
expect {should contain_file("/etc/ipsec.d/foobar.conf")
}.to raise_error(Puppet::Error, /$rightsubnets and $rightsubnet cannot be both empty/)end
endend
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
Style and lintingCatalogsrspec-puppet
rspec-puppet
Second examplerequire ’spec_helper’describe ’apache’, :type => :class do
let (:facts) { {:operatingsystem => ’CentOS’,:osfamily => ’RedHat’,
} }describe ’without parameters’ do
it { should create_class(’apache’) }it { should include_class(’apache::service’) }it { should contain_apache__listen(’80’) }it { should contain_apache__namevhost(’80’) }
endend
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
Style and lintingCatalogsrspec-puppet
rspec-puppet
• should, should_notshould, should_not• should contain_packageshould contain_package• contain_foo__bar(’baz’) (for foo::bar)contain_foo__bar(’baz’) (for foo::bar)
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
Integration with jenkins
• Pulling, testing and deploymentsPulling, testing and deployments• Push-Test-Package-DeployPush-Test-Package-Deploy• Continuous integrationContinuous integration• Continuous deliveryContinuous delivery
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
Jenkins pipelines
• Build pipelinesBuild pipelines• Overview of what happensOverview of what happens• Getting notified about what failedGetting notified about what failed• Promoted build pluginPromoted build plugin
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
Jenkins pipelines
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
Advantages of CI
• You trust your codeYou trust your code• ReproducabilityReproducability• You get metrics: number of warning, . . .You get metrics: number of warning, . . .• You have a backlogYou have a backlog• It is easy!It is easy!
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
Promotions
• Provides buttons you can clickProvides buttons you can click• Trigger actionsTrigger actions• deploy to other environmentsdeploy to other environments• Get a mail with the changesGet a mail with the changes• Have a log of who deployedHave a log of who deployed
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
Promotions
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
HomeworkConclusion
Homework
• Integrating tests with git hooksIntegrating tests with git hooks• Integrating tests with VIIntegrating tests with VI• github.com/philandstuff/fizzgiggithub.com/philandstuff/fizzgig
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
HomeworkConclusion
Conclusion
• Use nice & simple Puppet modulesUse nice & simple Puppet modules• Continuous integrationContinuous integration• Multiple environmentsMultiple environments• Readability & reusabilityReadability & reusability• Tools exist and work togetherTools exist and work together
Julien Pivotto Testing your puppet code
;
IntroductionTesting tools
JenkinsConclusion
HomeworkConclusion
Contact
Julien PivottoJulien [email protected]@inuits.eu@roidelapluie@roidelapluie
INUITS bvbaINUITS bvbaDuboisstraat 50Duboisstraat 502060 Antwerp2060 AntwerpBelgiumBelgium+32 473 441 636+32 473 441 636https://inuits.euhttps://inuits.eu
Julien Pivotto Testing your puppet code