Spiceweasel: Your Infrastructure as Code Table of Contents
Matt [email protected]
Twitter/IRC/GitHub: mattray
Infrastructure as Code
Enable the reconstruction of the business from nothing but
a source code repository, application data backup and
bare metal resources.
That's great but...how did I get here?
http://www.flickr.com/photos/mrchippy/443960682/
Spiceweasel
• Infrastructure deployment manifest
• Converts YAML/JSON to knife commands
• http://github.com/mattray/spiceweasel
http://www.flickr.com/photos/mrchippy/443960682/
YAML
• Simple
• Easy to read
• Easy to reason
• spiceweasel infrastructure.yml
http://www.flickr.com/photos/mrchippy/443960682/
JSON
• Patches welcomed.
• spiceweasel infrastructure.json
cookbooks:- annoyances:- apache2:- chef-client: - 1.1.5- java:- mysql:- tomcat:
knife cookbook upload annoyancesknife cookbook site download apache2 --file cookbooks/apache2.tgz tar -C cookbooks/ -xf cookbooks/apache2.tgzrm -f cookbooks/apache2.tgzknife cookbook upload apache2knife cookbook site download chef-client 1.1.5 --file cookbooks/chef-client.tgz tar -C cookbooks/ -xf cookbooks/chef-client.tgzrm -f cookbooks/chef-client.tgzknife cookbook upload chef-clientknife cookbook upload javaknife cookbook upload mysqlknife cookbook upload tomcat
Cookbooks
http://www.flickr.com/photos/mrchippy/443960682/
environments:- development:- qa:- preprod:- production:
knife environment from file development.rbknife environment from file qa.rbknife environment from file preprod.rbknife environment from file production.rb
Environments
http://www.flickr.com/photos/mrchippy/443960682/
Roles
roles:- base:- glance-single-machine:- nova-db:- nova-multi-compute:- nova-multi-controller:- nova-rabbitmq-server:- nova-single-machine:
http://www.flickr.com/photos/laenulfean/374398044/
knife role from file base.rbknife role from file glance-single-machine.rbknife role from file nova-db.rbknife role from file nova-multi-compute.rbknife role from file nova-multi-controller.rbknife role from file nova-rabbitmq-server.rbknife role from file nova-single-machine.rb
http://www.flickr.com/photos/mrchippy/443960682/
Data Bags
data bags:- users: - alice - bob - chuck- data: - *- passwords: - secret secret_key - mysql - rabbitmq
knife data bag create usersknife data bag from file users alice.jsonknife data bag from file users bob.jsonknife data bag from file users chuck.jsonknife data bag create dataknife data bag from file data dataA.jsonknife data bag from file data dataB.jsonknife data bag create passwordsknife data bag from file passwords mysql.json --secret-file secret_keyknife data bag from file passwords rabbitmq.json --secret-file secret_key
http://www.flickr.com/photos/75659300@N00/2615848530/
nodes:- ubuntu1-1004.vm ubuntu2-1004.vm ubuntu3-1004.vm: - role[base],role[tc],role[sample] - -i ~/.ssh/mray.pem -x ubuntu --sudo
knife bootstrap ubuntu1-1004.vm -i ~/.ssh/mray.pem -x ubuntu --sudo -r 'role[base],role[tc],role[sample]'knife bootstrap ubuntu2-1004.vm -i ~/.ssh/mray.pem -x ubuntu --sudo -r 'role[base],role[tc],role[sample]'knife bootstrap ubuntu3-1004.vm -i ~/.ssh/mray.pem -x ubuntu --sudo -r 'role[base],role[tc],role[sample]'
Nodes
http://www.flickr.com/photos/75659300@N00/2615848530/
nodes:- windows_winrm winboxA: - role[base],role[sqlserver] - -x Administrator -P 'super_secret_password'- windows_ssh winboxB winboxC: - role[base],role[iis] - -x Administrator -P 'super_secret_password'
knife bootstrap windows winrm winboxA -x Administrator -P 'super_secret_password' -r 'role[base],role[sqlserver]'knife bootstrap windows ssh winboxB -x Administrator -P 'super_secret_password' -r 'role[base],role[iis]'knife bootstrap windows ssh winboxC -x Administrator -P 'super_secret_password' -r 'role[base],role[iis]'
Nodes: Windows
http://www.flickr.com/photos/75659300@N00/2615848530/
nodes:- ec2 5: - role[base],role[tc],role[sample] - -S mray -i ~/.ssh/mray.pem -x ubuntu -I ami-0c6ebd65 -f m1.small- rackspace 5: - role[base],role[tc],role[sample] - --image 114 --flavor 2 -i ~/.ssh/mray.pem- openstack 5: - role[base],role[tc],role[sample] - --image 7 --flavor 2 -x ubuntu -i ~/.ssh/mray.pem
seq 5 | parallel -j 0 -v "knife ec2 server create -S mray -i ~/.ssh/mray.pem -x ubuntu -I ami-0c6ebd65 -f m1.small -r 'role[base],role[tc],role[sample]'"seq 5 | parallel -j 0 -v "knife rackspace server create --image 114 --flavor 2 -i ~/.ssh/mray.pem -r 'role[base],role[tc],role[sample]'"seq 5 | parallel -j 0 -v "knife openstack server create --image 7 --flavor 2 -i ~/.ssh/mray.pem -r 'role[base],role[tc],role[sample]'"
Nodes: Cloud Providers
Validation
• Cookbook versions and their dependencies exist
• Environment names and referenced cookbooks are correct
• Role names and referenced roles and cookbooks are correct
• Data bags parse, secret keys are present
• Node run lists items are correct and their Environment exists
http://www.flickr.com/photos/mrchippy/443960682/
What's the Point?
• Nodes are ephemeral
• Everything we do is in code.
• Everything is checked into version control.
• Basic roadmap for deployment.
http://www.flickr.com/photos/mrchippy/443960682/
Break it Down
How do we recreate our infrastructure?
http://www.flickr.com/photos/mrchippy/443960682/
Real World Example
• Whiteboard it out
• Break it down into components
• Comment it all out and start working
http://www.flickr.com/photos/mrchippy/443960682/
Here's the YAML
cookbooks:- annoyances:- chef-client:- java:- jpackage:- ntp:- sudo:- tomcat:- users:- sample:roles:- base:- tc:- sample:data bags: - users: - mraynodes:- ec2 1: - role[base],role[tc],role[sample] - -S mray -i ~/.ssh/mray.pem -x ubuntu -G default -I ami-0c6ebd65 -f m1.small- rackspace 1: - role[base],role[tc],role[sample] - --image 114 --flavor 2 -i ~/.ssh/mray.pem
http://www.flickr.com/photos/mrchippy/443960682/
Here it is on GitHub
http://www.flickr.com/photos/mrchippy/443960682/
Versioned Infrastructure
• Documentation for your deployment
• Iterate over it
• Stored in version control
• Share with others
http://www.flickr.com/photos/mrchippy/443960682/
What's Next?
•Extract from chef-repo
•Extract from Chef Server
•Execute as knife plugin
http://www.flickr.com/photos/mrchippy/443960682/
Thanks!
• gem install spiceweasel
• http://github.com/mattray/spiceweasel
• Twitter/IRC/GitHub: mattray