52
Chef for OpenStack Matt Ray [email protected] IRC/Twitter/GitHub: mattray www.opscode.com/openstack

Chef for OpenStack- Fall 2012.pdf

Embed Size (px)

DESCRIPTION

true

Citation preview

Page 1: Chef for OpenStack- Fall 2012.pdf

Chef for OpenStack

Matt [email protected]

IRC/Twitter/GitHub: mattraywww.opscode.com/openstack

Page 2: Chef for OpenStack- Fall 2012.pdf

Deploying and Managing OpenStack is not simple.

Page 3: Chef for OpenStack- Fall 2012.pdf

Chef makes it easier.

Page 4: Chef for OpenStack- Fall 2012.pdf

See Node

Application Server

Page 5: Chef for OpenStack- Fall 2012.pdf

See Nodes

Application Server

Application Database

Page 6: Chef for OpenStack- Fall 2012.pdf

See Nodes Grow

Application Server

Application Databases

Page 7: Chef for OpenStack- Fall 2012.pdf

Application Servers

Application Databases

See Nodes Grow

Page 8: Chef for OpenStack- Fall 2012.pdf

Application Servers

Application Databases

Load Balancer

See Nodes Grow

Page 9: Chef for OpenStack- Fall 2012.pdf

See Nodes Grow

Application Servers

Application Databases

Load Balancers

Page 10: Chef for OpenStack- Fall 2012.pdf

See Nodes Grow

Application Servers

Application Database Cache

Load Balancers

Application Databases

Page 11: Chef for OpenStack- Fall 2012.pdf

Tied together with Config

Application Servers

Application Database Cache

Load Balancers

Application Databases

Page 12: Chef for OpenStack- Fall 2012.pdf

Infrastructure is a Snowflake

Application Servers

Application Database Cache

Load Balancers

Floating IP?

Application Databases

Page 13: Chef for OpenStack- Fall 2012.pdf

Evolving Complexity

Load Balancers

Application Servers

NoSQL

Database Slaves

ApplicationCache

Database Cache

Database

Page 14: Chef for OpenStack- Fall 2012.pdf

Complexity Grows Quickly

DC1

DC3

DC2

Page 15: Chef for OpenStack- Fall 2012.pdf

http://www.flickr.com/photos/16339684@N00/2681435235/

And it Continues to Evolve

Tell me aboutChef!

Page 16: Chef for OpenStack- Fall 2012.pdf

Chef is Infrastructure as Code

http://www.flickr.com/photos/louisb/4555295187/

• Programmatically provision and configure

• Treat like any other code base

• Reconstruct business from code repository, data backup, and bare metal resources.

Page 17: Chef for OpenStack- Fall 2012.pdf

• Chef-Client generates configurations directly on nodes from their run list

• Reduce management complexity through abstraction

• Store the configuration of your programs in version control

http://www.flickr.com/photos/ssoosay/5126146763/

Nodes

Page 18: Chef for OpenStack- Fall 2012.pdf

Collections of Resources

• Networking

• Files

• Directories

• Symlinks

• Mounts

• Routes

• Users

• Groups

• Tasks

• Packages

• Software

• Services

• Configurations

• Other Stuffhttp://www.flickr.com/photos/stevekeys/3123167585/

Page 19: Chef for OpenStack- Fall 2012.pdf

Declarative Interface to Resources

• Define policy

• Say what, not how

• Pull not Push

http://www.flickr.com/photos/bixentro/2591838509/

Page 20: Chef for OpenStack- Fall 2012.pdf

Recipes and Cookbooks

• Recipes are collections of Resources

• Cookbooks contain recipes, templates, files, custom resources, etc

• Code re-use and modularity

• Hundreds already on Community.opscode.com

http://www.flickr.com/photos/shutterhacks/4474421855/

Page 21: Chef for OpenStack- Fall 2012.pdf

Ruby!

extra_packages = case node['platform'] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } endextra_packages.each do |pkg| package pkg do action :install endend

Page 22: Chef for OpenStack- Fall 2012.pdf

http://www.flickr.com/photos/kathycsus/2686772625

• IP addresses

• Hostnames

• FQDNs

• Search for nodes with Roles

• Find configuration data

Search

Page 23: Chef for OpenStack- Fall 2012.pdf

pool_members = search("node","role:webserver”)

template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]"end

Pass Results to Templates

Page 24: Chef for OpenStack- Fall 2012.pdf

# Set up application listeners here.listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%><% if node["haproxy"]["enable_admin"] -%>listen admin 0.0.0.0:22002 mode http stats uri /<% end -%>

Pass Results to Templates

Page 25: Chef for OpenStack- Fall 2012.pdf

Jboss App

Memcache

Postgres Slaves

Postgres Master

So when this

NagiosGraphite

Page 26: Chef for OpenStack- Fall 2012.pdf

Jboss App

Memcache

Postgres Slaves

Postgres Master

NagiosGraphite

Becomes this

Page 27: Chef for OpenStack- Fall 2012.pdf

Jboss App

Memcache

Postgres Slaves

Postgres Master

NagiosGraphite

Updates can be automatic

Page 28: Chef for OpenStack- Fall 2012.pdf

NagiosGraphite

Count the resources

Jboss App

Memcache

Postgres Slaves

• Load balancer config

• Nagios host ping

• Nagios host ssh

• Nagios host HTTP

• Nagios host app health

• Graphite CPU

• Graphite Memory

• Graphite Disk

• Graphite SNMP

• Memcache firewall

• Postgres firewall

• Postgres authZ config

• 12+ resource changes for 1 node addition

Page 29: Chef for OpenStack- Fall 2012.pdf

Build anything

• Simple internal applications

• Complex external applications

• Workstations

• Hadoop clusters

• IaaS infrastructure

• PaaS infrastructure

• SaaS applications

• Storage systems

• You name it

http://www.flickr.com/photos/hyku/245010680/

Page 30: Chef for OpenStack- Fall 2012.pdf

And manage it simply

http://www.flickr.com/photos/helico/404640681/

• Automatically reconfigure everything

• Linux, Windows, Unixes, BSDs

• Load balancers

• Metrics collection systems

• Monitoring systems

• Cloud migrations become trivial

Page 31: Chef for OpenStack- Fall 2012.pdf

The Chef Community

• Apache License, Version 2.0

• 900+ Individual contributors

• 160+ Corporate contributors

• HP, Dell, Rackspace, VMware, Calxeda, SUSE and many more

• 600+ cookbooks

• http://community.opscode.com

Page 32: Chef for OpenStack- Fall 2012.pdf

ResourcesChef for OpenStack

What's Out There?

Page 33: Chef for OpenStack- Fall 2012.pdf

Chef for OpenStack: Why

• Community for the automated deployment and management of OpenStack

• Reduce fragmentation and encourage collaboration

• Deploying OpenStack is not "secret sauce"

• Project not a product

• Apache 2 license

Page 34: Chef for OpenStack- Fall 2012.pdf

Chef for OpenStack: What

• Chef Repository for Deploying OpenStack

• Documentation for Chef for OpenStack

• Cookbooks

• Keystone

• Glance

• Nova

• Horizon

• Swift

• Knife OpenStack

Page 35: Chef for OpenStack- Fall 2012.pdf

Chef for OpenStack: Where

• opscode.com/openstack

• groups.google.com/group/opscode-chef-openstack

• #openstack-chef on irc.freenode.net

• github.com/opscode/openstack-chef-repo

• github.com/mattray/openstack-chef-docs

• github.com/opscode-cookbooks/

• keystone, glance, nova, horizon, swift

• github.com/opscode/knife-openstack

Page 36: Chef for OpenStack- Fall 2012.pdf

Chef for OpenStack: Who

Page 37: Chef for OpenStack- Fall 2012.pdf

Rackspace Private Cloud: Alamo

• Initial fork of current cookbooks

• github.com/rcbops/chef-cookbooks

• www.rackspace.com/cloud/private/

Page 38: Chef for OpenStack- Fall 2012.pdf

Chef for OpenStack: When

• Essex is working

• KVM

• Ubuntu 12.04

• Folsom has already started

• LXC

• Red Hat

• Grizzly and forward

Page 39: Chef for OpenStack- Fall 2012.pdf

github.com/mattray/openstack-chef-docs

Chef for OpenStack: How

Page 40: Chef for OpenStack- Fall 2012.pdf

Deploying OpenStack

• Chef ties it all together automatically

• Scaling changes how we deploy

• Interchangeable components

• Configurations shared, supported & documented

• Licensing makes it available to everyone

Page 41: Chef for OpenStack- Fall 2012.pdf

knife openstack

Page 42: Chef for OpenStack- Fall 2012.pdf

knife openstack

$ knife openstackAvailable openstack subcommands: (for details, knife SUB-COMMAND --help)

** OPENSTACK COMMANDS **knife openstack flavor list (options)knife openstack image list (options)knife openstack server create (options)knife openstack server delete SERVER [SERVER] (options)knife openstack server list (options)

Page 43: Chef for OpenStack- Fall 2012.pdf

knife openstack flavor list

$ knife openstack flavor listID Name Virtual CPUs RAM Disk 1 m1.tiny 1 512 MB 0 GB 2 m1.small 2 2048 MB 20 GB 3 m1.medium 2 4096 MB 40 GB 4 m1.large 2 8192 MB 80 GB 5 m1.xlarge 4 16384 MB 160 GB

Page 44: Chef for OpenStack- Fall 2012.pdf

knife openstack image list

$ knife openstack image listID Name 13 natty-server-cloudimg-amd64 12 natty-server-cloudimg-amd64-kernel 15 oneiric-server-cloudimg-amd64 14 oneiric-server-cloudimg-amd64-kernel

Page 45: Chef for OpenStack- Fall 2012.pdf

knife openstack server create --node-name ko1 --flavor 1 --image 13 -S trystack

knife openstack server create

Page 46: Chef for OpenStack- Fall 2012.pdf
Page 47: Chef for OpenStack- Fall 2012.pdf

$ ssh -i ~/.ssh/trystack.pem [email protected] authenticity of host '8.21.28.24 (8.21.28.24)' can't be established.RSA key fingerprint is 0c:d8:3e:34:d1:de:c4:ee:5f:bc:b5:89:11:0d:73:e0.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added '8.21.28.24' (RSA) to the list of known hosts.Welcome to Ubuntu 11.04 (GNU/Linux 2.6.38-13-virtual x86_64)

* Documentation: https://help.ubuntu.com/

System information as of Thu Feb 16 23:43:29 UTC 2012

System load: 0.08 Processes: 63 Usage of /: 40.8% of 1.35GB Users logged in: 0 Memory usage: 6% IP address for eth0: 8.21.28.24 Swap usage: 0%---------------------------------------------------------------------<snip>Get cloud support with Ubuntu Advantage Cloud Guest http://www.ubuntu.com/business/services/cloud

The programs included with the Ubuntu system are free software;the exact distribution terms for each program are described in theindividual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted byapplicable law.

To run a command as administrator (user "root"), use "sudo <command>".See "man sudo_root" for details.

ubuntu@ko1:~$

Page 48: Chef for OpenStack- Fall 2012.pdf

Chef for Infrastructure Portability

• knife openstack

• knife hp

• knife rackspace

• knife ec2

• ... and many others

Page 49: Chef for OpenStack- Fall 2012.pdf

• Documentation

• Hypervisors (LXC, Hyper-V)

• Databases (PostgreSQL)

• Operating Systems (RHEL, Debian, SUSE)

• HA Configurations

• Quantum (pluggable)

• Cinder (pluggable)

• Community Events (NYC Nov 13)

Chef for OpenStack Roadmap

Page 50: Chef for OpenStack- Fall 2012.pdf

• Cookbooks reusable outside of OpenStack

• TestKitchen

• Librarian

• Spiceweasel

• pxe_dust

• knife-rackspace/hp/dreamhost

• Crowbar

Chef for OpenStack Ecosystem

Page 51: Chef for OpenStack- Fall 2012.pdf

Chef for OpenStack TL;DL

• Opscode.com/openstack

• Project, not a product

• Lots of contributors with real deployments

• Essex works, Folsom started

• Features driven by demand (show up for what you want)

• Documentation with examples

Page 52: Chef for OpenStack- Fall 2012.pdf

Thanks!

Matt [email protected]

IRC/Twitter/GitHub: mattraywww.opscode.com/openstack