TXLF: Automated Deployment of OpenStack with Chef

  • Published on
    08-May-2015

  • View
    4.027

  • Download
    3

Embed Size (px)

DESCRIPTION

Automated Deployment of OpenStack with Chef talk given April 2, 2011 at the Texas Linux Fest.

Transcript

  • 1.Automated Deploymentof OpenStack with ChefTexas Linux FestApril 2, 2011 1

2. IntroductionsMatt RaySenior Technical Evangelistmatt@opscode.com@mattrayGitHub:mattray2 3. What is OpenStack? 3 4. Founders operate atmassive scale NASA 4 5. OpenStack: The Mission "To produce theubiquitousOpen Source cloud computing platform that will meet the needs of public and private cloud providers regardless of size, by being simple to implement and massively scalable."5 6. OpenStack Founding PrinciplesApache 2.0 license (OSI), open development processOpen design process, 2x year public DesignSummitsPublicly available open source code repositoriesOpen community processes documented andtransparentCommitment to drive and adopt open standardsModular design for deployment exibility via APIs 6 7. Community with Broad Support 7 8. Software to provision virtual machines onstandard hardware at massive scaleOpenStack Computecreating open source softwareto build public and privatecloudsSoftware to reliably store billions of objectsdistributed across standard hardware OpenStackObject Storage 8 9. OpenStack Compute Key Features ReST-based API Asynchronous eventually consistentcommunicationHorizontally andmassively scalableHypervisor agnostic:support for Xen ,XenServer, Hyper-V,KVM, UML and ESX is coming Hardware agnostic: standard hardware, RAID not required9 10. User ManagerCloud Controller: Global state ofsystem, talks to LDAP, OpenStackObject Storage, and node/storageworkers through a queueATAoE / iSCSIAPI: Receives HTTP requests,converts commands to/from APIformat, and sends requests to cloudcontrollerHost Machines: workersthat spawn instancesGlance: HTTP + OpenStack ObjectOpenStack Compute Storage for server images 10 11. Hardware Requirements OpenStack is designed to run on industry standard hardware, with flexible configurations Compute x86 Server (Hardware Virt. recommended) Storage flexible (Local, SAN, NAS) Object Storage x86 Server (other architectures possible) Do not deploy with RAID (can use controller for cache)11 12. Why is OpenStack important? Open eliminates vendor lock-in Working together, we all go faster Freedom to federate, or movebetween clouds12 13. What is Chef?13 14. Chef enables Infrastructure as Code Manage conguration as idempotent Resources. Put them together in Recipes. Track it like Source Code. Congure your servers. 14 15. At a High Level Library for conguration management Conguration management system Systems integration platform API for your entire Infrastructure 15 16. Fully automated Infrastructure16 17. Principles Idempotent Data-driven Sane defaults Hackability TMTOWTDI 17 18. Open Source and CommunityApache 2 licensedLarge and active communityOver 300 individual contributors(60+ corporate)Community is Important! 18 19. 19 20. How does it Work?20 21. How does it Work?Miracles!21 22. How does it Work?Miracles! (no really)22 23. Chef Client runs onyour System23 24. Chef Client runs onyour System ohai!24 25. Clients talk to the Chef Server25 26. The Opscode Platformis a hosted Chef Server26 27. We call each system you congure aNode27 28. Nodes have Attributes{"kernel": { "machine": "x86_64", "name": "Darwin", Kernel info! "os": "Darwin", "version": "Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010;root:xnu-1504.7.4~1/RELEASE_I386",}, "release": "10.4.0" Platform info!"platform_version": "10.6.4","platform": "mac_os_x","platform_build": "10F569","domain": "local","os": "darwin","current_user": "mray","ohai_time": 1278602661.60043,"os_version": "10.4.0", Hostname and IP!"uptime": "18 days 17 hours 49 minutes 18 seconds","ipaddress": "10.13.37.116","hostname": "morbo","fqdn": "morbomorbo.local","uptime_seconds": 1619358} 28 29. Nodes have a Run ListWhat Roles and Recipesto Apply in Order 29 30. Nodes have Roleswebserver, database, monitoring, etc.30 31. Roles have a Run ListWhat Roles and Recipesto Apply in Order 31 32. name "webserver"description "Systems that serve HTTP traffic"run_list("role[base]","recipe[apache2]","recipe[apache2::mod_ssl]")default_attributes("apache" => {"listen_ports" => [ "80", "443" ]})override_attributes("apache" => {"max_children" => "50"})3232 33. name "webserver"description "Systems that serve HTTP traffic"run_list("role[base]", Can include"recipe[apache2]", other roles!"recipe[apache2::mod_ssl]")default_attributes("apache" => {"listen_ports" => [ "80", "443" ]})override_attributes("apache" => {"max_children" => "50"})3232 34. Chef managesResources on Nodes 33 35. Resources Declare a description of the state a part of the node should be in34 36. Resources package "apache2" do version "2.2.11-2ubuntu2.6" action :install end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Declare a description of the state a part of the node should be in 34 37. Resources Have a type package "apache2" do version "2.2.11-2ubuntu2.6" action :install end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Declare a description of the state a part of the node should be in 34 38. Resources Have a type package "apache2" do version "2.2.11-2ubuntu2.6" action :install Have a name end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Declare a description of the state a part of the node should be in 34 39. Resources Have a type package "apache2" do version "2.2.11-2ubuntu2.6" action :install Have a name end template "/etc/apache2/apache2.conf" do Have parameters source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Declare a description of the state a part of the node should be in 34 40. Resources Have a typepackage "apache2" doversion "2.2.11-2ubuntu2.6"action :install Have a nameendtemplate "/etc/apache2/apache2.conf" do Have parameterssource "apache2.conf.erb"owner "root" Take action to put the group "root"mode 0644 resource in theaction :create declared state endDeclare a description of the state a part of the node should be in34 41. Resources take actionthrough Providers35 42. Recipes are lists of Resources 36 43. Recipes 1 package "apache2" do version "2.2.11-2ubuntu2.6" action :install endEvaluate and apply template "/etc/apache2/apache2.conf" doResources in the order source "apache2.conf.erb" owner "root"they appeargroup "root" mode 0644 action :create2 end 37 44. Order Matters38 45. Recipes are just Ruby!extra_packages = case node[:platform]when "ubuntu","debian"%w{ruby1.8ruby1.8-devrdoc1.8ri1.8libopenssl-ruby}endextra_packages.each do |pkg|package pkg doaction :installendend39 46. Cookbooks arepackages for Recipes 40 47. CookbooksDistributable, shareablecomunity.opscode.comInfrastructure as CodeVersionedHundreds 41 48. CookbooksRecipesFilesTemplatesAttributesMetadata 42 49. Data bags store arbitrary data43 50. A user data bag item...% knife data bag show users mray{"comment": "Matt Ray","groups": "sysadmin","ssh_keys": "ssh-rsa SUPERSEKRATS mray@morbo","files": { ".bashrc": {"mode": "0644","source": "dot-bashrc" }, ".emacs": {"mode": "0644","source": "dot-emacs" }},"id": "mray","uid": 7004,"shell": "/usr/bin/bash"} 44 51. Environments manageversioned infrastructure 45 52. Command-line APIutility, Knife http://www.ickr.com/photos/myklroventine/3474391066/ Copyright 2011 Opscode, Inc - All Rights Reserved 46 46 53. Search $ knife search node platform:ubuntu CLI or in Ruby search(:node, platform:centos) Nodes are searchable $ knife search role max_children:50 Roles are searchable search(:role, max_children:50) Recipes are$ knife search node role:webserver searchablesearch(:node, role:webserver) Data bags are$ knife users shell:/bin/bash searchablesearch (:users, group:sysadmins) 47 54. 48 55. HOW TO: Turn Racks ofStandard Hardware Into aCloud with OpenStack 49 56. What Works Today?50 57. Compute (Nova)Single machine installation Role: nova-single-machine MySQL, RabbitMQ Nova-(api|scheduler|network|objectstore|compute)Multi-machine Role: nova-multi-controller (1) Role: nova-multi-compute (N)51 58. Role: nova-single-machinename "nova-single-machine-install"description "Installs everything required to run Nova on a singlemachine"run_list("role[nova-multi-controller]","role[nova-multi-compute]")52 59. Role: nova-multi-controllername "nova-multi-controller"description "Installs requirements to run the Controller node in aNova cluster"run_list("role[nova-support-server]","role[nova-head]","role[nova-cloud-controller]","role[nova-super-user-setup]") 53 60. Role: nova-multi-computename "nova-multi-compute"description "Installs requirements to run a Compute node in a Novacluster"run_list("recipe[nova::compute]") 54 61. What does this look like?55 62. Crowbar Codename for the OpenStack installer from Dell Dell is releasing this under theApache 2 license Extension of the Chef server Jointly developed by Dell,Rackspace and Opscode56 63. Crowbar - What does it Do? Crowbar is a PXE state machine starts with bare metal hardware manages and congures BIOS and network settings network boot and installation nodes are congured with Chef deploys OpenStack, could be used for anything57 64. OpenStack Installation Cookbooks uploaded $ $ knife cookbook upload -a knife cookbook list $ rake roles Roles uploaded $ knife role list $ knife node list Nodes ready58 65. AMIsname "nova-ami-urls"description "Feed in a list URLs for AMIs to download"default_attributes("nova" => {"images" =>["http://192.168.11.7/ubuntu1010-UEC-localuser-image.tar.gz]})$ knife role from file roles/nova-ami-urls.rb Use an existing AMI Update URL to your own59 66. Assign the Roles$ knife node run_list add crushinator.localdomain "role[nova-ami-urls]"{"run_list": ["role[nova-ami-urls]"]}$ knife node run_list add crushinator.localdomain "role[nova-single-machine-install]"{"run_list": ["role[nova-ami-urls]""role[nova-single-machine-install]",]} 60 67. chef-clientmray@ubuntu1010:~$ sudo chef-client[Fri, 25 Feb 2011 11:52:59 -0800] INFO: Starting Chef Run (Version0.9.12)...[Fri, 25 Feb 2011 11:56:05 -0800] INFO: Chef Run complete in5.911955 seconds[Fri, 25 Feb 2011 11:56:05 -0800] INFO: cleaning the checksum cache[Fri, 25 Feb 2011 11:56:05 -0800] INFO: Running report handlers[Fri, 25 Feb 2011 11:56:05 -0800] INFO: Report handlers complete61 68. The Moment of Truthnova@$ nova-manage service listnova@$ euca-describe-imagesnova@$ euca-run-instances ami-h8wh0j17 -k mykey -t m1.tinynova@$ euca-describe-instancesnova@$ ssh -i mykey.priv ubuntu@10.0.0.2Linux i-00000001 2.6.35-24-virtual #42-Ubuntu SMP Thu Mar 3005:15:26 UTC 2011 x86_64 GNU/LinuxUbuntu 10.10Welcome to Ubuntu!See "man sudo_root" for details.ubuntu@i-00000001:~$ 62 69. How Did We Get Here? 63 70. Forked from Anso Labs CookbooksBootstrapped by OpscodeChef Solo/Vagrant installs for Developershttp://github.com/ansolabs/openstack-cookbooks 64 71. Whos involvedso far? 65 72. Whats Next? 66 73. Nova needed enhancements Pluggable/Modular Roles Database ObjectStore Network Virtualization Swift and Glance integration67 74. 68 75. Dashboard69 76. Knife http://github.com/opscode/knife-openstack Nova has same API as Amazon Fog supports OpenStack already knife openstack server create role [base] -i ami-a403f6xd -f m1.micro - A OpenStack instance70 77. Object Storage (Swift) Recipes originated from Anso Labs repository Will be managed with Chef and Crowbar Included in the bexar branch Untested so far (Cactus will tackle) 71 78. Image Registry (Glance) Recipes originated from Anso Labs repository Will be managed with Chef and Crowbar Included in the bexar branch Untested so far (Cactus!)72 79. Scaling changes howwe deploy OpenStack! 73 80. Deployment Scenarios Single machine is special case of multi-install Controller + Compute nodes is a known quantityfor small installations Nova + Swift + Glance in large installations Services separated and HA congurationssupported Documentation and Chef Roles will be thesolution 74 81. Cactus, Diablo, ...Development continues...Branches for each stable releaseDesign Summit later this monthDesign Summit in the Fall 75 82. Rackspace Cloud BuildersCommercial support and Training forOpenStack Opscode Dell Equinix Cloudscaling Citrix76 83. Get Involved!https://github.com/mattray/openstack-cookbooks/tree/bexarhttp://lists.openstack.orghttp://lists.opscode.com#chef on irc.freenode.net#openstack on irc.freenode.netmatt@opscode.comjordan@openstack.com77