Migrating Chef (10 to Chef 11)

© 2015 Jochen Hahn

© 2014 ResMed I

Introduction

•  DevOps (a portmanteau of "development" and "operations") is a software development method that stresses communication, collaboration (information sharing and web service usage), integration, automation and measurement of cooperation between software developers and other information-technology (IT) professionals.[1][circular reference][2][3] DevOps acknowledges the interdependence of software development and IT operations. It aims to help an organization rapidly produce software products and services and to improve operations performance - quality assurance.

•  HI Health informatics (also called health care informatics, healthcare informatics, medical informatics, nursing informatics, clinical informatics, or biomedical informatics) is informatics in health care. It is amultidisciplinary field that uses health information technology (HIT) to improve health care via any combination of higher quality, higher efficiency (spurring lower cost and thus greater availability), and new opportunities.

•  CI/CD Continuous Delivery (CD) is a design practice used in software development to automate and improve the process of software delivery. Techniques such as automated testing and continuous integration allow software to be developed to a high standard and easily packaged and deployed to test environments, resulting in the ability to rapidly, reliably and repeatedly push out enhancements and bug fixes to customers at low risk and with minimal manual overhead.

Courtesy of Wikipedia

2

© 2014 ResMed I

Agenda

•  Intro •  Motivation •  Before you start •  Strategy •  Components •  Process •  Pitfalls •  Result

3

© 2014 ResMed I

Motivation

•  New features •  Performance •  Stability •  Dependency Management •  Development Environment •  Clean up •  Chef 10(.8) Bugs

•  Frequent upload errors -> do it again and it works •  Provider naming cannot handle ‘-’ in cookbooks •  CouchDB has no auto compacting •  Chef 10 installation procedure is complicated •  ….

4

© 2014 ResMed I

Scenario

5

© 2014 ResMed I

New Features

6

Chef  10   Chef  11   Chef  12  Database   CouchDB   Postgres/CouchDB   Postgres  

Output   Basic  text  and  log  levels  

Visually  enhanced  and  vastly  improved  

Same  as  Chef11  

Codebase   Ruby  based   Completely  refactored  wriBen  in  Erlang  

OpenSource  Codebase  with  Speed  improvments  up  to  20  %  

Chef-­‐client   Ruby  >=  1.8.5   Refactored  to  add  beBer  aBribute  access  from  e.g.  Roles  Ruby  >=  1.9.1    

Ruby  >=  2.0    

Knife  essenOals   Needed  plugin  install   Added  to  core   Same  as  Chef11  

Ohai   Nodes  take  precedence  

Ohai  has  highest  priority  

Same  as  Chef11    

InstallaOon   Relies  on  prerequisits  and  needs  plugin  or  gems  to  complete  installaOon  

Simplified:  one  bundle   Same  as  Chef11    

© 2014 ResMed I

Chef Server vs. Solo

•  Chef Server •  Allows for UI based access to all current configs, node values, environment

values, and cookbook implementations •  Allows for searches over groups •  Allows for control of server groups (e.g. roles and environments) •  No additional toolset necessary

•  Chef Solo •  Simplified deployments •  Use Vagrant, Bamboo, Jenkins, etc. to drop in chef solo •  no centralized server or dependency

7

vs.  

© 2014 ResMed I

Stakeholders

•  Marketing •  Product Owner – Marketing decides on Features of the Product. There interest lies in improving the

Product •  Project Management

•  Priority of the upgrade must be clear and agreed on •  IT

•  Get support from IT – Chef server needs maintenance and support •  Developers / Engineers / Tester / Legal

•  They will need consideration and need to be informed / coordinated with.

8

© 2014 ResMed I

Strategies

Preperation: Gather requirements and set Priorities •  What do you need implemented to get the most seamless transition •  How „big“ is your project •  What can you do first / What‘s „nice to have“ •  How many customization do you have •  Do you need to upgrade dependant tools •  Do you have a backup / Plan B •  Read as many docs / blogs / websites / notes as you can

Prototyping: •  You want to make sure that your current implementation still works after upgrading. E.g. Cookbook

compatibility changes, Ruby syntax changes, .... •  Get familiar with the new toolset •  Consider implementing your prototype to become the ‚real-deal‘

Training / Documentation: •  Who and how many people need to be onboard? •  What is their skillset •  What processes are going to change?

9

© 2014 ResMed I

Components and Dependencies

Chef-client •  Your nodes should be updated. Opscode mentioned that e.g. Chef-Client 10 is compatbile with

Chef-server 11. But there are loads of changes in the new Chef Client.

Chef-server •  This is Mandatory and comes as a bundle. Its actually harder to get rid of the old Chef 10 server

properly than to install the new one. Knife/ChefDK

•  The development environment needs to updated to match the new chef server and new functionality. Specifically since the knife essentials made it into the core

Ruby •  Chef 10 and 11 (and 12) have different requirements for ruby versions.

RubyDK •  Same as above: Concerns only the development environment

Serverspec •  If not running test-kitchen …

Vagrant / Dockar / Base image •  These are hopfully indépendant from a chef upgrade. But you never know …

10

© 2014 ResMed I

Process - Server

In place upgrade •  For upgrading the server just:

•  Backup old database •  take a VM Snapshot (if you can) •  Remove the old version (Chef 10) or just •  Upgrade/Install the new version •  Restore the database

•  Client updates are one line commands (e.g. run knife ssh against all your clients to download new version)

•  But: you still need to test in advance

Parallel setup and Incremental switchover •  Install a new Chef server •  Upload your nodes, roles, cookbooks, databags, environments •  Attach your nodes (re-bootstrap) •  Run chef-client

11

© 2014 ResMed I

Installing the Server

12

Installing Chef10 •  Yum install dependencies:

yum install gcc gcc-c++ automake autoconf make curl libtool xulrunner-devel libicu-devel openssl-devel curl-devel make gcc libtool which

•  Download and make install ruby •  Download and install rubygems •  Download and install CURL version

7.20.1 (pre-req for CouchDB) •  Download and install Erlang (pre-req for

CouchDB) •  Download, install and configure

CouchDB •  Download and install RabbitMQ •  Install Gecode (Dependency for Chef-

Server) and chef. Install chef-solr •  Install Chef Gems •  ....

Installing Chef11:

1. Download Chef server curl -o chef-server-11.1.4-1.el6.x86_64.rpm https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-server-11.1.4-1.el6.x86_64.rpm    

2. Install chef server sudo rpm -ivh chef-server-11.1.4-1.el6.x86_64.rpm 3. Configure chef server sudo chef-server-ctl reconfigure

© 2014 ResMed I

Process - Nodes

•  Remove folders

rm –rf /usr/local/bin/chef-client

rm –rf /etc/chef rm –rf /var/chef

•  Detach from Chef 10

knife client delete <node_name>

•  Re-bootstrap (install new ruby and chef client and attach to chef11)

knife bootstrap <fqdn> -N <fqdn> -d <distro> -x chefsvc --sudo --bootstrap-proxy <proxy>

13

© 2014 ResMed I

Distros

14

© 2014 ResMed I

Development Environment

•  ChefDK for Windows: Installation is easy – just download and install •  http://www.getchef.com/downloads/chef-dk/windows/

•  (Optional) Download and install Ruby: use the version that your ChefDK supports •  http://rubyinstaller.org/downloads/

•  (Optional) Download and install RubyDevKit •  https://github.com/downloads/oneclick/rubyinstaller/DevKit-tdm-32-4.5.2-20111229-1559-sfx.exe

•  Setup your chef-repo e.g.: git clone git://github.com/chef/chef-repo.git •  Setup your .chef directory

•  Knife.rb •  Validator.pem •  User.pem

•  Run knife node list from within your chef repo and get

15

© 2014 ResMed I

Pitfalls

•  General: •  Most traps can be found before starting any migration of “real” environments

or hosts systems due to the intensive testing •  Prep time (Prototyping) takes longer than expected

•  Development environment / CI system •  Dependency Management: There is no “easy” way of mapping cookbook

versions to a particular release version. Ø  Define cookbook version in your repository

•  Bootstrapping Clients •  Either use distro templates or get your OS image setup properly.

•  Foodcritic / Serverspec / Chefspec / testkitchen •  Testkitchen depends on vagrant / its own VM

16

© 2014 ResMed I

Results

•  With Chef 11 •  Deployment Stability •  Improved traceability •  Stable Chef UI

•  With ChefDK/Berkshelf •  Dependency management: download from repository and upload to Chef •  In combination with vagrant: local testing •  Automatically pushing to chef

•  The most to gain from the combination of tools •  Dependency management •  GIT source control •  CI auto push to chef •  CI testing •  Local testing •  Standardized development •  …

•  And personally: you can be the hero that saves the product (*hopefully*)

17

© 2014 ResMed I

Key topics to take away

•  Preparation is everything •  Know what you want to achieve. •  What is the result you expect. •  What benefits will you and your team gain. •  What's your timeframe

•  Involve the stakeholders

•  Make people aware of the risks that come along with and upgrade of larger magnitude (such as upgrading chef) and get their buy-in.

•  Test your implementation •  You don’t want everything to go

18

© 2014 ResMed I

Thanks for listening

Thanks for listening

19

© 2014 ResMed I

References

•  Chef 11 Breaking Changes: •  https://www.chef.io/blog/2013/02/04/chef-11-released/

•  Chef 12: •  https://www.chef.io/blog/2014/09/07/chef-12-release-candidate/

•  Migrating to Chef 11: •  https://www.chef.io/blog/2013/03/12/5106/ •  http://www.ducea.com/2013/03/05/howto-migrate-to-chef-11/

•  Migrating to Chef 12: •  https://docs.chef.io/upgrade_server.html

•  Chef docs and infos: •  http://docs.chef.io/ •  http://tcotav.github.io/chefdk_getting_started.html •  http://foodfightshow.org/

•  ChefDK: •  https://docs.chef.io/install_dk.html

20