Devops – The Last Mile

Devops – The Last Mile

Workspace Management

Jay Flowers

jay.flowers@gmail.comhttp://jayflowers.com

Workspace Management

The act of defining, creating, and maintaining the software tool stack that comprises a

developer’s or tester’s computer configuration.

Why?

Problem?Why do we need Workspace

Management?Do we have a problem?

As workspace configurations drift the more pervasive and sever “works on my box” syndrome is encountered.They drift from each other as well

as away from production.

The Norm• Manually• Tribal knowledge• Some documentation, generally out of date

• Lots of options, room for variation

• Unreliable results• Difficult to reproduce same results

The Goal• Fully automated• Infrastructure as code• Institutionalized knowledge• Code as documentation• No options• Tested• Reliable, repeatable results

Needs• We need a way to unobtrusively manage all the developer/tester workspaces on a project

• We need all the software and configuration that makes up the workspace to be centrally managed

• We need all the workspaces to look and function identically

• We prefer to use the same tool to manage workspaces as we use to manage servers

Chef

• Automation for installing and configuring software

• Open Source• Supports Windows, OSx, and Linux• Why Chef and not Puppet?

– Internal DSL, more powerful than Puppet’s external DSL– 100% free, Puppet is only partially free

• Can bootstrap to create setup application• System level CM• Testable with Vagrant or EC2

– Higher quality from agile testing

DEMOHTTP://JAYFLOWERS.COM/MISC%20DOWNLOADS/WORKSPACE-SETUP.EXE

• Server/Client– Secure communication, authentication,

and authorization with public/private keys

– Servers control execution of clients• Solo– Client without a server– Execution is initiated at the command

line on the client

Chef Modes

When to use Solo and when to use Server/Client

• Solo for workspace management– Updates controlled by workspace owner, on

demand– Cookbooks stored in workspace

• Server/Client for everything else– Update controlled by server, pushed to clients– Cookbooks stored on server

• We can use Chef to deploy the applications we build!

Chef Anatomy

• Server/Client or Solo• Cookbooks (written to be idempotent)

– Resources and Providers – used to perform typical actions (make user, create directory, install package, etc…)

– Attributes – parameters specific to the cookbook• Nodes

– The client list of cookbooks to apply• Roles

– Lists of cookbooks to be applied– Helpful in organizing nodes into groups (e.g. frontend apache

server)• Databags

– Parameters specific to the target nodes and or organization

Bootstrap Solo Configurationrequire 'minitest-chef-handler'

file_cache_path "c:/tools/chef/cache"cookbook_path [File.dirname(__FILE__) + "/Chef/cookbooks"]file_backup_path "c:/tools/chef/backup"role_path "c:/tools/chef/roles"

json_attribs File.dirname(__FILE__) + "/node.json"

handler = MiniTest::Chef::Handler.new(:path => File.join(Chef::Config[:cookbook_path], "*", "test", "*test*.rb"))report_handlers << handler

log_level :debuglog_location STDOUT

Node.json{ "run_list": [ "recipe[git]", "recipe[tortoisegit]", "recipe[get_git_source]", "recipe[springsource_sts]" , "recipe[java]", "recipe[sysinternals]", "recipe[virgo]", "recipe[gradle]",

"recipe[notepadplusplus]","recipe[maven]","recipe[groovy]"

]}

Java Cookbook

windows_package "java" do source node['java']['url'] checksum node['java']['checksum'] action :install installer_type :custom options "/s /v \"/qn INSTALLDIR=#{node['java']['java_home']}\"" not_if { File.directory? node['java']['java_home'] }end

Git Clone Cookbook

dir = node['get_git_source']['dir']

directory dir do inherits true recursive true action :createend

Git Clone Cookbookruby_block "clone_git_repo" do block do url = node['get_git_source']['url']

Chef::Log.info("Cloning the repo #{url} to directory #{dir}, this could take a while...")

git = Chef::ShellOut.new("git clone #{url} #{dir}")git.run_commandputs git.stdoutif git.stderr != '' puts "error messages: " + git.stderrend# Raise an exception if it didn't exit with 0git.error!

end action :create

Git Clone Cookbook not_if do

Chef::Log.info("Checking if #{dir} is a working git repo") git = Chef::ShellOut.new("git status", :cwd => dir)

git.run_commandputs git.stdoutif git.stderr != '' puts git.stderrendif git.exitstatus == 0trueelsefalseend

endend

• Written to easily manage large numbers of servers, translates to workspaces too

• Installs and configures software and operating systems creating reliable and repeatable results

• Creation of new servers and workspaces becomes so easy allowing us to treat them as disposable resources

• Can be used to repair/heal broken workspaces and servers

Chef Foots the Bill

Benefits• What works on my box works everywhere else…

• Dramatically reduced time to create new workspace

• Workspace updates are non-events• Heal broken workspace• Can be managed by unskilled team members

• Pairs well with other scripting (e.g. automated deployments)

Cucumber• Test Tool• Open Source• Tests are written in plain text• Tests are implemented in Ruby, the same language as Chef is implemented in…

• Tests use virtualization libraries to create, control, and destroy test instances/environments

Simple Example

Cucumber Example

Minitest Example

Test Driven• These tools enable a test driven approach

• Tests can be written before Chef cookbooks are written

• Tests can be executed in a local workspace as well as on a CI server

Test Resources• ChefSpec - https://github.com/acrmp/chefspec

• MiniTest - https://github.com/calavera/minitest-chef-handler

• Test Kitchen - https://github.com/opscode/test-kitchen

• Examples– https://github.com/opscode-cookbooks/ap

ache2/

– https://github.com/opscode-cookbooks/mysql/

A LOOKUNDER THE HOOD

HTTPS://GITHUB.COM/JFLOWERS/VIRGO.SAMPLE-GREENPAGES