Embed Size (px)
DESCRIPTION
Presentation on topic of using virtualized environment in development and testing, especially for web applications. Prepared for Pycon Ukraine 2011
Citation preview
Virtual testing & development*
* batteries included
$ git clone git://server.com/project.git
$ run
Ideal world
$ git clone git://server.com/project.git
Real world
$ git clone git://server.com/project.git
... read help
$ mkdir, cp, vim config.conf, configure, make, make install
... ask guru
$ /project/hidden/blackmagic --fix_that --fix_this --undocumented_parameter=MAGIC_CONST_42
... %^ing @#$@#%@!!!!
Real world
$ git clone git://server.com/project.git
... read help
$ mkdir, cp, vim config.conf, configure, make, make install
... ask guru
$ /project/hidden/blackmagic --fix_that --fix_this --undocumented_parameter=MAGIC_CONST_42
... %^ing @#$@#%@!!!!
$ run
Real world
Why?
Operating System
User SpaceMusic
WebServer
DBAppServer
IMEditor
Browser
Problems
• No isolation (oh, wrong binary on path...?)
• Not repeatable (that README ain’t gonna run itself!)
• No guarantees (“..but it works on my computer!”)
Solution?
Operating System
Music
WebServer DBAppServer
IM Editor
BrowserUser Space
Virtualized OS
Benefits
• Isolation (project and version specific files only)
• Repeatable (snapshots, copies, automation)
• Guarantees (same OS different day)
Business benefits• Reduce costs (less hardware, lower energy
consumption)
• Backup and data protection
• Application availability
• Less time to respond to changing business needs
• Less time spent on routine tasks
• Better IT department perception
Time spend on routine tasks according to VMware small & medium business survey
Why haven’t we been doing this for a long time?
• Big companies have been
• Amazon EC2
• Microsoft
• Only recently possible on local machines
• Cheat hardware
• Cheap RAM
• Desktop virtualization API
Ok, let’s go virtual. Now what?
Vagrant is a tool for building and distributing virtualized development
environments (vagrantup.com)
VirtualBox is cross-platform virtualization software that works on all the major platforms
(virtualbox.org)
High level overview
• Describe environment with Vagrantfiles
• Command line interface to manage VM lifecycle
• Automate provisioning and configuration
• Share folder with host OS via NFS
• Provide SSH access to guest OS
• Network multiple VMs
• Package and distribute VMs
Vagrant boxes• Box = pre-packaged VM environment (~OS
image)
• A number of ready to use boxes @ http://vagrantbox.es
• Ubuntu 32 / 64
• CentOS
• Solaris
• RH
Vagrantfile• Vagrantfile is to Vagrant as Makefile is to
Make
• Describes VM in code
• One Vagrantfile per project (can contain multiple VMs)
• Commit it to version control
• Actually a Ruby code - easy to extent
Vagrantfile
Vagrant::Config.run do |config| config.vm.box = “base_ubuntu"end
Command line$ vagrant help
vagrant init vagrant up vagrant halt vagrant reload vagrant suspend vagrant resume vagrant destroy vagrant provision vagrant package …
• Isolation (project and version specific files only)
• Repeatable (snapshots, copies, automation)
• Guarantees (same OS different day)
Chef is an open-source systems integration framework built specifically for automating the cloud (opscode.com/
chef)
Puppet is a system for automating system administration tasks, works
on most UNIX-like operating systems (puppetlabs.com)
How Chef/Puppet works
• You create recipes
• Recipes describe desired system state
• Then you manage your server park
• Safely
• Repeatedly
• In automated way
Provisioning in Vagrant
• Use Chef, Puppet, bash scripts, etc. to manage VM configuration
• Repeatable!
• Use same tools for production - Chef, Puppet
Vagrant + Chef
Vagrant::Config.run do |config| config.vm.box = “base_ubuntu_box"
config.vm.provision :chef_solo do |chef| chef.recipe_url = "http://server/cookbooks.tar.gz"
chef.add_recipe(“aptitude") chef.add_recipe(“apache2") chef.add_recipe(“mysql-server") chef.add_recipe(“python27") chef.add_recipe(“pip") endend
Networking
• Assign IP to VM in Vagrantfile
• Forward ports
• Access VM from your browser
Networking
Vagrant::Config.run do |config| config.vm.box = "lucid32" config.vm.network("10.20.30.49") config.vm.forward_port("web", 80, 4567)end
• Isolation (project and version specific files only)
• Repeatable (snapshots, copies, automation)
• Guarantees (same OS different day)
Packaging
• When environment is stable
• Package and distribute prepared VMs
• Minimize setup time
• Isolation (project and version specific files only)
• Repeatable (snapshots, copies, automation)
• Guarantees (same OS different day)
Beyond that• Multi-VM Vagrantfile
• DB
• AppServer
• WebServer
• Use vagrant as library
• Call API functions to do the tasks from Ruby code
• Run custom SSH commands
Demo
github.com/dreamiurg/timetest
•Django application
•Apache2 + MySQL on single VM
•Additional Python packages like used (South, annoying)
•DB migrations handled by South
•Deployment process
• Vagrant is used to create VM, install required packages and configure services
• Fabric (fabfile.org) is used to populate DB, set up apache2 conf and install Python modules
Prerequisites
$ <install python>$ <install ruby>$ <install virtualbox>$ gem install vagrant
$ git clone http://server/project.git$ cd project
Demo: create VM, deploy project
00:00:00 $ vagrant box add base http://server/base.box
... base box for VM is copied to your computer
00:02:00 $ vagrant up
... vagrant creates VM from base box, installs and configures required packages – apt, apache2, mysql, python, git, …
00:05:00 $ fab deploy
... DB is populated with initial data, apache config updated, apache restarted
00:07:00 $ curl http://localhost:8080/
... site is up and running.
Real world
Our experience• 10+ projects during last 18 months
• From small (single webserver, no DB) to medium (several webservers + load balancer, master/slave DB replication)
• New developer start time reduced from 2-3 days to 1 hr on average
• Number of deployment/environment defects reduced by 75%
Links• http://vagrantup.com
• http://www.vagrantbox.es
• http://fabfile.org
• http://opscode.com
• http://puppetlabs.com
• http://github.com/dreamiurg/timetest
• http://www.vmware.com/files/pdf/VMware-SMB-Survey.pdf
