php[architect] Summit Series DevOps 2013 - Rock solid deployment of PHP apps

Pablo Godel @pgodelhttp://summits.phparch.com/http://joind.in/8966

Rock Solid Deploymentof PHP Applications

Tuesday, July 16, 13

Who Am I?

⁃ Born in Argentina, living in the US since 1999⁃ PHP & Symfony developer

⁃ Founder of the original PHP mailing list in spanish ⁃ Co-founder of ServerGrove

Tuesday, July 16, 13

Deployment

?Tuesday, July 16, 13

Deployment

Software deployment is all of the activities that makea software system available for use.

http://en.wikipedia.org/wiki/Software_deployment

Tuesday, July 16, 13

Deployment

A very important part of the application life-cycle

Tuesday, July 16, 13

Deployment

A very important critical part of the application life-cycle

Tuesday, July 16, 13

Deployment

It should not be an after thought

Tuesday, July 16, 13

Deployment

It should be predictable

Tuesday, July 16, 13

Deployment

The more you do it the better it goes

Tuesday, July 16, 13

Tuesday, July 16, 13

Deployment: Goals

Tuesday, July 16, 13

Deployment: Goals

One-click deploys

Tuesday, July 16, 13

Continuous deploys

Deployment: Goals

Tuesday, July 16, 13

Web AppsDeployment

Tuesday, July 16, 13

Web AppsDeployment

Tuesday, July 16, 13

Web AppsDeployment

Tuesday, July 16, 13

Anytime & Anywhere

Deployment: Goals

Tuesday, July 16, 13

Anyone

Deployment: Goals

Tuesday, July 16, 13

Reliable

Deployment: Goals

Tuesday, July 16, 13

Rollbacks

Deployment: Goals

Tuesday, July 16, 13

No downtime

Deployment: Goals

Tuesday, July 16, 13

Reusable

Deployment: Goals

Tuesday, July 16, 13

Scalable

Deployment: Goals

Tuesday, July 16, 13

• One-click / continuous deploys• Anytime & Anywhere• Anyone • No downtime• Predictable & Reliable• Rollbacks• Reusable• Scalable

Deployment: Goals

Tuesday, July 16, 13

Deployment Facts

Tuesday, July 16, 13

Deployment: Fact #1

Deployment starts with the developer

• Setup development environment to be as close as possible to productions servers

• Setup test/qa/staging servers• Use Vagrant to manage VMs• Use Puppet/Chef to manage OS packages/

configuration

Tuesday, July 16, 13

Deployment: Fact #2

Success linked to server OS setup

• Use Puppet/Chef to manage OS packages/configuration

• Create OS packages for 3rd party software• Setup your own package repositories

Tuesday, July 16, 13

Deployment: Fact #3

Monitoring is uptime

• Use monitoring tools to know what is going on with your servers (Ganglia, Cacti, Zabbix, etc.)

• Add monitoring and metrics to your app (Graphite, StatsD, New Relic)

• Use your logs wisely (Graylog, Logstash, Kibana)

Tuesday, July 16, 13

Deployment Methodologies

Tuesday, July 16, 13

Deployment Methodologies

• VIM-style• FTP uploads• rsync• source control (svn, git)• Build tools (ant, phing)• Specialized tools (capistrano, fabric, etc)• Package based (rpm, deb, etc)

Tuesday, July 16, 13

Web Apps Deployment: Steps overview

Tuesday, July 16, 13

Web Apps Deployment: First time

• Copy files to server(s)• Set server-side configurations• Load DB fixtures• Process and install assets• Warm up cache• “Enable” site

Tuesday, July 16, 13

• Copy files to server(s)• Apply DB updates (migrations)• Process and install assets• Warm up cache• “Enable” site

Web Apps Deployment: Subsequent times

Tuesday, July 16, 13

Deployment: Challenges

Tuesday, July 16, 13

Deployment: Challenges

• rsync• git pull• setup git repo on local network to save

bandwidth and avoid issues if git server is down (i.e. github)

Challenge:Fast & reliable copy of files

Solutions:

Tuesday, July 16, 13

Deployment: Challenges

• use a tool that allows to go from 1 to n servers easily (i.e. capistrano)

• pssh allows to send commands to n servers in parallel

• package your app in OS packages like .rpm/.deb to easily install across n servers

Challenge:Scalable

Solutions:

Tuesday, July 16, 13

Deployment: Challenges

• test!• tag releases• dedicated branches (master for production)• deploy each release in its own directory

Challenge:Rollbacks

Solutions:

Tuesday, July 16, 13

Deployment: Challenges

• use ssh based connections• don’t store passwords on source control• store sensitive strings (passwords) in server

environment variables

Challenge:Secure

Solutions:

Tuesday, July 16, 13

Deployment: Challenges

Challenge:DB migrations

Solutions:• Doctrine Migrations• Consider document oriented DBs like

MongoDB

“The best migrations are the ones you don’t have to do”

Tuesday, July 16, 13

Deployment: Challenges

Challenge:Static assets

Solutions:• YUICompress shrinks JS and CSS file sizes• Enable web server compression• Add versioning to static assets links (code.js?v=1)• Assetic combines multiple files into one• Run utilities locally or in a staging server, deploy

resultTuesday, July 16, 13

Deployment: Challenges

Challenge:Caching

Solutions:• Update one server while others handle load• Group servers and update group at a time• execute commands on “finalize” to clear up APC

cache

Tuesday, July 16, 13

Deployment: Challenges

Challenge:File permission conflicts

Solutions:• Run Apache/PHP with same user• Use php-fpm instead of mod_php• Create “deploy” user and add web server to the

group• Use setfacl to give write access to multiple users

Tuesday, July 16, 13

Web Apps Deployment: Other common pitfalls

Tuesday, July 16, 13

Web Apps Deployment: Other common pitfalls

• Case sensitive filesystems• Configuration differences• Outdated 3rd party software• Github down

$ git daemon --base-path=/git/repo/path/ --export-all

$ git clone git://127.0.0.1/repo

http://ozmm.org/posts/when_github_goes_down.htmlTuesday, July 16, 13

Web Apps Deployment: Examples

Tuesday, July 16, 13

Web Apps Deployment: Examples

Simplest continuous deployment ever!

<?php

exec(‘/usr/bin/env -i HOME=/var/www git pull’);echo “All done!”;

hook.php

screenshot

Tuesday, July 16, 13

Web Apps Deployment: Examples

Capistrano

• Ruby based• Very extensible• Large number of extensions• Simple client side installation

$ gem install capistrano

Tuesday, July 16, 13

Web Apps Deployment: Examples

Capistrano

set :application, "myapp" # Application nameset :deploy_to, "/var/www/myapp"

set :user, "deployer"set :use_sudo, false # sudo isn't required

set :deploy_via, :remote_cache set :repository, "git@github.com:user/repo.git"

role :web, "server.example.com", “server2.example.com”

Tuesday, July 16, 13

Web Apps Deployment: Examples

$ cap deploy:setup

Capistrano

Tuesday, July 16, 13

Web Apps Deployment: Examples

|-- releases`-- shared |-- logs `-- uploads

Capistrano

Tuesday, July 16, 13

Web Apps Deployment: Examples

Capistrano

$ cap deploy$ cap deploy:migrations$ cap deploy:rollback

Tuesday, July 16, 13

Web Apps Deployment: Examples

|-- current (symlink to releases/20130112)|-- releases| `-- 20130112`-- shared |-- logs `-- uploads

Capistrano

Tuesday, July 16, 13

Web Apps Deployment:Other options

• Fabric• Phing• Jenkins• WePloy

https://github.com/rlerdorf/WePloy• Magallanes

http://magallanes.zenreworks.com/• Idephix

http://getidephix.com/

Tuesday, July 16, 13

Web Apps Deployment: Tools

Tuesday, July 16, 13

Web Apps Deployment: Tools

App Metrics: StatsD & Graphite

Tuesday, July 16, 13

Web Apps Deployment: Tools

Logging: LogstashShip logs from any source, parse them, get the right timestamp, index them, and search them

Tuesday, July 16, 13

Web Apps Deployment: Tools

Logging: Logstash

Configure Apache to log jsonLogFormat "{ \"@timestamp\": \"%{%Y-%m-%dT%H:%M:%S%z}t\", \"@fields\": { \"client\": \"%a\", \"duration_usec\": %D, \"status\": %s, \"request\": \"%U%q\", \"method\": \"%m\", \"referrer\": \"%{Referer}i\" } }" logstash_json

# Write our 'logstash_json' logs to logs/access_json.logCustomLog logs/access_json.log logstash_json

{ "@timestamp": "2012-08-22T14:35:19-0700", "client": "127.0.0.1", "duration_usec": 532, "status": 404, "request": "/favicon.ico", "method": "GET", "referrer": "-" }

Result

Tuesday, July 16, 13

Web Apps Deployment: Tools

Logging: Graylog

Tuesday, July 16, 13

Web Apps Deployment: Tools

Logging: KibanaKibana is a user friendly way to view, search and visualize your log data

Tuesday, July 16, 13

Web Apps Deployment: Tools

Packaging: fpm

https://github.com/jordansissel/fpm

Build packages for multiple platforms (deb, rpm, etc) with great ease and sanity.

fpm -s dir -t rpm -n "myapp" -v 1.0 /var/www/myapp

fpm -s dir -t deb -a all -n myapp -v 1.0 /etc/apache2/conf.d/my.conf /var/www/myapp

Tuesday, July 16, 13

Web Apps Deployment: Summary

Tuesday, July 16, 13

•Stop using FTP

Tuesday, July 16, 13

•Stop using FTP•Plan early

Tuesday, July 16, 13

•Stop using FTP•Plan early•Practice

Tuesday, July 16, 13

•Stop using FTP•Plan early•Practice•Monitor

Tuesday, July 16, 13

•Stop using FTP•Plan early•Practice•Monitor•AUTOMATE!

Tuesday, July 16, 13

QUESTIONS?

Slides: http://slideshare.net/pgodelTwitter: @pgodel

E-mail: pablo@servergrove.com

Tuesday, July 16, 13

Thank you!

Slides: http://slideshare.net/pgodelTwitter: @pgodel

E-mail: pablo@servergrove.com

http://joind.in/8966

Tuesday, July 16, 13