Railsconf Virtualization 1212334054160801 9

Embed Size (px)

Citation preview

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    1/105

    Virtualization with

    Elastic Server On Demand

    Yan [email protected]

    http://elasticserver.com

    mailto:[email protected]://elasticserver.com/http://elasticserver.com/http://elasticserver.com/mailto:[email protected]:[email protected]
  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    2/105

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    3/105

    What is Virtualization

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    4/105

    What is Virtualization

    Why should I care?

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    5/105

    What is Virtualization

    Why should I care? How can I get started?

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    6/105

    Virtualization

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    7/105

    Virtualization

    Run multiple virtual computers on onephysical box.

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    8/105

    Virtualization

    Run multiple virtual computers on onephysical box.

    Desktop

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    9/105

    Virtualization

    Run multiple virtual computers on onephysical box.

    Desktop Server

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    10/105

    Virtualization

    Run multiple virtual computers on onephysical box.

    Desktop Server

    Data Center

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    11/105

    Virtualization

    Run multiple virtual computers on onephysical box.

    Desktop Server

    Data Center Cloud

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    12/105

    Lots of ways to do it

    Xen

    VMWare

    Parallels Amazon AMI

    Microsoft Hyper-V Clouds on the horizon

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    13/105

    Desktop

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    14/105

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    15/105

    Who Can Benefit?

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    16/105

    Who Can Benefit?Developers

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    17/105

    Who Can Benefit?

    Develop on a VM toavoid fubaring yourenvironment.

    Developers

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    18/105

    Who Can Benefit?

    Develop on a VM toavoid fubaring yourenvironment.

    Get new developers

    started quickly.

    Developers

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    19/105

    Who Can Benefit?

    Develop on a VM toavoid fubaring yourenvironment.

    Get new developers

    started quickly.

    Multi env. testing.

    Developers

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    20/105

    We're able to save time and money by having'standard' VMs for windows dev, rails dev,build machine, windows QA, etc. [...] It wasreally great when the customer handed overtwo VMs at the beginning of the project, andwe were able to just load up those VMs andstart our development.

    John McCaffrey

    Pathfinder (pathf.com)

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    21/105

    Server

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    22/105

    Who Can Benefit?

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    23/105

    Who Can Benefit?Software Vendors

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    24/105

    Who Can Benefit?

    Everyone knows how

    to run a Rails serverright?

    Software Vendors

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    25/105

    Who Can Benefit?

    Everyone knows how

    to run a Rails serverright?

    Absolutely NOT.

    Software Vendors

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    26/105

    Who Can Benefit?

    Everyone knows how

    to run a Rails serverright?

    Absolutely NOT.

    Ship it virtualized!

    Software Vendors

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    27/105

    Data Center

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    28/105

    Server Virtualization

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    29/105

    Server Virtualization

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    30/105

    Server Virtualization

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    31/105

    Server Virtualization

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    32/105

    You quickly forget about the virtualization except

    when you need it. When you need to migrate tonew hardware or increase RAM on a moment's

    notice. Ill never [again] install a Linux instance that

    doesn't start with Xen.

    Tom Mornini, EngineYard

    We provide you a product that is free fromtraditional hosting constraints. An object thatbecomes larger or smaller as needed, isduplicated at will and free to move around.

    Matt Tanase, SliceHost

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    33/105

    VMWare has over 20,000

    enterprise customers

    90% are in production

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    34/105

    Who Can Benefit?

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    35/105

    Who Can Benefit?Your Data Center

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    36/105

    Who Can Benefit?

    Save $$$ through

    increased utilization.

    Your Data Center

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    37/105

    Who Can Benefit?

    Save $$$ through

    increased utilization.Decrease IToverhead through

    encapsulation.

    Your Data Center

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    38/105

    Who Can Benefit?

    Save $$$ through

    increased utilization.Decrease IToverhead through

    encapsulation.Migrate slices andmanage resources.

    Your Data Center

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    39/105

    Who Can Benefit?

    Save $$$ through

    increased utilization.Decrease IToverhead through

    encapsulation.Migrate slices andmanage resources.

    Your Data Center

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    40/105

    Clouds

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    41/105

    http://www.flickr.com/photos/psd

    http://www.flickr.com/photos/psdhttp://www.flickr.com/photos/psd
  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    42/105

    Google

    AppEngine

    SliceHost Engine

    Yard

    Mosso

    FlexiScale

    Joyent

    Amazon

    EC2

    http://www.flickr.com/photos/psd

    http://www.flickr.com/photos/psdhttp://www.flickr.com/photos/psd
  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    43/105

    http://www.johnmwillis.com/groundwork/cloud-vendors-a-to-z/

    http://www.johnmwillis.com/groundwork/cloud-vendors-a-to-z/http://www.johnmwillis.com/groundwork/cloud-vendors-a-to-z/http://www.johnmwillis.com/groundwork/cloud-vendors-a-to-z/
  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    44/105

    http://www.johnmwillis.com/groundwork/cloud-vendors-a-to-z/

    http://www.johnmwillis.com/groundwork/cloud-vendors-a-to-z/http://www.johnmwillis.com/groundwork/cloud-vendors-a-to-z/http://www.johnmwillis.com/groundwork/cloud-vendors-a-to-z/
  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    45/105

    Cloud Approaches

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    46/105

    EngineYard - Managed Rails stack

    Cloud Approaches

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    47/105

    EngineYard - Managed Rails stack SliceHost - Choice of base OS images

    Cloud Approaches

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    48/105

    EngineYard - Managed Rails stack SliceHost - Choice of base OS images

    Google AppEngine - Python stack (for now)

    Cloud Approaches

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    49/105

    EngineYard - Managed Rails stack SliceHost - Choice of base OS images

    Google AppEngine - Python stack (for now) Amazon EC2 - self-service roll your own

    Cloud Approaches

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    50/105

    EngineYard - Managed Rails stack SliceHost - Choice of base OS images

    Google AppEngine - Python stack (for now) Amazon EC2 - self-service roll your own

    Private enterprise clouds...

    Cloud Approaches

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    51/105

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    52/105

    Animoto scales 50 slices to 3,500over 3 days with Amazon EC2

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    53/105

    Who Can Benefit?

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    54/105

    Who Can Benefit?Everyone

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    55/105

    Who Can Benefit?

    Your boss is risk averse.

    Everyone

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    56/105

    Who Can Benefit?

    Your boss is risk averse.

    Your IT doesnt know whatMongrel is.

    Everyone

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    57/105

    Who Can Benefit?

    Your boss is risk averse.

    Your IT doesnt know whatMongrel is.

    No money for hardware.

    Everyone

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    58/105

    Who Can Benefit?

    Your boss is risk averse.

    Your IT doesnt know whatMongrel is.

    No money for hardware.

    Cloudbursting.

    Everyone

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    59/105

    Who Can Benefit?

    Your boss is risk averse.

    Your IT doesnt know whatMongrel is.

    No money for hardware.

    Cloudbursting.

    Virtual test labs.

    Everyone

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    60/105

    Who Can Benefit?

    Your boss is risk averse.

    Your IT doesnt know whatMongrel is.

    No money for hardware.

    Cloudbursting.

    Virtual test labs.

    Run it in the cloud!

    Everyone

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    61/105

    Sweet! But...

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    62/105

    Making EC2 Images

    dd if=/dev/zero of=ubuntu.fs count=1024 bs=1M

    mke2fs -F -j ubuntu.fssudo mount -o loop ubuntu.fs /mntsudo debootstrap dapper /mntsudo cp /etc/apt/sources.list /mnt/etc/apt/sources.listsudo chroot /mnt

    now in the image:

    passwdaptitude updateaptitude upgradeaptitude install openssh-serveraptitude install [more packages ... ]cat > /etc/network/interfaces

    auto loiface lo inet loopback[hit ctrl^D]

    auto eth0iface eth0 inet dhcp

    sudo umount /mntec2-bundle-image -i ubuntu.fs -k [MYKEYFILE] -u [MYUSERID]ec2-upload-bundle -b my-ubuntu -m image.manifest -a [MYKEY] -s [MYSECRET]ec2-register my-ubuntu/image.manifest

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    63/105

    Making EC2 Images

    dd if=/dev/zero of=ubuntu.fs count=1024 bs=1M

    mke2fs -F -j ubuntu.fssudo mount -o loop ubuntu.fs /mntsudo debootstrap dapper /mntsudo cp /etc/apt/sources.list /mnt/etc/apt/sources.listsudo chroot /mnt

    now in the image:

    passwdaptitude updateaptitude upgradeaptitude install openssh-serveraptitude install [more packages ... ]cat > /etc/network/interfaces

    auto loiface lo inet loopback[hit ctrl^D]

    auto eth0iface eth0 inet dhcp

    sudo umount /mntec2-bundle-image -i ubuntu.fs -k [MYKEYFILE] -u [MYUSERID]ec2-upload-bundle -b my-ubuntu -m image.manifest -a [MYKEY] -s [MYSECRET]ec2-register my-ubuntu/image.manifest

    This is Hard,Annoying,and not

    Easily Reproducible

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    64/105

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    65/105

    A Job Opening?

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    66/105

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    67/105

    Making VMs Today

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    68/105

    Making VMs Today

    Create new VM, install new OS orstart with a base image

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    69/105

    Making VMs Today

    Create new VM, install new OS orstart with a base image

    Tweak, tweak, tweak, snapshot

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    70/105

    Making VMs Today

    Create new VM, install new OS orstart with a base image

    Tweak, tweak, tweak, snapshot

    Hopefully write down what you did

    and where its running

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    71/105

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    72/105

    What We Need

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    73/105

    What We Need

    Quick and reproducible VM provisioning

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    74/105

    What We Need

    Quick and reproducible VM provisioning

    Tracking contents and patches

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    75/105

    What We Need

    Quick and reproducible VM provisioning

    Tracking contents and patches Freedom of choice in hosting/cloud

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    76/105

    What We Need

    Quick and reproducible VM provisioning

    Tracking contents and patches Freedom of choice in hosting/cloud

    Testing locally, deploying globally

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    77/105

    What We Need

    Quick and reproducible VM provisioning

    Tracking contents and patches Freedom of choice in hosting/cloud

    Testing locally, deploying globally

    Manage running instances

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    78/105

    Elastic ServerOn Demand

    http://elasticserver.com

    http://elasticserver.com/http://elasticserver.com/
  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    79/105

    Elastic Server On Demand

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    80/105

    Elastic Server On Demand

    Virtualization manufacturing plant

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    81/105

    Elastic Server On Demand

    Virtualization manufacturing plant Component repositories

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    82/105

    Elastic Server On Demand

    Virtualization manufacturing plant Component repositories Agnostic about OS, virtualization, app stack

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    83/105

    Elastic Server On Demand

    Virtualization manufacturing plant Component repositories Agnostic about OS, virtualization, app stack Track server assemblies and instances

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    84/105

    Elastic Server On Demand

    Virtualization manufacturing plant Component repositories Agnostic about OS, virtualization, app stack Track server assemblies and instances

    Provisioning portals

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    85/105

    Elastic Server On Demand

    Virtualization manufacturing plant Component repositories Agnostic about OS, virtualization, app stack Track server assemblies and instances

    Provisioning portals Management console and API

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    86/105

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    87/105

    http://es.cohesiveft.com/site/rails2

    http://es.cohesiveft.com/site/rails2http://es.cohesiveft.com/site/rails2http://es.cohesiveft.com/site/rails2
  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    88/105

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    89/105

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    90/105

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    91/105

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    92/105

    Dev Tools

    VMWare Fusion/Player

    MacFUSE + MacFusion Elastic Server Capistrano Script

    http://es.cohesiveft.com/whitelabel/rails2/images/deploy.rb

    http://es.cohesiveft.com/whitelabel/rails2/images/deploy.rbhttp://es.cohesiveft.com/whitelabel/rails2/images/deploy.rbhttp://es.cohesiveft.com/whitelabel/rails2/images/deploy.rb
  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    93/105

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    94/105

    Adding CustomComponents

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    95/105

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    96/105

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    97/105

    Into The Clouds

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    98/105

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    99/105

    S

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    100/105

    Summary

    S

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    101/105

    Summary

    Save money in IT operations byconsolidating resources

    S

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    102/105

    Summary

    Save money in IT operations byconsolidating resources

    Increase dev agility through self-provisioning, reducing friction with ops

    S

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    103/105

    Summary

    Save money in IT operations byconsolidating resources

    Increase dev agility through self-provisioning, reducing friction with ops

    Use cloud resources to supplementinternal resources for traffic spikes(cloudbursting) and virtual test labs

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    104/105

    elasticserver.com

    U D

  • 8/3/2019 Railsconf Virtualization 1212334054160801 9

    105/105

    Usage Data