ChefConf 2013 Talk: Writing a Knife cloud plugin for your shiny VMWare vCloud Director

Writing a Knife Plugin for your shiny new VMWare vCloud

Director

Chirag Jog, CTO [email protected]

Innovation → Execution → Solution → Delivered

mailto:[email protected]

http://clogeny.com/

http://www.clogeny.com © 2012 Clogeny Technologies

About Me

Built cutting edge products:• Hybrid Cloud

Migration Appliances• Enterprise Java PaaS• Multi-cloud single

pane of glass Dashboard

Contributed to several knife plugins• VMWare vCloud

Director• Google Compute

Engine• Windows Azure• Citrix Cloudstack• Terremark• Amazon EC2

Co-founder/CTO at Clogeny

http://www.clogeny.com/

http://www.linkedin.com/company/clogeny

http://twitter.com/clogeny

http://blogs.clogeny.com/


Agenda

What is the knife plugins ?

The Knife cloud plugin framework

VMWare vCloud Director API Model

Code Walkthrough of the knife-vcloud

plugin

Demo of the knife vcloud plugin






What are knife cloud plugins ?

Extend Knife to support additional functionality

Virtual Machine Life cycle Server Create

Linux and Windows Bootstrapping

Server Destroy

List available cloud resources Images

Flavors / Server Types

Networks

Servers






Available Cloud Plugins






Knife Plugin Usage

gem install knife-<cloudname>






Knife Plugin Usage

gem install knife-<cloudname>knife <cloudname> server create






Knife Plugin Usage

gem install knife-<cloudname>knife <cloudname> server create• Flavor or Server size• Image• Zone/Region• Template/Distro• Chef Node Name• SSH params – ssh username, password, keypair,

keypair file• WinRM params – username, password,

AD/Kerberos params• Cloud Vendor Credentials






Knife Plugin Usage

knife <cloudname> server delete• Chef Node Name• Purge Node/Client

knife <cloudname> server listknife <cloudname> image listknife <cloudname> flavor list






High Level Diagram

...

knife-ec2

knife-rackspace

Knife-vcloud

….

AWS Driver

Rackspace Driver

Fog library

vCloud Driver






High Level Diagram

...

knife-ec2

knife-rackspace

Knife-vcloud

….

AWS Driver

Rackspace Driver

Fog library

vCloud Driver






fog.io

Knife Cloud plugins use fog to communicate with the Cloud services






Fog

Fog - Multi-Cloud Services Library

Open source, Community driven

Portable, Powerful, Established

Simulated Fog behavior using Mock

Organizes resources – servers, images,

networks, vapps as collection of models.






Using Fog

$ gem install fog

Fetching: fog-1.10.1.gem (100%)

Successfully installed fog-1.10.1

1 gem installed






Using Fog Interactive

$ fog Welcome to fog interactive!>> connection = Fog::Compute.new(:provider => <Cloud Provider>, credential params..) >> images = connection.images.all[<Fog::Compute::Cloud::Image id=…, name=“ubuntu”>, <Fog::Compute::Cloud::Image id=…, name=“centos”>…]>> servers = connection.servers.all>> server = connection.servers.create(server_def_params)



























High Level Diagram

...

knife-ec2

knife-rackspace

Knife-vcloud

….

AWS Driver

Rackspace Driver

Fog library

vCloud Driver






Knife plugin – Fog collections

knife-cloud

knife cloud server create

knife cloud server delete

knife cloud server list

knife cloud flavor list

knife cloud image list

servers

images

flavors

Fog Collections







knife-cloud






servers

images

flavors

Fog Collections






knife cloud server createclass Chef class Knife class CloudServerCreate < Knife

banner "knife vcloud server create (options)”

option :flavor, :short => "-f FLAVOR_ID", :long => "--flavor FLAVOR_ID", :description => "The flavor ID of server (m1.small, m1.medium, etc)", :proc => Proc.new { |f| Chef::Config[:knife][:flavor] = f }

option :image, :short => "-I IMAGE_ID", :long => "--image IMAGE_ID", :description => "The image ID for the server", :proc => Proc.new { |i| Chef::Config[:knife][:image] = i }

option :chef_node_name …. option :zone … Several bootstrapping options











Inherit from the Chef::Knife class











Usage Banner












Parameter Options like Server size,

Image, Zone and Bootstrapping

Options

Usage Banner







knife cloud server create (run)def run - validate compulsory parameters ….

connection = Fog::Compute.new(…) … Cloud provider based checks … server = connection.servers.create(params) … Cloud provider specific logic – Enable Networking, Associate IP Address, Reconfigure Server … if windows_image? wait_for_winrm else wait_for_sshd end

// Server is now up and runningBootstrap the Node

end









end

Knife commands are run by calling the `#run` method on an instance of your command class









end


Create the fog connection object









end



Create and configure the server









end




Wait for WinRM/SSH Service to start









end




Wait for WinRM/SSH Service to start

Windows or Linux Bootstrapping






Bootstrapping

Installs Chef and runs the chef-client to communicate with a Chef Server• via SSH• via Windows Remote

Management(WinRM)

Ref: http://www.lemen.com/imageBootstrap1.html







knife-cloud






servers

images

flavors

Fog Collections






knife cloud server deleteclass Chef class Knife class CloudServerDelete < Knife

banner "knife vcloud server delete (options)”

def run validate compulsory parameters …. connection = Fog::Compute.new(…)

Check whether server(s) exists or not@name_args.each do |server_id|

begin server = connection.servers.get(server_id) if !server

ui.warn(“Server does not exists”)next

end confirm("Do you really want to delete this server”) server.destroy ui.warn("Deleted server #{server.id}") #Destroy Chef Node # Destroy Chef Clientend


Usage Banner

Find the Server and Delete it







knife-cloud






servers

images

flavors

Fog Collections






knife cloud image listclass Chef class Knife class CloudImageList < Knife

banner "knife cloud image list (options)"

def run image_list = [ ui.color('ID', :bold), ui.color('Name', :bold) ]

connection.images.sort_by(&:name).each do |image| image_list << image.id.to_s image_list << image.name end

puts ui.list(image_list, :uneven_columns_across, 2) end end endend


Usage Banner

List Resource and Display it







knife-cloud






servers

images

flavors

Fog Collections






knife plugin directory structure

knife-vcloud├── Gemfile├── knife-vcloud.gemspec├── lib ├ chef├ knife-vcloud├ version.rb

├ knife├ vcloud_server_create.rb├ vcloud_server_delete.rb├ vcloud_image_list.rb├── spec├── README.md └── LICENSE









Parent Directory









Required to build the gem

Parent Directory









Version Info


Parent Directory









Actual Code

Version Info


Parent Directory





VMWare vCloud Director


VMWare vCloud Director

Reference: http://www.vmguru.nl/wordpress/2012/01/vmware-vcloud-director-design-guidelines/






vApps

Container of one or more VMs• Package up Multi-tier

Applications into vApps

• Operate on VMs as one unit

• Can be created by scratch

• Can be imported from outside the cloud

Reference: http://download3.vmware.com/vcat/documentation-center/index.html#page/Consuming%20a%20vCloud/3c%20Consuming%20a%20VMware%20vCloud.2.077.html






Catalogs

Catalogs are collections of vApps, vApps Templates and mediaExamples:• Infrastructure as Service Catalogs• Pre-installed Windows/Linux VMs• OS Media files• App Catalogs






vCloud Layered Network

External Network

Organization

Network

vApp Network

External Network : Providing a connection to the outside worldOrganization Network : Network contained within an organizationvApp Network : Network contained within a vApp

Ref: http://hol-prt.cloudfoundry.com/HOL-PRT-04_EN/HOL-PRT-04-m1/lessons/vCloud_Director_networking_and_Cisco_Nexus_1000V.html






knife-vcloud

Git repo: https://github.com/opscode/knife-vcloudSupport to provision one-VM vApps onto the Org-level networkSupported Commands:• knife vcloud server create• knife vcloud server list • knife vcloud server delete• knife vcloud image list• knife vcloud network list

https://github.com/opscode/knife-vcloud











knife-vcloud

Git repo: https://github.com/opscode/knife-vcloudSupport to provision one-VM vApps onto the Org-level networkSupported Commands:• knife vcloud server create• knife vcloud server list • knife vcloud server delete• knife vcloud image list• knife vcloud network list

Lets focus on it!












Deploying a vApp


Select the vApp Template.






Deploying a vApp


Select the vApp Template.Select the Network






Deploying a vApp


Select the vApp Template.Select the NetworkProvide a name, administrator Password to the vApp being deployed.






Deploying a vApp

Select the vApp Template.Select the NetworkProvide a name, administrator Password to the vApp being deployed.Configure the virtual machines inside the vApp, including • computer name, • CPU and Memory• Network and IP Assignment.






Deploying a vApp

Select the vApp Template.Select the NetworkProvide a name, administrator Password to the vApp being deployed.Configure the virtual machines inside the vApp, including • computer name, • CPU and Memory• Network and IP Assignment.

Bootstrapping!






Deploying a vApp(Deep-Dive)



































Deploying a vApp (Deep-Dive)






















Select the vApp Template.Select the NetworkBuild the server spec and deploy the vApp




























Additional Configuration



































Save Additional Configuration







Power On!







Wait until ready







Linux Bootstrapping!





















Windows Bootstrapping!







Windows Bootstrapping!






knife vcloud server create (Linux)knife vcloud server create --image <vApp-Linux-template ID>--node-name linux_node_test --network <Network ID>--ssh-user <SSH Username>--ssh-password <SSH Password>--distro chef-full






knife vcloud server create (Windows)

knife vcloud server create --image <vApp-Windows-template ID>--node-name windows_node_test--network <Network ID>--bootstrap-protocol winrm--winrm-password <Windows WinRM Password>--template-file ../knife-windows/lib/chef/knife/bootstrap/windows-chef-client-msi.erb -VV






Demo






knife vcloud server create

Source Code: https://github.com/opscode/knife-vcloud/blob/master/lib/chef/knife/vcloud_server_create.rb

https://github.com/opscode/knife-vcloud/blob/master/lib/chef/knife/vcloud_server_create.rb








knife vcloud server delete

Identify the vApp to deletePower Off the vAppDelete the vApp






knife vcloud server delete

Identify the vApp to deletePower Off the vAppDelete the vApp






Resource Listing

• knife vcloud server list• knife vcloud image list• Knife vcloud network list






knife vcloud server list






knife vcloud image list






knife vcloud network list






Demo






Chefs: now with Knife expertise!

Ref: http://s3.hubimg.com/u/1691762_f520.jpg





Thank You!

Innovation → Execution → Solution → Delivered

Feel free to reach out :Email: [email protected]

Twitter: @cheezo

mailto:[email protected]

http://clogeny.com/

Technology

ChefConf 2013 Talk: Writing a Knife cloud plugin for your shiny VMWare vCloud Director