Building Trusted Docker Images for Hybrid Cloud: What's New With Project Hammr?, Ludovic Queiroga, OW2con'17, Paris

Copyright © UShareSoft 2017 Copyright © UShareSoft 2017

Look Beyond the Cloud

Building Trusted Docker Images for Hybrid Cloud: What's New With Project Hammr?

Ludovic Queiroga, Software Development Engineer, UShareSoft

22

© Copyright 2017 UShareSoft© Copyright 2017 UShareSoft

What is ?

33


Hammr Command-Line ToolHammr Command-line Tool: Build Consistent Machine Images from a Single Configuration FileHammr Command-line Tool: Build Consistent Machine Images from a Single Configuration File

> Open source OW2 project> Written in Python> https://projects.ow2.org/bin/view/hammr/

http://www.hammr.io> Github: https://github.com/usharesoft/hammr

> Open source OW2 project> Written in Python> https://projects.ow2.org/bin/view/hammr/

http://www.hammr.io> Github: https://github.com/usharesoft/hammr

44


Hammr overall architecture

55


Why use Hammr ?

> You work in DevOps

> You hate working with web UI :-)

> You need to automate machine image creation and management

> Integrate with your favorite CI/CD tools > E.G. Jenkins, Puppet, Chef, Ansible…> Integrate with your own software repository

> You work in DevOps

> You hate working with web UI :-)

> You need to automate machine image creation and management

> Integrate with your favorite CI/CD tools > E.G. Jenkins, Puppet, Chef, Ansible…> Integrate with your own software repository

66


Create and Publish your Stacks

with

77


Hammr: Modeling the Stack from a Single File

Security parametersKernel ParametersPartitioningNetworkingKeyboard, TimezoneLicense Entitlement

Security parametersKernel ParametersPartitioningNetworkingKeyboard, TimezoneLicense Entitlement

JeOS ProfilingPackage Updates

OS Repository SearchPackage Time Machine

JeOS ProfilingPackage Updates

OS Repository SearchPackage Time Machine

Bundle CatalogBundle Catalog

Configuration scriptsConfiguration scripts

Installation Installation

OS OS

Bundles Bundles

ConfigurationConfiguration

88


Getting Started with Hammr

> Template file: configuration file (Yaml or Json)

> Describe your application stack using the stack keyword

> Generate machine or containers images by defining builders

> Publish the generated machine images using your cloud credentials or download its to your computer

> Template file: configuration file (Yaml or Json)

> Describe your application stack using the stack keyword

> Generate machine or containers images by defining builders

> Publish the generated machine images using your cloud credentials or download its to your computer

99


Creating the Template: nginx Example

$ vi nginx-template.yml

---

stack:

name: nginx

version: '1.0'

os:

name: Debian

version: '8'

arch: x86_64

profile: Minimal

pkgs:

- name: nginx

installation:

diskSize: 12288


---

stack:

name: nginx

version: '1.0'

os:

name: Debian

version: '8'

arch: x86_64

profile: Minimal

pkgs:

- name: nginx

installation:

diskSize: 12288

stack keyword defining the content of your stackstack keyword defining the content of your stack

os keyword defining the os profile and packagesos keyword defining the os profile and packages

installation keyword defining the “install profile”installation keyword defining the “install profile”

Defining your Stack

Create the Template: template create Create the Template: template create$ hammr template create --file nginx-template.yml

You provided a yaml file, checking...

Creating template from temporary [/tmp/hammr-15888/archive.tar.gz] archive ...

100%|#############################################################################|

OK: Template create: DONE

Template URI: users/ludovic/appliances/104

Template Id : 104

$ hammr template create --file nginx-template.yml

You provided a yaml file, checking...

Creating template from temporary [/tmp/hammr-15888/archive.tar.gz] archive ...

100%|#############################################################################|

OK: Template create: DONE

Template URI: users/ludovic/appliances/104

Template Id : 104

1010


Defining a Builder: Generate and Publish


---

...stack section omitted

builders:

- type: Docker

namespace: myNameSpace

repositoryName: my-uforge-container

tagName: latest

account:

file: "/home/ludovic/accounts/dockerhub-account.yml"


---


builders:

- type: Docker

namespace: myNameSpace

repositoryName: my-uforge-container

tagName: latest

account:

file: "/home/ludovic/accounts/dockerhub-account.yml"

builders keyword defining all the machine Images to buildbuilders keyword defining all the machine Images to buildImage formatImage format

Machine image registration informationMachine image registration information

Cloud account credentialsCloud account credentials

$ vi dockerhub-account.yml

---

accounts:

- type: Docker

name: My Docker Hub Account

endpointUrl: https://index.docker.io

login: my-login

password: my-password

$ vi dockerhub-account.yml

---

accounts:

- type: Docker

name: My Docker Hub Account

endpointUrl: https://index.docker.io

login: my-login


1111


Generating Machine Image and PublishGenerate the machine images: template build Generate the machine images: template build

$ hammr template build --file nginx-template.ymlyou provided a yaml file, checking...

Generating 'Docker' image (1/1)

|>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>| 100%: Done, created on ... |<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<|

OK: Generation 'Docker' ok

Image URI: users/ludovic/appliances/104/images/2

Image Id : 2

$ hammr template build --file nginx-template.ymlyou provided a yaml file, checking...

Generating 'Docker' image (1/1)

|>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>| 100%: Done, created on ... |<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<|

OK: Generation 'Docker' ok

Image URI: users/ludovic/appliances/104/images/2

Image Id : 2

Image ID: 2, used to register this generated machine image to the target cloud environmentImage ID: 2, used to register this generated machine image to the target cloud environment

Publishing the generated machine image: image publishPublishing the generated machine image: image publish$ hammr image publish --id 2 --file nginx-template.ymlyou provided a yaml file, checking...

|>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>| 100%: Done, published o... |<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<|

OK: Publication to My Docker Hub Account is ok

$ hammr image publish --id 2 --file nginx-template.ymlyou provided a yaml file, checking...

|>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>| 100%: Done, published o... |<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<|

OK: Publication to My Docker Hub Account is ok

1212


Supported Target Machine Image Formats

> ISO> PXE*> ISO> PXE*

> Hyper-V> KVM> Raw> QCOW2> Vagrant> VirtualBox> VHD> VMware Workstation> VMware ESXi> VMware vCenter

vSphere> Xen

> Hyper-V> KVM> Raw> QCOW2> Vagrant> VirtualBox> VHD> VMware Workstation> VMware ESXi> VMware vCenter

vSphere> Xen

> Abiquo> AWS (EC2)> CloudStack> Eucalyptus (EMI)> Flexiant> Google Compute

Engine> Microsoft Azure

(ARM)> Nimbula> OpenStack> VMware VCD> Fujitsu K5 cloud> Oracle Cloud*

> Abiquo> AWS (EC2)> CloudStack> Eucalyptus (EMI)> Flexiant> Google Compute

Engine> Microsoft Azure

(ARM)> Nimbula> OpenStack> VMware VCD> Fujitsu K5 cloud> Oracle Cloud*

PhysicalPhysical VirtualVirtual CloudCloud ContainerContainer> Docker> LXC> Docker> LXC

* Target Machine Image Formats under development* Target Machine Image Formats under development

1313


Docker and Fujitsu K5 cloud

Integration

1414


Hammr: Docker Image Support

Self-contained Docker Base Image(1 Docker image by hammr file)Self-contained Docker Base Image(1 Docker image by hammr file)

BenefitsBenefits

> Full software governance: keep control over all software packages and versions within your image

> Transparency: see at a glance which OS packages and software files make up your image

> Repeatability: generate a consistent Docker image each and every time

> Publishable: publish the generated Docker image directly to Docker Hub

> Full software governance: keep control over all software packages and versions within your image

> Transparency: see at a glance which OS packages and software files make up your image

> Repeatability: generate a consistent Docker image each and every time

> Publishable: publish the generated Docker image directly to Docker Hub

1515


Hammr: Fujitsu K5 Cloud Support

BenefitsBenefits

> K5 : Fujitsu IaaS based on OpenStack> Fujitsu is a Gold Member of OpenStack

> 10th contributor> Hammr now supports K5 as a target format

> Generate Machine Image> Publish

> K5 : Fujitsu IaaS based on OpenStack> Fujitsu is a Gold Member of OpenStack

> 10th contributor> Hammr now supports K5 as a target format

> Generate Machine Image> Publish

---


builders:

- type: K5

displayName: uforgeK5Image

domain: domainID

project: projectID

region: uk-1

account:

file: "/tmp/accounts/k5-account.yml"

---


builders:

- type: K5

displayName: uforgeK5Image

domain: domainID

project: projectID

region: uk-1

account:

file: "/tmp/accounts/k5-account.yml"

$ vi k5-account.yml

---

accounts:

- type: K5

name: My K5 Account

login: my-login


$ vi k5-account.yml

---

accounts:

- type: K5

name: My K5 Account

login: my-login


1616


Latest News and Outlook

1717


Hammr: Latest User Features

BenefitsBenefits

Software governance for Docker Images

Consistency between VM ↔ Docker Images

Software governance for Docker Images

Consistency between VM ↔ Docker Images

> Bundles enhancements> Bundles enhancements$ vi nginx-template.yml

bundles:

- name: "myvmwaretools"

version: "1.0"

restrictionRule: "Distribution#name=CentOS && Distribution#version=7 &&

Distribution#arch=x86_64 && TargetFormat#type=VMware vCenter"

files:

- name: "README"

ownerGroup: "root:root"

rights: "755"

tag: "softwarefile"

source: "/myvmwaretools/1.0/files/README"

files: []

- name: "init.sh"

bootOrder: 1

bootType: "firstboot"

tag: "bootscript"

source: "/myvmwaretools/1.0/files/init.sh"

files: []

- name: "vmware-tools-core-10.1.5-1.el6.x86_64.rpm"

tag: "ospkg"

source: "/myvmwaretools/1.0/files/vmware-tools-core-10.1.5-1.el6.x86_64.rpm"

files: []

sourceLogo: "/myvmwaretools/1.0/logo/vmwaretools.jpg"


bundles:

- name: "myvmwaretools"

version: "1.0"

restrictionRule: "Distribution#name=CentOS && Distribution#version=7 &&

Distribution#arch=x86_64 && TargetFormat#type=VMware vCenter"

files:

- name: "README"

ownerGroup: "root:root"

rights: "755"

tag: "softwarefile"

source: "/myvmwaretools/1.0/files/README"

files: []

- name: "init.sh"

bootOrder: 1

bootType: "firstboot"

tag: "bootscript"

source: "/myvmwaretools/1.0/files/init.sh"

files: []

- name: "vmware-tools-core-10.1.5-1.el6.x86_64.rpm"

tag: "ospkg"

source: "/myvmwaretools/1.0/files/vmware-tools-core-10.1.5-1.el6.x86_64.rpm"

files: []

sourceLogo: "/myvmwaretools/1.0/logo/vmwaretools.jpg"

Tagged as Simple file with permissions and owner/group

Tagged as Simple file with permissions and owner/group

Tagged as Bootscript (Configuration file)Tagged as Bootscript (Configuration file)

Tagged as Repository packageTagged as Repository package

Restriction: Bundle available only for CentOS 7 x86_64 machine images generated for Vmware vCenter

Restriction: Bundle available only for CentOS 7 x86_64 machine images generated for Vmware vCenter

1818


Hammr: Latest User Features

> Support Multiple Network Interface Cards (Multi-NIC)

> Support Windows Server 2016

> Migration: Scan (First step to migrate live system) without overlay:

> Scan full system and packages information only> No extra files downloaded on UForge server

> Support Multiple Network Interface Cards (Multi-NIC)

> Support Windows Server 2016

> Migration: Scan (First step to migrate live system) without overlay:

> Scan full system and packages information only> No extra files downloaded on UForge server

---

stack:

Installation:

diskSize: 12288

nics:

- name: “eth0”

...other field omitted

- name: “eth1”


---

stack:

Installation:

diskSize: 12288

nics:

- name: “eth0”


- name: “eth1”


1919


Hammr: Latest Technical Features

BenefitsBenefits

> Travis CI on Hammr github repository> Run Unit tests> Job launch for each Pull Request

> Compatibility check between Hammr and UForge> When launching Hammr> Error if Hammr version not compatible with target

UForge

> Travis CI on Hammr github repository> Run Unit tests> Job launch for each Pull Request

> Compatibility check between Hammr and UForge> When launching Hammr> Error if Hammr version not compatible with target

UForge$ hammr --url https://uforge.usharesoft.com/api -u username -p passwordERROR: Sorry but this version of Hammr (version = 'HAMMR_VERSION') is not compatible with the version

of UForge (version = 'UFORGE_VERSION').

ERROR: Please refer to 'Install Compatibility' section in the documentation to learn how to install a

compatible version of Hammr.

$ hammr --url https://uforge.usharesoft.com/api -u username -p passwordERROR: Sorry but this version of Hammr (version = 'HAMMR_VERSION') is not compatible with the version

of UForge (version = 'UFORGE_VERSION').

ERROR: Please refer to 'Install Compatibility' section in the documentation to learn how to install a

compatible version of Hammr.

2020


Hammr: Outlook

BenefitsBenefits

> New Target Machine Image formats support> Oracle cloud> PXE (Pre-boot eXecution Environment)

> Support of layers> Model layers> Assemble a stack from layers

> Possibility to launch a VM from Hammr> Provided gradually for each cloud platform

> Migration: One step scan to image> Automate 3 steps for migrating live system

> New Target Machine Image formats support> Oracle cloud> PXE (Pre-boot eXecution Environment)

> Support of layers> Model layers> Assemble a stack from layers

> Possibility to launch a VM from Hammr> Provided gradually for each cloud platform

> Migration: One step scan to image> Automate 3 steps for migrating live system

2121


Machine image & container builderfor the cloud

It's an open source OW2 project

Use it! Contributions welcome!

http://hammr.iohttps://projects.ow2.org/bin/view/hammr/

https://github.com/usharesoft/hammr


Come and see A DEMO

during the break


Ludovic [email protected]

www.usharesoft.com@usharesoft

Technology

Building Trusted Docker Images for Hybrid Cloud: What's New With Project Hammr?, Ludovic Queiroga, OW2con'17, Paris