DEPLOY ON CLOUD - WHERE ARE WE?

Qiming Tengtengqim@cn.ibm.com

AGENDA

Heat Basic

Heat SoftwareConfig

Heat BootConfig

Heat DockerCompose

HeatKubelet

Heat Docker Plugin

Heat Ansible

Senlin

Convergence

FROM TEMPLATE TO STACKversion: xxxparameters: key: mykeyresources: server: OS::Nova::Server key: {get: key} image: gold flavor: m1.small network: {get: network} volume: {get: volume} network: OS::Neutron::Network ... volume: OS::Cinder::Volume ...

Heat

Nova

Neutron

Cinder Volume

Instance

• orchestrator not bandmaster• deployment tool

NOT JUST A DEPLOYMENT TOOL, PLEASE!!!

version: xxxparameters: key: mykey

resources: server: OS::Heat::ServerGroup count: 5

volume: OS::Cinder::Volume ...

Heat

S0 S1 S2 S3 S4

(1) Parallelized Operation

(2) listen

Observed States

S0 S1 S2 S3 S4

Desired States

(3) converge

S2 S3

InstanceOS

App

SOFTWARE-CONFIG/SOFTWARE-DEPLOYMENT

version: xxxparameters: key: mykeyresources: config: OS::Heat::SoftConfig group: script config: # your script server: OS::Nova::Server key: {get: key} image: gold flavor: m1.small network: {get: network} volume: {get: volume} user_data: {get: config}

Heat

Nova

ChefPuppe

t

• OS::Heat::CloudConfig• OS::Heat::SoftwareConfig• OS::Heat::StructuredConfig• OS::Heat::SoftwareDeployment• OS::Heat::SoftwareDeployments• OS::Heat::SoftwareComponent

SOFTWARE-CONFIG/SOFTWARE-DEPLOYMENTversion: xxx

resources: config-1: OS::Heat::SoftConfig deploy-1: config: config-1 server: server-1 server-1: OS::Nova::Server

config-2: OS::Heat::SoftConfig deploy-2: depends_on: deploy-1 config: config-2 server: server-2 server-2: OS::Nova::Server

server-1 server-2

config-1 config-2

depends on

DECLARATIVEserver-1 server-2

config-1 config-2

depends on

InstanceOS

SOFTWARE-CONFIG/SOFTWARE-DEPLOYMENT

App

ChefPuppe

tQ: What is it?

[1] http://git.openstack.org/cgit/openstack/os-collect-config/[2] http://git.openstack.org/cgit/openstack/os-refresh-config/ [3] http://git.openstack.org/cgit/openstack/os-apply-config/ [4] http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/elements

A: THEY are collection of agents including

• os-collect-config [1]• os-refresh-config [2]• os-apply-config [3]• heat-config-script [4]• heat-config-puppet [4]• heat-config-docker-compose [4]• heat-config-kubelet [4]• ...

InstanceOS

SOFTWARE-CONFIG/SOFTWARE-DEPLOYMENT

App

ChefPuppe

tQ: How does the agent authenticate?

NOTE: There is a side path of generating EC2 tokens

A: Heat does secret job in the background

• heat domain• created during setup

• stack_domain_project• name stack id

• stack_domain_user• name resource name

• password ?• uuid.uuid4().hex

InstanceOS

SOFTWARE-CONFIG/SOFTWARE-DEPLOYMENT

App

ChefPuppe

tQ: How are the agents injected/installed?

[1] http://git.openstack.org/cgit/openstack/diskimage-builder/[2] http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/boot-config

A: There are two ways

• disk-image-builder (dib) • a TripleO project [1]• prebuilt images for use

• Heat boot-config [2]• install these agents on the fly

when VM boots up

goal install agents required to use certain software deployments in templates [1]

how it's used define an env yaml file with a Heat::InstallConfigAgent resource refer to this resource in your server.properties.user_data

HEAT BOOT-CONFIG

[1] http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/boot-config/

env yaml

inst-config

config-config

start-config

#!/bin/shyum install ..

#!/bin/shcat << EOF ...mkdir ...#!/bin/shsystemctl enable ...systemctl start ...

heat stack-create -f template -e environment mystack

template

server

user_data

environmentInstAgent

config

MIME

goal prepare guest environment for container deployment with docker-compose

HEAT CONTAINER AGENT

install_container_agent

write_image_pull_script

install_container_agent

#cloud-configwrite_files: # write a script that will # grab specified image via # 1. curl + docker load, or # 2. docker pull#!/bin/sh# 1. create a service:# heat-container-agent # 2. enable/start docker svc# 3. enable/start agent svc

template

server

user_data

environmentInstAgent

config

# execute the image pull script to get the <image>

# docker run --name heat-container-agent ... \ <image>

heat-container-agent service

sample image: http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/heat-container-agent/

goal a 'hook' that uses 'docker-compose' to deploy containers [1] an element that you will build into your guest image

HEAT DOCKER COMPOSE

[1] http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/elements/heat-config-docker-compose

template

server

user_data

config group:docker-compose inputs: [env_files] config: db: image: redis web: image: nginx

Instance

OS

os-collect-config docker-composeup -d --no-build

heat-config

heat-docker-compose

container

container

metadata

env files

yml file

template

server

user_data

config group: kubelet inputs: [env_files] config: containers: - name: doecho image: busybox command: ...

Instance

OS

os-collect-configmetadat

a

heat-config

config

hook-kubelet

goal a 'hook' that uses 'kubelet' agent from kubernetes to deploy containers

an element that you will build into your guest image

HEAT KUBELET

[1] http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/elements/heat-config-docker-compose

container

container

poll

kubelet.service

/opt/heat-docker/images.tar

preinstalled

preinstalled

poll

template

server

user_data

config group: ansible inputs: [...] config: # your ansible book # here

Instance

OS

os-collect-configmetadat

a

heat-config

playbook

goal a 'hook' that uses 'ansible' to configure A instance

HEAT CONFIG ANSIBLE

[1] http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/elements/heat-config-ansible

ansible-playbook -i localhost <file>

hook-ansible.pycontaine

rapplicati

on

A Resource Type (Contrib[1]) for Heat DockerInc::Docker::Container built on docker-py [2]

HEAT DOCKER PLUGIN

[1] http://git.openstack.org/cgit/openstack/heat/tree/contrib/heat_docker/[2] https://github.com/docker/docker-py

image: stringcommand: listhostname: stringuser: stringstdin_open: booleantty: booleanmem_limit: integerports: listenvironment: listdns: listvolumes: mapcpu_shares: integercpuset: string

CREATE

privileged: booleanbinds: map (volumes)volumes_from: listport_bindings: maplinks: maprestart_policy: mapcap_add: listcap_drop: listread_only: Booleandevices: list

START

infonetwork_infonetwork_ipnetwork_gatewaynetwork_tcp_portsnetwork_udp_portslogslogs_headlogs_tail

SHOW

SERVICES ROADMAP ON SUPERVESSEL

17

SuperVessel Cloud Infrastructure

SuperVessel Cloud Service

SuperVessel Big Data and

HPCService

Super Class

Service

OpenPOWER Enablement

Service

Super Project Team

Service

1.VM and container service

2.Storage service

3.Network service

4.Accelerator as service

5.Image service

1.Big Data: MapReduce (Symphony), SPARK

2.Performance tuning service

1.X-to-P migration: AutoPort tool

2.OpenPOWER new system test service

1.On-line video courses

2.Teacher course management

3.User contribution management

1.Project management service

2.DevOps automation

Storage IBM POWER serversOpenPOWER server FPGA/GPU

Docker

(Online) (Online) (Preparing)(Online)

SUPERVESSEL

HEAT -- TRIED NOT TO JUST A DEPLOYER Supports to High-Availability

OS::Heat::HARestarter recreates a resource when failure detected

Supports to Auto-Scaling OS::Heat::InstanceGroup OS::Heat::ResourceGroup OS::Heat::AutoScalingResourceGroup OS::Heat::ScalingPolicy AWS::AutoScaling::AutoScalingGroup AWS::AutoScaling::ScalingPolicy AWS::AutoScaling::LaunchConfiguration

AUTOSCALING REORG

BLUEPRINTS ON REWORKING HEAT AUTOSCALING

BP Priority Description

autoscaling-api-resources high Heat resources invoking AS APIs

as-api-group-resource high ScalingGroup resource wrapping AS API's group functionality

as-api-policy-resource high ScalingPolicy resource wrapping AS API's policy functionality

as-api-webhook-resource high Webhook resource wrapping AS API's execution of webhooks

autoscaling-api-client high A python client for Heat to interact with AS API

autoscaling-api - A separate service for the implementation of autoscaling w/ Heat

as-engine - A separate engine/service for autoscaling support AS API

as-engine-db - A DB dedicated to autoscaling, using schema created in as-lib-db

as-lib - A separate module to be used by the AS service

as-lib-db - A DB for autoscaling bookkeeping

DEPENDENCIES AMONG BPS

autoscaling-api-resources

as-api-group-resourcesas-api-policy-resources as-api-webhook-resources

autoscaling-api-client

autoscaling-api

as-engine

as-engine-db

as-lib

as-lib-db

OVERVIEW OF AUTOSCALING

A STRUGGLE BEFORE SENLIN STARTS Should we do this within Heat or outside Heat?

Within Heat pros

smooth transition; strict reviews better quality cons

long (maybe forever) code churn; eventually, a dedicated service is needed, thus the pain to switch over

Outside Heat pros

quick development; less code churn to Heat cons

high requirements of skills and cycles; eventual switch over, i.e. another animal to feed in the OpenStack zoo

We choose OUTSIDE HEAT There are Heat core team supports to this approach We see a lot potentials in a standalone clustering service We don't have to do everything from scratch we "borrow" and "steal" code

whenever license permits

WHAT DO WE REALLY NEED?

Scalable

Load-Balanced

Highly-Available

Manageable

......

of any (OpenStack) objects

-- What is missing from OpenStack

Senlin[Chinese Pinyin for Forest]

SENLIN ARCHITECTURE

Senlin Engine

Senlin API

Senlin Database

Senlin Client

REST RPC

Profiles

Policies

ER DIAGRAM

cluster

nameuuiduserprojectparentprofile_idstatus

profile

nameuuidtypespec

node

nameuuidcluster_idprofile_idindexstatuscreated_timeupdated_time

policy

nameuuidtypelevelspec

placement_policy

update_policy

deletion_policy

scaling_policy

health_policy create()delete()update()add()remove()

lb_policy

«policy_type»

os.nova.server

os.cinder.volume

os.keystone.user

os.heat.stack

cluster_policy

cluster_idpolicy_idenabledlevelcooldownpriority

«profile_type»action

contextactioninputsoutputs

webhook

targetactionuser

APIplugins

plugins

SENLIN OPERATIONS (ACTIONS)

Cluster• CREATE• DELETE• UPDATE• LIST• SHOW• ADD_NODES• DEL_NODES• SCALE_OUT• SCALE_IN• POLICY_ATTACH• POLICY_DETACH• POLICY_UPDATE

Node• CREATE• DELETE• UPDATE• LIST• SHOW• JOIN• LEAVE• MIGRATE

Policy• CREATE• UPDATE• DELETE• LIST• SHOW

Profile• CREATE• UPDATE• DELETE• LIST• SHOW

Action• LIST• SHOW

Event• LIST• SHOW

Webhook• CREATE• DELETE• LIST• SHOW

RELATION TO OTHER PROJECTS Senlin provides the array data type for cloud programming

Senlin CeilometerHeat

Nova Cinder Neutron Swift Keystone

Horizon

Primitive Data Types

Complex Data Types

struct person { int age; char name[0];}

person team[10]; // Senlin cluster of Heat stacks

// Senlin cluster of nova servers

// Heat stack containing senlin clusters

CURRENT STATUS

Code Base http://git.openstack.org/cgit/stackforge/senlin

including API design under doc subdirectory http://git.openstack.org/cgit/stackforge/python-senlinclient

IRC channel: #senlin

Date Milestone

2014-12-10 Initial Git Repository inside CRL

2014-12-25 Migration to github.com

2014-01-14 Introduction to IBM Heat Community

2015-01-19 Weekly conference call started

2015-02-06 Announcement on IBM openstack-dev mailinglist

2015-02-13 Email to OpenStack Heat Core team

2015-03-16 Senlin project accepted to OpenStack StackForge

2015-03-21 Senlin client project accepted to OpenStack StackForge

2015-03-26 Project announcement in community (link)

NEXT STEP Complete AutoScaling support

Cross-Region AutoScaling

Features Pipeline (draft)

High Priority Middle Priority Low Priority

Item W Item W Item W

Event Listening ** Horizon Plug-in * Metrics Collection *

Scavenger Process * User Defined Actions/Ansible *** AWS Compatible API ***

Multi-Engine Support * Quota Enforcement * Integration with Mistral **

Test Case Coverage ** Event Notification * Cluster suspend/resume **

Babican Support * Scheduled actions *

VPNaaS support ** Interaction with Congress *

Nova ServerGroup API * Integration with Tooz **

THANK YOU!