Devops Munich meetup

(Scripting) Orchestration

12.Februar

1. Short introduction

2. Tool usage at hybris

3. Orchestration in general

4. Introduction to adminportal

5. Short Demo

#1 “independent” commerce platform

Fastest growing global commerce ISV

Founded in 1997

Operations in 15 countries

Offices:

Munich*, Montreal*, Ams

terdam, Boston, Chicago, Gliwice

, Hong

Kong, London, Milan, Neuilly-Sur-

Seine,

Sao

Paolo, Sydney, Tokyo, Västerås,

Vienna, Zug

Why do we do this talk?

So, WHY this talk?

Getting feedback

Search the blind spot

We„re hiring

http://www.hybris.com/de/company/careers/jobs

… and WHO ?

Kim Neunert (@k9ert), hybris GmbH

Tobias Schuhmacher, hybris GmbH

1. Short introduction

2. Tool usage at hybris

3. Orchestration in general

4. Introduction to adminportal

5. Short Demo

Hybris has been a software vendor rather than a Service Provider, that ischanging right now

In the first place: Puppet (apply) for server-provisioning

Automated „puppet apply“ with the adminportal

MCollective and especially the MCollective-mongo-registry approachhttp://goo.gl/WubXM

For adhoc-tasks, we use ansible but:

Vagrant for development and Continous integration via jenkins

Puppi, a collection of bash-scripts provisioned via puppet fordeployments and a self introduced mongo-reporting-mechanism

Now heading towards a puppetmaster-approach being able to completelyreuse our modules/server-roles

Some context - tooling at hybris

1. Short introduction

2. Tool usage at hybris

3. Orchestration in general

4. Introduction to adminportal

5. Short Demo

Automation a part of an overall process

Orchestration Automating the whole overall process

Where are the typical limitations/borders of automation?

Orchestration, what„s that?

Deployment

Provisioning

Booting new

machines, configuring

infrastructure

Provisioning

In order for orchestration to really work, you need a single system calling the shots (although it may, or perhaps should, delegate some responsibilities to other systems).

This in turn requires that all the components … be manageable from that one platform.

There are two ways this gets done: the single-vendor approach and the open-standards approach.

With the single-vendor approach, [...]

The open-standards approach allows IT to mix and match resources from different vendors. As long as the orchestrator and management tools can talk to them, they can be folded in. [...]

-- http://goo.gl/2QN6V

The vision of orchestration

The downside of the imperative approach (orchestration)

Modeling the control-flow is error-prone, especially in mixedenvironments

No Stacktrace/„Exceptionhandling“

No complete control of the whole stack, maybe 3rd parties/other peopleinvolved

Difficult to decide how to continue in the case of errors

So, orchestration tends to fail

Whenever possible one switched to an declarative approach:

Parser-code generation, SQL, Regex, Rendering of UIs

Nodeconfiguration via Puppet (Chef?!)

The role of orchestration is reduced to just model dependencies

Let„s avoid orchestration model dependencies!

Cross-nodes dependencies Exported Resources in Puppet

You need to store state, that„s why exported Resources need a DB

Storing state or gathering it at runtime is very helpfull

PuppetDB

MCollective

R.I.Pienaar (author of mcollective) shows how to leverage MongoDB-Registration-Data in puppet manifests:

http://goo.gl/WubXM

Chris Spence (puppetlabs consultant) created a notify-mechanism whichwill handle cross-node-dependencies so that e.g. a Loadbalancer getconfigured afterwards immediately

http://goo.gl/NkHtS

Example: Cross node dependencies while provisioning

Long story short …

… you want to avoid orchestration

in favor of declarative dependency modeling

however, that will be a long journey ...

… in the meantime …

1. Short introduction

2. Tool usage at hybris

3. Orchestration in general

4. Introduction to adminportal

5. Short Demo

Features

Grails application (Spring, Groovy, Java...)

Execute scripts in a chain (Bash, Perl, Groovy, Python, PHP...)

Annotate scripts (metadata: description, author, parameters etc.)

#!/bin/bash

# @Scriptlet(author=“Tobi (my-email@somehwere.de)", \\

description=“Create Zerigo DNS entry")

#

#

# @Parameters([

# @Parameter(name=“ZERIGO_USER", type=ParameterType.CONSUME, \\

description=“The Zerigo uUsername", visible=false,

defaultValue=“my-zergio-username"),

#

# @Parameter(name=“FQDN", type=ParameterType.EMIT, \\

description=“The FQDN returned by Zerigo“),

# ])

List of required script parameters

Features

Features

Grouping of script chains

Per environment

Different parameters per group

Parameters exported via REST (Json, YAML – useful for Hiera)

Execution of script chains can be triggered via REST

Authorization / Authentication supported

Plugin mechanism: wrap a script and interact with Grails (e.g. DB access)

Execute scripts as cronjob

Track any output of a script chain

After the execution, it looks like this

1. Short introduction

2. Tool usage at hybris

3. Orchestration in general

4. Introduction to adminportal

5. Short Demo