OSDC 2014: Jonathan Clarke - Rudder

Normation – CC-BY-SAnormation.com

Rudder

A powerful and structuredCFEngine framework

Jonathan CLARKE – [email protected]@jooooooon42 (that's 7 'o's)

mailto:[email protected]

Normation – CC-BY-SAnormation.com 2

www.rudder.cmWho am I?

● Jonathan Clarke

● Title: Co-founder & Product lead at Normation

● Origins: Sysadmin, infrastructure management

● Now: Automation + “running a company”-stuff

● Contributor to free software:

– Co-creator of Rudder

– Contributor to CFEngine, OpenLDAP

● Co-organizer of events:


www.rudder.cmIntro

This presentationis about Lego

Photo CC BY-NC-SA 2.0 from https://www.flickr.com/photos/dillpixel/

https://www.flickr.com/photos/dillpixel/


www.rudder.cmIntro

Reminder

Photo CC BY-NC-SA 2.0 from https://www.flickr.com/photos/evaekeblad/ Photo CC BY-SA 2.0 from https://www.flickr.com/photos/georgivar/


www.rudder.cmBackground

A bunch of ops consultants

● From “plain old” infrastructure to configuration management● Multiple companies: small, large & huge● 5-10 years of doing this

We always got the same takeaways


www.rudder.cmTakeaway #1: Automated configuration rocks!

Automated configuration rocks!

ScalableManage 1 to > 100000 servers the same way

Save timeDeploy faster & be more responsive to changes

Improve reliabilityAvoid manual errors, harmonize configurations

The proper way

to manage systems


www.rudder.cmTakeaway #2: Getting everyone on board?

Getting everyoneon board for CM is hard

Frustration“I can do it quicker by hand or with a shell script”

Steep learning curveNew concepts, non obvious syntaxes, paradigm, ...

Lack of motivation“What do I have to gain from using this tool?”


www.rudder.cmFeedback #2: CFEngine is hard!

Getting started from lots of bricks is daunting.

Photo CC BY-NC-SA 2.0 from https://www.flickr.com/photos/strutta/


www.rudder.cmWhat can we do?

So how comeso many projects

do work out?



Thanks to a hero!

So how comeso many projects

do work out?

Photo CC BY-NC-ND 2.0 from https://www.flickr.com/photos/mwboeckmann/



Poor configuration management hero...




Hey, I'm trying to do this thing in config management,but I can't it to work, can you help me?




Hi, this is the supervision team.I'm sorry to disturb you at night, but we've got this error

in production, and I think it's related to a change in the CM tool,but I don't understand it. Can you help me?





How can we help?

This is clearly a problem.


www.rudder.cm

Steep learning curveNew concepts, non obvious syntaxes, paradigm, ...

Approach

1) Separate content and controls

2) Provide access to key parameters without having to edit {CFEngine,Puppet,Chef} code


www.rudder.cm

Lack of motivation“What do I have to gain from using this tool?”

Approach

1) Show the benefits to all users

2) Provide nice reports showing what works, how many machines are impacted


www.rudder.cm

Frustration“I can do it quicker by hand or with a shell script”

Approach

1) Make it easy and quick to achieve success

2) Provide ready-to-use configuration techniques and share in-house ones simply


www.rudder.cmWhy Rudder?

Make configuration management easyand increase its adoption

Extend benefitsof

configuration managementto

a wider population

ManagersJunior

sysadminsNon

experts

Lower entry barrierto

learn and use

configuration management

Easy to use Highly powerful


www.rudder.cm

Sane defaults, always configurable

Philosophy

Core principles

Plug and play

SmartEasy

Extensible& CustomizableOpen source


www.rudder.cmKey points

Specifically designed forautomation & compliance

Pre-packaged for:Linux, UNIX, Windows, Android

Open Source

Simplified user experiencevia a Web UI

Graphical reportingBased on CFEngine 3(don't reinvent the wheel!)

Vagrant config to test:https://github.com/normation/rudder-vagrant/



Right! Show me already!


www.rudder.cmOverview

Simplified configuration



Built-in reporting



Built-in reporting



Complete tracability


www.rudder.cmDesign choices

Design choices


www.rudder.cmDesign choices: CFEngine

#1: Why CFEngine?



CFEngine rocksMulti-platformLinux, Android, BSD, AIX, HP-UX, Solaris, Windows...

Open SourceGPLv3

Small footprint, scalableA few MB of RAM,just seconds to run...

Continuous checkingAgent based approach,no push

Resilient to errorsNetwork outages, failures,unavailable resources...



Continuous checkingEvery 5 minutes

Multi-platformLinux, Unix, Windows, Android...

Separate configuration from implementation

ReportingDone after the checks, separate process

High freqency, trust in compliance reporting

Reuse implementations, less bugs, shared code...Clear separation of roles

Cover as many systems as possible

Avoid bottleneckDifferent report types


www.rudder.cmDesign choices: Network architecture

#2: Network architecture?


www.rudder.cmDesign choices: Network architecture

Rudder server

Node Node Node

TCP - port 5309File metadata and files

Authentication and encryption (SSL)

TCP ports 80 and 514HTTP and syslog

Node Node

Isolated networkRelay server

Download info

→ Built upon CFEngine network architecture

All connections go→from nodes to server

Pull-based approach→


www.rudder.cmDesign choices: Workflow

#3: Typical usage


www.rudder.cmDesign choices: Workflow

Management

Definepolicy

Changes(fixes, upgrades...)

c c

Community Expert

Sysadmins

Configureparameters

Configuration agent

Initial applicationContinuous verification

REP

OR

TIN

G

Technical abstraction(method vs parameters)


www.rudder.cmDesign choices: Central validation

#4: Central validation



Validation workflow



Validation workflow● States:

● Pending validation

– Can be sent to: Pending deployment, Deployed, Cancelled.

● Pending deployment

– The change was validated, but now require to be deployed. Can be sent to: Deployed, Cancelled.

● Deployed

– The change is deployed. This is a final state, it can’t be moved anymore.

● Cancelled

– The change was not approved. This is a final state, it can’t be moved anymore.


www.rudder.cmDemonstration

Demo!


www.rudder.cmExtending & Customizing

Extending & Customizing


www.rudder.cmExtension

Techniques

Implemented inCFEngine syntax

+ metadata for

web configuration

Nodes

Search criteria oninventory data

Hardware/OS/Network/Software/Node name/

...

Directives

Rules

Apply Directives to a Group

Groups

Sysadmins

c c

Manager or sysadmins

Expert

Community



Techniques


+ metadata for

web configuration

Nodes



...

Directives

Rules


Groups

Sysadmins

c c


Expert

Community



Techniques


+ metadata for

web configuration

Nodes



...

Directives

Rules


Groups

Sysadmins

c c


Expert

Community

Write any configuration you like in a Techniqueand share them with co-workersby exposing a selection of parameters


www.rudder.cmResult

Example === 1000 words

With ncf (see http://www.ncf.io)

http://www.ncf.io/


www.rudder.cmResult

Example === 1000 words

With ncf + Rudder variables


www.rudder.cmOnline documentation

http://www.ncf.io/pages/reference.html


www.rudder.cmCurrent status

Project is now reliable & scalableBut needs more Techniques

Ohloh statistics:

Source: http://www.ohloh.net/p/rudder-project

h

Normation – CC-BY-SAnormation.com

Questions?

Check it out on:http://www.rudder.cm/

Jonathan CLARKE – [email protected]@jooooooon42 (that's 7 'o's)

mailto:[email protected]