Creating Packages that Run Anywhere with Chef Habitat

OPEN335

Nell Shamrell-Harrington

Platform Agnosticism with Habitat

What if…

What if you could build software once…


…and run it anywhere?

• Bare metal Server



• Bare metal Server• Virtual Machine



• Bare metal Server• Virtual Machine• Container



What if you could move a legacy application into the cloud…

What if you could move a legacy application into the cloud……without rewriting it?

What if you could empower your applications to recover from failure…

What if you could empower your applications to recover from failure……on their own?

You can!

Habitat is Application Automation

• Sr. Software Engineer at Chef Software• From Seattle, WA• @nellshamrell• [email protected]

Who am I?Nell Shamrell-Harrington

• Apache License• https://github.com/habitat-sh

Why am I here?Habitat is 100% Open Source!

Microsoft <3’s Open Source!Ken Thompson, Microsoft

Choice + Flexibility

Applications

Management

App Frameworks

Databases & Middleware

Infrastructure

Linux

PPT REM 8

Our Approach to Open Source in the Cloud

IntegrateEmbrace leading Open Source ecosystems and

integrate Microsoft products with agility and

consistency

ReleaseRelease key Microsoft technologies into the

Open Source domain to build a strong

ecosystem

ParticipateMicrosoft engineers to

participate in communities and

contribute to key Open Source projects

EnableEnable Linux and Open Source technology to be

first class citizens on Microsoft Platforms

Open Source Partners & Ecosystem

R Server.NET Core

RoslynTypeScript

F#

PowerShellautorestPowerBI Visuals

Office UI Fabric

Tools plugins

Why Habitat?

Building and running software is painful!

• Tied to particular runtimes and environments

Building and running software is painful


• Different packages for different OS’s



• Different packages for different OS’s• Different packages for bare metal/VMs and containers


Habitat builds and runs modern applications

Modern application

• Is immutable

Modern application

• Is immutable• Is agnostic to running environment

Modern application

• Is immutable• Is agnostic to running environment• Reduces complexity

Modern application

• Is immutable• Is agnostic to running environment• Reduces complexity• Enables scaling

Modern application

Building a modern application


Source

CodeRepo


Artifact

Source

CodeRepo


Artifact

Artifact

Repo

Source

CodeRepo


Artifact

Bare Metal

Container

Instance

VM

Artifact

Repo

Source

CodeRepo

What is Habitat?

Habitat is a new technology to build,

deploy, and manage applications…

…in any environmentfrom traditional datacenters

to containerized microservices

In Habitat, the application is the unit of automation

How does it work?

Packaging an Application with Habitat

USER


USER PLAN


Bash (available now)Powershell (soon!)

USER ARTIFACTPLAN


Bash (available now)Powershell (soon!)

Signed with Key


USER ARTIFACTPLAN DEPOT

(Optional)Bash (available now)Powershell (soon!)

Signed with Key

Public Depotapp.habitat.sh

Running an Application with Habitat


DEPOT

(Optional)


ARTIFACTDEPOT

(Optional)


ARTIFACTDEPOT

SERVICESUPERVISORBARE METAL

SERVICESUPERVISORCONTAINERS

SERVICESUPERVISORAMI

SERVICESUPERVISORVM

(Optional)


SERVICESUPERVISORVM

API


SERVICESUPERVISORVM

API

Load Balancer


SERVICESUPERVISORVM

SERVICESUPERVISORVM SERVICE

SUPERVISORVM

SERVICESUPERVISORVM


SERVICESUPERVISORVM


SUPERVISORVM

SERVICESUPERVISORVM

EncryptedEncrypted

EncryptedEncrypted

Habitat has many parts –many of which are still

in development

I’m here to talk aboutwhat you can do with

Habitat today

And together we can shape where

it goes in the future

• Habitat Supervisor (runs applications)

What we will discuss today

• Habitat Supervisor (runs applications)• Habitat Packaging Format (makes applications)


• Habitat Supervisor (runs applications)• Habitat Packaging Format (makes

applications)• Habitat with Containers (where Habitat shines)


Habitat Supervisor

• Intelligent runtime with deployment coordination and service discovery

Habitat supervisor

Habitat supervisorAllows you to run an application• Natively on HW

Habitat supervisorAllows you to run an application• Natively on HW• In a VM or Cloud Instance

Habitat supervisorAllows you to run an application• Natively on HW• In a VM or Cloud Instance• In a Container (Docker, Mesos, etc.)

Habitat supervisorProcess manager

Habitat supervisorProcess manager• Starts up and monitors package

Habitat supervisorProcess manager• Starts up and monitors package• Receives/implements configuration changes

Habitat supervisorProcess manager• Starts up and monitors package• Receives/implements configuration changes• Runs services

Habitat supervisorService• Habitat package running under a supervisor

Habitat supervisorService• Habitat package running under a supervisor• Simplest – 1 supervisor running 1 service

SERVICESUPERVISORVM

Habitat supervisor

Demo – running a Linux package with the Supervisor

Habitat supervisorService• Habitat package running under a supervisor• Simplest – 1 supervisor running 1 service• Beyond one service, need a supervisor ring

SERVICESUPERVISORVM

Habitat supervisorSupervisor Ring

Running MySQL

SERVICESUPERVISORVM


Also running MySQL


SUPERVISORVM

SERVICESUPERVISORVM



SUPERVISORVM

Encrypted Communication

SERVICESUPERVISORVM



SUPERVISORVM

Let’s elect a leader!(Uses built in

election algorithm)

SERVICESUPERVISORVM



SUPERVISORVM

Leader – receives write

requests

SERVICESUPERVISORVM



SUPERVISORVM


requests

Followers – receive read

requests

SERVICESUPERVISORVM



SUPERVISORVM

SERVICESUPERVISORVM



SUPERVISORVM

Cannot connect



SUPERVISORVM



SUPERVISORVM

Let’s elect a leader!(Uses built in

election algorithm)



SUPERVISORVM


requests



SUPERVISORVM

Follower – receives read

requests

Habitat assumes that failures will happen and

are normal

Healthy components will self-organize and

re-converge

Habitat supervisorSupervisor Topologies• Leader/Follower

Habitat supervisorSupervisor Topologies• Leader/Follower• Standalone

Habitat supervisorSupervisor Handles Package Updates

Habitat supervisorSupervisor Handles Package Updates• Detects when a new release is available on depot

Habitat supervisorSupervisor Handles Package Updates• Detects when a new release is available on

depot• Deploys it according to update strategy

Habitat supervisorSupervisor Update Strategies• All at once (only one currently supported)

Habitat supervisorSupervisor Update Strategies• All at once (only one currently supported)• Rollout (Coming Soon)

How does the Supervisor

get something to supervise?

Habitat Packaging Format

• HART format (because we <3 you)

Habitat packages

• HART format (because we <3 you)• Contain application itself

Habitat packages

• HART format (because we <3 you)• Contain compiled application itself• Contain everything needed to deploy and run the application

Habitat packages

Demo – Creating a Linux Package

Habitat plans use Bash for Packages that will run on

Linux

Habitat can also create plans for Windows using

Powershell(Preliminary)

Demo – Creating a Windows Package

Along with running hart packages with Habitat…

You can also export them

to other formats

Including container images, by

far the most popular is Docker

Habitat and Containers

Getting artifacts to run anywhere is very

difficult

Containers were supposed to solve this

problem, but…

Current Container Pain

• Major learning cliff between dev and prod


• Major learning cliff between dev and prod• Containers become black boxes – don’t always know what is in them.


• Major learning cliff between dev and prod• Containers become black boxes – don’t

always know what is in them.• Black boxes cause security issues


Libraries

Operating System

ApplicationApplication

Libraries

Containers with Habitat

Libraries

Operating System

Application

Application &

Libraries

OS


• Habitat starts with the app, minimal OS comes later



• App declares dependencies, dependencies are resolved from the application



• App declares dependencies, dependencies are resolved from the application

• Exposes API for external services

When you create a container image with

Habitat…

You know exactly what went

into the container andwhat is configurable about the container

Demo – Exporting package to Docker image

Deploying Containers with HabitatYou can deploy the container using…


• Container Scheduling services• Kubernetes, Mesosphere, Docker Swarm


• Container Scheduling services• Kubernetes, Mesosphere, Docker Swarm

• Cloud based container services• AWS ECS, Azure Container Services

Azure Container ServicesKen Thompson, Microsoft

Azure Container ServicesStandard Docker tooling and API supportApache Mesos, Docker Swarm, Kubernetes Linux and Windows Server containersAzure and Azure Stack

DC/OS Swarm Kubernetes

Azure

ACS EngineOpen Source Project on GitHubConfigure Template to Meet Your NeedsBasis for the Azure Container Service Resource ProviderContributions from the Community“Upstream” – New features here first

DC/OS Swarm Kubernetes

Azure

Look for a blog post on using Habitat with Azure Container

Servicescoming soon!

Wrap Up

You can build softwareonce and run it

(almost) anywhere

You can move a legacyapplication into the

cloud(without rewriting it)

You can empower your applications to recover

fromfailure on their own…

You can do all these things with

Habitat is Application Automation

• https://github.com/habitat-sh• Apache License

Why am I here?Habitat is 100% Open Source!

How do I get involved?

• Habitat.sh/community• Stop by the Chef booth!





Thank You!



Any Questions?

Engineering

Creating Packages that Run Anywhere with Chef Habitat