Scared of Huge Kubernetes Ecosystem?Adopt it incrementally!

Mazedur Rahman

About Me

● Advocate building repeatable and testable infrastructure

● Love to automate everything

● Following container ecosystem since 2013

● Enjoy reading/writing papers on cloud computing

● Teach classes in spare time

Kent Beck’s 3X

● eXplore○ Embarking on the greenfield project○ Risky search for a viable return○ If unexpectedly successful → leads to

● eXpand○ Now things are going nuts○ Unanticipated bottlenecks appear○ Growth becomes routine → leads to

● eXtract○ Now problem and solution spaces are clear○ Known cost pattern: $1 in → $3 out○ Playbook emerge, economies of scale matters

Explore

Expand

Extract

Success

Pay

off

THE PROBLEM

The Problem

● Deploy/Manage a set of stacks○ ∑(LB, Web Server, App, DB, ...)

● Needs○ Highly available

○ Scale horizontally

○ Rolling deploy, Blue/Green, …

○ Fast rollback

○ Secure runtime

○ ...

AmazonDynamoDB

THE CHALLENGES

The Challenges: VM is Too Heavy

Host OS

Hypervisor

GuestOS

GuestOS

GuestOS

Bin / Libs

Bin / Libs

Bin / Libs

App 1 App 3App 2

Host/Guest OS

Container Engine

Bin / Libs

Bin / Libs

Bin / Libs

App 1 App 3App 2

Infrastructure Infrastructure

VM

Co

nta

iner

The Challenges

● How do we○ deploy new version of apps?

○ solve service discovery?

○ avoid downtime during rollouts?

○ auto-heal services from failures?

○ patch host OS without disruptions?

○ implement load balancing?

○ automate certificate deployment?

○ orchestrate storage?

○ ….

LIFE WITHOUT ORCHESTRATOR

Life without Orchestrator: Ad-hoc Style

● Poor man’s orchestrator○ manual ‘docker run …’

○ ‘docker run …’ via scripts

○ docker-compose, rancher, ...

○ custom (home-grown) tooling

○ ansible, chef, puppet, …

○ Jenkins

○ systemd, fleet, …

○ …

Life without Orchestrator: Outcome

● Likely consequences○ ad-hoc everything → snowflakes

○ random failures → firefighting

○ human in the loop → slow process

○ inextensible → business blockers

ENOUGH IS ENOUGH

The Contenders

AWS ECS Kubernetes Marathon Nomad Swarm

The Winner

AWS ECS Kubernetes Marathon Nomad Swarm

✅

Kubernetes Brings...

Au

to-h

ealin

g

Service D

iscovery

Secret & Config mgmt

Auto binpacking

Horizontal Scaling

Load Balancing

Rolling d

eploy

The Ecosystem

...

DESIGN FOR CHANGE

Design for Change: Layered Architecture (1)

● Business and Tech Change Inevitable → Design in Layers○ Keep lock-in low → Easy to move around

○ Easy to upgrade/patch

○ Swappable

Network

K8s Cluster

K8s Master K8s Worker

Services

Design for Change: Layered Architecture (2)

● An ExampleServices

(Containers, FaaS, …)

Agents(Logging, Monitoring, …)

Container Engine(Docker, rkt, runc, …)

VM(OS Image, AMI, …)

Network(VPC, Route, SG, …)

Spec

ific

Agn

ost

ic Orchestration(K8s, Swarm, …)

Provision(Terraform, Pulumi, Provider tooling, …)

Design for Change: Workflows, not Tools

● Tools will most likely change○ But workflow remains same

Build Test Deploy Manage

DESIGN FOR STABILITY

Design for Stability: API Selection

● Opt for Stable APIs Initially○ v1 > v1beta1 > v1alpha1

Design for Stability: Service Selection

● Leading Candidates○ Non-user-facing API services

○ Predictable traffic/client

○ Stateless

○ ...

Online StoreFrontend

Account Cart Catalogue Order ...

KILL THE SCARE

● Simple but Powerful Workflows

Kill the Scare: Automate Workflow

Build Test Deploy Manage

$ ./stack build $ ./stack test $ ./stack deploy $ ./stack manage

Human or Bot

● Automated tests○ Assertions for cluster/service/resource provisioning/deployment

○ Leverage automated test frameworks

● Automated health checks○ Assertions for runtime resources, cross-communication, ...

Kill the Scare: Guarantee Repeatability

● Same Process Throughout○ Dev/Prod parity

○ Instrumentation

○ Troubleshooting

Kill the Scare: Boost Confidence

Traditional App Twelve-Factor App

Time between deploys Weeks Hours

Developers vs. Deployers Different people Same people

Dev vs. Prod environment Divergent As similar as possible

● Safe Rollout/Rollback○ Rolling update

○ Blue/Green

○ Canary

○ A/B testing

○ ...

Kill the Scare: Stress-Free Release

github.com/ContainerSolutions/k8s-deployment-strategies

ADOPT INCREMENTALLY

● Defer as many tooling as possible○ Prometheus

○ Istio

○ Calico

○ ...

Adopt Incrementally: Go Minimalist Initially

Explore

Expand

Extract

Success

Pay

off

Rolling update(Off-the-shelf)

Blue/Green(via simple tooling)

Canary(Envoy, Istio, …)

App Code Rollout

● Minimal templating footprint○ Avoid too many tooling initially

○ Good for getting familiar with K8s stack

○ Supported by core community

○ ...

Adopt Incrementally: Start with YAML

Explore

Expand

Extract

Success

Pay

off

YAML

Helm, Charts, ...

client-go, draft, ...

K8s Manifests

● Offload Responsibilities as much as possible○ Master component management

○ Monitoring, logging, ...

○ ...

Adopt Incrementally: Start with Managed K8s if an Option

Explore

Expand

Extract

Success

Pay

off

Whatever you have

Datadog, New Relic, ...

Prometheus, ...

Monitoring

● Reduce host OS management burden○ Minimalist OS pre-packaged with base tooling

■ e.g. managed workers, CoreOS, etc.

● Reduce attack surface○ Avoid bloated OS

Adopt Incrementally: Choose Right Host OS

Explore

Expand

Extract

Success

Pay

off

Supervised patching

Semi-automatic

Automatic

Upgrade/Patch Rollout

● Undecided on orchestration?○ Go for Kubernetes if you must pick one

■ May feel a bit overwhelming initially

■ Investment pays off

Final Thoughts

Thanks!