Kubernetes 101 Workshop

Containing Container Chaos with Kubernetes

Bret McGowenGoogle@bretmcg

Carter MorganGoogle@_askcarter

Workshop setup: http://github.com/bretmcg/kubernetes-workshop

2@kubernetesio @bretmcg @_askcarter

Agenda09:00 - 10:30 Containers and Kubernetes overview

10:30 - 10 :45 - BREAK

10:45 - 12:00 - Kubernetes 101

12:00 - 01:00 - Lunch!

01:00 - 02:30 - Kubernetes in Production

02:30 - 02:45 - BREAK

02:45 - 04:00 - Kubernetes in Production, cont’d

33

What’s in this for you...

44

Let's go back in time...

5

Shared machines Chroots, ulimits, and nice

Noisy neighbors: a real problemLimited our ability to share

The fleet got largerInefficiency hurts more at scale

Share harder!

ca. 2002 App-specific machine poolsInefficient and painful to manage

Good fences make good neighbors

6

Everything we do is about isolation

Namespacing is secondaryc.f. github.com/google/lmctfy

We evolved our system, made mistakes, learned lessons

Docker

The time is right to share our experiences, and to learn from yours

ca. 2006 Google developed cgroupsInescapable resource isolationEnables better sharing

7

job hello_world = {

runtime = { cell = 'ic' } // Cell (cluster) to run in

binary = '.../hello_world_webserver' // Program to run

args = { port = '%port%' } // Command line parameters

requirements = { // Resource requirements

ram = 100M

disk = 100M

cpu = 0.1

}

replicas = 5 // Number of tasks

}

10000

Borg - Developer View

8

web browsers

BorgMaster

link shard

UI shardBorgMaster

link shard

UI shardBorgMaster

link shard

UI shardBorgMaster

link shard

UI shard

Scheduler

borgcfg web browsers

scheduler

Borglet Borglet Borglet Borglet

Config file

BorgMaster

link shard

UI shard

persistent store (Paxos)

Binary

Borg

What justhappened?

9

Hello world!

Hello world!

Hello world!

Hello world!Hello

world! Hello world! Hello

world!

Hello world!

Hello world!

Hello world!

Hello world!

Hello world!

Hello world!

Hello world!

Hello world!

Hello world!

Hello world!Hello world!

Hello world!

Hello world!

Hello world!

Hello world!

Hello world! Hello

world!

Hello world!

Hello world!

Hello world!

Image by Connie Zhou

Hello world!

Hello world!

Hello world! Hello

world!

Hello world! Hello

world!

Hello world!

Hello world!

Hello world!

Hello world!

Hello world! Hello

world!

Hello world! Hello

world!

Hello world!

Hello world!

Hello world!

Hello world!

Hello world! Hello

world!

Hello world! Hello

world!

Hello world!

Hello world!

10

Developer View

11

Data center as one machineMachines are just resource boundaries


The App (Monolith)

nginx

monolith


The App (Microservices)

nginx

helloauth

1414

Containers


Old Way: Shared Machines

No isolation

No namespacing

Common libs

Highly coupled apps and OS

kernel

libs

app

app app

app


Old Way: Virtual Machines

Some isolation

Inefficient

Still highly coupled to the guest OS

Hard to manage app

libskernel

libs

app app

kernel

app

libs

libskernel

kernel


New Way: Containers

libs

app

kernel

libs

app

libs

app

libs

app


But what ARE they?

Containers share the same operating system kernel

Container images are stateless and contain all dependencies▪ static, portable binaries▪ constructed from layered filesystems

Containers provide isolation (from each other and from the host) Resources (CPU, RAM, Disk, etc.) Users Filesystem Network

19

Why containers?

• Performance• Repeatability• Isolation• Quality of service• Accounting• Portability

A fundamentally different way of managing applications

late binding vs. early binding

Images by Connie Zhou

2020

Packaging and Distributing Apps demo

2121

LabWorkshop setupandContainerizing your applicationhttp://github.com/bretmcg/kubernetes-workshop

2222

But that's just one machine!

Discovery

ScalingSecurity

Monitoring Configuration

SchedulingHealth

23

https://www.flickr.com/photos/greeblie/2224507899

We’ve been there...

23

Now that we have containers...Isolation: Keep jobs from interfering with each other

Scheduling: Where should my job be run?

Lifecycle: Keep my job running

Discovery: Where is my job now?

Constituency: Who is part of my job?

Scale-up: Making my jobs bigger or smaller

Auth{n,z}: Who can do things to my job?

Monitoring: What’s happening with my job?

Health: How is my job feeling?


Kubernetes

Manage applications, not machines

Open source, container orchestrator

Supports multiple cloud and bare-metal environments

Inspired and informed by Google’s experiences and internal systems

Design principles

Declarative > imperative: State your desired results, let the system actuate

Control loops: Observe, rectify, repeat

Simple > Complex: Try to do as little as possible

Modularity: Components, interfaces, & plugins

Legacy compatible: Requiring apps to change is a non-starter

Network-centric: IP addresses are cheap

No grouping: Labels are the only groups

Bulk > hand-crafted: Manage your workload in bulk

Open > Closed: Open Source, standards, REST, JSON, etc.

2727

Kubernetes Made Easy demo

2828

Pods


PodsLogical Application

Pod


PodsLogical Application• One or more containers

Pod



Pod

nginx

monolith



and volumes

Pod

nginx

monolith



and volumes

Pod

nginx

monolith

NFSiSCSIGCE



and volumes• Shared namespaces

Pod

nginx

monolith

NFSiSCSIGCE



and volumes• Shared namespaces• One IP per pod

Pod

nginx

monolith

NFSiSCSIGCE

10.10.1.100




Pod

nginx

monolith

NFSiSCSIGCE

10.10.1.100

3737

LabCreating and managing podshttp://github.com/bretmcg/kubernetes-workshop

3838

Health checks


Monitoring and Health Checks

Node

Kubelet PodPodapp v1



Hey, app v1... You alive?

Node

Kubelet Podapp v1app v1



Node

Kubelet Nope!Pod

app v1app v1



OK, then I’m going to restart you...

Node




Node

Kubelet Pod



Node

Kubelet Podapp v1



Node

Kubelet


Podapp v1



Node

Kubelet Yes!Pod

app v1



Node

Kubelet Podapp v1

4848

LabMonitoring and health checkshttp://github.com/bretmcg/kubernetes-workshop

4949

Secrets


Secrets and Configmaps

Kubernetes Master

etcdAPI

Server

Node

Kubeletsecret

$ kubectl create secret generic tls-certs --from-file=tls/



Kubernetes Master

etcdAPI

Server

Node

Kubeletpod

$ kubectl create -f pods/secure-monolith.yaml



Kubernetes Master

etcdAPI

Server

Node

KubeletAPI

Server

Node

Kubelet Pod

Pod



Kubernetes Master

etcdAPI

Server

Node

KubeletAPI

Server

Node

Kubelet Pod

Podsecret



Kubernetes Master

etcdAPI

Server

Node

KubeletAPI

Server

Node

Kubelet Pod

Pod

/etc/tls

secret



Kubernetes Master

etcdAPI

Server

Node

Kubelet

Node

Kubelet Pod

Pod

/etc/tls/etc/tls

10.10.1.100

secret

API Server



Kubernetes Master

etcdAPI

Server

Node

KubeletAPI

Server

Node

Kubelet Pod

Pod

/etc/tls

nginx

10.10.1.100

secret

5757

LabManaging application configurations and secretshttp://github.com/bretmcg/kubernetes-workshop

5858

Services


Services

Node1 Node3Node2

Podhello

Service

Podhello

Podhello


ServicesPersistent Endpoint for Pods

Node1 Node3Node2

Podhello

Service

Podhello

Podhello


Services

Node1 Node3Node2

Podhello

Service

Podhello

Podhello

Persistent Endpoint for Pods• Use Labels to

Select Pods


LabelsArbitrary meta-data attached to Kubernetes object

Pod

hello

Pod

hello

labels: version: v1 track: stable

labels: version: v1 track: test


Labelsselector: “version=v1”

Pod

hello

Pod

hello




Labelsselector: “track=stable”

Pod

hello

Pod

hello




ServicesPersistent Endpoint for Pods• Use Labels to

Select Pods• Internal or

External IPsNode1 Node3Node2

Podhello

Service

Podhello

Podhello

6666

LabCreating and managing serviceshttp://github.com/bretmcg/kubernetes-workshop

6767

Recap


Kubernetes

Manage applications, not machines

Open source, container orchestrator Supports multiple cloud and bare-metal

environments

Inspired and informed by Google’s experiences and internal systems


machine-1

machine-2

machine-3

frontend middleware backend

Physical Infrastructure


frontend

middleware

backend

Kubernetes API: Unified Compute Substrate

Logical Infrastructure


Goal: Write once, run anywhere*

Don’t force apps to know about concepts that are cloud-provider-specific

Examples of this:● Network model● Ingress● Service load-balancers● PersistentVolumes

* approximately

Workload Portability


Top 0.01% of all GitHub projects

1200+ externalprojects based on

k8s

Companies Contributing

Companies Using

690+unique contributors

Community




Pod

nginx

monolith

NFSiSCSIGCE

10.10.1.100




Node




Kubernetes Master

etcdAPI

Server

Node

Kubeletsecret

$ kubectl create secret generic tls-certs --from-file=tls/


ServicesPersistent Endpoint for Pods• Use Labels to

Select Pods• Internal or

External IPsNode1 Node3Node2

Podhello

Service

Podhello

Podhello


LabelsArbitrary meta-data attached to Kubernetes object

Pod

hello

Pod

hello



Kubernetes in Production

7979

Deployments


Drive current state towards desired stateDeployments

Node1 Node2 Node3

Podhello

app: helloreplicas: 1



Node1 Node2 Node3

Podhello




Node1 Node2 Node3

Podhello


Podhello

Podhello



Node1 Node2 Node3

Podhello


Podhello



Node1 Node2 Node3

Podhello


Podhello

Podhello



Node1 Node2 Node3

Podhello


Podhello

Podhello

Podhello



Node1 Node2 Node3

Podhello


Podhello

Podhello

8787

LabCreating and managing deploymentshttp://github.com/bretmcg/kubernetes-workshop

8888

Rolling Updates


Rolling Update

Node1 Node3Node2

ghostPod

app v1

Service

ghost

Podapp v1

Podapp v1


Rolling Update

Node1 Node3Node2

ghostPod

app v1

Service

ghost

Podapp v1

Podapp v1

Podapp v2


Rolling Update

Node1 Node3Node2

ghostPod

app v1

Service

ghost

Podapp v1

Podapp v1

Podapp v2


Rolling Update

Node1 Node3Node2

ghostPod

app v1

Service

ghost

Podapp v1

Podapp v1

Podapp v2


Rolling Update

Node1 Node3Node2

Service

ghost

Podapp v1

Podapp v1

Podapp v2


Rolling Update

Node1 Node3Node2

Service

ghost

Podapp v1

Podapp v1

Podapp v2

Podapp v2


Rolling Update

Node1 Node3Node2

Service

ghost

Podapp v1

Podapp v1

Podapp v2

Podapp v2


Rolling Update

Node1 Node3Node2

Service

ghost

Podapp v1

Podapp v1

Podapp v2

Podapp v2


Rolling Update

Node1 Node3Node2

Service

Podapp v1

Podapp v2

Podapp v2


Rolling Update

Node1 Node3Node2

Service

Podapp v1

Podapp v2

Podapp v2

Podapp v2


Rolling Update

Node1 Node3Node2

Service

Podapp v1

Podapp v2

Podapp v2

Podapp v2


Rolling Update

Node1 Node3Node2

Service

Podapp v1

Podapp v2

Podapp v2

Podapp v2


Rolling Update

Node1 Node3Node2

Service

Podapp v2

Podapp v2

Podapp v2

102102

LabRolling out updateshttp://github.com/bretmcg/kubernetes-workshop

103103

Implementing a CI/CD Pipeline on K8s


1. Check in code

2. Build an Image

3. Test Image

4. Push Image to registry

5. Apply change to manifest files

Automating Deployments

105105

LabImplementing a CI/CD Pipeline on Kuberneteshttps://github.com/GoogleCloudPlatform/continuous-deployment-on-kubernetes

Thank you!

kubernetes.io

@bretmcg @_askcarter

Technology

Kubernetes 101 Workshop