Tales of Training: Scaling CodeLabs with Swarm Mode and Compose

Damien DUPORTALTraining Engineer - CloudBees

@DamienDuportal - github.com/dduportal

AgendaWhat is this talk about?

● Trench Story● Drinking champagne● Training / CodeLabs● VM workload● Container workload● Orchestration

What is this talk NOT about?

● Toolings wars:○ Kubernetes vs. Swarm○ AWS vs. Azure

● Showing code (20 min only!)○ Help yourself (GitHub):

http://bit.ly/2yoYG0v

Drinking my own champagneLearn by TRYING:● Experimentation never wastes

time● Build something that works

and iterate

“CodeLab”?● Part of a training (learning experience)● Practical Exercise● Using an isolated environment ● Dedicated to a single user ● Synonym of “workshop” or

“hands on lab”

CodeLab == ProductionConstraints:● Network● Configuration diversity● User Error (Learning process)● Time boxed Sessions

CodeLab VM Content: ● Jenkins Training:

○ Store Git SCM (Gitea - Go)○ Jenkins Master + Build Agents (JVMs)○ Web Command Line (TTYDs - Python)○ WebIDE (Codiad - PHP)○ Store Artifacts (Artifactory - JVM)○ Docker Registry (Go)

Challenges● CodeLab defined by service list

○ Docker-compose● Portability

○ Docker as blueprint● Explicit definition

○ Definition as code

Step 1: VMs

Consul Servers

AWS EC2 VM

Host OS: Ubuntu

Docker & Compose

CodeLab Service Stack

Consul

AWS EC2 VM

Host OS: Ubuntu

Docker & Compose

CodeLab Service Stack

Consul

AWS EC2 VM

Host OS: Ubuntu

Docker & Compose

CodeLab Service Stack

Consul

AWS EC2 VM

Host OS: Ubuntu

Docker & Compose

CodeLab Service Stack

Consul

AWS EC2 VM

Host OS: Ubuntu

Docker & Compose

CodeLab Service Stack

Consul

AWS EC2 VM

Host OS: Ubuntu

Docker & Compose

CodeLab Service Stack

Consul

SSH Bastion

Reverse Proxies

Register into Consul

End User

Trainer

Deploys with Terraform

Load-Balancer

Reconfigure with Consul-Template

Step 1: Take AwaysIt works!Easy to begin with and to operateCost (1 attendee per VM)TTP (Time To Play)

Road to Native Orchestration● Solving Costs Issue: More attendees per VM● Solving TTP Issue:

○ Moving to container-based workload(Look at PWD speed!)

○ Un-tie Infra from Services

Step 2.1: Provide Infra

SSH Bastion“Infra Admin”

Deploys with Terraform

Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)

Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)

Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)

Swarm Manager (EC2 VM)Swarm Manager (EC2 VM)

Swarm Manager (EC2 VM)

Step 2.2: Provide Services

SSH Bastion“Infra Admin”

Deploys with Docker

Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)

Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)

Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)

Swarm Manager (EC2 VM)Swarm Manager (EC2 VM)

Swarm Manager (EC2 VM)

Deploys with Docker

Trainer

CodeLab Service Stack

Support Services

Support Services

Step 2.3: Benefits

Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)

Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)

Swarm Worker (EC2 VM)Swarm Worker (EC2 VM)

Swarm Manager (EC2 VM)Swarm Manager (EC2 VM)

Swarm Manager (EC2 VM)

CodeLab Service Stack

Support Services

Support Services

End User

Step 2: Positive Side Effect

● Relying on Docker ecosystem● Less side-kicks to maintain

Step 2: Nice problems to have● CodeLab Scaling Pattern

○ (Deploy N times, NOT Scaling)● Monitoring / Metrics● Resource Management

CodeLab Scaling Pattern● Compose and Swarm:

○ Define a stack of services○ Scale each horizontally

● Concern here: ○ Deploy N times the same stack

Monitoring/MetricsIt is NOT a challenge

● Should have done it at Step 1● Feedback loop● A LOT of “Working Out

of the box” tools

Resource Management● Resource sharing patterns:

○ VM workloads: 1 stack for 1 VM resources○ Container workload: N stacks for

N VMs resources● JVM pattern: Capping Memory (OOM)● Need to measure and iterate

○ Automation

Step 2: Take Aways● It still works!● Easier to begin with and to operate● Cost: 4x less (4 attendees per VM)● TTP: ~10 s instead of ~10 min● Maintaining: 5 tools instead of 8

Thank you and enjoy DockerCon!