42
Scalable Environments using Kubernetes and VMSS on Azure Cloud Noam Shochat

Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

  • Upload
    others

  • View
    24

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Scalable Environments using

Kubernetes and VMSS on Azure Cloud

Noam Shochat

Page 2: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1
Page 3: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1
Page 4: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1
Page 5: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1
Page 6: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Infrastructure Distribution

London US

West

North

Europe

West

Europe

US

East

@noamshochat

Page 7: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Base Code

@noamshochat

Page 8: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

CI / CD, Configuration management tools and scripting languages

@noamshochat

Page 9: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Agenda

● Challenge #1 - Development and QA

● Challenge #2 - QA Automation

● Challenge #3 - Production Readiness

@noamshochat

Page 10: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Challenge #1Development and QA

Personal environments

@noamshochat

Page 11: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Challenge #1Development and QA

● Growing Number of QA employees

○ Each QA person needs 4

servers (For now), 1 for each

environment.

● Limited number of VMs

○ High Costs

○ Resource Management

@noamshochat

Page 12: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

New employee engagement

Page 13: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

SolutionPersonal environment by demand

How?

Docker on Kubernetes

@noamshochat

Page 14: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Kubernetes

@noamshochat

Page 15: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Kubernetes on Azure (AKS)

● Managed

● Dynamic Scale

● Simple Deployment

@noamshochat

Page 16: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

front front

assetsassets

Ingress

Service_user 1 Service_DGService_user 2Service_user n . . .

@noamshochat

Page 17: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Services

Ingress

Traefik

@noamshochat

Page 18: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Traefik

A reverse proxy / load balancer that's easy,

dynamic, automatic, fast, full-featured, open source,

production proven, provides metrics, and integrates

with every major cluster technology...

@noamshochat

Page 19: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Jenkins Pipeline

@noamshochat

Page 20: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Jenkins Pipeline

@noamshochat

Page 21: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Users Private DNS

● DNS-1: front${USER_ID}.int.costco.com

● DNS-2: front${USER_ID}.dev.costco.com

● DNS-3: front${USER_ID}.prod.costco.com

● DNS-3: front${USER_ID}-2.prod.costco.com

@noamshochat

Page 22: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

@noamshochat

Page 23: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

front front

assetsassets

Ingress

Service_user 1 Service_DGService_user 2Service_user n . . .

@noamshochat

Page 24: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Benefits

● Self service

● Better use of resources

● Almost unlimited environments

● Faster deployment times

● Simplicity

● If it’s not working Blame the

developer :-)@noamshochat

Page 25: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Challenge #2QA Automation

@noamshochat

Page 26: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Challenge # 2QA Automation

● Simulating client behavior

○ Variable number of VMs for

different runs

○ Hundreds of tests

○ Windows OS VMs

○ Make it cost effective and fast

@noamshochat

Page 27: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Solution

● Azure VMSS Solution

@noamshochat

Page 28: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Virtual Machine Scale Set

VMSS is an Azure solution for managing a group of identical, load balanced

VMs with elastic autoscale

● Easy to create and manage multiple VMs

● Provides high availability and application resiliency

● Dynamic and large scale support

● Automatic (templates, events driven, mgmt)

@noamshochat

Page 29: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

QA Automation Flow1. Initiate test scenario

2. Creating VMSS cluster

3. VMs connect to the master as Jenkins slaves

4. Download test code to each VM simultaneously

5. Destroy VMs once tests are complete and results are reported

@noamshochat

Page 30: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

QA Automation Tests results

@noamshochat

Page 31: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

QA Automation Time Saved

26 hours → 1.5 hour

@noamshochat

Page 32: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Challenge # 3Production Readiness

@noamshochat

Page 33: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Challenge # 3Production Readiness

● 1...N VMs

● move from Classic Cloud service

● Make it cost effective

● Redundant

● Scale in/out

● Self service

@noamshochat

Page 34: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Solution

● Azure VMSS Solution

@noamshochat

Page 35: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Production Deployment Flow

@noamshochat

Page 36: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Deployment

Template

@noamshochat

Page 37: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Multi Regional Architecture

@noamshochat

Page 38: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Multi Regional Architecture

@noamshochat

Page 39: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Multi Regional Architecture

Page 40: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Benefits● Large number of VMs

● Repeatable deployment

● Easy rollback

● Faster deployment times

● Cost effective

● Infrastructure as code

● Easy to maintain

● Automatic scale

@noamshochat

Page 41: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

Thank You

@noamshochat

Page 42: Scalable Environments using Kubernetes and VMSS on Azure Cloud · Kubernetes and VMSS on Azure Cloud Noam Shochat. Infrastructure Distribution ... @noamshochat. Agenda Challenge #1

If you Q I will A

@noamshochat