28
www.scout24.com To Infinity and Beyond Handling heterogenous container clusters in AWS AWS Summit | Feb 2019 | Christine Trahe, Cloud Platform Engineering

To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

  • Upload
    others

  • View
    2

  • Download
    0

Embed Size (px)

Citation preview

Page 1: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

www.scout24.com

To Infinity and Beyond

Handling heterogenous container clusters in AWS

AWS Summit | Feb 2019 | Christine Trahe, Cloud Platform Engineering

Page 2: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Title | Your name

2

Page 3: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

3

Containerized Applications

running on AWS

Page 4: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Enabling product engineers to

focus on delivering value to

Scout24 consumers and

customers.

4

Platform Engineering, Scout24

Page 5: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Platform Setup

Page 6: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Multi-Account Strategy

6

Page 7: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Multi-Account Strategy

7

Page 8: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Keeping it simple

8

Custom resource

Basic configuration

Mandatory Tagging

Page 9: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

What the user sees

9

Page 10: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Under the hood: Centralized compute infrastructure

10

Page 11: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Under the hood: Logging and Monitoring

11

Page 12: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

What the user sees

12

Page 13: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Learnings

• Services affecting eachother

• CloudFormation Application deployment

• Tooling around ECS

Page 14: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Services Affecting Eachother

Page 15: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

But it doesn‘t scale to Infinity just yet…

15

Too much CPU consumption

on the container instances

Load Balancer response time (latency)

increases, causing real time customer impact

Page 16: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Sharing resources not ideal…

16

Page 17: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Application differences

17

Page 18: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Categorizing Services to their needs

18

Page 19: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Deploying Applications through CloudFormation

Page 20: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

It‘s CloudFormation stacks all the way down…

20

Page 21: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Stuck in Rollback State

21

Page 22: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Deploying directly with EKS K8S API

22

Page 23: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Tooling around ECS

Page 24: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Cross Account Permissions for a Service

24

Page 25: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Many open source tools for K8S

25

Page 26: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Key Takeaways

Page 27: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

27

• Lower level building block

• Non-heterogenous workloads

• Smaller community

• Not for application deployment (API calls, AWS

CodeDeploy)

• Infrastructure deployment

• Platform abstraction

• Higher level of abstraction

• Heterogenous workloads

• Large open source community

• AWS abstracts complexity

Amazon

EKS

• What level to abstract

• Standards and best practices

Platform

Engineering

Page 28: To Infinity and Beyond - aws-de-marketing.s3-eu-central-1 ...... · • Not for application deployment (API calls, AWS CodeDeploy) • Infrastructure deployment • Platform abstraction

Christine Trahe

AWS Summit Feb 2019| Cloud Platform Engineering, Scout24

https://jobs.scout24.com/#/department-positions/PlatformEngineering

Thank you! Questions?