© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Solutions Architect, Amazon Web Services Japan

Containers and the Evolution of ComputingRyosuke Iwanaga

July 2016, LinuxCon+ContainerCon Japan

Scaling Applications

Order UI User UI Shipping UI

Order Service

User Service

Shipping Service

Order UI User UI UI

Order Service Service Shipping

Service

Order UIOrder UI

User UI UIShipping UI

Order ServiceOrder

ServiceService

ServiceService

ServiceUser

Service

Shipping Service

What is DevOps?

DevOps = efficiencies that speed up this lifecycle

developers customers

releasetestbuild

plan monitor

delivery pipeline

feedback loop

Software development lifecycle

Monolith development lifecycle

developers

releasetestbuild

delivery pipelineapp

Microservice development lifecycle

developers delivery pipelinesservices

releasetestbuild

releasetestbuild

releasetestbuild

releasetestbuild

releasetestbuild

releasetestbuild

Services

• Not only for "microservices"; • Division, New business, Internal/External, etc.

• There tends to be a lot of "services"• From startup to enterprise

• So many services, so many pipelines/devops

DevOps reality

Build Test ProductionSource

Application Artifact

All my need is Code…?

DevOps reality

Build Test ProductionSource

Application Artifact

Provision

Config

Maintain development environment…

Environment difference between dev, test, prod…

Demands for test workload is diverse…

Auto scaling…Node fail...

OK, all my need is everything…

Multiple DevOps Reality

DevOps difficulty

• Too many things to handle• "Unicorn" person / team

• Too many different pipelines• Services, Languages, Frameworks, Versions, etc.

What are Containers?

OS virtualization

Process isolation

Images

Automation Server

Guest OS

Bins/Libs Bins/Libs

App2App1

Container Advantages

Portable

Flexible

Fast

EfficientServer

Guest OS

Bins/Libs Bins/Libs

App2App1

Containers are Natural for Microservices

Simple to model

Any app, any language

Image is the version

Test & deploy same artifact

Stateless servers decrease change risk

DevOps with Docker

Build Test ProductionSource

Application Image

Provision

Config

All my need is Code! Yah!

Server

Guest OS

Bins/Libs Bins/Libs

App2App1

Managing One Host is Straightforward

Managing a Fleet is Hard

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

Server

Guest OS

AZ 1 AZ 2

AZ 3

What is Amazon ECS?

Amazon EC2 Container Service (Amazon ECS) is a highly scalable, high performance container management service. You can use Amazon ECS to schedule the placement of containers across your cluster. You can also integrate your own scheduler or third-party scheduler to meet business or application specific requirements.

Container Management at Any Scale

Nothing to run

Complete state

Control and monitoring

Scale

Flexible Container Placement

Long running applications

Batch jobs

Multiple schedulers

Integration with the AWS Platform

Elastic Load Balancing

Amazon Elastic Block Store

Amazon Virtual Private Cloud

Amazon CloudWatch

AWS Identity and Access Management

AWS CloudTrail

Container Management

What is a Container Manager?

• Maintains available resources

• Tracks resource changes

• Accepts resource requests

• Guarantees accuracy and consistency

CPU

Memory

Ports

Disk space

Disk IOPS

Network bandwidth

Resources

How do you model your applications?

{"environment": [],"name": "simple-demo","image": "my-demo","cpu": 10,"memory": 500,"portMappings": [

{"containerPort": 80,"hostPort": 80

}],"mountPoints": [

{"sourceVolume": "my-vol","containerPath": "/var/www/my-vol"

}],"entryPoint": [

"/usr/sbin/apache2","-D","FOREGROUND"

],"essential": true

},

{"name": "simple-demo","image": "my-demo","cpu": 10,"memory": 500,"portMappings": [

{"containerPort": 80,"hostPort": 80

}],"entryPoint": [

"/usr/sbin/apache2","-D","FOREGROUND"

],"essential": true

},

“Task Definitions”

Tasks

Shared Data Volume

Containers

launchContainer Instance

Volume Definitions

Container Definitions

Schedulers

What is a Scheduler?

• Determine desired state

• Check against current state

• Perform action

Cluster, Scheduler, Task Scheduler

ManagerCluster

Task Definition

Task

Agent

ECS Agent

Docker

Task

Container Instance

Container

ECS Agent

Task

Container

https://github.com/aws/amazon-ecs-agent

Instance Registration

register-container-instance --total-resources

[{

“name” : “cpu”,“type” : “integerValue”,“integerValue” : 2048

},…

]

Starting a Task

API

User / Scheduler

StartTask

Starting a Task

API

User / Scheduler

StartTask

Cluster Management Engine

Starting a Task

API

User / Scheduler

StartTask

Cluster Management Engine

Agent Communication

Starting a Task

API

User / Scheduler

StartTask

Cluster Management Engine

Agent Communication

Docker

Container Instance

ECS Agent

TaskContainer

WebSocket

Starting a Task

API

User / Scheduler

StartTask

Cluster Management Engine

Agent Communication

DockerTask

Container Instance

Container

ECS Agent

TaskContainer

SubmitStateChange

Tracking Resource Changes

Terminated Task

API

User / Scheduler

StartTask

Cluster Management Engine

Agent Communication

DockerTask

Container Instance

Container

ECS Agent

SubmitStateChange

Missing Container Instance

API

User / Scheduler

StartTask

Cluster Management Engine

DockerTask

Container Instance

Container

ECS Agent

?

Agent Communication

Terminated Container Instance

API

User / Scheduler

StartTask

Cluster Management Engine

Agent Communication

Termination Notifier

DockerTask

Container Instance

Container

ECS Agent

Guaranteeing Accuracy and Consistency

Amazon ECS Under the Hood

IDN-1 IDN IDN+1 IDN+2 IDN+3 IDN+4 IDN+5

IDN+6

IDN+5

WRITE

READ

Amazon ECS Under the Hood

IDN-1 IDN IDN+1 IDN+2 IDN+3 IDN+4 IDN+5

IDN+6IDN+3

IDN+5IDN+2

WRITE WRITE

READREAD

Scalable

Multiple Schedulers

Amazon ECS: Scheduling• Each scheduler periodically queries the current cluster state

Copy of cluster stateScheduler A Scheduler B

Cluster

Amazon ECS: Scheduling• Each scheduler allocates tasks on the cluster• Each scheduler updates the current cluster state

Run a taskRun a task

Amazon ECS: Scheduling• If the resource is already claimed, the request will be rejected

Run a task on the same resource=> Transactional

Amazon ECS: Scheduling

• Shared state optimistic scheduling• All schedulers can see the current cluster state at all times

Amazon ECS Service Scheduler

What is a Service?

• Models a long-running application

• Maintains desired state

• Optionally runs behind an Elastic Load Balancing load

balancer

Scheduling Containers: Long-running App

Deploy using the least space:minimumHealthyPercent = 50%, maximumPercent = 100%

Old version New version

Scheduling Containers: Long-running AppDeploy quickly without reducing service capacity: minimumHealthyPercent = 100%, maximumPercent = 200%

Old version New version

Task Auto Scaling

Task Auto Scaling

• Service Scheduler is integrated with Auto Scaling• CloudWatch Alarm => Policy => Modify Desired count

• Useful CloudWatch metrics:• CPU/Memory Utilization per Service

• How much each task consumes reserved resources?• CPU/Memory Utilization per Cluster

• How much actual cluster-wide resources are consumed?• CPU/Memory Reservation per Cluster

• How much cluster-wide resources are reserved?

Monitoring with Amazon CloudWatch Dashboards

Case studies

“Moving to Amazon ECS significantly improved our service performance. We reduced service response times in the 99th percentile by 50%”

Jason FischlVP of Engineering

SegmentTo collect customer data in a single hub for later use in analytics, marketing, and for other purposes.

"Switching to Amazon ECS has greatly simplified running a service without needing to worry about provisioning or

availability."

Calvin French-OwenCofounder and Chief Technology Officer

Before• Instance based• Manual setup• Wrong config / Out of sync

After• Easy maintaining, stateless• CI/CD pipeline automated• Focus on developing

https://aws.amazon.com/solutions/case-studies/segment/

To recap

{"environment": [],"name": "simple-demo","image": "my-demo","cpu": 10,"memory": 500,"portMappings": [

{"containerPort": 80,"hostPort": 80

}],"mountPoints": [

{"sourceVolume": "my-vol","containerPath": "/var/www/my-vol"

}],"entryPoint": [

"/usr/sbin/apache2","-D","FOREGROUND"

],"essential": true

},

“Task Definitions”

Scheduler

ManagerCluster

Task Definition

Task

Agent

Thank you!