How to Scale Kubernetes in AWS - Amazon Web Services, Inc. · The DevOps Institute (DOI) is the continuous x learning community dedicated to advancing the human elements of DevOps

How to Scale Kubernetes in AWS

xThe DevOps Institute (DOI) is the continuouslearning community dedicated to advancing thehuman elements of DevOps success.

Our mission is based on a framework of skills,

knowledge, ideas and learning (SKIL) that

practitioners need to advance DevOps and their

careers. All of our work, including accreditations,

research, events and continuous learning

programs – is focused on providing the “know-

how” to make DevOps possible.

Join our FREE community at www.devopsinstitute.com

About DevOps Institute

Speakers

Kubernetes allows development teams to move

their application workloads without having to

redesign their applications or completely rethink

their infrastructure thus providing flexibility and

the ability to standardize on a platform.

However, as you start to operate your

Kubernetes environments at scale, it becomes

more complex and requires special focus on

monitoring for performance and health.

Today you will discover how to proactively

approach the health and performance monitoring

of your Kubernetes environments—at any scale

and at any level of complexity. You will also

discover tools you can use to get useful answers

to fundamental questions, such as “What is the

health of my cluster?” and “What is the hierarchy

of the elements within my cluster?”

Synopsis

A Transformation

The Modern Architecture of containers, Kubernetes

and microservices is a massive shift from

monolithic practices. It will impact:

1) The way we develop software.

2) The way we deliver software and our CI/CD

Pipeline.

3) The way we manage and monitor our ‘data

center.’

In essence - everything is being transformed.

A Tsunami

In order to achieve success,

it will require an open mind, a

willingness to learn, and the

ability to shift our thinking

from monolithic practices to a

new microservice practice.

In other words, we will need to surf this wave into

a new form of software development.

Kubernetes Container

Orchestration:

An open source platform

from Google for

orchestrating containers

across clusters of servers

providing auto scaling and

fault tolerance.

Containers and Docker:

A container is a standard

unit of software that

packages up code and all

dependencies so the

application runs quickly

and reliably.

Microservices:

An architectural style that

structures an application as a

collection of loosely coupled

services.

In a microservices architecture,

services are fine-grained and

independently deployable.

Let's Start at the Beginning

A container provides an

isolated context in which an

application, together with its

environment can run.

A container provides an isolated context

in which an application, together with its


• Need to be managed

• Networking is hard

• Must scheduled, distributed and load balanced.

• Data must persist somewhere.

Why Kubernetes and Not Just Containers?

Kubernetes Orchestration





Kubernetes ensures that the

cluster continues to serve users

no matter how its environment is

altered, with minimal intervention

from operations teams.

Self-Healing• Finds and restarts failed containers.

• Finds and reschedules failed Nodes.

• Destroys unresponsive containers

Nodes are a VM for Pods.

Kubernetes Parts and Pieces





“Pods run your containers. There

is at least one container for every

Pod.

The Pod controls the execution of

that container. When the

container exit, the pod dies too.”

Phippy Goes to the Zoo –A Kubernetes

Story, by Matt Butcher & Karen Chu,

Illustrated by Bailey Beougher, Renee

French

Pods and Nodes

Kubernetes Cluster





Node

POD POD

Container

Container

Container

Container

Node

POD POD

Container

Container

Container

Container

Node

POD POD

Container

Container

Container

Container

Node

POD POD

Container

Container

Container

Container

Node

POD POD

Container

Container

Container

Container

Kubernetes Parts and Pieces





Replica Sets

ReplicaSet ensures that a set of identically configured Pods are running at the desired replica

count. If a Pod drops off, the ReplicaSet brings a new one online as a replacement.”

Phippy Goes to the Zoo –A Kubernetes Story, by Matt Butcher & Karen Chu, Illustrated by Bailey Beougher,

Renee French

Kubernetes Parts and PiecesDeployments

Deployments controls deploying and maintaining a set of Pods. It uses a ReplicaSet to keep Pods

running, but offers sophisticated logic for deploying, updating, and scaling a set of Pods within a cluster.”

Phippy Goes to the Zoo –A Kubernetes Story, by Matt Butcher & Karen Chu, Illustrated by Bailey Beougher, Renee

French

Kubernetes Parts and PiecesDeamonSets

DaemonSets provide a way to ensure that a copy of a Pod is running on every node in a cluster.

As a cluster grows and shrinks, the DaemonSets spreads these specially labeled Pods across all of the

nodes.”

Phippy Goes to the Zoo –A Kubernetes Story, by Matt Butcher & Karen Chu, Illustrated by Bailey Beougher, Renee French

Kubernetes Parts and PiecesIngresses

“Ingresses provide a way to declare that traffic ought to be channeled from the outside of the cluster into

destination points within the cluster. One single external Ingress point can accept traffic destined to

many different internal services.”


Kubernetes Parts and PiecesCustom Resource Definitions

CRDs provide an extension mechanism that cluster operators and developers can use to create their own

resource types.“


Kubernetes Parts and PiecesLabels, Named Spaces & Volumes

“Kubernetes uses labels as “nametags” to

identify things. Labels are open-ended. You

can use them to indicate roles, stability, or

other important attributes.” The Illustrated

Children’s Guide to Kubernetes

“A namespace functions as a grouping mechanism inside of

Kubernetes. Services, pods, replication controllers, and volumes

can easily cooperate within a namespace, but the namespace

provides a degree of isolation from the other parts of the cluster.”

The Illustrated Children’s Guide to Kubernetes

“A service tells the rest of the

Kubernetes environment (including

other pods and replication controllers)

what services your application

provides. While pods come and go,

the service IP addresses and ports

remain the same. And other

applications can find your service

through Kubernetes service

discovery.”

The Illustrated Children’s Guide to

Kubernetes

And Then There are Microservices

“Microservices are a software

development technique—a variant of

the service-oriented architecture

architectural style that structures an

application as a collection of loosely

coupled services. In a microservices

architecture, services are fine-grained

and the protocols are lightweight.”

Wikipedia

APIAPI APIAPI

API

APIAPI API

API

API

API

Loosely Coupled

Shifting Configuration

Source Code

Version

Repository

Monolithic CI/CD

Compile and Link

DEV PROD

DEV PROD

Microservices CI/CD

Microservice

Version

Repository

API Link

Microservices CI/CD

DEV

DEV Prod

Prod

Your New Challenge – The Death Star

DEV PROD

DEV PROD

Microservices CI/CD

A Whole New World

DEV PROD

DEV PROD

Microservices CI/CD

Big ComplexityThis is a complex environment but delivers big benefits:

• Auto Scaling

• Fault Tolerance

Making it SimpleNew cloud native solutions will simplify the Kubernetes platform from

configuration management to monitoring. Your current tools for Dev and

Ops will need to be modified or upgraded to meet the challenges of this

new modern architecture.

©2008–20 New Relic, Inc. All rights reserved

How to scale Kubernetes in AWSStephen ZornStijn Polfliet


Traces

Metrics

Events

Logs

NRDB

NR Proprietary

Agents

METRICS

TRACESLOGS

APPS

SERVICES

HOST

Open

Telemetry

The New Relic Observability PlatformA new target architecture for Digital Transformation


What is Observability

In control theory

Internal state inferred from external data

In software

A connected, real-time view of all performance data in one place, in order to:

Pinpoint issues faster

Understand what caused an issue and why

Proactively tie data to business outcomes

Monitoring tells you when something is wrong. Observability lets you ask why.

Root causeRoot cause

!

Symptom

Symptom

Symptom

Problem

!

!


Monitoring Practices Have Evolved

Many Monitoring

Tools

On-Prem Hybrid Cloud /SaaS

Thousands of Microservices

Multiple Deployments

Daily


Taking a Measured Approach to Cloud AdoptionC

usto

me

r V

alu

e

Cloud Adoption Over Time

PilotInitial deployment and experimentation

in the cloud Feasibility study

FoundationFull stack instrumentation and observability

Pilot modern IT and financial management

MigrationMore predictable project outcomes Accelerated

time to Value Improved Service Delivery

Continual Modernization / OptimizationFocus on customer experience Increase cadence of release

Increased standards and governance across teams

Thriving Digital BusinessDigital Intelligence prioritizing investment compete

against disruptors improved business outcomes

Cloud Journey with New Relic


Transformation: Speed vs. Complexity

Increasing Complexity

Evolving Teams & Practices

Seconds Not MinutesMore Data from More Sources


Digital Transformation Requires Observability

Open Connected Programmable



5 Complete Service Observability

2 Dynamic Cluster Behavior

3 Microservices Communication

4 Log Correlation

1 Overall Kubernetes Health & Capacity

Observability - 5 things to care about


2-click troubleshooting


1. Overall Kubernetes health

Install kube-state-metrics for overall health & state of each Kubernetes object

Use Kubernetes secrets to storelicense keys & passwords

Deploy Observability agent as a DaemonSet


1. Overall Kubernetes health & capacity

Set resource requests & limitsto control cluster capacity

Good to know:Pod over cpu limit - throttledPod over memory limit - terminated


2. Dynamic cluster behavior

Kubernetes API server tracks all eventsUse kubectl get events

Use health checks

Readiness probes When is my pod ready to accept traffic?

Liveness probesIs my pod still alive?


3. Microservices communication

Use Distributed Tracing to understand communication between microservices

Get automatic distributed tracing by adding the New Relic APM agent to your containers


4. Log correlation

Fluent Bit is lightweight

Use the New Relic Fluent Bit plugin


5. Complete Service observability

Scrape any Prometheus endpoint

Use prometheus.io/scrape label and the New Relic Prometheus OpenMetrics integration

Build a MELT dashboard

Metrics, Events, Logs, Traces

Monitor Redis, RabbitMQ (and many more)

with New Relic service integrations


2-click troubleshooting!


5 Complete Service Observability

2 Dynamic Cluster Behavior

3 Microservices Communication

4 Log Correlation

1 Overall Kubernetes Health & Capacity

Observability - 5 things to care about

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Why Kubernetes on AWS?

Chris Chapman

Partner Solutions Architect, AWS


#1 place to run containers

Source: https://nucleusresearch.com/research/single/guidebook-containers-and-kubernetes-on-aws/

80%of all containerized

applications running in

the cloud run on AWS

84%of all Kubernetes

applications running in

the cloud run on AWS

https://aws.amazon.com/containers/


Containers on AWS

Amazon Elastic

Kubernetes Service (EKS)

Amazon Elastic

Container Service (ECS)

Amazon Elastic

Container Registry (ECR)AWS Fargate

AWS Deep Learning

Containers

AWS Elastic

Beanstalk

AWS Batch


New Relic integration


Optimize container spend with New Relic

Track and forecast AWS spend with dashboards

View and optimize your host health from any AWS attribute

Accurate, real-time view of all your dynamic assets

Failures and downtime are also a cost!


Gaining cluster visibility

Scalable custom

environments

Simplified and

accelerated

deployment

Full visibility into

infrastructure health

70% cost reduction

News UK transitions customer Kubernetes deployments to EKS

“New Relic gives our infrastructure and application teams the ability to dive through the entire stack, diagnose the

issue, and then resolve it as rapidly as possible.” – Martin Cuber, Senior Cloud DevOps Engineer, News UK

Custom infrastructure visibility


Get started on AWS Marketplace

Flexible consumption and contract models

Quick and easy deployment

Helpful humans to support you


Points to consider

Easily experiment with software using AWS Marketplace – without license lock-in.

Leverage expertise of others so you can focus on the fundamentals of your business.

You need to be monitoring performance to make the decisions that drive your

iterative evolution.

Kubernetes and microservices are a massive shift from monolithic practices.


Resources

• AWS Marketplace: aws.amazon.com/marketplace

• AWS Kubernetes Workshop: github.com/aws-samples/aws-workshop-for-kubernetes

• New Relic Quick Start: aws.amazon.com/quickstart/architecture/new-relic-infrastructure/

• Questions: [email protected]


Q&A

Use GoToWebinar’s Questions tool to

submit questions to our panel.

Please specify whether your question is

for Tracy (DevOps Institute), Stijn (New

Relic), or Chris (AWS).

Documents

How to Scale Kubernetes in AWS - Amazon Web Services, Inc. · The DevOps Institute (DOI) is the continuous x learning community dedicated to advancing the human elements of DevOps