Application Architecture Summit - Monitoring the Dynamic Cloud

Monitoring the Dynamic CloudApplication Architecture Summit – Feb 23, 2017

Lee Atchison ∙ Senior Director Strategic Architecture at New Relic, Inc.leeatchison@leeatchison

We want better apps faster

Better Data Center

Dynamic Environment

How do we use the cloud to accomplish this?

Better Data Center

Better Data Center

Dynamic Environment

Cloud as a “Better Data Center”

Resources are allocated to uses, just like in a data

center

Provisioning process is faster

Lifetime of components is relatively long

Capacity planning is still important and

still applies

Why use a “Better Data Center”?

Add new Capacity(faster)

Improve Application Availability(redundancy)

Compliance

Who is impacted?

Better Data CenterCan I scale my server fleet?

Can apps run anywhere?

How do they perform in the cloud?

Operations Development

Data center is a data center…

Similar to monitoring any other data center…

How do I monitor it?

Monitoring an application

• Application & Application Microservices• Server OS• Hardware (virtual)

Typical Server / EC2 Instance

EC2 Instance

Server OS

Server (Virtual)Hardware

Application &Application Microservices

BrowserMobile

EC2 Instance

Server OS

Server (Virtual)Hardware

Application &Application Microservices

BrowserMobile

AWS Monitoring

• Server OS• Memory / Filesystem• Processes• Configuration• Application

- Latency- Error rates

• EC2 instance• Virtualization• Hardware• [CPU / Disk / Networking]

CloudWatch

Monitors

Doesn’t know about:

CloudWatch

AWS CONSOLE

EC2 Instance

Server OS

Server (Virtual)Hardware

Application &Application Microservices

BrowserMobile

CloudWatch

AWS CONSOLE

DASHBOARDS

New Relic Monitoring

• Virtualization

• How O.S. is performing• Configuration• Processes• Hardware

• App health• App performance• Microservices

New Relic

Monitors (Server):

Monitors (Application):

Doesn’t know

New RelicApplicationMonitoring

New Relic Infrastructure

Monitoring

EC2 Instance

Server OS

Server (Virtual)Hardware

Application &Application Microservices

BrowserMobile

CloudWatch

AWS CONSOLE

DASHBOARDS

New RelicApplicationMonitoring

New Relic Infrastructure

Monitoring

AWSNew Relic Monitoring

• Visibility into virtualization• CPU / Disk / Networking

• CPU / Disk / Networking• Memory / Filesystem• Processes

- Infrastructure components• Application / Microservices:

- Latency- Error rates- App insights

AWS / CloudWatch

New Relic

New Relic Monitors

CloudWatch monitors

Dynamic Cloud

Better Data Center

Dynamic Environment

Cloud as a “Dynamic Tool for Dynamic Apps”

Use Only the Resources you need

Cloud as a “Dynamic Tool for Dynamic Apps”

Use Only the Resources you need

Allocate / de-allocateresources on the fly

Cloud as a “Dynamic Tool for Dynamic Apps”

Use Only the Resources you need

Allocate / de-allocateresources on the fly

Resource allocation is an integral part of your

application architecture

Dynamic Cloud

Resources are: Application in charge:

Allocated Application is aware of and is controlling traditional OPs resources

Consumed De-allocated

Dynamic Usage Example…Docker Container Age

(Count vs. Hours)

1 Hour

200 days 833 days

Dynamic Usage Example…Docker Container Age

(by Minute and Hour)

1,200,00011% under one minute

Container age (minutes)

Dynamic Cloud Technologies

Dynamic Cloud is about scaling

EC2 Auto Scaling

Mobile / IoT Dynamic routing

Load balancing

Queues and notifications

Docker

Dynamic Cloud has unique monitoring requirements…

How do I monitor the Dynamic Cloud?

What is a Dynamic Cloud Application?

• Application & Application MicroservicesResponsible for the parts you care about

• Infrastructure• Allocation/Provisioning• Scaling

Let cloud manage rest

Server OS

Server (Virtual)Hardware

Application & Application

Microservices

Provisioning

Application & Application

Microservices

Application & Application

Microservices

BrowserMobile

Server OS

Server (Virtual)Hardware

Application & Application

Microservices

Provisioning

Application & Application

Microservices

Application & Application

Microservices

BrowserMobile

Monitoring Dynamic Cloud Applications

DASHBOARDS

AWS CONSOLE

CloudWatch

Server OS

Server (Virtual)Hardware

Application & Application

Microservices

Provisioning

Application & Application

Microservices

Application & Application

Microservices

BrowserMobile

AWS InfrastructureNew Relic work together

CloudWatch

AWS CONSOLE

New RelicApplicationMonitoring

New Relic Infrastructure

Monitoring

DASHBOARDS

Server OS

Server (Virtual)Hardware

Application & Application

Microservices

Provisioning

Application & Application

Microservices

Application & Application

Microservices

BrowserMobile

CloudWatch

AWS CONSOLE

New RelicApplicationMonitoring

New Relic Infrastructure

Monitoring

DASHBOARDS

AWS InfrastructureNew Relic work together

New Relic Monitors

CloudWatch & AWS monitors

Server OS

Server (Virtual)Hardware

Application & Application

Microservices

Provisioning

Application & Application

Microservices

Application & Application

Microservices

BrowserMobile

How do you monitor this?

?How do you

monitor this?

Where did it go? It was just here!!

The thing you monitored 10 minutes ago…

...doesn’t exist anymore!?

Dynamic Cloud

1,200,00011% under one minute

Docker Container Age(by Minute and Hour)

Container age (minutes)

Monitoring the Dynamic Cloud

Monitor the Cloud Components themselves Monitor the lifecycle of the Cloud Components

Very different than monitoring traditional Data Center components

Who is impacted?

Better Data CenterCan I scale my server fleet?

Can apps run anywhere?

How do they perform in the cloud?

Operations Development

Data center is a data center…

Who is impacted?

Operations DevelopmentDynamic CloudWhat is a container?

Why do I care??

It was just here, where did it go??

Cloud architecture is integral to the application architecture

*

Developers deeply involved in cloud activities

Changing World

Ops

Previous - STATIC World

Changing World

Previous - STATIC World

Ops

Dev

Now - DYNAMIC World

Ops

Change is speeding up

Traditional Data Center Cloud Data Center Dynamic Cloud

Dynamic Cloud enables better applications faster.

Good Better Best

The way you’ve done things in the past won’t work in the future.

Dynamic Cloud

Server running application/ processes

Process running a command

Things happen faster because of…

This is HARD

EC2 Docker Container

Dynamic Cloud

Server running application/ processes

Process running a command

Function performing a task or operation

EC2 Docker Container Lambda

Things happen faster because of…

The Future with Lambda

Microcomputing & AWS Lambda

• Newest entrance to the “dynamic cloud”

• Provides event driven compute capabilities

• No infrastructure to provision

• Massively shared infrastructure

Why use Lambda?

Run in response to a state change or action in the cloud

Stateless, “filters”Perform quick actions

Virtually no startup/shutdown cost

Lambda scripts

AWS Lambda

S3Bucket

DynamoDB

APIGateway

SQS

RESOURCESSOME

• Takes an event from an AWS resource (A Trigger)

AWS Lambda

S3Bucket

DynamoDB

APIGateway

SQS

RESOURCESSOME

LambdaScript

• Takes an event from an AWS resource (A Trigger)

• Creates an instance to execute

Lambda Instance

AWS Lambda

S3Bucket

DynamoDB

APIGateway

SQS

RESOURCESSOME

S3Bucket

APIGateway SQS

RESOURCESSOME

LambdaScript

• Takes an event from an AWS resource (A Trigger)

• Creates an instance to execute

• Can impact original or different AWS Resource

Lambda Instance

AWS Lambda

S3Bucket

DynamoDB

APIGateway

SQS

RESOURCESSOME

S3Bucket

APIGateway SQS

RESOURCESSOME

LambdaScript

LambdaInstances

• Takes an event from an AWS resource (A Trigger)

• Creates an instance to execute

• Can impact original or different AWS Resource

• Any number of instances can run at a time

Photo Management App

Lambda example #1

Photo management application

UploadFile• Photos uploaded to S3

S3 BucketImageImport

User

Photo management application

UploadFile

S3 BucketImage

Thumbnails

View Thumbnail

• Photos uploaded to S3

• Lambda script creates thumbnails

S3 BucketImageImport

LambdaScript

User

Photo management application

ImageDatabase

UploadFile

S3 BucketImageImport

S3 BucketImage

Thumbnails

View Thumbnail

LambdaScript

• Photos uploaded to S3

• Lambda script creates thumbnails

• Lambda script updates metadata in database User

LambdaScript

Photo management application

Application

UploadFile

S3 BucketImageImport

S3 BucketImage

Thumbnails

LambdaScript

AppInteractions

View Thumbnail

LambdaScript

• Photos uploaded to S3

• Lambda script creates thumbnails

• Lambda script updates metadata in database

• Application only has to deal with metadata editing, not photo file management

User

ImageDatabase

Mobile Game App

Lambda example #2

• Cloud platform hosts an API for mobile app

- API Gateway

• Lambda scripts implement the API

• Lambda scripts manipulate database

• Extremely high scale possible

- No infrastructure

Mobile game platform

APIGateway

LambdaScript

LambdaScript

LambdaScript

Mobile PhoneApplication Users

Database

Monitoring Lambda Scripts

Less like infrastructure monitoring / More like web application monitoring

We Care About We Don’t Care About

• Run time (average, extremes – TP90/TP99)

• Statistical metrics

• Error rates and other deviations from norm

• “Drill down” into individual “runs”

• Details about all ”runs”

• Server / infrastructure metrics

Monitoring LambdaMore like application performance monitoring than infrastructure monitoring

Monitoring LambdaMore like application performance monitoring than infrastructure monitoring

Monitoring just the server

EC2 Instance

Server OS

Server (Virtual)Hardware

Application &Application Microservices

AWS CONSOLE

CloudWatch

Worked when rate of change was low…

Monitoring just the server

• Rate of change is faster• Problems come up quicker• “Server” isn’t a server anymore• “Provisioning” isn’t provisioning anymore

Insufficient in the cloud:

Server OS

Server (Virtual)Hardware

Application & Application

Microservices

Provisioning

Application & Application

Microservices

Application & Application

Microservices

BrowserMobile

Server OS

Server (Virtual)Hardware

Application & Application

Microservices

Provisioning

Application & Application

Microservices

Application & Application

Microservices

BrowserMobile

New Relic – Full Stack Monitoring

New RelicApplicationMonitoring

New Relic Infrastructure

Monitoring

DASHBOARDS

• Top to bottom monitoring…• Full stack accountability...• Dynamic infrastructure control...

You need:

New Relic enables full stack accountability between your code & cloud

Customers On-Premises On Premises Relational Data

AWS RDS

Synthetic Customers

S3

Service API

EC2

NoSQL

Browser / Mobile / Apps

Data Driven Digital Business

Customer Experience Mgmt Application Performance Mgmt Dynamic Infrastructure Mgmt

Thank youLee Atchison ∙ Senior Director Strategic Architecture at New Relic.

leeatchison@leeatchison

Architecting for ScaleBy: Lee AtchisonPublished by: O’Reilly Mediawww.architectingforscale.com

This document and the information herein (including any information that may be incorporated by reference) is provided for informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc. (“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby is proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission.

Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the statement will include words such as “believes,” “anticipates,”, “expects” or words of similar import.

Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date hereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement or success of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations, and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual results, performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further information on factors that could affect such forward-looking statements is included in the filings we make with the SEC from time to time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at http://ir.newrelic.com or the SEC’s website at www.sec.gov.

New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.

Safe Harbor