Monitoring the Dynamic Nature of the Cloud [FutureStack16 NYC]

New Relic Template 2015

Monitoring the Dynamic Nature of the CloudHow Docker and the Cloud bring your application to life

Lee Atchison, Principal Cloud Architect at New Relic, Inc.

@leeatchison leeatchison


Safe HarborThis document and the information herein (including any information that may be incorporated by reference) is provided forinformational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of NewRelic, Inc. (“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any informationprovided hereby is proprietary to New Relic and may not be replicated or disclosed without New Relic’s express writtenpermission.

Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that isnot a historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizationsof future events is a forward-looking statement. These forward-looking statements can often be identified as such because thecontext of the statement will include words such as “believes,” “anticipates,”, “expects” or words of similar import.

Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the datehereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other thirdparties transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. Theachievement or success of the matters covered by such forward-looking statements are based on New Relic’s currentassumptions, expectations, and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes incircumstances that may cause the actual results, performance, or achievements to differ materially from those expressed orimplied in any forward-looking statement. Further information on factors that could affect such forward-looking statements isincluded in the filings we make with the SEC from time to time. Copies of these documents may be obtained by visiting NewRelic’s Investor Relations website at http://ir.newrelic.com or the SEC’s website at www.sec.gov.

New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law.New Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.


Who am I?

Specialize in:

Cloud computing

Services & Microservices

Scalability, Availability

29 years in industry7 in Amazon Retail & AWS(Built SW/VG AppStore, AWS Elastic Beanstalk)

4 in New Relic(Architecture Lead, Cloud, Service Migration)



DynamicEnvironment

BetterData Center

We want better apps fasterHow do we use the cloud to accomplish this?


Cloud as a Better Data Center


Cloud as a “Better Data Center”

Resources are allocated to uses, just like in a data center




Provisioning process is faster




Lifetime of components is relatively long





Lifetime of components is relatively long


Capacity planning is still important and

still applies


Why use a “Better Data Center”?



Add new Capacity(faster)



Improve ApplicationAvailability

(redundancy)




ComplianceImprove ApplicationAvailability

(redundancy)



Who is impacted?

Better Data Center

Operations Development


Who is impacted?

Can I scale my server fleet?

Can apps run anywhere?How do they perform in the cloud?

Better Data Center



Who is impacted?

Data center is a data center…



Better Data Center



Who is impacted?

Better Data Center Faster Application Launch / Deploy=

Better Data Center



How do I monitor it?Similar to monitoring any other data center…


Monitoring an application

Typical Server / EC2 Instance

• Application & Application Microservices

• Server OS

• Hardware (virtual)

EC2 Instance

Application & Application

Microservices

Server OS

Server (Virtual)Hardware


AWS Monitoring

CloudWatch

Monitors:• EC2 instance• Virtualization

• Hardware• [CPU / Disk / Networking]

Doesn’t know about:• Server OS• Memory / Filesystem• Processes• Application

• Latency• Error rates• Internal insights

EC2 Instance


Microservices

Server OS


AWS CONSOLE

CloudWatch


New Relic Monitoring

New Relic

Monitors (Server):• How O.S. is performing• Processes• Hardware

Monitors (Application):• App health• App performance

• Microservices

Doesn’t know• Virtualization

EC2 Instance


Microservices

Server OS


AWS CONSOLE

New RelicApplicationMonitoring

New Relic Server

Monitoring

DASHBOARDS

CloudWatch


EC2 Instance


Microservices

Server OS


AWS CONSOLE


New Relic Server

Monitoring

DASHBOARDS

AWS çè New Relic work together

AWS / CloudWatch

• Visibility into virtualization• CPU / Disk / Networking

New Relic

• CPU / Disk / Networking

• Memory / Filesystem• Processes

• Infrastructure components• Application / Microservices:– Latency

– Error rates– App insights

CloudWatch

New Relic monitors

CloudWatchmonitors


Cloud as a Dynamic Tool


Cloud as a “Dynamic Tool for Dynamic Apps”

Use only the resources

you need




you need

Allocate / de-allocateresources on

the fly




you need

Resource allocation is an integral part

of your applicationarchitecture

Allocate / de-allocateresources on

the fly


Dynamic Cloud

Resources are:

Application is aware of and is controlling traditional OPs

resources

De-allocatedConsumedAllocated

Auto Scale


Dynamic Cloud

Quicker / cheaper startup / shutdown than EC2 instances

Easy to consume Docker containers much like you do

“processes”

Perform jobs:

• In a script

•Off of some queue of job requests


Dynamic Cloud

833 days

Docker Container Age(Count vs. Hours)

Confidential ©2008-16 New Relic, Inc. All rights reserved. 29

200 days

1 Hour


Dynamic Cloud

1,200,00011% under one minute

Docker Container Age(by Minute Under and Hour)



EC2 Auto Scaling

Dynamic Cloud Technologies

Mobile / IoTDynamic routing

Load balancing

Data transport

Queues and notifications

Docker

Dynamic Cloud is about scaling


How do I monitor the Dynamic Cloud?Dynamic Cloud has unique monitoring requirements…


Responsibility (Dynamic Cloud)

Responsible for the parts you care about

• Application & Application Microservices

Let cloud manage rest

• Infrastructure

• Allocation/Provisioning• Scaling

Server OS



Microservices

Provisioning


Microservices


Microservices


AWS Infrastructure çè New Relic work togetherDASHBOARDS

Server OS



Microservices

Provisioning


Microservices


Microservices

CloudWatch

AWS CONSOLE



Server OS



Microservices

Provisioning


Microservices


Microservices

CloudWatch

AWS CONSOLE




Server OS



Microservices

Provisioning


Microservices


Microservices

CloudWatch

AWS CONSOLE


New Relic monitors

CloudWatch & AWSmonitors


How do you monitor this?

?How do you

monitor this?

Server OS



Microservices

Provisioning


Microservices


Microservices


Server OS



Microservices

Provisioning


Microservices


Microservices

Where did it go? It was just here!!!

? The thing you monitored 10 minutes ago…...doesn’t exist anymore!


Dynamic Cloud

1,200,00011% under one minute

Docker Container Age(by Minute Under and Hour)



Monitoring the Dynamic Cloud

Very different than monitoring traditional Data Center components

Monitor theCloud Components

themselves

Monitor the lifecycle of the

Cloud Components


Who is impacted?

Operations


Development



Better Data Center


Who is impacted?

Operations


Development



Better Data CenterDynamic Cloud

What is a container?

Why do I care??It was just here, where did it go???

Cloud architecture is integralto the application architecture

Developers deeply involved in cloud activities


Change is speeding up

Dynamic Cloud enables better applications faster.

TraditionalData Center

Good

CloudData Center

Better

DynamicCloud

Best

The way you’ve done things in the past won’t work in the future.


Dynamic Cloud

Things happen faster because of…

EC2

Server running application /

processes

Docker container

Process running a command

This is HARD


Dynamic Cloud

EC2

Server running application /

processes

Docker container

Process running a command

Lambda

Function performing a task or operation


The Future with Lambda


Microcomputing & AWS Lambda

• Newest entrance to the “dynamic cloud”

• Provides event driven compute capabilities


Microcomputing & AWS Lambda

• No infrastructure to provision

• Massively shared infrastructure


Why use Lambda?

Lambda scripts

Run in response to a state change or action in the cloud

Stateless, “filters”Perform quick actions

Virtually no startup/shutdown cost


AWS Lambda

S3Bucket

DynamoDB

APIGateway

SQS

RESOURCESSOME

• Takes an event from an AWS resource (A Trigger)


AWS Lambda

S3Bucket

DynamoDB

APIGateway

SQS

RESOURCESSOME

LambdaScript


• Creates an instance to execute

LambdaInstance


AWS Lambda

S3Bucket

DynamoDB

APIGateway

SQS

RESOURCESSOME

S3Bucket

APIGateway SQS

RESOURCESSOMELambdaScript



• Can impact original or different AWS Resource

LambdaInstance


AWS Lambda

S3Bucket

DynamoDB

APIGateway

SQS

RESOURCESSOME

S3Bucket

APIGateway SQS

RESOURCESSOMELambdaScript

LambdaInstances• Takes an event from an

AWS resource (A Trigger)


• Can impact original or different AWS Resource

• Any number of instances can run at a time


Lambda example #1:

Photo Management App


Photo management application

S3 BucketImageImport

• Photos uploaded to S3Upload

File

User




S3 BucketImage

Thumbnails

LambdaScript

• Photos uploaded to S3

• Lambda script creates thumbnails

UploadFile

View Thumbnail

User



ImageDatabase


S3 BucketImage

Thumbnails

LambdaScript

LambdaScript



• Lambda script updates metadata in database

UploadFile

View Thumbnail

User



ImageDatabase

Application

UploadFile


S3 BucketImage

Thumbnails

LambdaScript

AppInteractions

View Thumbnail

LambdaScript



• Lambda script updates metadata in database

• Application only has to deal with metadata editing, not photo / file management

User


Lambda example #1:

Mobile Game App


Mobile game platform

APIGateway

LambdaScript

LambdaScript

LambdaScript

Mobile PhoneApplication Users

Database

• Cloud platform hosts an API for mobile app– API Gateway

• Lambda scripts implement the API

• Lambda scripts manipulate database

• Extremely high scale possible

– No infrastructure


Monitoring Lambda Scripts

Less like server monitoring / More like web application

monitoring

We Care About

We Don’t Care About

• Run time (average, extremes – TP90/TP99)

• Statistical metrics

• Error rates and other deviations from norm

• “Drill down” into individual “runs”

• Details about all ”runs”

• Server / infrastructure metrics


Monitoring Lambda

More like application performance monitoring than infrastructure monitoring


Monitoring Lambda

More like application performance monitoring than infrastructure monitoring


Change is speeding up

Dynamic Cloud enables better applications faster.The rate of change is increasing…

TraditionalData Center

Good

CloudData Center

Better

DynamicCloud

Best

The way you’ve done things in the past won’t work in the future.


Monitoring just the server

Worked when rate of change was low…EC2 Instance


Microservices

Server OS


AWS CONSOLE

CloudWatch


Monitoring just the server

Insufficient in the cloud:

• Rate of change is faster• Problems come up quicker• “Server” isn’t a server anymore• “Provisioning” isn’t provisioning anymore

You need top to bottom monitoring…

You need full stack accountability...

Server OS



Microservices

Provisioning


Microservices


Microservices


New Relic enables accountability betweenyour code & AWS

Customers

Mobile / Apps / Browser

On-PremisesOn Premises

Relational Data

EC2

AWSContainer Service

AWS RDSSynthetic Customers

S3

NoSQL

AWS Storage Services

Your Responsibilities§ Application code & 3rd party

services

§ App architecture

§ Customer experience

§ On-Premises components

NoSQLAWS Responsibility:§ Service Availability & Performance

§ Provisioning & operation of DC

§ Isolation & security of resources

§ Service maintenance & innovation


Architecting for ScaleBy: Lee AtchisonPublished by: O’Reilly Mediawww.architectingforscale.com

Want to Learn More?



Thank you

Lee Atchison, Principal Cloud Architect and Advocate at New Relic, Inc.


Architecting for ScalePublished by: O’Reilly Media, www.architectingforscale.com

Technology

Monitoring the Dynamic Nature of the Cloud [FutureStack16 NYC]