Scaling Patterns for November 2019 Netflix’s Edge › system › files › presentation-slides ›...

Justin Ryan, Senior Software Engineer @ NetflixTwitter: quidryan, LinkedIn: www.linkedin.com/in/justin-charles-ryan

Scaling Patterns for Netflix’s EdgeNovember 2019

Scaling Patterns for Netflix’s EdgeJustin Ryan

Laundry

Space-Time Compromise

Laundry

ScalingQ

Scaling

Scaling Problems

— Debuggability

— Infrastructure

— Managing

Scalability

Trade Offs

Policies

Accessible

Five Use Cases

— Melnitz— Mantis— Passport— Device Types— Sharding

Logged In

Cookies

Melnitz

Trust Cookie Cookie Expiration

Cookies L7 Proxy Account

Service

ExpiredYes

Melnitz

Copyright Columbia Pictures

Melnitz

Set<CustomerId> Cookie Expiration

Melnitz Bloom Filter: Compact Set which

can answer the question is an object

“Possibly in Set”?

Melnitz

Melnitz L7 Proxy Account

Service

Bloom Filter

ExpiredYes

NoMaybe

MelnitzAccount Service Auth

Melnitz A

Melnitz B

L7 Proxy

Melnitz

Melnitz Trade Offs

— Probabilistic Data Structure

— Off the Shelf Components

Five Use Cases

— Melnitz - Distributed Bloom Filter— Mantis— Passport— Device Types— Sharding

Mantis

Mantis 2 million request per second *

3 kilobytes per request *

10 micro-services =

57 GB/s =~ 4.6 PB / day

Naive Approach

Service Elastic SearchKibana

Mantis

Mantis Master

Mantis

Service

MantisAgent

Mantis Source Job

Elastic Search

Kibana

Mantis Master

Service

Mantis

Mantis Source Job

Elastic Search

Browser

MantisAgent

Avoid Work

Mantis Trade Offs

— Mantis Query Language

— Scale Solution to Zero

Five Use Cases

— Melnitz - Distributed Bloom Filter— Mantis - Event Querying— Passport— Device Types— Sharding

Passport

Service Plan

Mid Tier A

Mid Tier B

Mid Tier C

Passport

Service Plan

Mid Tier A

Mid Tier B

Mid Tier C

Service Plan

Mid Tier A

Mid Tier B

Mid Tier C

Mid Tier A

Mid Tier B

Mid Tier C

Service Plan

Mid Tier A

Mid Tier B

Mid Tier C

Service Plan

Passport

Passport Trade Offs

— Data Passing

— Heavy Data Structures

Five Use Cases

— Melnitz - Distributed Bloom Filter— Mantis - Event Querying— Passport - Heavy Data Structure— Device Types— Sharding

Device Types

Device TypesNFAPPL-01-IPAD7=5-09BEC57F64CA5F837C75F924B94D

NFAPPL-01-IPAD3=4-15B2D9B33A994CC373017A9C6586

NFAPPL-02-IOS2=4-FT7WHG8H2UR2MCJ4YKR3P0DFH6

NFANDROID1-PRV-P-GOOGLPIXEL=2-7351-0F441DF98B8D99074

NFANDROID1-PRV-4339-0E50FC7582396F5C4496ED935F2F48ED

13951007

Device Types

Mid-Tier F

Mid-Tier E Mid-Tier D

Mid-Tier C

Mid-Tier BMid-Tier A

Device Types

Mid-Tier CMid-Tier BMid-Tier A

DTS PubSub

Device Types

Mid-Tier CMid-Tier BMid-Tier A

DTS PubSub

Device Types Device TypesDevice Types

Device Types

Device Type Trade Offs

— Business Tuned Fallbacks

— Leverage Existing Infrastructure

Five Use Cases

— Melnitz - Distributed Bloom Filter— Mantis - Event Querying— Passport - Heavy Data Structure— Device Types - Device Grouping— Sharding

Sharding

MSL — Message Security Layer == MSL

— Secure Messaging Framework

Sharding MSL

roxy APIAPIAPIAPIAPIAPI

Sharding MSL

APIAPIAPI

CPU Cost Per Request

- CPU to RPS ratio- Load Average

31% decrease

Latency

30% decrease

20% p99 decrease

Sharding MSL- % of Wall Clock in GC- GC Pause Time

Sharding

3 = 1 + 1

Sharding Trade Offs

— Operational Overhead

Five Use Cases

— Melnitz - Distributed Bloom Filter— Mantis - Event Querying— Passport - Heavy Data Structure— Device Types - Device Grouping— Sharding - Runtime Refactoring

Bonus Trade Offs

— Do the laundry

Justin Ryanjryan@netflix.com

Twitter: @quidryan

Thank You.

References — github.com/Netflix◆ eureka◆ zuul◆ mantis◆ msl

— Spinnaker— Apache Pulsar— Google Guava (Bloom Filter)

Scaling Patterns for November 2019 Netflix’s Edge › system › files › presentation-slides ›...

Documents

Cassandra Day SV 2014: Netflix’s Astyanax Java Client Driver for Apache Cassandra

Scaling impact: a primer For nonproFit S now allows us to solve old problems with new techniques and to ... We're on the edge of a new frontier. ... Symposium on Scaling Impact,

Netflix’s Layered Approach to Reducing Risk of Credential ... · Netflix’s Layered Approach to Reducing Risk of Credential Compromise WILL BENGTSON SENIOR SECURITY ENGINEER TRAVIS

UK FFAG Plans Introduction to FFAGs Scaling vs non-scaling Non-scaling FFAGs Non-scaling POP Why the interest? UK plans

Unified MPLS: Advanced Scaling for Core and Edge Networksd2zmdbbm9feqrf.cloudfront.net/2012/usa/pdf/BRKSPG-2405.pdf · ... Advanced Scaling for Core and Edge Networks BRKSPG-2405

Measurement and Scaling of Trailing-Edge NoiseM. Herr > 01.10..2009 Measurement and Scaling of Trailing-Edge Noise * Michaela Herr Institute of Aerodynamics and Flow Technology DLR

LESSON 14. Scaling-up and Scaling-down - … Lesson 1… · BIOREACTOR SCALING-UP AIMS FOR TODAY’S LESSON BIORREACTOR SCALING-UP Definition Starting point Scaling-up or numbering-up?

Scaling The Edge Bridge Address Table In Datacenter Networks

Three-dimensional localization of ultracold atoms in an ... · According to the celebrated scaling theory3, AL dependsonthedimension,andinthethree-dimensional(3D)case a mobility edge

Scaling edge packet networks to 100Gbit/s

NTT’s Use-cases and Plans for Stratum•Flexible service function chaining •Auto chaining/scaling •In-band telemetry between VNFs Service Chaining and Scaling Edge Gateway Stratum

1321347500_vlsi Implementation of an Edge Oriented Image Scaling Processor

Scaling Up on the Edge of the Rift Valley

March 3 - 6, 2019 Mesa, Arizona Archive...–Density scaling –Speed scaling –Power scaling –Cost scaling •CPU architectures are not scaling –Processor frequency scaling ended

Non ComparativeNon comparative scaling techniques.ppt Scaling Techniques

Scaling Agile is scaling people

SCALING NEW HEIGHTS SCALING - IIA Indonesia

SCALING VIDEO ANALYTICS ON CONSTRAINED EDGE N › static › paper_files › mlsys › 2019 › 19… · Scaling Video Analytics on Constrained Edge Nodes ... up to a given layer

Scaling - qcontokyo.comqcontokyo.com/tokyo-2013/data_2013/MartyWeiner_Q... · Scaling Pinterest Mar 2010 Jan 2011 Jan 2012 May 2012 · Amazon EC2 + S3 + Edge Cast, ELB · 135 Web

Netflix’s Recommendation ML Pipeline using Apache Spark · Netflix’s Recommendation ML Pipeline using Apache Spark DB Tsai Spark Summit ... from our recommendations Recommendations