72
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Zuul @ Netflix By Mikey Cohen - Manager Cloud Gateway, Netflix @moldfarm 1

Netflix's Edge Gateway Using Zuul

Embed Size (px)

Citation preview

Page 1: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Zuul @ NetflixBy Mikey Cohen - Manager Cloud Gateway, Netflix

@moldfarm

1

Mike Cohen
picture here
Page 2: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 2

Page 3: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Global Streaming of TV Shows and Movies

3

Page 4: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Over 42 Billion Hours of Streaming in 2015

4

Page 5: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Over 83 Million Subscribers

In nearly every country5

Page 6: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

The Gateway : From the Internet to Services in the Cloud

GatewayGatewayGATEWAY

Origin (API)Origin (API)

API

Origin (API)Origin (API)

Website

6

Page 7: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Our Gateway (Zuul) @ Netflix

• Handles most netflix.com hosts• More than 1000 device types

– Hundreds of permutations of protocols and device versions

• Fronted by over 50 elbs • Handling tens of billions of requests per day• 3 AWS regions• Over 20 production Zuul clusters• Fronting about 10 “Origin” systems

7

Page 8: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

• Evolution• Scale• Success• Failure

Our Journey to Zuul

8

Page 9: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Why Have a Gateway?

9

Daniel Jacobson
great image. do you provide attribution for the images used?
Mike Cohen
All images I used are open license without restriction
Daniel Jacobson
ok, great. just to be sure, you mean they dont even require or request attribution?
Page 10: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Why have a Gateway?

10

Page 11: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Isn’t (fill-in-the-blank) Good Enough?

11

Page 12: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Netflix’s Idea of a Gateway●Deeply rooted in Microservice ecosystem●Dynamic routing logic●Deep Insights●Load balancing●Availability focused●Service protection●Quality assurance tool

12

Page 13: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Evolving to a Gateway...Evolving to the Cloud

13

Page 14: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Netflix’s Public API (2008)

14

Page 15: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

...The world was a simpler place...

15

Page 16: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

A Public Developer’s Gateway

• Simple static rule-based routing• API portal• Access Control• Request authentication (OAuth)• Throttling - request caps• Basic Monitoring and Analytics• Caching• 3rd Party managed and developed• Weeks and months development cycles

16

Page 17: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Streaming Devices using public API

• Early Streaming Devices - 2009 – Windows Media Center– XBox– PS3

17

Page 18: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Migration to AWS - 2010

• Apigee• Device traffic, not

public• Controlling DC ->

cloud migration• Running in AWS• Netflix controlled

18

Page 19: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

2011 - Streaming Success / General Instability

• Massive Growth• Rapid device

expansion• Pushing AWS limits• Netflix Nascent in

AWS

19

Page 20: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Instability makes way for Innovation

20

Page 21: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Shortcomings of what we had

• Ecosystem Mismatch• Static configuration• Deployment time - Hours• Manual deployment - Error prone• Critical bugs go through vendor• Automated testing not possible

21

Daniel Jacobson
getting to this point, i am now thinking more that you dont need the deep history stuff. you just need enough to express the pains that trigger changes in our approach.
Page 22: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Netflix’s Microservice Ecosystem

Netflix Platform (Karyon, Hystrix)

Data Pipeline (Kafka, etc)

Discovery (Eureka)

Real-time analytics(Mantis)

Monitoring(Atlas)

Dynamic Properties (ARCHAIS)

Deployment Pipeline

AWS

EC2 S3

CryptoAuthentication Database (Cassandra)

Micro Service(Zuul)

22

Page 23: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Spring Platform Microservice Ecosystem

Spring Platform (Karyon, Hystrix)

Discovery (Eureka)

Monitoring(Atlas)

Dynamic Properties (ARCHAIS)

Deployment Pipeline

AWS

EC2 S3

Micro Service(Zuul)

23

Page 24: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

2012 - Zuul Created

24

Page 25: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Zuul in a Nutshell

25

Page 26: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Request Lifecycle of a Request

26

Page 27: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Example Filterclass DeviceDelayFilter extends ZuulFilter { def static Random rand = new Random() @Override String filterType() { return 'pre' } @Override int filterOrder() { return 5 } @Override boolean shouldFilter() { return RequestContext.getRequest().getParameter("deviceType")?equals("BrokenDevice"):false } @Override Object run() { sleep(rand.nextInt(20000)) //Sleep for a random number of seconds between [0-20] }}

27

Page 28: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

What Zuul Got Us:

• Handle Edge Cases• React quickly• Service Resiliency• Move fast• Microservice Insights

28

Page 29: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

What Engineers Saw:• Handle Edge Cases

• React quickly!• Service Resiliency

• Move fast! ←• Microservice Insights

29

Page 30: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Zuul : Early Successes!

• Easy and convenient• Instant results• High adoption• Happy customers

30

Page 31: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Zuul - A Victim of Success

• Business logic in Zuul• Affects system

resiliency • Gateway team in

critical path

31

Page 32: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Creating a Gateway Strategy

32

Page 33: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Principles of Netflix’s Gateway Strategy • Creative Routing• Dynamic Routing• Delivery Focused• Traffic Shaping• React Fast • Insights

33

Page 34: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Creative Routing - Subclusters with Purpose

GatewayGateway

Gateway

Origin (API)

v1

v2

test

debug

Instrumented

squeeze

“sticky” canarybaseline

“sticky” baseline

v1

v2

test

debug

baseline canary

“sticky” canary

“sticky” baselineFIT

Instrumented

squeeze

34

Page 35: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Red / Green Deployments

GatewayGateway

Gateway

Origin (API)

v1

v2

test

debug

canary

Instrumented

squeeze

“sticky” canarybaseline

“sticky” baseline

v1

v2

test

debug

baseline canary

“sticky” canary

“sticky” baselineFIT

InstrumentedInstrumented

squeezesqueeze

35

Page 36: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Developer Test Branches

GatewayGateway

Gateway

Origin (API)

v1

v2

test

debug

canary

Instrumented

squeeze

“sticky” canarybaseline

“sticky” baseline

v1

v2

test

debug

baseline canary

“sticky” canary

“sticky” baselineFIT

InstrumentedInstrumented

squeezesqueeze

36

Page 37: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Instrumented Clusters (Trickling traffic)

GatewayGateway

Gateway

Origin (API)

v1

v2

test

debug

canary

Instrumented

squeeze

“sticky” canarybaseline

“sticky” baseline

v1

v2

test

debug

baseline canary

“sticky” canary

“sticky” baselineFIT

Instrumented

squeezesqueeze

37

Page 38: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Squeeze Testing

GatewayGateway

Gateway

Origin (API)

v1

v2

test

debug

canary

Instrumented

squeeze

“sticky” canarybaseline

“sticky” baseline

v1

v2

test

debug

baseline canary

“sticky” canary

“sticky” baselineFIT

Instrumented

squeeze

38

Page 39: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Targeted Routing

GatewayGateway

Gateway

Origin (API)

v1

v2

test

debug

canary

Instrumented

squeeze

“sticky” canarybaseline

“sticky” baseline

v1

v2

test

debug

baseline canary

“sticky” canary

“sticky” baselineFIT

Instrumented

squeeze

39

Page 40: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Service “Canarying”

GatewayGateway

Gateway

Origin (API)

v1

v2

test

debug

canary

Instrumented

squeeze

“sticky” canarybaseline

“sticky” baseline

v1

v2

test

debug

baseline canary“sticky” canary

“sticky” baselineFIT

Instrumented

squeezesqueeze

40

Page 41: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

“Sticky” Canary

GatewayGateway

Gateway

Origin (API)

v1

v2

test

debug

canary

Instrumented

squeeze

“sticky” canarybaseline

“sticky” baseline

v1

v2

test

debug

baseline canary

“sticky” canary

“sticky” baselineFIT

Instrumented

squeezesqueeze

41

Page 42: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Failure Injection Testing

GatewayGateway

Gateway

Origin (API)

v1

v2

test

debug

Instrumented

squeeze

“sticky” canarybaseline

“sticky” baseline

v1

v2

test

debug

baseline canary

“sticky” canary

“sticky” baselineFIT

Instrumented

squeezesqueeze

42

Page 43: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Degraded Experience Testing

GatewayGateway

Gateway

Origin (API)

v1

v2

test

debug

Instrumented

squeeze

“sticky” canarybaseline

“sticky” baseline

v1

v2

test

debug

baseline canary

“sticky” canary

“sticky” baselineFIT

Instrumented

squeezesqueeze

43

Page 44: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Gateway Features of Deployment Pipeline

Gateway Features Completing the Continuous Delivery Pipeline

Version Control Build Unit

TestsFunctional TestingDeploy Service

Canary

Sticky Canary

Failure Injection Testing

Squeeze Testing

Instrumented Servers

Production Push

44

Page 45: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Traffic Shaping

45

Page 46: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

A Global Cloud Deployment

Persistence Tier

Business services Tier

Presentation Tier

Network Tier

Websites API

Zuul

DB

Persistence Tier

Business services Tier

Presentation Tier

Network Tier

Websites API

Zuul

DB

Persistence Tier

Business services Tier

Presentation Tier

Network Tier

Websites API

Zuul

DB

US-West-2

US-East-1 EU-West-1

46

Page 47: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Global Cloud Routing

Persistence Tier

Business services Tier

Presentation Tier

Network Tier

Websites API

Proxy

DB

Persistence Tier

Business services Tier

Presentation Tier

Network Tier

Websites API

Proxy

DB

Persistence Tier

Business services Tier

Presentation Tier

Network Tier

Websites API

Proxy

DB

Zuul

Zuul

Zuul

US-West-2

US-East-1 EU-West-1

47

Page 48: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

A Failing Region

Persistence Tier

Business services Tier

Presentation Tier

Network Tier

Websites API

Proxy

DB

Persistence Tier

Business services Tier

Presentation Tier

Network Tier

Websites API

Proxy

DB

Persistence Tier

Business services Tier

Presentation Tier

Network Tier

Websites API

Proxy

DB

Zuul

Zuul

Zuul

US-West-2

US-East-1 EU-West-1

48

Page 49: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Gateway routing to other regions

Persistence Tier

Business services Tier

Presentation Tier

Network Tier

Websites API

Proxy

DB

Persistence Tier

Business services Tier

Presentation Tier

Network Tier

Websites API

Proxy

DB

Persistence Tier

Business services Tier

Presentation Tier

Network Tier

Websites API

Proxy

DB

Zuul

Zuul

Zuul

US-West-2

US-East-1 EU-West-1

49

Page 50: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Attack Detection & Prevention

GatewayGateway

Gateway

Origin (API)Origin (API)

API

Origin (API)Origin (API)

Website

50

Page 51: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Other Zuul Responsibilities @ Netflix• Top Level request context

– Geolocation– Cookie / Token decryption

• Authentication– Hand off to Authentication Services

• Request/Response “normalization”– Device specific “weirdness”

• Chunked Encoding• Header truncations• URL fixes

• Testing / Debug support– Verbose Headers

– Geolocation overrides– Error Context

51

Page 52: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Smart Load Balancing

GatewayGateway

Gateway

Origin (API)

52

Page 53: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Smart Load Balancing - Bad Nodes

GatewayGateway

Gateway

Origin (API)

53

Page 54: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Gateway Backoff and Blacklists Bad Nodes

GatewayGateway

Gateway

Origin (API)

54

Page 55: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Zone Failure - Blacklist the Zone automatically

GatewayGateway

Gateway

Origin (API)

55

Page 56: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

React Quickly - Runtime Filter changes

GatewayGateway

Gateway

Origin (API)Origin (API)

API

Origin (API)Origin (API)

Website

Runtime Filter Injection

56

Page 57: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

A Room with a View - Insights

GatewayGateway

Gateway

Origin (API)Origin (API)

API

Origin (API)Origin (API)

Website

Insights

57

Page 58: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

What’s new on with Zuul

58

Page 59: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Zuul 2!!

• Zuul 2.0– Netty (non-blocking, async)– RxJava chained filters– Coming to OSS soon.

59

Page 60: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Why Zuul 2?

• 100M+ Persistent connections– Push notifications– Events

• Transport protocol agnostic– HTTP/2– Websocket– HTTP– etc..

60

Page 61: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Async Non-blocking vs Blocking

61

Page 62: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

What’s Next?

• Auto-remediation • Gateway as a service

–Self-service dynamic routing / route validation

–Control APIs for special routing functions

62

Page 63: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Top Lessons Learned

63

Page 64: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Ask Why?

64

Page 65: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Build for handling Failures

65

Page 66: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Take Advantage of your Microservice Ecosystem

66

Page 67: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Use Routing Creatively

67

Page 68: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Shard to Reduce Blast Radius

68

Page 69: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Devices are WeirdProtocols are Weird

69

Page 70: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Devices are ForeverProtocols are Forever

70

Page 71: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Keep Business Logic out of your Gateway

71

Page 72: Netflix's Edge Gateway Using Zuul

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

Learn More. Stay Connected.

Mikey Cohen - @moldfarm

https://github.com/Netflix/zuulhttp://techblog.netflix.com

@springcentralspring.io/blog

@pivotalpivotal.io/blog

@pivotalcfhttp://engineering.pivotal.io

72