48
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. November 29, 2016 Case Study: How Startups like Mapbox, Ring, Hudl, and Veristor Use AWS to Punch Above Their Weight STG204

AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Embed Size (px)

Citation preview

Page 1: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

November 29, 2016

Case Study: How Startups like Mapbox,

Ring, Hudl, and Veristor Use AWS to

Punch Above Their Weight

STG204

Page 2: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)
Page 3: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)
Page 4: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Maps

Directions Geocoding

Mobile

Developer tools

Analysis

Page 5: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)
Page 6: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)
Page 7: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)
Page 8: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)
Page 9: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Serving maps at scale on AWS

• Built from the ground up using AWS since 2007

• Powers over 5,000 apps in categories ranging from social

to mobility

• Reaches more than 200 million users each month and

growing

Page 10: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Amazon S3 is our storage workhorse

Processing

EC2s

Amazon

S3

Mobile Apps

Satellites

Drones

Vehicles

Any Sensor

Mobile AppsAPI EC2s Amazon

CloudFront

Page 11: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

AWS KMS + S3

• Encrypts data on Amazon S3 using a hardware encryption

device

• Data is automatically encrypted as it's written and

decrypted as it's read

Page 12: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Massively scalable

• Over 2 petabytes of street and satellite data stored in S3

• 70 billion requests to S3 each month

• Burstable - temporarily scale up during heavy data

processing and scale back down

• At times we've had 5 petabytes of data in S3 but can

scale that back to save money

Page 13: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Built-in cost saving features

• Infrequent access - save on data that you don't access

often - like raw data

• Reduced redundancy storage - save on data that you can

easily regenerate from raw data

• Combine these features for maximum cost savings

Page 14: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

In short

• Integration with other AWS services

• Scalability

• Cost-saving features

Page 15: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)
Page 16: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Rob Hruska

Engineering Director

Page 17: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)
Page 18: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)
Page 19: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)
Page 20: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

8+ petabytes

4,300,000,000+ objects

35 hours video/minute

145 k amateur + elite teams

Page 21: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Today

• Video Upload/Storage Pipeline

• Amazon S3 Transfer Acceleration

• Standard – Infrequent Access

Page 22: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Hudl Video Lifecycle

Browsers

Amazon

CloudFront

Hudl Uploader

Upload-Only

Bucket

Application

Video

Processing

Mobile Apps

Metadata / Object Locations (HTTP)

VideoAmazon

SQS

Permanent

Bucket

Page 23: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Amazon

CloudFront

Application

Video

Processing

Browsers

Mobile Apps

Metadata / Object Locations (HTTP)

Amazon

SQS

Permanent

Bucket

Hudl Uploader

Upload-Only

Bucket

Video

Hudl Video Lifecycle

Page 24: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Amazon

CloudFront

Application

Video

Processing

Browsers

Mobile Apps

Metadata / Object Locations (HTTP)

Amazon

SQS

Permanent

Bucket

Hudl Uploader

Upload-Only

Bucket

Video

Transfer

Acceleration

Hudl Video Lifecycle – Transfer Acceleration

Page 25: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Hudl Video Lifecycle

Browsers

Amazon

CloudFront

Hudl Uploader

Upload-Only

Bucket

Application

Video

Processing

Mobile Apps

Metadata / Object Locations (HTTP)

VideoAmazon

SQS

Permanent

Bucket

Page 26: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Browsers

Amazon

CloudFront

Hudl Uploader

Upload-Only

Bucket

Application

Video

Processing

Mobile Apps

Metadata / Object Locations (HTTP)

VideoAmazon

SQS

Permanent

Bucket

Hudl Video Lifecycle – Standard-IA

Page 27: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Finding an ideal Standard-IA transition lifecycle

Video

Views

Weeks After Created

Basketball Video Usage Over Lifetime

Page 28: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Standard-IA> 50% transitioned

Page 29: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Teams

N-N

Teams

N-N

Teams

N-N

Game

Video

Highlight

Video

Recruit

Video

Bucket Organization

Page 30: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)
Page 31: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)
Page 32: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Ring Products: Practical Uses for the IoT

Page 33: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Ring Neighborhoods: Network Effects in Practice

Wilshire Park study with LAPD:

Ring installed on 10% of homes

Burglaries down 55% for the

entire community in 6 months

Burglars want an easy hit and go

elsewhere if you’re home

Alarms are reactive, not proactive

Traditional systems don’t link up,

so protection ends at your door

Page 34: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Devices installed in nearly every country on Earth

Millions of connected apps and devices

Over 1 billion videos and rapidly increasing

High growth brings big challenges, even month-to-month

Page 35: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Ring Video Pipeline

Raw

Buckets

Final

(Standard)

S3 Logs

Amazon

CloudFront

Ring App(s)

AWS

Lambda

Viewers

Amazon

SQS

Owner(s)

Visitor

Application

Servers

Ring Device

GPU

Farm

Final

(IA)

Lifecycle

Transitions

Event

Triggers

Live Video

Page 36: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Intelligently Determining Class

Page 37: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Infrastructure Savings in Amazon S3

Page 38: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Global Concerns

Page 39: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Extreme Performance is Easy

S3 will automatically partition if you use good keys – or just add more buckets

Amazon CloudFront as a CDN for GET heavy loads and faster downloads

Faster uploads with Transfer Acceleration

TCP Window Scaling - without it, 64 KB window kneecaps long fat networks

TCP SACK is good for fast but lossy connections like mobile connections

examplebucket/2134857/data/start.png

examplebucket/2134857/data/resource.rsrc

examplebucket/2134857/data/results.txt

examplebucket/2134858/data/start.png

examplebucket/2134858/data/resource.rsrc

examplebucket/2134858/data/results.txt

examplebucket/2134859/data/start.png

examplebucket/2134859/data/resource.rsrc

examplebucket/2134859/data/results.txt

examplebucket/7584312/data/start.png

examplebucket/7584312/data/resource.rsrc

examplebucket/7584312/data/results.txt

examplebucket/8584312/data/start.png

examplebucket/8584312/data/resource.rsrc

examplebucket/8584312/data/results.txt

examplebucket/9584312/data/start.png

examplebucket/9584312/data/resource.rsrc

examplebucket/9584312/data/results.txt

Page 40: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

S3 Scaling on H-Day

Page 41: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)
Page 42: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)
Page 43: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

DevOps Software

Real-Time Splunk for JIRA Service Desk

Splunk Real-Time Search integration with JIRA

Service Desk for automated issue creation.

JIRA Service Desk links to Splunk Real-Time

search results for rapid troubleshooting.

Page 44: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Atlassian Cloud to Splunk

Forty8Fifty

Atlassian

Connect Add-on

Page 45: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

AWS Benefits

Fault Tolerant

HA Infrastructure

Rapid AWS

Innovation

Supports Our

Innovation

Scalable for Growth Cost/Margin

Management

Page 46: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)
Page 47: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Thank you!

Page 48: AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Others Use AWS to Punch above Their Weight (STG204)

Remember to complete

your evaluations!