© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
November 29, 2016
Case Study: How Startups like Mapbox,
Ring, Hudl, and Veristor Use AWS to
Punch Above Their Weight
STG204
Maps
Directions Geocoding
Mobile
Developer tools
Analysis
Serving maps at scale on AWS
• Built from the ground up using AWS since 2007
• Powers over 5,000 apps in categories ranging from social
to mobility
• Reaches more than 200 million users each month and
growing
Amazon S3 is our storage workhorse
Processing
EC2s
Amazon
S3
Mobile Apps
Satellites
Drones
Vehicles
Any Sensor
Mobile AppsAPI EC2s Amazon
CloudFront
AWS KMS + S3
• Encrypts data on Amazon S3 using a hardware encryption
device
• Data is automatically encrypted as it's written and
decrypted as it's read
Massively scalable
• Over 2 petabytes of street and satellite data stored in S3
• 70 billion requests to S3 each month
• Burstable - temporarily scale up during heavy data
processing and scale back down
• At times we've had 5 petabytes of data in S3 but can
scale that back to save money
Built-in cost saving features
• Infrequent access - save on data that you don't access
often - like raw data
• Reduced redundancy storage - save on data that you can
easily regenerate from raw data
• Combine these features for maximum cost savings
In short
• Integration with other AWS services
• Scalability
• Cost-saving features
Rob Hruska
Engineering Director
8+ petabytes
4,300,000,000+ objects
35 hours video/minute
145 k amateur + elite teams
Today
• Video Upload/Storage Pipeline
• Amazon S3 Transfer Acceleration
• Standard – Infrequent Access
Hudl Video Lifecycle
Browsers
Amazon
CloudFront
Hudl Uploader
Upload-Only
Bucket
Application
Video
Processing
Mobile Apps
Metadata / Object Locations (HTTP)
VideoAmazon
SQS
Permanent
Bucket
Amazon
CloudFront
Application
Video
Processing
Browsers
Mobile Apps
Metadata / Object Locations (HTTP)
Amazon
SQS
Permanent
Bucket
Hudl Uploader
Upload-Only
Bucket
Video
Hudl Video Lifecycle
Amazon
CloudFront
Application
Video
Processing
Browsers
Mobile Apps
Metadata / Object Locations (HTTP)
Amazon
SQS
Permanent
Bucket
Hudl Uploader
Upload-Only
Bucket
Video
Transfer
Acceleration
Hudl Video Lifecycle – Transfer Acceleration
Hudl Video Lifecycle
Browsers
Amazon
CloudFront
Hudl Uploader
Upload-Only
Bucket
Application
Video
Processing
Mobile Apps
Metadata / Object Locations (HTTP)
VideoAmazon
SQS
Permanent
Bucket
Browsers
Amazon
CloudFront
Hudl Uploader
Upload-Only
Bucket
Application
Video
Processing
Mobile Apps
Metadata / Object Locations (HTTP)
VideoAmazon
SQS
Permanent
Bucket
Hudl Video Lifecycle – Standard-IA
Finding an ideal Standard-IA transition lifecycle
Video
Views
Weeks After Created
Basketball Video Usage Over Lifetime
Standard-IA> 50% transitioned
Teams
N-N
Teams
N-N
Teams
N-N
Game
Video
Highlight
Video
Recruit
Video
Bucket Organization
Ring Products: Practical Uses for the IoT
Ring Neighborhoods: Network Effects in Practice
Wilshire Park study with LAPD:
Ring installed on 10% of homes
Burglaries down 55% for the
entire community in 6 months
Burglars want an easy hit and go
elsewhere if you’re home
Alarms are reactive, not proactive
Traditional systems don’t link up,
so protection ends at your door
Devices installed in nearly every country on Earth
Millions of connected apps and devices
Over 1 billion videos and rapidly increasing
High growth brings big challenges, even month-to-month
Ring Video Pipeline
Raw
Buckets
Final
(Standard)
S3 Logs
Amazon
CloudFront
Ring App(s)
AWS
Lambda
Viewers
Amazon
SQS
Owner(s)
Visitor
Application
Servers
Ring Device
GPU
Farm
Final
(IA)
Lifecycle
Transitions
Event
Triggers
Live Video
Intelligently Determining Class
Infrastructure Savings in Amazon S3
Global Concerns
Extreme Performance is Easy
S3 will automatically partition if you use good keys – or just add more buckets
Amazon CloudFront as a CDN for GET heavy loads and faster downloads
Faster uploads with Transfer Acceleration
TCP Window Scaling - without it, 64 KB window kneecaps long fat networks
TCP SACK is good for fast but lossy connections like mobile connections
examplebucket/2134857/data/start.png
examplebucket/2134857/data/resource.rsrc
examplebucket/2134857/data/results.txt
examplebucket/2134858/data/start.png
examplebucket/2134858/data/resource.rsrc
examplebucket/2134858/data/results.txt
examplebucket/2134859/data/start.png
examplebucket/2134859/data/resource.rsrc
examplebucket/2134859/data/results.txt
examplebucket/7584312/data/start.png
examplebucket/7584312/data/resource.rsrc
examplebucket/7584312/data/results.txt
examplebucket/8584312/data/start.png
examplebucket/8584312/data/resource.rsrc
examplebucket/8584312/data/results.txt
examplebucket/9584312/data/start.png
examplebucket/9584312/data/resource.rsrc
examplebucket/9584312/data/results.txt
S3 Scaling on H-Day
DevOps Software
Real-Time Splunk for JIRA Service Desk
Splunk Real-Time Search integration with JIRA
Service Desk for automated issue creation.
JIRA Service Desk links to Splunk Real-Time
search results for rapid troubleshooting.
Atlassian Cloud to Splunk
Forty8Fifty
Atlassian
Connect Add-on
AWS Benefits
Fault Tolerant
HA Infrastructure
Rapid AWS
Innovation
Supports Our
Innovation
Scalable for Growth Cost/Margin
Management
Thank you!
Remember to complete
your evaluations!