25
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Henry Zhang, Senior Product Manager, AWS December 15, 2016 Collect, Analyze, and Move Extremely Large Data Sets with AWS Snowball Edge and AWS Snowmobile

Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

Embed Size (px)

Citation preview

Page 1: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Henry Zhang, Senior Product Manager, AWS

December 15, 2016

Collect, Analyze, and Move Extremely Large Data Sets with

AWS Snowball Edge and AWS Snowmobile

Page 2: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

Data Migration, Batch and Stream Processing

Direct Connect

Snowball, Snowball Edge,

Snowmobile

3rd Party Connectors

Transfer Acceleration

Storage Gateway

Kinesis Firehose

File

Amazon EFS

Block

Amazon EBS (persistent)

Object

Amazon GlacierAmazon S3 Amazon EC2 Instance Store

(ephemeral)

Page 3: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

The AWS data transport solution

AWS Snowball—petabyte scale data transport

• Rugged 8.5G impact case

• Rain and dust resistant

• Data encryption end-to-end

• 50 TB capacity/10G networkE-ink shipping

label

RE:INVENT 2015 LAUNCH

Page 4: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

How Snowball moves data into and out of AWS

Createa job

Connect theSnowball

Copy data tothe Snowball

Your data moved to

Amazon S3

In transit to you Delivered to you Delivered to AWS At AWS

Job created Job completed

Page 5: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

Snowball to date

SPEED

COSTLOGISTICS

SECURITY • Circled the earth 100x

• 63 total launches

• 10 AWS Regions

100x

Page 6: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

When to use AWS Snowball

Cloud Migration

Disaster Recovery

DatacenterDecommission

ContentDistribution

Page 7: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

Customer use case: S3 as a data lake

Amazon EMR Amazon Kinesis

Amazon Redshift

Amazon DynamoDB

Amazon RDS

AWS Data Pipeline

Spark StreamingCassandra Storm

Amazon S3

Page 8: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

Snowball key features released

• Data export• 80 TB Snowball• Snowball SDK, S3 Adaptor• HDFS import• HIPAA eligibility

Page 9: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

How fast is Snowball?• Less than 1 day to transfer 50 TB via a 10 G connection with

Snowball, less than 1 week including shipping• Number of days to transfer 50 TB via the internet at typical utilizations

Internet Connection SpeedUtilization 1Gbps 500Mbps 300Mbps 150Mbps

25% 19 38 63 12650% 9 19 32 6375% 6 13 21 42

Page 10: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

How fast is Snowball?• Less than 1 day to transfer 250 TB via 5x10 G connections with 5

Snowballs, less than 1 week including shipping• Number of days to transfer 250 TB via the Internet at typical utilizations

Internet Connection SpeedUtilization 1Gbps 500Mbps 300Mbps 150Mbps

25% 95 190 316 63250% 47 95 158 31675% 32 63 105 211

Page 11: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

How is my data transported securely?

• 256-bit encryption• Data is encrypted by the

Snowball client before it reaches the Snowball appliance

• Keys are managed by AWS KMS and are never sent to the Snowball

Page 12: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

How is my data transported securely?

• Strong chain of custody• Tamper-resistant case• Tamper-resistant electronics

(TPM)• Each Snowball is erased

according to NIST 800-88 media sanitization guidelines between every job

Page 13: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

AWS Snowball EdgePetabyte-scale hybrid device with onboard compute and storage

• 100 TB local storage

• Local compute equivalent to an Amazon EC2 m4.4xlarge instance

• 10GBase-T, 10/25Gb SFP28, and 40Gb QSFP+ copper, and optical networking

• Ruggedized and rack-mountable

RE:INVENT 2016 LAUNCH

Page 14: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

Snowball Edge key features

S3-compatible endpoint

File interface (NFS)

Clustering

Run AWS Lambda functions

Faster data transfer

Encryption

Page 15: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

Collect data Create job Copy data Moved to S3

Hybrid capabilities beyond data migrationM

IGR

ATIO

NC

OLL

ECTI

ON

Create job Copy data Moved to S3

Page 16: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

Snowball Edge – data collection & processing

Protectedand stored

CO

LLEC

TIO

N

Accessible and consumable

Shorttime frame

Migrate tothe cloud

Page 17: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series
Page 18: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

Snowball Edge can…

Extend of your data center

Process data Expedite move

Encrypted, secure, and embedded

compute

Write data directly as data is generated

Offers a fast and cost effective way to ensure data can be

quickly transferred to and from the cloud

Simplifydata transfer

Uses standard and familiar tools

for the data transfer process

Page 19: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

Snowball Edge use cases

Offline Staging

Local Tiering and ComputeIoT Local

Transformation

Page 20: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

What’s your bright idea?

Page 21: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

AWS storage migration for exabytes:AWS Snowmobile

Page 22: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

What• 10-100PB in a 45 foot-long, secure (256-bit) ruggedized container truck

Where & When• Can be made available in all AWS regions

How• Data transferred via multiple 40Gbps interfaces up to 1Tb/s (100PB in a few weeks)• Appears as NFS mount point• Customer orders a Snowmobile, we dispatch it to their site, they hook it up and fill it, it returns

How much does it cost• $0.005/GB/mo based on provisioned capacity (from site departure to AWS ingestion completion)

Page 23: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

Snowmobile key features

Massively Scalable

Customized for your needs

Faster data transfer

Strong Encryption

Rugged, Durable, and Secure

Page 24: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series

Getting startedLearn more about Snowmobile and Snowball Edge

https://aws.amazon.com/snowmobile

Contact AWS Sales

Go to the management

console

https://aws.amazon.com/snowball-edge

Snowmobile Snowball Edge

Page 25: Announcing AWS Snowball Edge and AWS Snowmobile - December 2016 Monthly Webinar Series