View
1.726
Download
2
Category
Preview:
Citation preview
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Data Transfer ServicesAccelerating Large-Scale Data Ingest into the AWS Cloud
Erik Durand, Principal BDM
Tom Johnston, Senior Manager, S3 Product Management
Amazon EFS
File
Amazon EBSAmazon EC2
Instance Store
Block
Amazon S3 Amazon Glacier
Object
Data Transfer
AWS Direct
Connect
AWS
Snowball
ISV Connectors Amazon
Kinesis
Firehose
Transfer
Acceleration
Storage
Gateway
Storage is a platform
New ways to transfer data into the cloud:
AWS Import/Export Snowball
• Now holds 60% more
– New 80 TB model, $250/job
– 50 TB still available in US West and US East for $200/job
• New regional availability*
– Currently in US West (Oregon) and US East (N. Virginia)
– US West (N. California), GovCloud (US), Asia Pacific (Sydney), and EU
(Ireland) with other regions expected by the end of 2016
*80 TB model only
What is Snowball? Petabyte scale data transport
E-ink shipping
label
Ruggedized
case
“8.5G Impact”
All data encrypted
end-to-end50TB & 80TB
10G network
Rain & dust
resistant
Tamper-resistant
case & electronics
Use cases: AWS Import/Export Snowball
Cloud
Migration
Disaster
Recovery
Data Center
Decommission
Content
Distribution
How fast is Snowball?
• Less than 1 day to transfer 50 TB via a 10 G connection with
Snowball; less than 1 week including shipping
• Number of days to transfer 50 TB via the Internet at typical utilizations
Internet Connection Speed
Utilization 1Gbps 500Mbps 300Mbps 150Mbps
25% 19 38 63 126
50% 9 19 32 63
75% 6 13 21 42
How fast is Snowball?
• Less than 1 day to transfer 250 TB via 5x10 G connections with 5
Snowballs’ less than 1 week including shipping
• Number of days to transfer 250 TB via the Internet at typical utilizations
Internet Connection Speed
Utilization 1Gbps 500Mbps 300Mbps 150Mbps
25% 95 190 316 632
50% 47 95 158 316
75% 32 63 105 211
How is my data transported securely?
• All data is encrypted with 256-bit
encryption by the Snowball client
• Keys are managed by AWS KMS and
are never sent to the Snowball
• Strong chain of custody
• Tamper-resistant case
• Tamper-resistant electronics (TPM)
• Each Snowball is erased according to
NIST 800-88 media sanitization
guidelines between every job
Where can I use Snowball?
us-east-1
(Virginia)us-west-2
(Oregon)
us-gov-west-1
(PDT)
us-west-1
(SFO)
ap-southeast-2
(SYD)
eu-west-1
(DUB)
Pricing
Dimension Price
Usage Charge per Job $200.00 (50 TB)
$250.00 (80 TB)
Extra Day Charge (First 10 days* are free) $15.00
Data Transfer In $0.00/GB
Data Transfer Out $0.03/GB
Shipping** Varies
Amazon S3 Charges Standard storage and request
fees apply
* Starts one day after the appliance is delivered to you. The first day the appliance is received at your site and the last day the appliance is shipped out are also free
and not included in the 10-day free usage time.
** Shipping charges are based on your shipment destination and the shipping option (e.g., overnight, 2-day) you choose.
Complicated
Setup and Management
Hard to Optimize
Performance
Expensive
Optimizing Internet Performance Is Hard
Prioprietary
Introducing Amazon S3 Transfer Acceleration
S3 BucketAWS Edge
Location
Uploader
Optimized
Throughput!
Up to 300% faster
Change your endpoint, not your code
54 global edge locations
No firewall exceptions
No client software required
Amazon
Route 53
Resolve
b1.s3-accelerate.amazonaws.com
HTTPS PUT/POST
upload_files.zip
HTTP/S PUT/POST
“upload_files.zip”
Service traffic flowClient to S3 Bucket example
S3 Bucket
b1.s3-accelerate.amazonaws.com
EC2 Proxy
AWS Region
AWS Edge Location
Customer Client
1
2
3
4
Rio DeJaneiro
Warsaw New York Atlanta Madrid Virginia Melbourne Paris LosAngeles
Seattle Tokyo Singapore
Tim
e [h
rs]
500 GB upload from these edge locations to a bucket in Singapore
Public Internet
How fast is Transfer Acceleration?
S3 Transfer Acceleration
We have customers uploading large files from all over
the world.
We’ve seen performance improvements in excess of
500% in some cases.
- Emery Wells, Cofounder/CTO ”
“
Use case: media uploads
We loved how easy it was to get started with S3 transfer
acceleration — just a simple endpoint change in our
application and done.
S3 transfer acceleration reduces the average time it takes for
us to ingest videos from our global user base by almost half.
This gives our customers the ability to edit and share videos
sooner where speed is a critical factor.
All this for a fraction of the cost of the solution we evaluated
before.
- Brian Kaiser, CTO ”
“
Use case: media uploads
S3 transfer acceleration is way faster than we expected.
It’s removed the international distance barrier when
uploading video.
Our customers now have more time to focus on
producing great videos rather than waiting for a video to
upload.- Domagoj Filipovic, CTO
“
Use case: media uploads
”
Getting Started
1. Enable S3 transfer acceleration on your S3
bucket
2. Update your application/destination URL to <bucket-name>.s3-accelerate.amazonaws.com
3. Done!
How much will it help me?
https://s3.amazonaws.com/s3-accelerate-speedtest/cheetah-speedtest.html
Pricing*
Dimension Price / GB
Data Transfer In from Internet** $0.04 (Edge location in US, EU, JP)
$0.08 (Edge location in rest of the world)
Data Transfer Out to Internet $0.04
Data Transfer Out to Another AWS Region $0.04
Amazon S3 Charges Standard data transfer charges apply
*Plus standard Amazon S3 data transfer charges apply
**Accelerated performance or there is no bandwidth charge
AWS Snowball Transfer Acceleration
When do I use what?
• Large, infrequent uploads
• Tens of TBs of upload from a
centralized location
• 7–10 day tolerance
• Recurring, frequent uploads
• GBs or TBs of upload from
distributed locations
• Long geographic distances
Recommended