Upload
hyun-woo-park
View
1.617
Download
0
Embed Size (px)
DESCRIPTION
AWS 한국 사용자 그룹 2013년 1회 세미나에서 발표한 자료입니다.
Citation preview
Using CloudFront with S3at SMARTSTUDY
Amazon Web Service Korean User GroupPark Hyun-woo, a SCV of SMARTSTUDY / [email protected]
What is S3 and CF?
● S3 : Simple Storage Service○ Extreme durability.○ Multiple copies in selected region.
■ vs EBS : Multiple copies in single available zone.
● CF : CloudFront○ Content Delivery Network(CDN) service.○ Distribute from origin to AWS Edge Network
■ http://aws.amazon.com/cloudfront/#details■ About 30 edges in global. (only in Earth, lol)
Why choose these?
● We serve large multimedia contents to global market in mobile application era.
● We need redundant, fast and reasonable price content delivery service aims globally.
● We have not enough human resource to maintain various system.
Flash Movie
S3 Bucket( tokyo )
s3cmd
Package( encrypted )
Thumbnails
Video
npk
Transcoder
Gnash
FFmpeg
qtfaststart
Flash
Movie
CloudFront{
Edge Edge Edge Edge Edge
S3 Bucket( tokyo )
s3cmd
Package( encrypted )
Thumbnails
Video
Transcoder
Gnash
FFmpeg
qtfaststart
Flash
Movie
npk
How it works?
● Copy to S3○ ~100 files, just use Cyberduck or other GUI tool.○ Or, you have to use s3fs or s3cmd.○ If you need sync or invalidation, use s3cmd.○ We have ~30,000 files. Thus, s3cmd.○ We chose Tokyo bucket for faster uploading.
● s3cmd○ Awesome!○ Works on S3 and CloudFront.
How it works? (cont.)
● s3cmd○ Can do almost everything as like as AWS console.○ Support INI-style configuration file.○ And, magical 'sync' command. (yeah!)
● s3cmd sync○ Support glob-style exclude / include option.○ ... rexclude / rinclude for regular expression.○ Support automatic invalidation request.○ ... with rsync-like 'dry-run' option. (-n, --dry-run)
● s3cmd sync○ Basically, sync based on MD5 information.○ (!) Over 15MB files, using only filesize.○ Our 30,000 files, 60GB = 8 min to dry-run.
( env : i7 2500K + SATA HDD to bucket in tokyo )
● s3cmd sync --cf-invalidate○ Call invalidation API automatically.○ (!) Simultaneous invalidation is up to 3.○ (!) Free for ~1,000 files per month.○ Check it up by s3cmd cfinvalinfo
How it works? (cont.)
So, is it really?
FASTREDUNDANTNOT EXPENSIVE
{
It's fast enough.
8.0MB/s
It's very redundant.
BETTER THAN YOUR OWN
SERVER, EVER
And
NOT EXPENSIVE
* without transfer fee for deploying origin bucket to edges
Bill for
CloudFrontBased on
Total traffic
● 10TB/month
● $0.209/GB
= $2,090*
Bill for
Domestic CDNBased on
Peak traffic
● 350Mbps/@peak
● $6.00/Mbps
= $2,100
STILL EXPENSIVE?
* Photo from Linkedin page of Lee, Jung-in / Amazon Korea
For over 10TB,
RESERVEDCapacity Pricing
● Smaller○ Encode media with optimal options.○ Use dedicated option for each platform.○ Storage is cheaper than transfer fee!
● Static○ Do not deploy volatile objects to CloudFront.○ Invalidation needs ~10min, often.○ We're still using Nginx for them.
● Streamable○ Mobile networks are still slow.○ Do not make users to wait until finishing download.
Tips
● Gnash○ http://www.gnu.org/software/gnash/
● FFmpeg○ http://www.ffmpeg.org/
● qtfaststart○ https://github.com/danielgtaylor/qtfaststart
● npk○ https://github.com/lqez/npk
● s3cmd○ http://s3tools.org/
Link
Using CloudFront with S3at SMARTSTUDY
Amazon Web Service Korean User GroupPark Hyun-woo, a SCV of SMARTSTUDY / [email protected]