Drupal Amazon Integration

Hosting Drupal – Getting Started!

Drupal Integration with Amazon S3 & Cloudfront

Mark Schoonover – ThetaJoinmark@thetajoin.comSandCamp 2010

Content Delivery Networks

● What do they do?

http://www.thetajoin.com

Creative Commons Attribution ShareAlike 3.0

http://www.thetajoin.com/

http://en.wikipedia.org/wiki/Creative_Commons

http://creativecommons.org/licenses/by-sa/3.0/

● Content Delivery Networks (CDN) are used to move bandwidth intensive files off your Drupal hosting server and put them closer to end user computers.

● Content can be audio, video, photos, software, other documents, or static files.

● Excellent not just for distributing large files, but frequently accessed smaller files.

● Excellent for streaming audio and video, especially if your Drupal server doesn't support streaming.

● Popular CDNs

– Akamai– Amazon Cloudfront– Amazon Simple Storage Service (S3)– CacheFly– Many more...

We'll be concentrating on integrating Drupal 6 with Amazon Simple Storage Service – S3 & Amazon Cloudfront

Requirements

http://www.thetajoin.comCreative Commons Attribution ShareAlike 3.0

Requirements

– Amazon Web Services Account– Signup for Amazon S3 & Cloudfront– Amazon S3 file management tools– Comfortable installing & configuring Drupal modules– Comfortable modifying Drupal settings.php file– Comfortable modifying template.php theme file– Know how to configure ImageCache

Benefits of Amazon S3

Benefits of Amazon S3– Unlimited, pay as you go storage– Geographical data centers in US, Northern CA, Europe

too– Many S3 file managment tools– Keeps your Drupal files directory free of large infrequently

changed files– Supports BitTorrent

Drawbacks of Amazon S3– Not as many datacenters as Cloudfront– No CCK for S3– Manual integration with Drupal content– Data in two places

Benefits of Amazon Cloudfront

Benefits of Amazon Cloudfront– Unlimited temporary storage– Uses Amazon S3 as origin storage– More datacenters than S3– Now supports audio & video content streaming

● Unlimited, Private YouTube anyone??Unlimited, Private YouTube anyone??

United States

* Ashburn, VA

* Dallas/Fort Worth, TX

* Los Angeles, CA

* Miami, FL

* Newark, NJ

* Palo Alto, CA

* Seattle, WA

* St. Louis, MO

Europe

* Amsterdam

* Dublin

* Frankfurt

* London

Asia

* Hong Kong

* Tokyo

Drawbacks of Amazon Cloudfront– 24 hour limit on files at edge, must recopy from S3– Charged for every time file is copied from S3 to

Cloudfront.– Charged when file requested from Cloudfront– Revisions of files with same name

Copy Content to Amazon S3

Copy Content to Amazon S3– Amazon S3 Drupal module

● Can be found: http://drupal.org/project/amazon_s3– Firefox S3Fox add-on

● Can be found: http://www.s3fox.net/– There are others...

Sign Up for Amazon Web Services

– http://aws.amazon.com/

Sign Up for Amazon S3 & Cloudfront

Amazon S3 Access Keys

Install & Enable Amazon S3 Module

– Download– Untar into /path/to/drupal/sites/all/modules– Enable

● Administer->Site Building->Modules

Amazon S3 Drupal Module Configuration

Administer>Site Configuration>Amazon S3 Configuration

Amazon S3 Drupal Module Usage

Amazon S3 Drupal Module– Draw backs

● Can't create subdirectories● Very basic S3 control● Can't modify existing buckets● Can't create Cloudfront distributions

Toplevel bucket name needs to be unique across ALL of S3!

– cdn.example.com

S3Fox – Firefox Addon

S3Fox – Firefox Addon– www.s3fox.net

S3Fox Account Manager

S3Fox Configuration

S3Fox Usage

Files are now on Amazon S3!

Drupal & Amazon S3 Integration

Drupal & Amazon S3 Integration● Obtain URL to file on S3

– Copy URL to Clipboard● Insert URL into Content

Amazon S3 File URL

http://cdn.thetajoin.com.s3.amazonaws.com/HostingGuide/CactiLogin01.png

Paste S3 URL

Drupal & Amazon S3 Integration Complete!

Oh, this URL

http://cdn.thetajoin.com.s3.amazonaws.com/HostingGuide/....

Oh, this URL

http://cdn.thetajoin.com.s3.amazonaws.com/HostingGuide/....

Oh SO ugly!

DNS CNAME trick

New URL:

http://cdn.thetajoin.com/HostingGuide/CactiLogin01.png

Amazon Cloudfront Integration

Amazon Cloudfront Integration● Upload files to S3● Create Cloudfront distribution

– Right click on bucket->manage distribution

– Right click on bucket->manage distribution● Add CNAME like cf.example.com in DNS

– cf.example.com

– cf.example.com– http://cf.example.com/directory/file.pdf

– cf.example.com– http://cf.example.com/bucket/file.pdf

● Amazon Cloudfront handles requesting content from S3

Amazon Cloudfront Integration● Determine URL● Copy N Paste URL into Drupal

Amazon Cloudfront Integration● Determine URL● Copy N Paste URL into Drupal● You're golden!

Drupal and Amazon Cloudfront Automation● Use Cloudfront module

Drupal Amazon Cloudfront Module● Install Cloudfront module, enable, permissions

– http://drupal.org/project/cloudfront– Depends on ImageCache & ImageAPI modules

● Add Access & Secret Keys to settings.php● Add 'Send to Cloudfront' Image Cache action &

Configure● Override theme_imagecache function in template.php● Cron configuration & security

Drupal Amazon Cloudfront Module● Add Access & Secret Keys to settings.php

Cloudfront Config

Override theme_imagecache function in template.php

How does it work?● imagecache inserts path data into cloudfront_files

table

table● CRON job http://localhost/cloudfront/processqueue

– Pushes images to S3– Updates cloudfront_send_queue table with updated

path info

table● CRON job http://localhost/cloudfront/processqueue

– Pushes images to S3– Updates cloudfront_send_queue table with updated

path info● garland_imagecache() rewrites IMG tag URL to

point to Cloudfront

Becareful flushing imagecache!

– Could take significant time to reprocess images, copy to S3!

Becareful flushing imagecache!– Could take significant time to reprocess images,

copy to S3!– Remember Cloudfront and revisions, there's no

way to manually flush Cloudfront

Slides of this talk available on: http://www.thetajoin.com - SandCAMP

Thank you for attending my SandCAMP 2010 session. I hope you enjoyed the session as much as I did preparing and presenting.

For attendees of my sessions & people who download my slides, I'm offering a free Drupal performance analysis of your site! I have to limit this offer to a single site only, so pick your worst! Just email or call to setup an appointment.

Hope you had a great time at SandCAMP 2010!

Regards,Mark Schoonover

http://www.thetajoin.com - The Drupal Hosting & Performance CompanyEmail: mark@thetajoin.com :: Voice: 619-928-4473 :: Fax: 619-374-3130

Drupal Amazon Integration

Documents

Drupal as integration hub.odp

Drupal Continuous Integration Workflow

Drupal Integration with Solr for Fabulous CMS Search

Open (Government) Data mit Drupal und Integration mittels ... & Opdedata - OGDI.pdfOpen (Government) Data mit Drupal und Integration mittels Windows Azure . 15.06.2011 / Page 2 2 Why

Behaviour Testing and Continuous Integration with Drupal

Drupal on the AWS Cloud - aws … · This guide discusses best practices for deploying Drupal on AWS using Amazon Elastic ... (Amazon EFS) provides simple, scalable file ... Shared

Anton Shloma. Drupal as an integration platform

Drupal Integration Best Practices

Continuous integration with drupal

Planning & Executing Custom Drupal Integration Projects

Things Made Easy: One Click CMS Integration with Solr & Drupal

2016 amazon-integration

Amazon Marketplace Integration - Broadgate Infonet · Amazon Marketplace Integration Amazon Marketplace Integration y integrating Amazon Marketplace with your business software you

Drupal & Continous Integration - SF State Study Case

Continuous Integration and Drupal

Alfresco based Publishing Management Solution by Integration MongoDB & Drupal

osCaddie Drupal/Alfresco Integration Technical Update

Drupal Continuous Integration and devops - Beyond Jenkins

DrupalCamp Delhi - Magento Integration with Drupal

Stress Test Drupal on Amazon EC2 vs. RackSpace cloud