39
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lee Atkinson, Solutions Architect December 1, 2016 Taking DevOps to the AWS Edge CTD302

AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Embed Size (px)

Citation preview

Page 1: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Lee Atkinson, Solutions Architect

December 1, 2016

Taking DevOps to the AWS EdgeCTD302

Page 2: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

What to Expect from the Session

• Integrate Amazon CloudFront and other AWS edge

services into your agile development process.

• Follow best practices to improve security, performance,

and reduce costs.

• Automate edge deployment and testing.

• Monitor and analyze usage of AWS edge services.

Page 3: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Agenda

1. What is the AWS Edge?

2. The application edge

3. Patterns for the DevOps Edge

Page 4: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

What is the AWS Edge?

Page 5: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

AWS edge locations

68 Edge locations

Page 6: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

AWS edge services

• Amazon CloudFront

• Content Delivery Network

• Amazon Route 53

• DNS registration and hosting, health checks

• AWS WAF

• Web Application Firewall

Page 7: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

The application edge

Page 8: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Your Application =

EC2/ELB/S3 Route 53/CloudFront/AWS WAF

HTML5/Mobile/Desktop

Origin+ Edge+ Client

Page 9: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

ClientClient

Amazon Backbone

Edge location Edge location Edge location

CloudFront helps improve user experience

Origin

Page 10: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

CloudFront helps reduce costs

Free data transfer cost AWS

origin to CloudFront

Less load on origin

Lower data transfer cost CloudFront to Internet

compared to AWS Region to Internet

Client

Edge location Edge location Edge location

Origin

Page 11: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Amazon Route 53 health checks

Amazon Route 53

health checks

Endpoint

Endpoint

www.test Amazon

CloudWatch

Amazon

SNS

Page 12: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Patterns for the DevOps Edge

Page 13: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Patterns for the DevOps Edge

1. Cache as much as possible

2. Forward as little as possible

3. Validate efficiently

4. Automate the edge

5. Monitor the edge

Page 14: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Patterns for the DevOps Edge

1. Cache as much as possible

2. Forward as little as possible

3. Validate efficiently

4. Automate the edge

5. Monitor the edge

Page 15: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

HTTP Cache-Control

Origin

Client

Cache-Control: max-age:1800Cache-Control: max-age:1800,s-maxage:900Cache-Control: max-age:1800,privateCache-Control: no-cacheCache-Control: no-store

HTTP/1.1 200 OKLast-Modified: Fri, 02 Dec 2016 01:00:00 GMT

ETag: "TGVlQXRraW5zb25Xb3onRXJl"

GET /barchart.jpg HTTP/1.1

If-None-Match: "TGVlQXRraW5zb25Xb3onRXJl"

HTTP/1.1 304 Not ModifiedHTTP/1.1 200 OKLast-Modified: Fri, 02 Dec 2016 02:00:00 GMT

ETag: "TGVlQXRraW5zb24nRXJlVG9v"

Amazon

CloudFront

Page 16: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Cache-Control

s-maxage?

Cache-Control

max-age?

Expires?

> MinTTL?< MaxTTL?

Use DefaultTTL Use MaxTTL Use MinTTL Use Origin-Defined TTL

The origin has defined

the TTL for the object

CloudFront TTL

Page 17: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Use the client in increase cacheability

Page 18: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Patterns for the DevOps Edge

1. Cache as much as possible

2. Forward as little as possible

3. Validate efficiently

4. Automate the edge

5. Monitor the edge

Page 19: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Forwarding

GET /search?query=widget&foo=bar HTTP/1.1

Host: www.test

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-GB,en;q=0.7,en-US;q=0.3

Accept-Encoding: gzip, deflate

Cookie: session=cjZYuh3fVXzf6rXGAwV2; theme=modern; currency=gbp

GET /search?query=widget&foo=bar HTTP/1.1

Host: www.test

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101 Firefox/49.0

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-GB,en;q=0.7,en-US;q=0.3

Accept-Encoding: gzip, deflate

Cookie: session=cjZYuh3fVXzf6rXGAwV2; theme=modern; currency=gbp

GET /search?query=widget&foo=bar HTTP/1.1

Host: www.test

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101 Firefox/49.0

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-GB,en;q=0.7,en-US;q=0.3

Accept-Encoding: gzip, deflate

Cookie: session=cjZYuh3fVXzf6rXGAwV2; theme=modern; currency=gbp

GET /search?query=widget&foo=bar HTTP/1.1

Host: www.test

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101 Firefox/49.0

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-GB,en;q=0.7,en-US;q=0.3

Accept-Encoding: gzip, deflate

Cookie: session=cjZYuh3fVXzf6rXGAwV2; theme=modern; currency=gbp

GET /search?query=widget&foo=bar HTTP/1.1

Host: www.test

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101 Firefox/49.0

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-GB,en;q=0.7,en-US;q=0.3

Accept-Encoding: gzip, deflate

Cookie: session=cjZYuh3fVXzf6rXGAwV2; theme=modern; currency=gbp

GET /search?query=widget&foo=bar HTTP/1.1

Host: www.test

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101 Firefox/49.0

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-GB,en;q=0.7,en-US;q=0.3

Accept-Encoding: gzip, deflate

Cookie: session=cjZYuh3fVXzf6rXGAwV2; theme=modern; currency=gbp

GET /search?query=widget&foo=bar HTTP/1.1

Host: www.test

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-GB,en;q=0.7,en-US;q=0.3

Accept-Encoding: gzip, deflate

Cookie: session=cjZYuh3fVXzf6rXGAwV2; theme=modern; currency=gbp

GET /search?query=widget&foo=bar HTTP/1.1

Host: www.test

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101 Firefox/49.0

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-GB,en;q=0.7,en-US;q=0.3

Accept-Encoding: gzip, deflate

Cookie: session=cjZYuh3fVXzf6rXGAwV2; theme=modern; currency=gbp

GET /search?query=widget&foo=bar HTTP/1.1

Host: www.test

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101 Firefox/49.0

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-GB,en;q=0.7,en-US;q=0.3

Accept-Encoding: gzip, deflate

Cookie: session=cjZYuh3fVXzf6rXGAwV2; theme=modern; currency=gbp

Page 20: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Tips when forwarding

• Only forward when it affects the response

• Reduce variability in forwarded values

• Use CloudFront logs or beacon for tracking

• Authorize using signed URLs/cookies, CloudFront

restrictions, and AWS WAF

• Use Vary response header

Page 21: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Forward User-Agent?

GET / HTTP/1.1

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101 Firefox/49.0User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/601.6.17 (KHTML, like Gecko) Version/9.1.1 Safari/601.6.17User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0

HTTP/1.1 307 Temporary RedirectLocation: /user-agent?return-url=/Vary: Cookie

origin

client

GET /user-agent?return-url=/ HTTP/1.1User-Agent: Mozilla/5.0 (Macintosh; …

HTTP/1.1 307 Temporary RedirectLocation: /Set-Cookie: OS=MacOSXVary: User-Agent

GET / HTTP/1.1Cookie: OS=MacOSX

CloudFront-Is-Mobile-Viewer

CloudFront-Is-Tablet-Viewer

CloudFront-Is-Desktop-Viewer

CloudFront-Is-SmartTV-Viewer

Cache-Behavior for “/user-agent”:

Forward Header ‘User-Agent’

Default Cache Behavior:

Forward Cookie ‘OS’

Page 22: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Responsive web design

/* Use CSS3 media-selectors */ @media only screen and (max-width: 768px) {

…}

Page 23: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Patterns for the DevOps Edge

1. Cache as much as possible

2. Forward as little as possible

3. Validate efficiently

4. Automate the edge

5. Monitor the edge

Page 24: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

HTTP does not provide invalidation

• CDN invalidation is not recognized by downstream

caches:

• Origin ✓

• CloudFront ✓

• ISP / corporate / hotel caches ✗

• Browser caches ✗

Page 25: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

HTTP does provide validation

• Use conditional HTTP requests• If-Modified / If-None-Match / If-Match

• Origin should efficiently generate 304s

• Use low s-maxage and higher max-age

• Use URL versioning to ‘cache-bust’

Page 26: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Patterns for the DevOps Edge

1. Cache as much as possible

2. Forward as little as possible

3. Validate efficiently

4. Automate the edge

5. Monitor the edge

Page 27: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

AWS edge services automation

• Route 53, CloudFront, and AWS WAF have APIs

• AWS SDK available for 11 languages and platforms

• Deploy DNS, CDN, and WAF using AWS

CloudFormation

• Change configuration as your application develops

• Use AWS IAM to control access to your resources

Page 28: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Amazon

Route 53

AWS WAF

Amazon

CloudFront

Elastic Load Balancing EC2

EU-WEST-1

Amazon S3

Corporate data center

Elastic Load Balancing EC2

US-WEST-1

Amazon

Route 53

DNS, CDN, and WAF ‘as code’

AWS Lambda

AWS

Certificate Manager

Client

Page 29: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

AWS WAF automation

• Automate AWS WAF using your application or Lambda

• ‘AWS Answers’ AWS WAF

Security Automations

https://aws.amazon.com/answers/security/aws-waf-security-automations/

Page 30: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Patterns for the DevOps Edge

1. Cache as much as possible

2. Forward as little as possible

3. Validate efficiently

4. Automate the edge

5. Monitor the edge

Page 31: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

AWS CloudTrail

• Route 53

• CloudFront

• AWS WAF

Page 32: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Amazon CloudWatch metrics & alarms

• Route 53

• CloudFront

• AWS WAF

Page 33: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

CloudFront reporting

Page 34: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

CloudFront access logs

Amazon

CloudFrontS3 bucket

AWS

LambdaS3 bucket

Amazon

QuickSight

#Version: 1.0 #Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agen2014-05-23 01:13:11 FRA2 182 192.0.2.10 GET d111111abcdef8.cloudfront.net /view/my/file.html 200 www.displaymyfiles.com Mozilla/4.0%20(c2014-05-23 01:13:12 LAX1 2390282 192.0.2.202 GET d111111abcdef8.cloudfront.net /soundtrack/happy.mp3 304 www.unknownsingers.com Mozilla/

AWS Data

PipelineAmazon

Redshift

W3C Log

Processor

Amazon

CloudWatch

Logs

Page 35: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Patterns for the DevOps Edge

1. Cache as much as possible

2. Forward as little as possible

3. Validate efficiently

4. Automate the edge

5. Monitor the edge

Page 36: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Key takeaways

1. Consider the AWS Edge as a part of your application

2. Optimise and use caching efficiently

3. Automate configuration of the AWS Edge

4. Monitor the AWS Edge using CloudTrail, CloudWatch,

and services logs

Page 37: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Thank you!

Page 38: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Remember to complete

your evaluations!

Page 39: AWS re:Invent 2016: Taking DevOps to the AWS Edge (CTD302)

Related Sessions

1. CTD301 - Amazon CloudFront Flash Talks: Best

Practices on Configuring, Securing, and Monitoring your

Distribution

Friday 09:30, Venetian, Level 2, Venetian D

2. SAC316-R - Security Automation: Spend Less Time

Securing Your Applications

Friday 10:30, Venetian, Level 3, Lido 3003