46

Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release
Page 2: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Learning To Fly: How Angry

Birds Reached the Heights of

Store Performance

Page 3: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Insert photo of speaker

here

891 pixels h x 688 pixels w

Learning To Fly:

How Angry Birds

Reached the

Heights of Store

Performance

Lead Magento Developer

Fabrizio Branca

Page 4: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

San Francisco, CA

Janine

Fiona

that’s me

It’s a boy!

Page 5: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

AOE media, a Magento Gold Partner and leading provider for

Open Source-based Portal development, Web Content

Management and e-Commerce with its headquarters in San

Francisco, California and Wiesbaden, Germany and more than

120 people in seven offices world-wide.

Page 6: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Magento Imagine 2012 Meet Magento 2012 Meet Magento 2011

Part II Part I

Page 7: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release
Page 8: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Lessons Learned ...from running a

High Performance Magento Store

original title of this presentation!

Page 9: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release
Page 10: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

PC games (download)

Plush toys, shirts, ...

Page 11: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

> 24.000.000 likes > 500.000 followers

Social Media Impact

Page 12: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

1.000.000 unique visitors. per day.

Page 13: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

concurrent users.

Page 14: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Orders.

10

5 6 7

8

9

per second. Yes, per second!

Page 15: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release
Page 16: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

S3 Assets Storage

Varnish Array EC2

Frontend Array

CloudFront CDN for Skin

RDS MySQL

S3 Deployment

Cache

ELB Load Balancer

CloudFront CDN for Assets

Monitoring

Backend Array EC2 EC2

EC2

Worker Array EC2

Braintree

Mail DRM

Giftcards

Logging

EC2

Route 53 DNS-Service

shop.angrybirds.com

Page 17: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Varnish Array

Frontend Array Worker Array EC2

Deployment x

ELB Load Balancer

Varnish ArrayEC2

Frontend ArrayBackend Array EC2 EC2

EC2

EC2

Deployment x+1 Release Y

ELB Load Balancer

Route 53 DNS-Service

shop.angrybirds.com

Deployment

Page 18: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Downtime

Number of lost carts

Number of lost sessions and logins

Page 19: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release
Page 20: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Stores …and GeoIP redirection!

Page 21: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Themes

Page 22: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Growing Data

Trimming the fat!

Orders

Promotions

cms / banners

Aoe_CacheCleaner

Aoe_QuoteCleaner

Page 23: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Varnish Reverse Proxy

Magento

Protecting Magento

more traffic, more stores,

more products, more urls,…

less requests hitting Magento!

Page 24: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

90%

Varnish Tweaking

Cache Hitrate

Page 25: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

80%

Varnish Tweaking

Cache Hitrate

Page 26: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

98.6%

Varnish Tweaking

Cache Hitrate

Page 27: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Request a page (HTTP Request)

deliver static page

Request

Delivering static content

Placeholders for dynamic parts.

Cache-Control headers

Ajax call request dynamic parts

and transfers the id of the currently visited product

JSON response with dynamic content

Replace placeholder

Write session id to cookie

Render the page

Browser Varnish Magento

Aoe_Static

Page 28: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Reduce Number of Requests Do ajax requests instead of full page loads! »Add to basket

»Displaying the mini cart

Don’t do ajax requests! »Login status and username

»(Number of) cart items

»Geo location

Cache 404 pages and redirects!

Normalize urls!

… hitting Magento

Use cookies or localStorage

Page 29: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Cache in a Cache

Magento

Varnish

Aoe_AsyncCache Aoe_VarnishAsyncCache

Updates

Page 30: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Bottleneck Database

Page 31: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Database Tuning

db.m2.4xlarge

Provisioned IOPS Storage

Percona Toolkit

This is not an iPhone!

“High-Memory Quadruple

Extra Large DB Instance”

Page 32: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

RDS

(MySQL)

RDS

(MySQL)

Cache Backend

Aoe_DbCache

Database Splitting

Page 33: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Sessions

FPC RDS

(MySQL) Redis

Cache Backend

Redis

Page 34: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Bottleneck File I/O

Page 35: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

0

lib

community

core

local

Number of *.php files by location

Less file operations

Page 36: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Aoe_ClassPathCache Magento

Compiler

Solution File I/O

…also check Magento 2!

Page 37: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Report Generation

Don’t try this at home! on production

Read Replica Background

Process

Delta Pre-

processing

Page 38: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Processing Crons

cron.sh

cron.php

Aoe_Scheduler with multiple cron groups

will block other crons

performance issues, race conditions!

SCHEDULER_WHITELIST='a,b,c’

cron.sh cron2.php

SCHEDULER_BLACKLIST='a,b,c'

cron.sh

Aoe_Scheduler

Page 39: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Code Quality Modules

Don’t let them bring you down!

Uh-Oh!

Page 40: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Code Quality Modules

Not enterprise

ready

Not production

ready

Not cloud

ready

Bad

performance

Security

Page 41: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Continuous Quality Assurance

Code Reviews Profiling,

stress tests Automate

Aoe_Profiler

Everything sounds better with “continuous”

Question everything!

Part of the dev process!

Page 42: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Broken Testcases

Switch to Selenium 2

Nobody pays attention to test results

Menta

Page 43: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Lesson Learned Plan extra time!

Number of unexpected problems

Number of components

Page 44: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

What’s next?

Move quotes Cache Backend

+ php-fpm

+ PHP 5.4

Web server EC2 instance

sizes

MySQL? MongoDB?

Less bigger instances vs.

more smaller ones?

Page 45: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Thank you, Team! You’re awesome!

Page 46: Learning To Fly: How Angryinfo2.magento.com/rs/magentoenterprise/images/... · shop.angrybirds.com . Varnish Array Frontend Array EC2 Worker Array Deployment x ELB Load Balancer EC2Release

Thank you! Any questions?

[email protected]

http://www.aoemedia.com

http://www.fabrizio-branca.de

@fbrnc Follow me on twitter!

My blog