Black Friday Brazil 2013 web performance metrics

FRIDFRIDFRIDFRIDAAAAYYYY

BRAZiLweb performance metrics

BLABLABLABLACKCKCKCK

49494949Brazil ecommerce sites

5555

3333data centers

4444mobile carriers

1111K+K+K+K+last mile

agentspurchasetransactions

Key take away points32% average page load time increase during Black Friday Brazil 2013.

Average homepage page load times by network access technology:

Data Center (Backbone) 9 seconds

Last Mile (Cable, DSL, Wi-Fi) 15 seconds

Mobile cellular (3G) 22 seconds

32% Baseline average page load time increase between Black Friday Brazil 2012 & 2013

96% availability during Black Friday Brazil 2013

24 seconds peak third party content (Facebook)

Some sites implemented wait queue’s during peak usage times, using custom error screens

Last Mile:

Slowest state: Amapa 30 sec.

Fastest state: Acre 8.5 sec.

BRAZIL BLACK FRIDAY 2013 web performance metrics

secondsBackbone

averageresponse time9999

aggregated

by site

Fastest site: 1.84s

Slowest site: 16.97s


range: 28 Nov 9AM - 30 Nov 2AM @ 30min interval

% Backboneavailability9999

9999

aggregated

by site

Highest availability: 100%

Lowest availability: 70.83%



increase in average page load times on Nov 29

from previous week

32323232cumulative

average

%Black Friday seems to have started

earlier this year, registering a spike

on Monday November 25th.

increase in

response time

range: 22 November 9AM through 30 November 9AM 1 week prior


was November 23rd

Black Friday 2013

Black Friday 2012

35353535% Reduction in page peak times:

2012 peak of 80 seconds

2013 peak of 52 seconds


range: Back Friday 2012 and 3 days before, 3 days after

range: Back Friday 2013 and 3 days before, 3 days after

2013 average

(+3 days before/after Black Friday):

7.800 seconds

The peaks are lower but the baseline

is higher by about 2 seconds

2012 average

(+3 days before/after Black Friday):

5.871 seconds

The peaks are higher by

about 3 seconds but the baseline is

lower

32323232% Increase in average baseline

Page load times

Black Friday 2013

Black Friday 2012


sec.Mobile

averageresponse time22222222

Remember, users are expecting the same

response times for mobile as for PC’s



range: 28 Nov 9AM - 30 Nov 2AM @ 1hr interval

sec.Last Mileaverage

response time15151515


cumulative

average range: 28 Nov 9AM - 30 Nov 2AM @ 1hr intervalSite homepage only

single site homepage avg. response time

single site homepage page weight


range: 28 Nov 9AM - 30 Nov 2AM @ 1 hr interval

Backbone vs Last Mile vs Mobile comparison:

Third party components: FACEBOOK (including Akamaized content)

sec.3rd Party peakresponse time24242424



Detail Friday November 29, starting 28 Nov 9AM through November 30 2AM

Backbone

Wait queue’s implemented

between 1 and 8AMFirst peak: midnight Nov 29

Last peak 9PM Nov 29


cumulative

average range: 28 November 9AM through 30 November 2AM @ 30min interval

CROWD Controlimplementation of wait lines queues

page weight changed during maximum utilization

displaying “wait in queue” messages

availability drops at the same time

response time

page weight

availability


one mobile formatted site web page (not native app) in São Paulo

Mobile average response time & availability


response time by carrier

Availability by carrier


Site behaviorScreen Captures

FRIDFRIDFRIDFRIDAAAAYYYYBLABLABLABLACKCKCKCK

Server errorsScreenshots of errors encountered by different sites during Black Friday, this is what the end

users would have seen. Providing custom error screens with better explanations can help.


Server errorsScreenshots of application exception errors encountered during Black Friday on different sites.

These errors can be further analyzed and prevented with an internal APM monitoring tool.


Network errors: Screenshots of what users would see on their browsers when

encountering network errors, like DNS and timeouts. These

were captured during Black Friday.


Instead of generic (or none) error messages like this:

Error Messages


or minimalistic like this:

Some sites provided visitors with informative custom messages:


Error Messages

RICARDOELETRO

Page was not displayed correctly, showing just text and images with

no style sheet.


sample error screen captured of site during Black Friday

com

ple

te w

eb

pa

ge

ima

ge

WALMART.com.br

Many product images loading at the same time.

When there’s low response time this will make the page

seem empty, users might hesitate taking next step.



com

ple

te w

eb

pa

ge

ima

ge

LEADER

After several seconds the page still shows “loading” message.

Is the page ready?



com

ple

te w

eb

pa

ge

ima

ge

FAST Shop

Same issue as seen in other pages, no style sheet, just raw images

and text. Makes site appear longer than it really is, more scrolling

needed by user to find items.



com

ple

te w

eb

pa

ge

ima

ge

EXTRA

Page missing style sheet, just showing images and text.

This forces the user to reload page, retry later.

Most will not scroll down this very long page.



com

ple

te w

eb

pa

ge

ima

ge

DI SANTINNI


Another example of missing style sheet, site just showing images

and text, forcing user to reload page. There’s a risk of the user

actually abandoning the page and going elsewhere.


com

ple

te w

eb

pa

ge

ima

ge

POLISHOP

Heavy content like videos takes much longer to load and

will give the appearance of the page not being ready.



com

ple

te w

eb

pa

ge

ima

ge

COLOMBO

Heavy flash content takes longer to load.

Take into account size and load time of this type of

Component, is it really worth the hit in performance?



com

ple

te w

eb

pa

ge

ima

ge

CASA E VIDEO


Some images missing at the bottom of the page,

in this specific example nothing significant lost.


com

ple

te w

eb

pa

ge

ima

ge

PONTOFRIO


Too many item images loading at the same time

if the server where these images reside is slow, it

will slow the entire page. This can make a user

abandon the page, check response times for

image components.


com

ple

te w

eb

pa

ge

ima

ge

AMAZON.com.br


Application accessing visitor history, potential

bottleneck that can slow down page load time.

Is it really needed if you’re not logged in yet?


com

ple

te w

eb

pa

ge

ima

ge

� Blank space, nothing displayed

takes too long to load �

PASSARELA




purchasetransaction& Last Mile


Site A Site B Site C Site D Site E

Number of steps 10 9 9 8 8

Typical steps are:

Homepage

Search

View product detail

Purchase / warranty info

Login

Confirm ship-to address

Payment method

- Back to shopping cart

- Empty shopping cart

- Exit

- ( the last three steps are necessary to replenish product stock )


Purchase transaction steps

Just measuring the performance of an e-commerce homepage will not give you a complete

picture of the user experience. That’s why a full purchase script was recorded for 5 top sites

that would give us more insight into specific steps, for example the login – where the site

has to check the user db, authenticate it via SSL/HTTPS – this touches different elements of

the infrastructure.

For all steps of purchase script from last mile locations in all of Brazil:

Average response times from last mile


Average response time for login step (different step # on scripts)

Average response time for step 1 on all sites, homepage


Average response time for step 2 on all sites, search for same product

Average response time for step 3 on all sites, show product detail

Average response time by Brazil state, homepage only

Last Mile measurements


Average throughput by Brazil state, homepage only

Last Mile measurements



NetworkComponents


As expected, First byte times are the major contributor to the increases in response times.

This is the time it takes the infrastructure of the site to send the first byte of content back to

the client, this includes Web, App and Database servers as well as network devices.

Proper load-testing can help identify weak areas within the infrastructure prior to a big

calendar event. Ideally, 1st byte response times should remain flat even with increased load.

Network component times


cumulative

SSL encryption connection times

were also stable.

Network connection times stable

throughout the day, ISP’s did a good

job by providing sufficient bandwidth

DNS times increased slightly after initial peak

Other network component times


cumulative

Page weightExtremely heavy pages, 4MB and 3MB!

Some sites had as many as 70 hosts just on their homepage, this means a lot of

third party content and tags that slow down page load times.

# of Hosts


metrics collected using Compuware synthetic monitoring

Backbone: Terremark / Verizon (SP), Level3 (SP), Alog (RJ)Last Mile, over 1K agents available throughout Brazil

Mobile 3G networks: Claro, Vivo, Oi, TIM

purchase transactions recorded using the Compuware Gomez Recordererror screens captured using SCoE (Screen Capture on Error)

Ricardo Aguilera, December 2, 2013

for more information contact: [email protected]


Technology

Black Friday Brazil 2013 web performance metrics