Embed Size (px)
DESCRIPTION
Keynote has a private monitoring agent called "Cloud Application Perspective" (CApP). This agent was used, plus some public Keynote agents, to monitor the response of a test website during an actual Keynote load testing event during 2012. I presented these results in this Performance Analysis, typically to potential customers who wanted to better understand what monitoring could tell them about their website performance--in this case, under load. This critical Proof of Concept technically sold the service.
Citation preview
Gaining Insight into Web Performance and Availability
Keynote Demonstration & Example Overview and Data Review for
Glenn Huang, Solutions Consultant
What Keynote Measured
3 Steps using a public-facing Keynote Demonstration server called TSPDemo04 from 2/15/12 to 2/16/12
1. TSPDEMO04 Home Page http://tspdemo04.keynote.com
2. Click Link for About Page
3. Submit Comment on About Page
During that time, there were two Load Tests run against this same TSPDEMO04 server.
1. 2/15/12 @6:15 PM 15 minute ramp
2. 2/15/12 @8:40 PM 15 minute ramp
Using an emulated browser (ApP) 5 US Locations every 30 minutes (average one measurement every 6 minutes)
1. Dallas SBC ApP
2. Chicago SBC ApP
3. San Francisco Sprint ApP
4. Denver CenturyLink ApP
5. New York Cogent ApP
Using an emulated browser private agent (CApP) San Mateo HQ Location every 5 minutes
1. (Keynote) CApP Glenn_Huang_CApP
November 9, 2013
2
Path Screenshots
November 9, 2013
3
1 2
3
What was the Performance and Availability measured?
• Track ongoing response time patterns and availability %
• Results of all measurements of all steps averaged From 2/15@5 PM to 2/16@2PM PST
• We see the load test had a direct impact on availability for those periods.
• We note too that there was additional response time between 4AM to 7:30 AM; possibly another load test without comments.
• Database server was down 2/16@12:20-12:47 PM
November 9, 2013
4
Where did it Happen?
November 9, 2013
5
Average performance by Agent over the time period
Where did it Happen?
• Another more detailed view of each agent as they experienced these load tests
• There were other page errors during this time due to the database server being down.
• Let’s see what steps were affected during this time
November 9, 2013
6
Average performance by Agent over time
On Which Steps?
• For External Measurements only
• Note that the Home Page was very sensitive to database connectivity and load tests
• How did it look from a local internal view?
November 9, 2013
7
Average performance by Step over time
On Which Steps?
• For Internal CApP Measurements only
• Note that the higher 5 minute granularity allows us to see when the Home Page was down due to the 1st load test.
• We also see that the About page was also down during the 2nd load test and the Homepage response time jumped.
November 9, 2013
8
Average performance by Step over time
Scatter Plot: Page and Content Errors
This scatterplot shows all points taken from both Internal and External agents 2/15/12 from 5-11:59 PM PST.
Again, Keynote picked up all server errors during the load test periods.
November 9, 2013
9
Drill Down: Page Summaries
• Comparing measurements
• Top is before Load Test
• Bottom is during the Load Test
• Note the time scales are not identical between top and bottom graphs
• We see Increased 1st Byte Times under load
• Redirect on Step 3 also increased from .23s to 1.2s, a 5X increase
November 9, 2013
10
Before Load
During Load
Drill Down: Page Summary
• This data point was not during one of the two load tests but was one of a few outliers noted for the CApP agent.
• In every data point on the Comments step where there was an increase in time, the Redirection component was the reason, adding about 5-6 seconds.
November 9, 2013
11
Slow CApP result
Page-Detail Level Performance Before Load Test
November 9, 2013
12
This is the Homepage from Chicago SBC ApP at 2012-02-15 17:54:54 PST
Note First Byte of Base Page.
Page-Detail Level Performance During First Load Test
November 9, 2013
13
This is the Homepage from Chicago SBC ApP at 2012-02-15 18:24:54 PST
Note 5X increase in First Byte time of Base Page.
Summary
Keynote measures end-user web performance from either inside or outside the firewall via Private or Public agents, respectively
Detailed measurement data facilitates:
• Tracking of response times and availability over time
• Troubleshooting of performance issues before they become critical
• Enforcement of SLAs for your own content, 3rd party or CDN-delivered content
• Virtual Pages can be provided to segregate this information
November 9, 2013
14
Keynote Web Monitoring Solutions
Application Perspective (ApP)
• Used for general Web performance and availability monitoring
• Supports any Web browser (IE, Firefox, Safari, Opera, Chrome, etc)
• Employs an emulated browser to playback measurement scripts
Cloud Application Perspective (CApP)
• Just like ApP but a private agent installable behind firewalls, load balancers, in virtualized environments.
• Can be placed wherever a public agent is not available.
November 9, 2013
15
