Upload
ntatsumi
View
541
Download
2
Tags:
Embed Size (px)
Citation preview
Automation:xpLor Performance Test
2
Performance Test AutomationObjectives
• Enable developers to run performance tests against any specified xpLor instances to proactively create performance aware culture
• Provide comprehensive reports, analysis, and visualization for xpLor performance test results that are persisted forever
• Create continuous feedback of performance and script breakage against the latest xpLor builds
3
Performance Test AutomationAutomation Types
• Continuous: Runs nightly at 6pm EST against the latest successful build in Bamboo
• Ad hoc: Triggered by developers against a desired xpLor instance
4
ArchitectureGalileo + Fusion + Ansible + JMeter
5
xpLor
6
Ansiblehttp://www.ansibleworks.com/
• GPL licensed software for configuration management, application deployment, and continuous delivery
• Written in Python• Manages servers over SSH – doesn’t require any agents• YAML for reusable system descriptions
https://github.com/blackboard/perf-test-tools/tree/develop/ansible-playbooks/xplor
7
JMeterData + Load Generation
Server poolr6x64o11-pv087, r6x64o11-pv088, r6x64o11-pv089, r6x64o11-pv090, r6x64o11-pv091, r6x64o11-pv092
Data modelhttps://github.com/blackboard/perf-test-tools/tree/develop/jmeter/xplor/config/datamodels
Test scenariohttps://github.com/blackboard/perf-test-tools/tree/develop/jmeter/xplor/config/scenarios
8
Test ResultsReports and Notifications
• Automated analysis + notifications• *Overview statistics• *Cumulative response time rate of change• *Vusers vs. response time – top 5,000 slowest transactions• Transactions details statistics• Continuous trend
* These reports have the ability to view results in categories of all, realtime, or long-running transaction types
9
Test ResultsAutomated Analysis + Email Notifications
• High transaction response time: 90th percentile = 3 seconds
• Low script quality: HTTP 500 = 50 requests; HTTP 400 = 75 requests; failure rate > 1%
• Top 5 Transaction Response Time Degradation with 20% threshold
• 100% failed transaction(s) found• Uneven app load distribution• Crashed JMeter generator
10
Test ResultsAutomated Analysis + Email Notifications
11
Test ResultsOverview Statistics
12
Test ResultsCumulative response time rate of change
13
Test ResultsVusers vs. Response Time – Top 5,000 Slowest Transactions
14
Test ResultsTransaction Details Statistics
15
Test ResultsNavigating to Your Test Results
• From process ID (PID) link in email notification• Galileo “Browse” tab
16
Test ResultsCustomizing Your Test Results Dashboard
17
Test ResultsContinuous Trend
18
Fusion – Task SchedulerTemplate - http://tinyurl.com/perftest-xplor
19
Fusion – Task SchedulerHow to run test against your server
1. Go to Fusion tab on Galileo (http://galileo.pd.local)2. Create “New Fusion Process”
1. Project: jmeter2. Process: My Test
3. Click on the star icon to create a Fusion task from a template
4. Choose the group icon to select “xpLor – Ad hoc Test” template
5. Modify PID variables accordingly (app.server.hosts, app.server.ports, file.server.hosts, file.server.ports, loadtest.duration)
20
Fusion – Task SchedulerHow to run test against your server
xpLor Configuration Requirements:• Set isTesting to true• Set exports.development to false;• Set exports.timestampExpirySeconds to 99999
(We don’t run ping requests in JMeter to keep sessions live)
21
Zabbix MonitoringTemplates
22
Next StepsWhat’s coming?
• Instructional videos for running ad hoc testing• Institute performance aware culture• Shared ownership of JMeter scripts• JMeter training