Embed Size (px)
Citation preview
Continuous Testing in the Agile AgeTesting as part of the Continuous Delivery process
785 Castro Street, Mountain View, CA 94041 | 1.855.445.2285 | [email protected]
BlazeMeter
ITAY MENDEL Devop
MICHAEL REDMANCustomer Success
ANDREY POKHILKOChief Scientist
ALON GIRMONSKYFounder & CEO
R
Waterfall Agile
2015 CURRENT STATE OF AFFAIRSMoving from Waterfall to Agile
“Continuous Delivery (CD) is rapidly emerging as the ‘new normal’ in software development, with approximately 80% of SaaS companies and 51% of non-SaaS companies adopting this practice”
Perforce Software Study
QA wasn’t invited to the party
THE TESTING BOTTLENECK
TESTING IS COMPLICATED!Why?
Developers• Unit
• Troubleshooting
CI/DevOps QA Operations• API
• Functionality
• Performance
• Regression
• Load
• Soak
• Regression
• Acceptance
• Load
• Monitoring
• Discrete VS continuous
• Frequent code commits
• Domain expertise required
• Multiple test points
ON THE VERGE OF CONTINUOUS DELIVERYTesting is a Challenge
TIM
E TO
REL
EASE
RELEASE NUMBER / TIME
20 40 60 80 100 120 140 160 180 200 220 240
2wks
1wks
3wks
4wks
5wks
THE OUTCOMEAny Agile Process is Most Likely to Fail
Test Early
Test Often
Test Everywhere
Automate it
CONTINUOUS TESTING PROCESS
Test Early
Test Often
Test Everywhere
Automate it(and get everyone on the same toolset)
CONTINUOUS TESTING PROCESS
CONTINUOUS TESTING
REDUCED TIME-TO-TEST
INCREASED # OF ITERATIONS
AUTOMATION
TIME-TO-RELEASE QUALITY
TIM
E TO
REL
EASE
RELEASE NUMBER / TIME
20 40 60 80 100 120 140 160 180 200 220 240
2wks
1wks
3wks
4wks
5wks
Before
After
CONTINUOUS TESTING FACILITATES AGILE
Developer
Devops
QA
OPs
OPs
Commit Code +
Test Representation
Configure
Test
Acceptance Test
Configure
Version Control Repository
Comprehensive Testing
Post-Production Monitoring
Continuous Deployment
Continuous Integration
PEOPLE
Red Arrow = Test Failed Green Arrow = Test Succeeded Blue Arrow = Sent to Testing
SYSTEMS TESTING ALERT
CONTINUOUS TESTING FACILITATES AGILE
Leve
rage
Pre
vio
us
Test
Rep
rese
nta
tio
n
Co
nti
nu
ou
s T
es
tin
g
IS POST-PROD MONITORING A PART OF TESTING?
Developers CI/DevOps QA Operations• GitHub
• JSON tests
• Jenkins
• JSON tests
• + JMeter tests
• + New Relic
• JSON tests
• JMeter tests
• New Relic
• Load & Soak
• Deploy
• Test
• Monitor
CD STACKWhat Toolset Will We Use?
Demo?We suggest you watch this 7 minute movie to see BlazeMeter in action: https://www.youtube.com/watch?v=6Oih-v-LICE
Please view in HD quality.
• Testing starts @ Dev
• The module’s test configuration is maintained by the developer in a JSON format
• Version control friendly
• Devop configures CI
• Tightly integrated with CI environments
Alert on Failure + Artifacts✖
ENTERPRISE CONTINUOUS TESTINGUse Case #1
PERFORMANCE LAB IN THE CLOUDUp to 1M users, multi-GEO, 100% Apache JMeter compatible
ON PREMISE + CLOUD TRAFFIC GENERATIONFull hybrid model
• Allocate as many on-premise servers as needed
• Group them into clusters (e.g. R&D, QA, OPs)
• Assign permissions
• Have each cluster appear as yet another GEO location
ANY SCRIPT, ANY SCENARIOComprehensive scripting at your fingertips
JMeter is the most commonly used open source web, mobile and API backend testing solution. BlazeMeter is 100% compatible with JMeter
HTTP / HTTPS
Login / Signup
Form POSTing / Credit card transactions
Rest / Web services / JSON
Parameters extraction ( JSON / RegEx / Xpath )
Mobile
Mongo, MSSQL, Oracle SQL, MySQL
Multiple user profiles
PRODUCT: JUST A TASTEO
n-P
rem
ise
A JSON Test Representation VS GUI
On
-Pre
mis
e
PRODUCT: JUST A TASTEA JSON Test Representation VS GUI
THRESHOLDS Setting global and per URL thresholds (Automation)
THRESHOLDS Setting global and per URL thresholds (UI)
CONTINUOUS INTEGRATIONConnect with leading CI applications
JENKINSLeverage inherent reporting
NEWRELIC INTEGRATION
Capture Monitoring DataCapture Load Testing Data
Ask the Test Data any Question
AWS CLOUDWATCHFetch CW related data associated with selected AWS resources
QUICKLY RECORD ANY SCENARIO
• Easily Record:
Web traffic
Mobile web or apps
HTTP/HTTPS
APIs
• No JMeter knowledge or installation required
REST API AND DSL SCRIPTINGBuilds into any continuous delivery process
THANK YOU FOR YOUR TIME!
Q & A
For more information:
BlazeMeter.com/blog/ [email protected] @BlazeMeter
