Upload
ranorex
View
1.390
Download
2
Embed Size (px)
Citation preview
IntegratingRanorex and NeoLoad
Time to make peace between functional and non-functional testing
Welcome to the Ranorex-Neotys Webinar
Christian Breitwieser, RanorexInnovation Manager
Henrik Rexed, NeotysTechnical Evangelist
Agenda• Quick Company and Tool Intro• Performance Matters in User Experience• Benefits of Automated Functional Testing• Benefits of Combining Automated Functional and Load
Testing• Live Demonstration
• Functional test automation for desktop, web and mobile applications
• Cross-technology end-to-end testing• Tool for testers and developers
• Functional test automation for desktop, web and mobile apps
• Cross-technology end-to-end testing
Tool for testers and developers
Code Module
Action Based Modules
Ranorex Studio
• Deliver faster applications faster through continuous performance validation
- Who We Are
- Who We AreMission: Deliver faster applications faster through continuous performance validation
- Who We AreMission: Deliver faster applications faster through continuous performance validation
User Demands Process Revolutions
We’re Driven by Your Challenges
Technology Transitions
User Demands Process Revolutions
We’re Driven by Your Challenges Technology Transitions
User Demands Process Revolutions
We’re Driven by Your Challenges Technology Transitions
User Demands Process Revolutions
We’re Driven by Your Challenges Technology Transitions
Docker
Load and Performance Testing
24x7 Performance Monitoring
Continuous Performance Validation Solution
Load and Performance Testing
24x7 Performance Monitoring
Continuous Performance Validation Solution
Continuous Performance Validation Solution
DevelopmentContinuous
Integration & Component Testing
Continuous Performance Validation Solution
DevelopmentContinuous
Integration & Component Testing
Continuous Performance Validation Solution
Pre-productionSystem-wide
Testing
DevelopmentContinuous
Integration & Component Testing
Continuous Performance Validation Solution
Pre-productionSystem-wide
Testing
DevelopmentContinuous
Integration & Component Testing
Continuous Performance Validation Solution
Pre-productionSystem-wide
Testing
DevelopmentContinuous
Integration & Component Testing
Continuous Performance Validation Solution
Pre-productionSystem-wide
Testing
Production
Proactive Monitoring
DevelopmentContinuous
Integration & Component Testing
Continuous Performance Validation Solution
Pre-productionSystem-wide
Testing
Production
Proactive Monitoring
DevelopmentContinuous
Integration & Component Testing
Continuous Performance Validation Solution
Pre-productionSystem-wide
Testing
Production
Proactive Monitoring
DevelopmentContinuous
Integration & Component Testing
Agenda• Quick Company and Tool Intro• Performance Matters in User Experience• Benefits of Automated Functional Testing• Benefits of Combining Automated Functional and Load
Testing• Live Demonstration
User Experience is Crucial• 57% of users use their mobile devices to access
applications
• 40% of users move towards a competitor after a bad user experience
• 44-61% of users share their feelings on social networks
What isUser Experience? Usability
Ergonomics
Performance
Accessibility
Functionality
User Experience Impacts the Business
Case StudyImpact of the response time on the conversion rate:
Agenda• Quick Company and Tool Intro• Performance Matters in User Experience• Benefits of Automated Functional Testing• Benefits of Combining Automated Functional and Load
Testing• Live Demonstration
Why Functionalityis important Usability
Ergonomics
Performance
Accessibility
Functionality
Otherwise…
Test AutomationPyramid
Unit
Integration
Functional
Manual
Manual & Integration TestingManual testing• Time-consuming• Error-prone• Cannot „run“ overnight
Integration testing• No black box• Hard to cross technology
borders• Good coding skill required
Benefits of Automated Functional Testing• Quick feedback• Parallel test execution• Scalable
• Codeless functional testing• Increased test coverage
Numbers & FactsTomTom – Case Study:
• Testing time: from 10 days down to 4h• Over 90% of time saved with Ranorex automated tests
Siemens – Case Study:• Due to the use of C#, new test cases can be implemented
more effectively• Only 60% of the time needed with Ranorex compared to
previously used tool
Agenda• Quick Company and Tool Intro• Performance Matters in User Experience• Benefits of Automated Functional Testing• Benefits of Combining Automated Functional and Load Testing
• Live Demonstration
Separated Functional & Performance Testing
• Functional testing ensures functionality… but not when the AUT is under load
• Load/Performance testing ensures a fluent user experience… but does not guarantee a functional UI
Ranorex Customer Experience Survey No Idea
Yes No54% 37%
9%
Do you combine Ranorex test automation with non-functional tests?
In addition to functional test automation you perform…
Other
Load Testing
Database Testing
Web-Service-Testing
Performance Testing
Unit Testing
4%
19%
22%
27%
31%
33%
Functional – Performance Tests
Golden Rule: Automated performance testing should not take place if functional regression tests failed.
Performance tests require that the application or part of the system is:
• Operational: No systematic errors or exceptions
• Representative : Enough data available in the test environment to determine the behavior under load conditions
How Can We Increase Risk Coverage?
100 test cases
40 test casescovering 80% of the risknot necessarily selected for performance testing
We prioritize test cases based on risk. These test cases are validated through :- Automated regression testing- Exploratory testing
Validate Critical Business Case Under Load
This guarantees:• The stability of the crucial use cases under real usage
conditions• High user experience• The identification of potential bottle-necks, even across
technology borders
The Initial Workflow…Automated functional
testing
Exploratory testing
Load testing
Exploratory testing
under load
…
Applying the Ranorex-NeoLoad Integration…
00:00:00 00:00:30 00:01:00 00:01:30
Load
NeoLoad
Application Server
App Error Rate
Virtual Users Load
Automated functional
tests
UserExperience
… You‘ll Find Bugs Earlier!Automated functional
testing
Automated functional
testing under load
Exploratory testing
Exploratory testing under
load
…
Agenda• Quick Company and Tool Intro• Performance Matters in User Experience• Benefits of Automated Functional Testing• Benefits of Combining Automated Functional and Load
Testing• Live Demonstration
Further Resources
Combining Automated Functional and Load Testinghttp://www.ranorex.com/blog/combining-functional-and-load-testing
How to Combine Ranorex and NeoLoad Testshttp://www.ranorex.com/blog/combine-ranorex-and-neoload-tests
Questions?
Henrik Rexed: [email protected]
Download NeoLoad Free Edition
Visit our neotys.com
Follow me on Twitter @hrexed
Christian Breitwieser: [email protected]
Download free Ranorex trial
Visit www.ranorex.com
Follow us on Twitter @Ranorex