Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Welcome!
If you haven‘t already set up, please come and get a USB stick!
You will need:
1. An installation of Jubula 7.2
2. The example projects
3. To perform the pre-tutorial checks in the PDF
11.03.2014 EclipseCon 2014 1
Jubula TutorialAlex Schladebeck
11.03.2014 2EclipseCon 2014
Agenda
Quick introduction to Jubula
Exercises: • Hands-on test specification and execution
• Automated testing in the project lifecycle
• Code coverage
• Headless test execution
Show & Tell• Reporting
• ALM integration
Further aspects
11.03.2014 3EclipseCon 2014
Philosophy 101:
Why do we test?• To gain information with the intention of making a decision
• Information can be:
• Acceptance – “this works“
• Regression – “this doesn‘t work (anymore)“
• Further questions – “is this a problem?“
Why do we automate?• To gain specific information quickly, frequently, reliably
• To reduce gaps in knowledge
11.03.2014 EclipseCon 2014 4
Constant feedback
Automated testing• Info during development
• Fewer nasty surprises
• Safety net
• Lower error costs
UI testing• User perspective at all times
11.03.2014 EclipseCon 2014 5
What is Jubula?
11.03.2014 EclipseCon 2014 6
What is Jubula?
Automated functional (black box) testing• Swing, SWT/RCP/GEF, HTML, .NET, iOS, JavaFX
Tests through the GUI• No / minimal difference to manual test
• All layers tested
• Complete workflows
• Intelligent tests
• Information about acceptance and regression
Jubula == GUIdancer
11.03.2014 7
http://www.eclipse.org/jubula
EclipseCon 2014
How is Jubula different?
11.03.2014 EclipseCon 2014 8
Quick demo: writing tests
Specify via drag and drop
Add data
Do object mapping
Execute and analyse
11.03.2014 EclipseCon 2014 9
Let‘s go!
Start the ITE
Import the projects
Open the EclipseCon project
Connect to the embedded AUT Agent
Check for any strange red squares
11.03.2014 EclipseCon 2014 10
Set up your perspective
11.03.2014 EclipseCon 2014 11
The AUT (Application Under Test)
DVD Manager (extraordinaire)
In 3 different versions• “Project“ lifecycle
11.03.2014 EclipseCon 2014 12
Version 1
Execute an existing test
Write a simple test
Write a less simple test
Finish off a test and run it
11.03.2014 EclipseCon 2014 13
0: Run an existing test
Project is imported and opened
Connected to AUT Agent
Start the AUT for your system in Version 1• E.g. DVDTool (Win_V1): Win_V1@localhost
Wait for AUT to start
Start Test Suite FIRSTTEST• Yes to relevant
• Yes to change perspective
• Hands off the keyboard!
11.03.2014 EclipseCon 2014 14
1: Create a simple test
Consists of actions on application and menu• Which means no object mapping
Use Case:• Open Waiting Dialog from Edit menu
• Unbound modules > Actions (basic) > Select > Menu Bar
• Wait for dialog
• Unbound modules > Actions (basic) > Wait > Application
• Wait for dialog to close
• Unbound modules > Actions (basic) > Wait > Application
11.03.2014 EclipseCon 2014 15
2: Create slightly less simple test
This time with other components (object mapping)
Use Case:• Open Help/Info
• Wait for Info Dialog
• Modules > Generic > Dialogs
• Check version number
• ubm > Actions (basic) > Check > Component with Text
• Click OK
• ubm > Actions (basic) > Click
11.03.2014 EclipseCon 2014 16
Sounds like weneed a module!
3: Finish off test for name uniqueness
Category names should be unique
Open Test Case: Check uniqueness of category names
Look at what we‘ve got already
Add final test actions:• Check enablement of OK button
• Ubm > Actions (basic) > Check
• Close dialog with cancel
• Modules > Generic > Dialogs
Do mapping and run FULLTEST
11.03.2014 EclipseCon 2014 17
Version 2
Check that previous bug is gone
Make any necessary changes
Write a new test
11.03.2014 EclipseCon 2014 18
4:Rerun uniqueness test on new version
Stop Version 1 AUT
Start Version 2 AUT
Rerun FULLTEST (to check that the bug is fixed)
Do any changes need to be made?
11.03.2014 EclipseCon 2014 19
5: Comment in deletion test and run
In the FULLTEST Test Suite• comment in the Delete Single Category Test Case
Run the test
Is everything ok?
11.03.2014 EclipseCon 2014 20
What do we do about errors?
Can we fix or deal with this error?
Yes:• RETRY Event Handler (later)
No / Don‘t Know• RETURN Event Handler
• Use restart / reset AUT to put AUT into correct state
11.03.2014 EclipseCon 2014 21
About event handling
Event Types• Action error
• Component not found
• Check failed
• Configuration error
Reentry types (test continuation)• Continue
• Exit
• Return
• Retry
• Pause
• Break
11.03.2014 EclipseCon 2014 22
Why do we need Event Handlers?
What if an error occurs?
11.03.2014 EclipseCon 2014 23
Why do we need Event Handlers?
11.03.2014 EclipseCon 2014 24
Using Event Handlers
11.03.2014 EclipseCon 2014 25
6: Adding a global Event Handler
Create normal Test Case : Restart• Add a restart action
• ubm > Actions (basic) > Restart Application > Restart
Add this Test Case as an Event Handler• To the Delete Category Test Case
• Error Type Action Error
• Reentry Type Return
Run the test
11.03.2014 EclipseCon 2014 26
..if an action error occurs in this Test Case, then restart the AUT, and continue the test at the nextTest Case in the hierarchy –it will be ready for you!
Version 3
Check that previous bug is gone
Check that there are no side effects
11.03.2014 EclipseCon 2014 27
7: Run test on new version
Stop version 2
Start version 3• (Mac Java 7 users: switch context menu action for menu action)
Re-run FULLTEST on latest version
11.03.2014 EclipseCon 2014 28
Return to Event Handling
What if the category to delete contains DVDs?
Check for non-existence of confirmation dialog• If it is there, close it and check again
11.03.2014 EclipseCon 2014 29
Retry EH
8: Code Coverage
Stop the AUT
Open the Project Properties• Select AUTs from the left
• Select DVDTool and select „Edit“
• Find your system_v3_CC configuration and select edit
• Open the Expert properties
• In the Jacoco Configuration
• Browse to the examples/AUTs/DVDTool directory on your system for:• The installation directory
• Click all the OKs
Start the AUT configuration for code coverage
11.03.2014 EclipseCon 2014 30
8: Code Coverage
Run the FULLTEST
Open the Reporting Perspective
See the code coverage values
Open the report
To discuss over a beer:• What should we use code coverage for?
• http://testing.bredex.de/code-coverage.html
http://www.eclemma.org/jacoco/
11.03.2014 EclipseCon 2014 31
What do we do with test results?
11.03.2014 EclipseCon 2014 32
Gathering and analysing test results
Daily test result analysis• Check e.g. Hudson jobs
• Ensure all tests ran (collectResults)
• Gather overview of status (Test Execution Overview)
• Analyse any tests that failed in the ITE / Dashboard / HTML Reports
• Add comments to Test Results Summary View
• Create issues in external systems from ITE
• Automatically report test results to external systems (v8)
• Print out comments report for e.g. standup discuss and react!
Long-term test trend overview• E.g. History report with code coverage
11.03.2014 EclipseCon 2014 33
http://www.eclipse.org/birt/phoenix/
Continuous Integration
11.03.2014 EclipseCon 2014 34
Develop
Commit
Check out
Build anddeploy
Setup environment
Run tests
Gather andanalyse results
Discussion andfeedback
testexec
ITE ResultHTML ResultDashboard ResultAutomated comment to ALM
(AUT Agent)
9: Headless execution (now/later)Locate the config.xml file
Make changes:• Change the autconfig parameter if you‘re not using Windows• Mac users – change ~ to actual path• Enter a writeable path for the datadir and resultdir parameters
Mac users – use an absolute path in the AUT configuration
Stop the AUT
Disconnect from the AUT Agent*
Minimize all windows
Via the command line, navigate to:• <jubulaInstallation>/jubula
Enter testexec(.exe) -c <pathToConfigFile>/config.xml
11.03.2014 EclipseCon 2014 35
* we‘re cheating by using the running embedded Agent. For a real CI test, use a separately started agent or, in V8, theembedded agent support in testexec
Closing some gaps with Mylyn
View, edit, create tasks in the ITE
Automatically report test results to external ALM systems
11.03.2014 EclipseCon 2014 36
www.eclipse.org/mylyn
Summary
Complete workflows tested based on acceptance criteria• Encourages early and constant discussion about quality
• Even before the application is available
• Automated regression testing
Manual intelligence incorporated• Test almost indistinguishable from user working with system
• Synchronization
• Reacting to clues in AUT
• Error handling
Reuse instead of copy• Tests made up of similar units – reuse to improve maintainability
11.03.2014 EclipseCon 2014 37
Current status
Version 8.0 will be released before Easter• JavaFX support
• Comment to ALM system after headless run
• Support for HP ALM connection
• Support for embedded Agent in testexec
JavaFX support will also be released with Luna
11.03.2014 EclipseCon 2014 38
Don‘t forget…
Please evaluate the talk!
For more testing fun, join me on Wednesday:• Grand Peninsula C - 11:15 to 11:50
Exploratory testing and session-based test management
11.03.2014 EclipseCon 2014 39
Homework: Continue writing tests
Startup
Import DVD library
Select category (Science Fiction)
Select DVD from table
Select technics tab
Select region from combo box
Select multiple languages
Select content tab
Enter content information
11.03.2014 EclipseCon 2014 40
Thanks!
Community help• Forum
• Mailing lists
• Webinar
• Examples & Articles
Professional services• Training
• Proof of Concept
• Reviews, Consulting
• Custom extensions
• Sponsored development
@alex_schl
http://testing.bredex.de
www.bredexsw.com
www.eclipse.org/jubula
@EclipseJubula
11.03.2014 EclipseCon 2014 41
Sneak preview JavaFX
Simple Adder
Brick Breaker
Planned components for first release• Buttons, Radio Boxes, Check Boxes, Toggle Buttons
• Labels, Text
• Text input controls
• Combo boxes
• Content controls: Lists, Trees, Tables
• Accordeons
11.03.2014 EclipseCon 2014 42