Upload
dkulinich
View
223
Download
0
Embed Size (px)
Citation preview
8/8/2019 DMT Automation UI-Testing Practice7
1/25
DMT UI-testing automationDMT UI-testing automationpracticespractices
8/8/2019 DMT Automation UI-Testing Practice7
2/25
DMT problemsDMT problems
Neither manual nor automatedtest cases cover 100% of code
No instrument to measure testcases coverage
8/8/2019 DMT Automation UI-Testing Practice7
3/25
Deeper view on problemDeeper view on problem
System tests are not flexible Testing whole functionality build to
built is inaccessible and
unnecessary Time spent for manual testing of regression tests could be aimedonly on testing new functionality
There is no information what exactlycould be affected by particular codechange
There is no guaranties we test all
required peaces of code
8/8/2019 DMT Automation UI-Testing Practice7
4/25
System TestsSystem Tests
Manual test cases that aimed to coverwhole functionality and itsdependencies
Problem
Coverage: is unknown and scattered overwhole system Maintainability: very low due to linkage
with several requirements Execution: hard to execute because tests
executed in one way and pauses couldcause lost of this test way
Dependency: every step is dependentfrom data used in case before or fromdump data
Automation: not automatable (support of those cases will be unacceptable
8/8/2019 DMT Automation UI-Testing Practice7
5/25
System Test CaseSystem Test Case
8/8/2019 DMT Automation UI-Testing Practice7
6/25
Functional TestsFunctional Tests
Atomic test cases that aimed to verifyparticular functionality
Benefits
Coverage: measurable! Maintainability: extremely high maintainability!
Case covers atomic peace of requirement Execution: easy to execute! Every case could be
executed separately even in different days by
different test engineers Dependency: test cases are independent! Every
case prepares preconditions (application, data) forits run
Automation: extremely easy to automate! Just walk
through cases and convert it to automationframework API language
8/8/2019 DMT Automation UI-Testing Practice7
7/25
Document for automationDocument for automation
8/8/2019 DMT Automation UI-Testing Practice7
8/25
TraceabilityTraceability
Traceability matrix allows linking Issue for new functionality with
specification document Specification document with
functional test case Functional test case with automated
test script Black box coverage assuming bymanual / automated tests
8/8/2019 DMT Automation UI-Testing Practice7
9/25
Traceability matrixTraceability matrix
8/8/2019 DMT Automation UI-Testing Practice7
10/25
ChangeabilityChangeability
New improved test cases designallows be flexible withcontinuously improvedrequirements:
Easily find exact place in testcase/test script that should beupdated
Create new test case/test scriptfor new functionality withoutaffecting existing test cases
Measure not only code coveragebut also requirements coverage
8/8/2019 DMT Automation UI-Testing Practice7
11/25
MaintainabilityMaintainability
8/8/2019 DMT Automation UI-Testing Practice7
12/25
Summary 1Summary 1
Atomic test cases Traceability matrix binds all
artifacts Easy to support test cases Easy to handle test cases execution
results Test cases are ready to be
automated
8/8/2019 DMT Automation UI-Testing Practice7
13/25
How to automate?How to automate?
DMT automated testingframework
WebUI interaction API Page object pattern
Pre-requisites creation API
u to mat ed Te st Ca se
-Pre requisitescreation
Test Case body
Verifications
Activities
DBUnit
WebDriver
TestNG
use
use
use
being ran by
DMT entities creation API
DMT specific,orms widgets
wrappers
8/8/2019 DMT Automation UI-Testing Practice7
14/25
Why DBUnit?Why DBUnit?
Test cases pre-requisitescreation is significant part of either manual or automatedapproach (time consuming)
Solutions XML based DB fixtures
Data that should be already in databaseto pass test case
XML based GUI fixtures Data that should be entered into form to
pass test case
8/8/2019 DMT Automation UI-Testing Practice7
15/25
Why Selenium/WebDriver?Why Selenium/WebDriver?
Open source Java support
Developers can help to automate testcases
Developers can write functional testson defects
Cross platform/browser
8/8/2019 DMT Automation UI-Testing Practice7
16/25
Build process integrationBuild process integration
You need only maven + browserto run test cases
WebDriver supports: IExplorer Chrome Firefox
8/8/2019 DMT Automation UI-Testing Practice7
17/25
PerformancePerformance
Individual test case takes about90 sec.
About 480 test cases per night
8/8/2019 DMT Automation UI-Testing Practice7
18/25
Summary 2Summary 2
Atomic test cases are easy toautomate and maintain
Pre-requisites creation takesmilliseconds
Test cases can be run againstany browser without changes
8/8/2019 DMT Automation UI-Testing Practice7
19/25
How to measure?How to measure?
Problems mentioned in the verybeginning (just to remind): No guaranties that code in production
is covered by test cases No instruments to measure functional
test case coverage over sourcecode
8/8/2019 DMT Automation UI-Testing Practice7
20/25
Rich UI based apps test coverageRich UI based apps test coveragetooltool
Automated test cases (using WebUIinteraction API)
or QA m anually interact w ith application withinCoverage tool through browser
Coverage tool tracks the source code lines affect edby those activities
overagetool
Browser
- -
Auto test cases
Manual TCexecution App DB
8/8/2019 DMT Automation UI-Testing Practice7
21/25
Coverage tool collecting coverageCoverage tool collecting coverage
Step Action
1.1:1.2 User/WebDriver activities
2.1:2.2 Communication with server
3.1 Cobertura tracks the code execution
4. 1:4.2 Dev Plugin controls the code execution onCode Server
5.1 Emma tracks the code execution
6.1 The stop-tracking command request7.1:7.2 Get client side coverage
8.1:8.2 Get Emma runtime coverage
9.1:9.2 Get Cobertura runtime coverage
10.1:10.2 Persist combined coverage for an activity toDB
8/8/2019 DMT Automation UI-Testing Practice7
22/25
Code Coverage by Test CcasesCode Coverage by Test Ccases
Test Cases
toExecute
Not tested code
8/8/2019 DMT Automation UI-Testing Practice7
23/25
Summary 3Summary 3
We can measure the codecovered by test cases being raneither manually or byautomated tool
We can determine the code notcovered by test cases
We can reduce the testing timeby executing the only testcases against affected source
code
8/8/2019 DMT Automation UI-Testing Practice7
24/25
8/8/2019 DMT Automation UI-Testing Practice7
25/25
Issues & problemsIssues & problems
How to build pre-requisites(database?)
Test execution performanceoptimization (initialloging/loading)
Traceability matrix has to beintegrated with RTH or otherdocument tracking system