View
10.225
Download
1
Category
Preview:
DESCRIPTION
An overview of how to use QTP and Quality Center for daily automated regression test Type: BTO Applications, using a case study of ATP
Citation preview
©2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice
Session ID: BTOT-TU-1800/1Twitter hashtag #HPSWU
©2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice
Speaker Name: Test Manager Maja Usbeck Cao and Test Specialist Claus Haslund AndersenDate: 30. november 2010Session ID: BTOT-TU-1800/1
How to use QTP and Quality Center for daily automated regression test Type: BTO Applications
Agenda ATP – the company and the test organisation
System landscape and regression test
Requirements for our testing tools and automated test
setup
The setup for automated testing
Communication with relevant stakeholders
Daily mail with graphs
Daily follow-up on the nightly runs
Progress on developing automated tests
How did we get there? Prerequisites for our autotest
setup
Brief presentation of ATP: Company tasks
Bankruptcy(pay compensation)
Maternity(salaries refunding)
Occupational disease(Workers’ compensation insurance)
HolidayHoliday pay)
Pension(Pension services)
Vocational training (internship grants)
Pension DanmarkJØP (Unit Link)
DA-Barsel
AERAESFerieKontoLGBarsel.dk
ATPSPSUPPLD
Pension og SecuritySecurityPension
Regulated tasks
Customers contact with the ATP Group
4.4 million Danes come in contact with the ATP Group's products. Total productsensures our customers a basic economic security during and after their working
lives.
Test OrganizationThe test resources
• 3 Head of Test
• 6 Test Managers (RTM, TI and TM)
• 9 Test coordinators
• 3 Test specialists (automated test and test data)
• Up to 4 IT-Testers and students workers (ITU and DTU)
• 23 Test consultants
Responsibility area
• Test in projects and Sustained Engineering
• Test methods, TDM and test tools (QC, QTP and Loadrunner)
Test Organization - automatisationCentralized automation team
• Building principles and guidelines• Coordination across the organization• Upgrade & support of the tool• Building regression test • Maintenance of automated test• Responsibility for the daily runs
Sometimes QTP developers are lent to projects for a period of time
ATP Business Platform - Logical overview
Execution of:• GUI transactions• Service requests • Batch runs
WebsphereWebsphere
Mainframe DB2
Portal
Kerne/.NET/DB2
Broker / Websphere MQ
SAP Doc./Papyrus/WAS Masterdata/WAS WFM/WAS
Client
Client
HTTP
Client
Portal 360o
SAP clientGuitalis
Regression test purposesRegression testing aims to test whether any errors occurs in already implemented functionality, when implementing new or changed functionality.
The extent of regression test and risk of missing regression test increases in proportion to the implementation of new functionality (see illustration).
Missing/incomplete regression test represent a quality risk
Release 1 2 3 54 6 7
Test of new functionality
Regression test of previously delivered functionality
Requirements for our Testing Tools• We want to use our current tools Quality Center and QTP
• We have to be able to run our test set on a scheduled basis: Daily, weekly and
individual dates
• Automatic re-running tests that fails the first time (based on percentage of failed
tests in a test set)
• Distribution of loads on x number of machines (so we continuously can expand our
setup)
• Grouping of servers, so test on different releases is possible (Sustained Engineering
and release)
• Troubleshooting - current screenshots of the machines and embedded error
detection in scripts
• We need to know the expected execution time for a test set (Crucial when time is
short eg. emergencies)
Don’t think: What you can get – Think : What you want!
Automation focus areas
Testdata Creation of data Deleting data
Environments Component Integration Pilot
ServicesAliveFunctional
Batch runs (via gui)Start of batch runCheck result
System testingDeep functional
Test of Service requests - solutionThere are 6 main systems providing services.
Application for communication (Broker).
The portal using these services.
WebsphereWebsphere
Portal
Kerne/.NET/DB2 SAP Doc./Papyrus/WAS Masterdata/WAS WFM/WAS
Client
HTTP
Portal 360o
Broker / Websphere MQ
One automated BPT component in Quality Center.
QTP
QCATSA
Automation Test environmentAutomated Test Scheduler Application (ATSA)
What is the ATSA and how does it work?
Daily status on the test executionDaily mail with information on the execution of automated tests
Daily monitoring of test executionThe main reason that there are 26 failed tests, time out and changed data.
Errors are described in details below (note that you can not compare the numbers directly, as part of the failed setup test cases are not counted astest cases)
Portal:PD: Passed.DFO: Errors due to environmental instability. There is created POB-action on that.SUPP: Passed
Service requests:Alive test:ATP: 2 Broker Time out (Defect 16159). PD: 5 Broker Time out (Defect 16159). Functional:AER: Passed.ATP: 3 Broker Time out (Defect 16159). PD: 2 fails because of environment setup (service call is send to wrong environment).
1 Broker Time out (Defect 16159). SUPP: 1 Broker Time out (Defect 16159).
Kernel:ATP: 2 errors in death batch job because of time out of the stakeholder replication
3 data / environment errors on K060.PD: 3 kernel time outs
5 Data / rounding errors on the letters and H1503 fails due to customer agreement does not work (H150). Defect has been created
SUPP: 9 Data / rounding error on the letters and F024 fails because of data / setup
Number of automated tests over time
8 10 11 13 16 18 22 23 25 26 27 28 29 30 32 33 34 35 36 38 39 40 41725
775
825
875
925
975
1025
1075
1125
Automated regression test progress over time
Week
Tes
tcas
es
Approach on automated test Standard for BPT (content, size and structure)
It is mandatory to use BPT in all regression tests
Education in BPT components (workshops on our
customers applications)
Review of the BPT components (both manual and
automated)
Code guideline
Code review
Automation team located in the same room
(communication)
Data discipline in QC is crucial (data for our graphs)
Test data and change of test environments
Test data used for the automated tests must be
reserved in all test environments
Solution established for deleting and reload of data in
our component test environments
The automated test generates data and cleans up data
How did we get this far Management focus on automation - it has taken a long time to
reach this far (maturity)
Visibility on automated testing - we get that by sending out a daily
mail with graphs and mark all defects found
We can test a large part of our regression tests automatically in
one night and it can be used for bug fixes in production
Over time - lower costs on our regression test
Centralized automation team is crucial for our success
Important to focus on maturity of the company (we are now TMMi
level 2 – moving towards level 3)
Continue the conversation with your peers at the HP Software Community hp.com/go/swcommunity
Recommended