View
224
Download
0
Category
Tags:
Preview:
Citation preview
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 1
LCG-SPI: SW-TestingLCG Applications Area
GridPP 7th Collaboration Meeting Manuel.Gallas@cern.ch
LCG/SPILCG Software Process & Infrastructure
LCG Software Process & Infrastructure
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 2
Outline and Context
SPI SW-Testing
• Overview• Test FrameWorks• User support• Test policies• Test documents
LCG Infrastructure
SPI project
http://spi.cern.ch/
LHC grid software
applications (LHC experiments, projects,
etc)LCG Application Area
POOLPI
SIMU
SEAL
SW-Testing
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 3
Overview
•Software testing will be an integral part of the software development process.•All level of software testing should be run as part of an automatic process.
GOAL
Software testing
Integration test
System test
Acceptance test
Unit test
Sw-testing team
IntegrationTests
System Tests
ExamplesLHC experiments
Use in Exp.Test FrameWorks
SW-testing doc
SW- testing support
SPI SW-Testing
SW-testing polices
Code Documentation
Software developer
CODE
Work PackageTest
Automated testing
Nightly Building
System
SW-Testing
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 4
Test FrameWorks
Aim: to help developers:
• to produce code for testing • to run tests in automatic way
Our inputs: • Contacts within HEP-community.• What is available as free open source code.
Our constrains: • Avoid commercial software and licensing problems. • Avoid “do it yourself solutions”•Try to adopt commonly used open-source software.
SW-Testing
Requirements: • Work with different languages (C++, Python, …)• Allow two ways of testing: “test the output of the code” and “test inside the testing-code”.• Easy way to integrate existing tests.• Allow the creation of dependencies among tests• Organize the tests by components or packages.• Regression testing• Easy integration with the Nightly Building System.• Provide also a graphical interface.
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 5
•Scope: Unit –testing
•The same “assertion style” in different languages.
•Provides:- The name of the test case that failed.
-The name of the source file that contains the test.
-The line number where the failure occurred
• Different platforms/compilers: (Linux/Solaris/Windows)
Test FrameWorks: X-Unit family
X-Unit family
JUnit
QtUnit
CppUnit
PyUnit
A simple test:
1. Subclass the TestCase CppUnit or PyUnit class
2. Override the method runTest().
3. When you want to check a value, call the (CppUnit or PyUnit) ASSERT(bool) and pass in an expresion that is true if the test succeeds
SW-Testing
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 6
Test FrameWorks: Oval
Oval:• validation and regression • used in CMS
•Can be use for Unit-test to Validation tests.
•Compare the output log file with a given reference file (Smart comparison of those lines which start with “[Oval]”)
• It is possible to set different run environments.
• Can run external scripts and external binaries.
Ovalfile(configuration)
ExecutableTest
Source Code
Referencefile
Log file
Oval run
Ovaldiff
Oval build
SW-Testing
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 7
Test FrameWorks: QMTest
QMTest
•Uses a graphical interface for creating and running tests
• Runs tests in parallel
• Organizes tests hierarchically
• Supports execution of a single test or many at once
• Records dependencies between tests
• Can be run in batch mode -> easy integration with the Nightly-Building systems
SW-Testing
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 8
Test FrameWorks: Global picture
TestScripts
Valgrind
Aim: to help developers:
• to produce code for testing • to run tests in automatic way
OvalFile
Oval
Logfile
Reffile
?
• Automated tests
• Regression test
CppUnit
PyUnit
X-Unit family
JUnit
QtUnit
QMTest
• Uses a graphical interface for creating and running tests• Runs tests in parallel• Organizes tests hierarchically• Supports execution of a single test or many at once• Records dependencies between tests
In LCG AppArea runs what is in the OvalFile and uses oval.
Oval
SW-Testing
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 9
SPI SW-Testing
• Overview• Test FrameWorks• User support• Test policies• Test documents
SW-Testing
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 10
User support: Web information
http://spi.cern.ch/
Test FrameWorks
HowTo
Test doc
Policies
SW-Testing
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 11
User support: Tools supported
CppUnit
PyUnit
Provided as free software by the SPI External Software Service
•Different platforms: (Linux/Solaris/Windows)
•Different compilers•Integrated with SCRAM and ready to be used•Installed in AFS/DFS•Instructions to install them in a local machine
How to use them ?
Oval
QMTest
SW-Testing
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 12
1
User support: How-To
2
3 Follow sw-testing policies
*
SW-Testing
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 13
SPI SW-Testing
• Overview• Test FrameWorks• User support• Test policies• Test documents
SW-Testing
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 14
Sw-Testing policies
http://spi.cern.ch/software_development.html
For the projects
For the developersFor the testing responsibl
e
Sw-testing policies are needed to ensure:
• Software is created together with the tests.• Test directory structure and naming should be homogeneous and this will imply easy life to have automatic tools.• Common and well integrated tools
SW-Testing
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 15
SPI SW-Testing
• Overview• Test FrameWorks• User support• Test policies• Test documents
SW-Testing
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 16
Test Documents
• For all project is needed a Test Plan in order to have a clear idea of the type of tests, schedule, environment and responsibilities.• We use a template•Test Case template collects information about the existing tests and how they cover the expected functionality of the software.
SW-Testing
M Gallas CERN EP-SFT
LCG-SPI: SW-Testing 17
Thanks to:• PPARC• LHC experiments• LCG
Feedback and interaction are always welcome!!
SPI SW-Testing
• Overview• Test FrameWorks• User support• Test policies• Test documents
SW-Testing
Conclusions:Software testing is an important task and we have tools in place to do it.
Recommended