Upload
nikita-knysh
View
735
Download
3
Embed Size (px)
DESCRIPTION
Citation preview
Software Testing Foundations #2Testing in the Software Lifecycle
Nikita [email protected]://www.facebook.com/groups/istqb/
Agenda• Levels of Test: The V-model•Testing New Product Versions•Types of Testing•Testing Related to Changes
Development Activities•Req specification: needs & reqs of the customer,
desired characteristics and features of the system.•Func system design: reqs are mapped onto system
functions & dialogs.•Technical system design: implementation of system is
designed (architecture, interfaces, etc.).•Component specification: task, behavior, inner
structure & interfaces are defined for each subsystem.•Programming.
Levels of Test: The V-modelRequirements
Definition
Functional System Design
Technical System Design
Component Specification
Programming
Component Test
Integration Test
System Test
Acceptance Test
Implementation &Integration
Validation
Verification
V-model: Test Activities #1• Component test: test functionality, robustness, efficiency
and maintainability of components. Techniques: white box, black box, test-driven development.
• Integration test: test of collaboration between components after assembling to expose faults in interfaces and interaction between components. Test drivers, stubs and monitors are used. Integration in the large – integration of several software systems.▫ Integration strategy has to be designed by test manager
taking into account system architecture, project plan and test plan constraints. Top-down, bottom-up, ad-hoc, backbone.
V-model: Test Activities #2• System test: check if the integrated product meets the
specified requirements. System is examined from the point of view of customer and future user. Environment should be as close to the operation environment as possible, but this should NOT be the customer’s operational environment.
• Acceptance test: “Formal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system.” [After IEEE 610]. Customer must be involved.
V-model: Acceptance Test on Lower Levels
•A commercial off-the-shelf software product (COTS) may be acceptance tested when it is installed or integrated.
•Acceptance testing of new functionality may come before system testing (using a prototype).
V-model: Forms of Acceptance Test•According to contract•User acceptance testing•Operational testing•Field testing▫alpha – at the producer’s site▫beta – at the customer’s site
Extent of acceptance test depends on application risk.
Testing New Product Versions•Software maintenance:▫testing after maintenance (anything new or changed is
tested),▫testing after change of environment,▫testing for retirement (data archiving or data migration
into future system).•Release: full regression test (if possible) to find side
effects of changes to the software.
Testing New Product Versions #2•Testing in incremental development: several V-
models after each other, where every next "V" reuses existing test material and adds the tests for everything new, or for higher reliability require ments.
Types of Testing: Overview•Functional•Non-functional•Testing of Software Structure•Testing related to Changes
Types of Testing: FunctionalTest what the system does. Tests are based on functional specs, check system’s input-output behavior and use black-box methods.•Requirements-based testing focuses on single
system functions.•Business-process-based testing focuses on whole
process consisting of many steps.
Types of Testing: Non-functional #1Test how well the (partial) system carries out its functions.
[ISO 9126]: reliability, usability, efficiency, maintainability, portability.
Types of Testing: Non-functional #2• Load test: measuring system behavior when working with the
number of users, number of transaction at a time.• Performance test: measuring system processing speed and
response time (depending on increasing load).• Volume test: observation of system behavior on large amount of
data (e.g. processing very large files).• Stress test: observation of system behavior when it is
overloaded.• Security test: against unauthorized access, denial of service
attacks, etc.• Stability (reliability) test during permanent operation, e.g. time
between failures or failure rate within a given user profile.
Types of Testing: Non-functional #3• Robustness test for response to operating errors, wrong programming
hardware failure, etc., as well as examination of exception handling and recovery.
• Testing of compatibility (to other systems) and data conversion (import / export).
• Testing of different configurations of the system, e.g. OS version, UI language, hardware platform, etc.
• Usability test: ease of learning the system, ease and efficiency of operation, etc. with respect to a specific group of users.
• Checking of documentation for compliance with the system behavior (e.g. user manual and GUI).
• Checking of maintainability: is documentation understandable and up-to-date? Does the system have modular structure, etc.
Types of Testing: Testing Software Structure
Using white box techniques to test object’s internal code or architecture.
•Goal is to cover all structure items.•Most used in component and integration testing.
Testing Related to Changes•Re-test of changed (modified or repaired) parts. •Regression test of whole program (part) after
modification to ensure that no faults are introduced as side effects of changes.
Volume of Regression Test•Defect retest (confirmation test): rerunning of tests
that detected faults that have been then fixed.•Testing of altered functionality: testing of all
program parts that were changed or corrected.•Testing of new functionality: testing of all newly
integrated program parts.•Complete regression test: testing of the whole
system.
Shrinking Regression Test EffortDeciding which test cases we can omit:•Only high-priority test cases (according to test plan)
are repeated.• In the functional test, certain variations (test cases)
are omitted.•Restriction of the test to certain configurations only
(on one OS only, in one language only, etc.)•Restriction of the test to certain subsystems or test
levels.
Thank you!
http://www.facebook.com/groups/istqb/