View
218
Download
0
Embed Size (px)
Citation preview
TCS2411 Software Engineering 1
Software Testing Strategies
“How do you organise your software tests?”
TCS2411 Software Engineering 2
Lecture ObjectivesTo describe the process of planning
for software testingTo understand the different levels of
testing that can be doneTo describe the different approaches
to debugging software
TCS2411 Software Engineering 3
Software Testing Strategy
The approach towards software testingA set of tests that is planned and
conducted to ensure successful construction of software
Guideline or “road map” of testing activities - planning, design, execution, data collection, evaluation
TCS2411 Software Engineering 4
Examples of Software Testing Strategies
Testing begins at module level and works “outward” towards integration of the entire computer-based system
Different testing techniques at different points in time
Testing is conducted by developer and an independent test group
Testing and debugging are different activities, but debugging is accommodated
TCS2411 Software Engineering 5
A Software Testing Strategy
SystemEngineering
SystemEngineering
RequirementsRequirements DesignDesign CodeCode
UnitTestUnitTest
IntegrationTest
IntegrationTest
ValidationTest
ValidationTest
SystemTest
SystemTest
TCS2411 Software Engineering 6
Software Testing Organization
Developer : unit testing Ensure each functions performs according to
its design Sometimes conduct integration testing
Independent testing group (ITG) Report to SQA team
Developer and ITG work closely to ensure thorough test is well conducted
Developer must be present during testing to correct uncovered errors.
TCS2411 Software Engineering 7
Software Testing Steps
High-order tests
Integration test
Unit test
Testingdirection
code
design
requirements
Testingdirection
code
design
requirements
TCS2411 Software Engineering 8
Unit TestingVerification of the smallest unit in
software - the moduleWhite box orientedCan be conducted in parallel for
multiple modulesUnit test cases designed based on
source code - examine likely causes for errors
Test environment includes driver and/or stubs
TCS2411 Software Engineering 9
Unit Test Environment
Driver
Module tobe tested
StubStub …..
TCS2411 Software Engineering 10
Unit Testing Considerations
Interface - tested to ensure that information properly flows into and out of the unit
Local data structures - examined to ensure that data stored temporarily maintains its integrity in an algorithm’s execution
Boundary conditions - tested to ensure that module operates properly at boundaries
TCS2411 Software Engineering 11
Unit Testing Considerations (Continued)
Independent paths - to ensure all statements in a module executed at least once
Error-handling paths clear and correct messages/descriptions correct processing of errors not interrupted by system sufficient information to locate cause of
error
TCS2411 Software Engineering 12
Unit Test Environment
Driver
Interface Local data structures Boundary conditionsIndependent pathError Handling path
Test cases
Module to be tested
Stub Stub
•Act as main program•Accept test data•Past to the component to be tested•Print results
•Replace module that are subordinate (called by) component to be tested•Use subordinates module interface•Do minimal data manipulation•Print verification entry•Return control to the module
Notes :•Driver and stub are software that must be written.•Both are not delivered with final product
Notes :•Driver and stub are software that must be written.•Both are not delivered with final product
RESULTSRESULTS
TCS2411 Software Engineering 13
Integration TestingSystematic technique for constructing
software while testing to uncover errors associated with interfacing
Take unit-tested modules and build the program structure based on the design
Nonincremental (“big bang”) - all modules combined in advanced chaos!
Incremental integration - part by part, e.g. top-down or bottom-up
TCS2411 Software Engineering 14
Top-Down IntegrationBeginning with main module, modules
are integrated by moving downward through the control hierarchy
Subordinate modules are added into the structure either depth-first or breadth-first
Verifies major control or decision points early in testing
Problem when lower level processes needed to test upper levels
TCS2411 Software Engineering 15
Top-Down Integration Diagram
M1
M2 M3 M4
M5 M6
M8
M7
TCS2411 Software Engineering 16
Top-Down Integration Process
Beginning at main module, stubs are replaced one at a time with actual modules
Tests are conducted as each module is integrated
On the completion of each set of tests, another stub is replaced with real module
Regression testing (conduct all or some of previous tests) may be done to ensure no new errors have been introduced
TCS2411 Software Engineering 17
Bottom-Up Integration
Begins construction and testing with atomic modules (lowest level in hierarchy)
No need for stubs, all subordinate modules tested and available
Test drivers required and can be shared as integration move towards the higher levels
TCS2411 Software Engineering 18
Bottom-Up Integration Diagram
Mc
D1 D2 D3
Ma Mb
Cluster 1
Cluster 2
Cluster 3
TCS2411 Software Engineering 19
Bottom-Up Integration Process
Low-level modules are combined into clusters
A driver is written to coordinate test case input and output
The cluster is testedDrivers are removed and clusters are
combined moving upward in the program structure
TCS2411 Software Engineering 20
Regression Testing
New module or major changes in software.Helps to ensure changes does not introduce
Unintended behavior Additional errors
Re-execute the subset of all test cases Most advised : Use automated capture-play
back toolsThis tool enable the software engineer to capture test
cases and results for subsequent playback and comparison.
TCS2411 Software Engineering 21
Validation Testing
A series of black box tests that demonstrate conformity with requirements
Test that software functions as expected by customer
Test plan and details can be based on software requirements specification
TCS2411 Software Engineering 22
Alpha Test
Conducted at developer’s site by customer
Software is used in a natural setting with the developer “looking over the shoulder”
Errors and usage problems recorded, correction can possibly be done quickly
Test conducted in a controlled environment
TCS2411 Software Engineering 23
Beta TestConducted at one or more customer sites
by end user(s)Developer generally not present“Live” application of software in
uncontrolled environmentCustomer records all problems and
reports them to developer at regular intervals
Software modified based on problems reported before final release to customers
TCS2411 Software Engineering 24
System Testing
A series of tests to fully exercise a computer-based system
Verify that all system elements have been properly integrated and perform allocated functions
Software is one element to be integratedSoftware engineer to anticipate potential
interfacing problems
TCS2411 Software Engineering 25
Recovery Testing
Force the software to fail in a variety of ways
Verify that recovery is properly performedIf recovery is automatic, re-initialisation,
checkpointing mechanisms, data recovery, and restart are evaluated for correctness
If recovery requires human intervention, mean time to repair is evaluated
TCS2411 Software Engineering 26
Security Testing
Verify that protection mechanisms built into the system will protect it from improper penetration
Tester plays the role(s) of individual who desires to penetrate the system
System designer to make penetration cost greater than the value of information obtained
TCS2411 Software Engineering 27
Stress & Performance Testing
Stress Testing Execute system in a manner that
demands resources in abnormal quantity, frequency or volume
Performance Testing Test run-time performance of software
within context of an integrated system
TCS2411 Software Engineering 28
Debugging
To find and correct the cause of a software error
Attempt to match symptom to causeBug characteristics:
Symptom and cause may be geographically remote
Symptom disappear (temporarily) when another error is corrected
TCS2411 Software Engineering 29
Debugging (Continued)
Symptom caused by nonerrors Symptom caused by human errors Symptom a result of timing problems Difficult to accurately reproduce input
conditions Symptom may be intermittent Symptom due to causes that are
distributed across a number of tasks on different processors
TCS2411 Software Engineering 30
Debugging Process
TestCases
ExecutionOf cases
Results
Debugging
Additional test
Suspectedcauses
Regression test
CorrectionsIdentifiedcauses
TCS2411 Software Engineering 31
Debugging Approaches
Brute force “let the computer find the error” memory dumps, run-time traces, lots of
WRITE statements clues to errorBacktracking
trace source code backwards from errorCause elimination
induction/deduction, “binary partitioning” isolate potential causes, test each one
TCS2411 Software Engineering 32
Debugging AidsDebugging Tools
Debugging compiler Dynamic debugging aids (“tracers”) Automatic test case generators Memory dumps Cross-reference maps
Good design documentationClear source codeOther people!
TCS2411 Software Engineering 33
References
“Software Engineering: A Practitioner’s Approach” 5th Ed. by Roger S. Pressman, Mc-Graw-Hill, 2001
“Software Engineering” by Ian Sommerville, Addison-Wesley, 2001