Upload
vijayasekar
View
219
Download
0
Embed Size (px)
Citation preview
8/14/2019 Proficience - SQAM - Module 3b
1/26
SQAM Course, Proficience, IISc 1
Module 3: Software TestingModule 3: Software Testing
3.1 Fundamentals of Software Testing
3.2 Testing Strategy and Planning
3.3 Testing Methods3.4 Economics of Testing
3.5 Testing Metrics
3.6 Defect Tracking3.7 Templates and Checklists
3.8 Case Study
8/14/2019 Proficience - SQAM - Module 3b
2/26
SQAM Course, Proficience, IISc 2
TestingTesting
Goal is to find faults
What kind of faults?
Algorithmic
Computation and Precision
Documentation
Overload
Capacity
Timing
Performance Recovery
System
Standards
8/14/2019 Proficience - SQAM - Module 3b
3/26
SQAM Course, Proficience, IISc 3
Codes logic does not produce the proper output for a
given input Some examples
Branch under wrong conditionsFail to set loop invariant
Fail to initialize variablesComparing inappropriate data types
Formula is implemented incorrectly
Computation not performed to the required accuracy
Data structures filled past their capacity Buffer overflow
Dimension of tables larger than code can handle
Length of queue too great
ExamplesExamples
8/14/2019 Proficience - SQAM - Module 3b
4/26
SQAM Course, Proficience, IISc 4
Some more..Some more.. Performance becomes unacceptable as activity reaches system limit
E.g., system capable of handling 1,000,000 records becomes slower andslower as more records are added
May be examined in relation to disk access, number of processes, etc. System does not perform at the speed prescribed by the requirements Problem with underlying hardware or operating system
Disk failure Out of memory Process table full
Software doesnt adequately handle the system fault Interface
Fault in interacting with another component or driver Checking
Fault in logic that fails to validate data properly before it is used Build/Package/Merge
Fault that occurs because of problems in repositories and version control
8/14/2019 Proficience - SQAM - Module 3b
5/26
SQAM Course, Proficience, IISc 5
Testing TypesTesting Types
Goal is to find the faults
Different types of testing
Unit testing Integration testing
Functional testing
Performance testing Acceptance testing
Installation testing
8/14/2019 Proficience - SQAM - Module 3b
6/26
SQAM Course, Proficience, IISc 6
Testing PhasesTesting Phases
Requirements
Specification
System
Specification
System
Design
Detailed
Design
Unit code and
Test
Sub-system
Integration test
System
Integration test
Acceptance
TestService
Acceptance
Test Plan
System
Integration
Test Plan
Sub-system
Integration
Test Plan
8/14/2019 Proficience - SQAM - Module 3b
7/26
SQAM Course, Proficience, IISc 7
Unit TestingUnit Testing
The first opportunity to test whether a particular module orcomponent is complete
Often performed by the developer
Can choose test cases in many ways. Some sample to get
broad test coverage: Statement testing : Every statement is executed at least once
Branch testing : Every decision point is seen at least once.
Path testing : Variation of above, each path through the
code tested at least once (visualize code as a flow chart) Definition-use path testing : Every path from every
definition of every variable to every use of that definition is
exercised
8/14/2019 Proficience - SQAM - Module 3b
8/26
SQAM Course, Proficience, IISc 8
Integration TestingIntegration Testing
Integration
Piecing together
individual components
to make a workingsystem
Individual components
should already have
passed their unit tests
Integration Testing Tests to ensure the
system functions when
components pieced
together
Bottom-up
Low-level first, with test
harness
Top-Down Stubs
Big-Bang
Put them together and see if
it works
Sandwich System viewed as three
layers
Top-down in top layer,
bottom-up in lower,
converge in the middle
8/14/2019 Proficience - SQAM - Module 3b
9/26
SQAM Course, Proficience, IISc 9
Top-Down Integration TestingTop-Down Integration Testing
Modules are integrated by
moving downward
through control hierarchy
Modules subordinate to
main control module are
incorporated
Depth-first
Breadth-first
M1
M3 M4M2
M7M6M5
M1
8/14/2019 Proficience - SQAM - Module 3b
10/26
SQAM Course, Proficience, IISc 10
Top-Down Integration TestingTop-Down Integration TestingStrategy:
Verifies major control or decision early in the test process. Allows early recognition of problems Depth first strategy allows a complete function of the software to be
implemented Stubs replace lower-level modules so no significant data can flow upward
Delay some tests until have actual modules Add code to simulate module
Steps:
The main control module is used as a test driver, and stubs are substituted forall components directly subordinate to the main control module.
Depending on the integration approach selected, subordinate stubs arereplaced one at a time with actual components.
Tests are conducted as each component is integrated. On completion of each set of tests, another stub is replaced with a real
component . Regression testing may be conducted to ensure that new errors have not been
introduced.
8/14/2019 Proficience - SQAM - Module 3b
11/26
SQAM Course, Proficience, IISc 11
Bottom-Up IntegrationBottom-Up Integration Begins construction and testing with atomic modules
Need for stubs is eliminated
Operational modules tested thoroughly
Steps Low-level components are combined into clusters (builds) to perform specific sub-function
A driver is written to coordinate test case input and output
Cluster is tested
Drivers are removed and clusters are combined moving upward in program structure.
Level N Level N Level N Level N Level N
Level N-1 Level N-1 Level N-1
Test drivers
Test
drivers
Testing
sequence
8/14/2019 Proficience - SQAM - Module 3b
12/26
8/14/2019 Proficience - SQAM - Module 3b
13/26
SQAM Course, Proficience, IISc 13
Comparison of Integration StrategiesComparison of Integration Strategies
MediumEasyHardEasyAbility to test
particular paths
MediumHighLowMediumWork parallelism at
beginning
YesNoYesNoStubs needed
YesYesNoYesComponent drivers
needed
EarlyLateEarlyLateTime to basic working
program
EarlyLateEarlyEarlyIntegration
SandwichBig-BangTop-DownBottom-Up
8/14/2019 Proficience - SQAM - Module 3b
14/26
SQAM Course, Proficience, IISc 14
System TestingSystem Testing
Software incorporated with hardware, people, information to form
system
System testing is a series of different tests whose primary purpose is to
fully exercise the computer-based system
Types Recovery testing
Security testing
Stress testing
Performance testing
8/14/2019 Proficience - SQAM - Module 3b
15/26
SQAM Course, Proficience, IISc 15
Validation (Product) andValidation (Product) and
Acceptance TestingAcceptance Testing
Validation test
Series of tests that demonstrate conformity with requirements
Test plan and procedure ensure that
all functional requirements are satisfied
all behavioral characteristics achieved all performance requirements attained
documentation is correct
usability and other requirements are met
Acceptance test Alpha test: test at developers site by end-users
Beta test: test conducted at end-user sites
8/14/2019 Proficience - SQAM - Module 3b
16/26
SQAM Course, Proficience, IISc 16
Results of Acceptance TestsResults of Acceptance Tests
Ideally: acceptance testing uncovers discrepancies in requirements
Reality: by working with the system users discover aspects of the
problem which they were not aware
New requirements New development and features
Other direction:
Some requirements may not be needed
8/14/2019 Proficience - SQAM - Module 3b
17/26
SQAM Course, Proficience, IISc 17
Installation TestingInstallation Testing
Installing the system at the user site
System-specific configuration
Co-existence test with other software
E.g. does it work with Office software?
Does anything break if the software is
removed?
8/14/2019 Proficience - SQAM - Module 3b
18/26
SQAM Course, Proficience, IISc 18
Regression TestingRegression Testing
Adding new or changingmodule impacts the system
New data flow paths
established
New I/O may occur
New control logic invoked
Regression testing is re-
execution of subset of tests that
have already been conducted
Ensures changes have notpropagated unintended side
effects
Approaches:Manual testing
Capture/Playback tools: capture test
cases and results for subsequent
playback and comparison
Test suite contains following classes of
test cases:
Representative sample of tests that
exercises all software functions
Focus on functions likely affectedby change
Focus on components that have been
changed
8/14/2019 Proficience - SQAM - Module 3b
19/26
SQAM Course, Proficience, IISc 19
Test CasesTest Cases
Want a broad set of test cases to cover the range of possible values and
code paths
Closed Box
Apply all possible inputs, compare with expected output accordingto requirements
Includes out of range inputs
Open Box
View codes internal structure
Generate tests based on this structure, e.g. give values on both
sides of an if-then else test
8/14/2019 Proficience - SQAM - Module 3b
20/26
SQAM Course, Proficience, IISc 20
Who Should Test?Who Should Test?
In your case, you (the developer) will test
But the developer is often too close to the code to be objective
and recognize some subtle faults
Difficult for developer to separate implementation structure andrequired function
Independent tester or test team is preferable
Familiar with how to break software
Familiar with testing methods and tools
Users should be involved throughout the process
Might catch missing requirements at early stages
8/14/2019 Proficience - SQAM - Module 3b
21/26
SQAM Course, Proficience, IISc 21
Automated Testing ToolsAutomated Testing Tools
Programs that will help test your code
Static Analysis, e.g. lint
Code Analysis
Check syntax, of a construct is fault-prone, etc.
Structure Checker
Tool to depict logic flow, check for structural flaws Data analyzer
Review data structures, illegal data usage, improper linkage
Sequence checker
Highlight events in the wrong sequence
Dynamic Analysis, e.g. VTune
Monitor and collect statistics as program is running
How many times a statement was executed, branches covered, memory use,
etc.
Useful for performance analysis to find the hotspots of the code
8/14/2019 Proficience - SQAM - Module 3b
22/26
SQAM Course, Proficience, IISc 22
How Much Testing?How Much Testing?
How do we know when to stop?
When all tests pass?
When enough tests pass meeting some threshold
Myers (1979)
As the number of detected faults increases, the probability of the
existence of more undetected faults increases
Suggests that if the number of detected faults is decreasing, we
may not have too many left and can stop soon
Fault Seeding
Deliberately add faults to the process
Use their detection as a measure of the remaining faults
8/14/2019 Proficience - SQAM - Module 3b
23/26
SQAM Course, Proficience, IISc 23
Test MetricsTest Metrics
Effective TestMetrics
8/14/2019 Proficience - SQAM - Module 3b
24/26
SQAM Course, Proficience, IISc 24
Defect TrackingDefect Tracking
ug defect life cycl
Defect RemovalEffectiveness
8/14/2019 Proficience - SQAM - Module 3b
25/26
SQAM Course, Proficience, IISc 25
Test Plan Outline
esting Health Car
Testing CRMTesting ERP
Web Testing
Road map fortesting
Case studies, Checklists, Templates
reating Checklistfor testing
oad Plan templat
V&V Template
8/14/2019 Proficience - SQAM - Module 3b
26/26
SQAM Course, Proficience, IISc 26
ReferencesReferences
Effective Methods of Software Testing,
W.Perry
Managing Testing Process, Rex Black