Transcript
Page 1: Software Testing Strategies 2 Software Testing

1

Software Testing StrategiesSoftware Testing Strategies

Page 2: Software Testing Strategies 2 Software Testing

2

Software Testing StrategyStrategy

Integration of software test case design Integration of software test case design methods into a well-planned series of stepsmethods into a well-planned series of steps

Successful software constructionSuccessful software construction Provides a road map that describes Provides a road map that describes

The steps to be conducted as part of testingThe steps to be conducted as part of testing How much effort, time and resources will be How much effort, time and resources will be

requiredrequired Must incorporate test planning, test case Must incorporate test planning, test case

design, test execution, and resultant data design, test execution, and resultant data collection and evaluationcollection and evaluation

Page 3: Software Testing Strategies 2 Software Testing

3

Software Testing StrategyStrategy

Should be flexible enough to promote a Should be flexible enough to promote a customized testing approachcustomized testing approach

Must be rigid enough to promote reasonable Must be rigid enough to promote reasonable planning and management tracking as the planning and management tracking as the project progressesproject progresses

Different test methods are beginning to cluster Different test methods are beginning to cluster themselves into distinct approaches and themselves into distinct approaches and philosophiesphilosophies

Page 4: Software Testing Strategies 2 Software Testing

4

Software Testing What is it?What is it?

To uncover errorsTo uncover errors How do we conduct the test?How do we conduct the test?

Do we develop a formal plan?Do we develop a formal plan? Should we test the entire program as a whole or run Should we test the entire program as a whole or run

tests on a small part of it?tests on a small part of it? Should we rerun tests we have already conducted as Should we rerun tests we have already conducted as

we add new components to a large system?we add new components to a large system? When should we involve customer?When should we involve customer? These questions must be answered when we develop a These questions must be answered when we develop a

software testing strategy.software testing strategy.

Page 5: Software Testing Strategies 2 Software Testing

5

Software Testing Who does it?Who does it?

The strategy is developed by the project The strategy is developed by the project manager, software engineers, and testing manager, software engineers, and testing specialists.specialists.

Why is it importantWhy is it important Testing often accounts for more project effort Testing often accounts for more project effort

than any other software engineering activitythan any other software engineering activity If conducted haphazardlyIf conducted haphazardly

time is wastedtime is wasted Unnecessary effort is expendedUnnecessary effort is expended Errors sneak through undectedErrors sneak through undected

Page 6: Software Testing Strategies 2 Software Testing

6

Software Testing What are the steps?What are the steps?

Begins in the small and progresses to the Begins in the small and progresses to the largelarge

Focus on a single component or a small group of Focus on a single component or a small group of related componentsrelated components

After components are tested they may be integrated After components are tested they may be integrated until the complete system is constructeduntil the complete system is constructed

As errors are uncovered, they must be diagnosed As errors are uncovered, they must be diagnosed and corrected using a process that is called and corrected using a process that is called debuggingdebugging

Page 7: Software Testing Strategies 2 Software Testing

7

Software Testing What is the work product?What is the work product?

A test specification documentA test specification document Defining a plan that describes overall strategyDefining a plan that describes overall strategy A procedure that defines specific testing steps and A procedure that defines specific testing steps and

the tests that will be conductedthe tests that will be conducted

How do I ensure that I’ve done it right?How do I ensure that I’ve done it right? By reviewing the Test specification prior to By reviewing the Test specification prior to

testingtesting Assess the completeness of test cases and Assess the completeness of test cases and

testing taskstesting tasks

Page 8: Software Testing Strategies 2 Software Testing

8

Software Testing A Strategic ApproachA Strategic Approach

Testing is a set of activities that can be planned Testing is a set of activities that can be planned in advance and conducted systematicallyin advance and conducted systematically

Conduct effective formal technical reviewsConduct effective formal technical reviews Begin at the component level and work “outward” Begin at the component level and work “outward”

toward the integration of the entire computer-based toward the integration of the entire computer-based systemsystem

Different testing techniques at different points in timeDifferent testing techniques at different points in time Usually conducted by the software developer and for Usually conducted by the software developer and for

large projects by an independent test grouplarge projects by an independent test group Testing and debugging are different activities, but Testing and debugging are different activities, but

debugging must be accommodated in any testing debugging must be accommodated in any testing strategystrategy

Page 9: Software Testing Strategies 2 Software Testing

9

Software Testing Verification and ValidationVerification and Validation

Software testing is often referred to as V&VSoftware testing is often referred to as V&V Verification refers to the set of activities that Verification refers to the set of activities that

ensures that software correctly implements a ensures that software correctly implements a specific functionspecific function

Are we building the product right?Are we building the product right?

Validation is a different set of activities that ensure Validation is a different set of activities that ensure that the software that has been built is traceable to that the software that has been built is traceable to customer requirementscustomer requirements

Are we building the right product?Are we building the right product?

Page 10: Software Testing Strategies 2 Software Testing

10

Software Testing Verification and ValidationVerification and Validation

Encompass a wide array of Software Quality Encompass a wide array of Software Quality Assurance (SQA) activities includingAssurance (SQA) activities including

Formal technical reviewsFormal technical reviews Quality and configuration auditsQuality and configuration audits Performance monitoringPerformance monitoring SimulationSimulation Feasibility studyFeasibility study Documentation reviewDocumentation review Database reviewDatabase review Algorithm analysisAlgorithm analysis Development, usability, installation testingDevelopment, usability, installation testing

Page 11: Software Testing Strategies 2 Software Testing

11

Software Testing Organizing for Software TestingOrganizing for Software Testing

Psychological point of viewPsychological point of view The software engineer analyzes, models, and then The software engineer analyzes, models, and then

creates a computer program and its documentationcreates a computer program and its documentation From the software engineers perspective testing is From the software engineers perspective testing is

destructive as it tries to break the thing that SE has destructive as it tries to break the thing that SE has builtbuilt

There are often a number of misconceptionsThere are often a number of misconceptions The developer of the software should no do testThe developer of the software should no do test Software should be tossed over the wall to strangers Software should be tossed over the wall to strangers

who will test it mercilesslywho will test it mercilessly Testers get involved with the project only when the Testers get involved with the project only when the

testing steps are about to begintesting steps are about to begin

Page 12: Software Testing Strategies 2 Software Testing

12

Software Testing Organizing for Software TestingOrganizing for Software Testing

The role of the independent test groupThe role of the independent test group Remove the inherent problems associated with Remove the inherent problems associated with

letting the builder test the thing that has been builtletting the builder test the thing that has been built Removes the conflict of interestRemoves the conflict of interest The software engineer, however, does turn the The software engineer, however, does turn the

program over to the ITG and walk away; both have program over to the ITG and walk away; both have to work closely throughout the projectto work closely throughout the project

The developer must be available to correct The developer must be available to correct uncovered errorsuncovered errors

Page 13: Software Testing Strategies 2 Software Testing

13

Software Testing Strategy for conventional software Strategy for conventional software

architecturearchitecture Unit TestingUnit Testing

Concentrates on each unit (i.e. component) of the Concentrates on each unit (i.e. component) of the software as implemented in the source codesoftware as implemented in the source code

Integration TestingIntegration Testing Focus is on design and the construction of the software Focus is on design and the construction of the software

architecture architecture Validation TestingValidation Testing

Requirements are analysis are validated against the Requirements are analysis are validated against the software that has been constructedsoftware that has been constructed

System TestingSystem Testing Software and other elements are tested as a wholeSoftware and other elements are tested as a whole

Page 14: Software Testing Strategies 2 Software Testing

14

Software Testing Strategy for Object Oriented ArchitectureStrategy for Object Oriented Architecture

Focus on “testing in the small” and work Focus on “testing in the small” and work outward toward “testing in the large”outward toward “testing in the large”

Testing in the small changes from an individual Testing in the small changes from an individual module to a class encompassing attributes and module to a class encompassing attributes and operations; implies communication and collaborationoperations; implies communication and collaboration

A series of regression tests are run to uncover errors A series of regression tests are run to uncover errors due to communication and collaboration between due to communication and collaboration between classesclasses

Page 15: Software Testing Strategies 2 Software Testing

15

Software Testing Strategic IssuesStrategic Issues

Specify product requirements in a quantifiable manner Specify product requirements in a quantifiable manner long before testinglong before testing

A good strategy not only looks for errors, but also asses A good strategy not only looks for errors, but also asses other quality characteristics such as other quality characteristics such as

PortabilityPortability MaintainabilityMaintainability UsabilityUsability

State testing objectives explicitlyState testing objectives explicitly The specific objectives of testing should be stated in The specific objectives of testing should be stated in

measurable termsmeasurable terms Test effectivenessTest effectiveness The cost to find and fix defectsThe cost to find and fix defects Test coverageTest coverage

Page 16: Software Testing Strategies 2 Software Testing

16

Software Testing Strategic IssuesStrategic Issues

Understand the user of the software and Understand the user of the software and develop a profile for eachdevelop a profile for each

Use-cases that describe the interaction scenario for Use-cases that describe the interaction scenario for each class of user can reduce overall testing effort by each class of user can reduce overall testing effort by focusing testing on actual use of the projectfocusing testing on actual use of the project

Build robust software that is designed to test Build robust software that is designed to test itselfitself

Software should be designed in a manner that uses Software should be designed in a manner that uses anti-bugging techniques. Software should be capable anti-bugging techniques. Software should be capable of diagnosing certain classes of errors. The design of diagnosing certain classes of errors. The design should be accommodated automated testing and should be accommodated automated testing and regression testingregression testing

Page 17: Software Testing Strategies 2 Software Testing

17

Software Testing Strategic IssuesStrategic Issues

Use effective formal technical reviews as a Use effective formal technical reviews as a filter prior to testingfilter prior to testing

Reviews can reduce the amount of testing effort that Reviews can reduce the amount of testing effort that is required to produce high-quality softwareis required to produce high-quality software

Develop a continuous improvement approach Develop a continuous improvement approach The testing strategy should be measured. The The testing strategy should be measured. The

metrics collected during testing should be used as metrics collected during testing should be used as part of a statistical process control approach for part of a statistical process control approach for testingtesting

Page 18: Software Testing Strategies 2 Software Testing

18

Software Testing Test Strategies for Conventional SoftwareTest Strategies for Conventional Software

There are many strategies for testing software. There are many strategies for testing software. A software team could wait until the system is fully A software team could wait until the system is fully

constructed and then conduct tests on the overall system. constructed and then conduct tests on the overall system.

At the other extreme software engineer can conduct tests At the other extreme software engineer can conduct tests on a daily basis, whenever any part of the system is on a daily basis, whenever any part of the system is constructedconstructed

A testing strategies falls between the two extremesA testing strategies falls between the two extremes Begin with the testing of individual program unitsBegin with the testing of individual program units Moving to tests designed to facilitate the integration of the Moving to tests designed to facilitate the integration of the

unitsunits

Page 19: Software Testing Strategies 2 Software Testing

19

Software Testing Test Strategies for Conventional SoftwareTest Strategies for Conventional Software

Unit TestingUnit Testing Focuses verification effort on the smallest unit of Focuses verification effort on the smallest unit of

software design – the software component or modulesoftware design – the software component or module Integration TestingIntegration Testing

““If they all work individually, why do you doubt that If they all work individually, why do you doubt that they’ll work when we put them together?”they’ll work when we put them together?”

The problem is “putting them together” – interfacingThe problem is “putting them together” – interfacing Data may be lostData may be lost One module can have an inadvertent, adverse One module can have an inadvertent, adverse

effect on another; sub-function when combinedeffect on another; sub-function when combined Global data structure can present problemsGlobal data structure can present problems

Page 20: Software Testing Strategies 2 Software Testing

20

Software Testing Test Strategies for Conventional SoftwareTest Strategies for Conventional Software

Regression TestingRegression Testing Each time a new module is added as part of Each time a new module is added as part of

integration testing, the software changesintegration testing, the software changes New data flow paths are establishedNew data flow paths are established New I/O and control logic is invokedNew I/O and control logic is invoked Regression testing is the re-execution of some Regression testing is the re-execution of some

subset of tests that have already been conducted to subset of tests that have already been conducted to ensure that changes have not propagated ensure that changes have not propagated unintended side effectsunintended side effects

Smoke testingSmoke testing An integration testing An integration testing

Page 21: Software Testing Strategies 2 Software Testing

21

Software Testing Test Strategies for Conventional SoftwareTest Strategies for Conventional Software

Smoke testingSmoke testing An integration testing An integration testing Software components that have been translated into Software components that have been translated into

code are integrated into a “build” – data files, code are integrated into a “build” – data files, libraries, reusable modules, and engineered libraries, reusable modules, and engineered components that are required to implement one or components that are required to implement one or more productmore product

The intent of testing here is to uncover show The intent of testing here is to uncover show stoppers – errors that have the highest likelihood of stoppers – errors that have the highest likelihood of throwing the software project behind schedulethrowing the software project behind schedule

The build is integrated with other builds and the The build is integrated with other builds and the entire product is smoke tested dailyentire product is smoke tested daily

Page 22: Software Testing Strategies 2 Software Testing

22

Software Testing Test Strategies for OBJECT-ORIENTED Test Strategies for OBJECT-ORIENTED

SOFTWARESOFTWARE Unit TestingUnit Testing

An encapsulated class is the focus of unit testingAn encapsulated class is the focus of unit testing Operations within the class are, however, the smallest Operations within the class are, however, the smallest

unitunit Integration TestingIntegration Testing

Thread based testingThread based testing Integrates a set of classes required to respond to one input Integrates a set of classes required to respond to one input

or event for the systemor event for the system Used based testingUsed based testing

Begins the construction of the system by testing Begins the construction of the system by testing independent classes – use very few if any server classesindependent classes – use very few if any server classes

Testing dependent classes – which use independent classesTesting dependent classes – which use independent classes

Page 23: Software Testing Strategies 2 Software Testing

23

Software Testing Test Strategies for OBJECT-ORIENTED Test Strategies for OBJECT-ORIENTED

SOFTWARESOFTWARE Integration TestingIntegration Testing

Cluster TestingCluster Testing A cluster of collaborating classes (determined by A cluster of collaborating classes (determined by

examining the CRC) to uncover errors in collaborationexamining the CRC) to uncover errors in collaboration

Page 24: Software Testing Strategies 2 Software Testing

24

Software Testing Validation TestingValidation Testing

At this level the distinction between conventional and At this level the distinction between conventional and object-oriented software disappearsobject-oriented software disappears

Testing focuses on user visible actions and user Testing focuses on user visible actions and user recognizable output from the systemrecognizable output from the system

Can be defined in many waysCan be defined in many ways A simple definition is that validation succeeds when A simple definition is that validation succeeds when

software functions in a manner that can be reasonably software functions in a manner that can be reasonably expected by the customer.expected by the customer.

Alpha TestingAlpha Testing Conducted at the developer site by end-usersConducted at the developer site by end-users

Beta TestingBeta Testing Conducted at end user sites – the developer is generally Conducted at end user sites – the developer is generally

not presentnot present

Page 25: Software Testing Strategies 2 Software Testing

25

Software Testing System TestingSystem Testing

Ultimately, software is incorporated with other system Ultimately, software is incorporated with other system elements elements

Hardware, people, informationHardware, people, information A series of system integration and validation tests are A series of system integration and validation tests are

conductedconducted A classic system testing problem – finger pointingA classic system testing problem – finger pointing

Software engineers should anticipate potential interfacing Software engineers should anticipate potential interfacing problemsproblems

Design error handling paths that test all information Design error handling paths that test all information coming from other elements of the systemcoming from other elements of the system

Conduct series of tests to simulate bad dataConduct series of tests to simulate bad data Record the results of testing as evidenceRecord the results of testing as evidence Participate in planning and design of systems tests to Participate in planning and design of systems tests to

ensure that software is adequately testedensure that software is adequately tested

Page 26: Software Testing Strategies 2 Software Testing

26

Software Testing System TestingSystem Testing

Recovery TestingRecovery Testing Is a system test that forces the software to fail in a Is a system test that forces the software to fail in a

variety of ways and verifies that recovery is properly variety of ways and verifies that recovery is properly performedperformed

If automatic recoveryIf automatic recovery Re-initializationRe-initialization Check-pointing mechanismsCheck-pointing mechanisms Data recoveryData recovery Restart are evaluated for correctnessRestart are evaluated for correctness

If manualIf manual Mean time to repair (MTTR) is evaluatedMean time to repair (MTTR) is evaluated

Page 27: Software Testing Strategies 2 Software Testing

27

Software Testing Security TestingSecurity Testing

It verifies that protection mechanisms built into a It verifies that protection mechanisms built into a system will, in fact protect it from improper system will, in fact protect it from improper penetrationpenetration

Stress TestingStress Testing It executes a system in a manner that demands It executes a system in a manner that demands

resources in abnormal quantity, frequency, or volumeresources in abnormal quantity, frequency, or volume Performance TestingPerformance Testing

Are often coupled with stress testing and usually Are often coupled with stress testing and usually require both hardware and software instrumentation. require both hardware and software instrumentation.

To measure resource utilizationTo measure resource utilization


Recommended