View
57
Download
1
Category
Tags:
Preview:
DESCRIPTION
Software Engineering Muhammad Fahad Khan fahad@uettaxila.edu.pk. University Of Engineering & Technology Taxila,Pakistan. Today’s Class Chapter 14 Software Testing Techniques R.S.Pressman. Overview. What is it? To design a series of test cases that have high likelihood of finding errors. - PowerPoint PPT Presentation
Citation preview
1
Software Engineering
Muhammad Fahad Khan
fahad@uettaxila.edu.pkUniversity Of Engineering & University Of Engineering &
Technology Taxila,PakistanTechnology Taxila,Pakistan
Today’s Class Chapter 14
Software Testing TechniquesSoftware Testing Techniques R.S.Pressman
3
OverviewOverview What is it?What is it?
To design a series of test cases that have high likelihood of finding errors.To design a series of test cases that have high likelihood of finding errors. Testing techniques provide systematic guidance of designing tests that Testing techniques provide systematic guidance of designing tests that (1) exercise the internal logic and interface of every software component(1) exercise the internal logic and interface of every software component (2) exercise the input and output domains of the program to uncover (2) exercise the input and output domains of the program to uncover
errors in program function, behavior, and performance.errors in program function, behavior, and performance. Who does it?Who does it?
Early stage- software engineerEarly stage- software engineer Later – testing specialistsLater – testing specialists
Why important?Why important? In order to find the highest possible number of error, test must be In order to find the highest possible number of error, test must be
conduct systematically and test cases must be designed using disciplined conduct systematically and test cases must be designed using disciplined techniquestechniques
Steps?Steps? Work products?Work products?
Test casesTest cases
4
TestabiliTestabilitytyHow easily a program can be tested?How easily a program can be tested?
OperabilityOperability—the better it works, the more efficiently it can be —the better it works, the more efficiently it can be testedtested
ObservabilityObservability—the results of each test case are readily —the results of each test case are readily observed, source code, variables are visible? observed, source code, variables are visible?
ControllabilityControllability—the degree to which testing can be automated —the degree to which testing can be automated and optimizedand optimized
DecomposabilityDecomposability—testing can be targeted. Software is built as —testing can be targeted. Software is built as independent modules that can be tested independentlyindependent modules that can be tested independently
SimplicitySimplicity—reduce complex architecture and logic to simplify —reduce complex architecture and logic to simplify teststests
StabilityStability—few changes are requested during testing—few changes are requested during testing UnderstandabilityUnderstandability—of the design—of the design
5
OutlineOutline Software testing fundamentalSoftware testing fundamental White-box testingWhite-box testing Basic path testingBasic path testing Control structure testingControl structure testing Black-box testingBlack-box testing Object-oriented testing methodsObject-oriented testing methods
6
What is a “Good” Test?What is a “Good” Test?
A good test has a high probability of A good test has a high probability of finding an errorfinding an error
A good test is not redundant.A good test is not redundant. A good test should be “best of breed” A good test should be “best of breed” A good test should be neither too A good test should be neither too
simple nor too complexsimple nor too complex
7
Test Case Test Case DesignDesign
"Bugs lurk in corners"Bugs lurk in corners and congregate atand congregate at boundaries ..."boundaries ..."
OBJECTIVEOBJECTIVE
CRITERIACRITERIA
CONSTRAINTCONSTRAINT
to uncover errorsto uncover errors
in a complete mannerin a complete manner
with a minimum of effort and timewith a minimum of effort and time
8
Exhaustive TestingExhaustive Testing
loop < 20 Xloop < 20 X
There are 10 possible paths! If we execute oneThere are 10 possible paths! If we execute onetest per millisecond, it would take 3,170 years totest per millisecond, it would take 3,170 years to
test this program!!test this program!!
1414
9
Selective TestingSelective Testing
loop < 20 Xloop < 20 X
Selected pathSelected path
10
Software TestingSoftware Testing
Methods
Strategies
white-boxmethods
black-box
methods
11
White-Box White-Box TestingTesting
... our goal is to ensure that all... our goal is to ensure that all statements and conditions havestatements and conditions have been executed at least once ...been executed at least once ...
12
White-Box TestingWhite-Box Testing Glass-box testingGlass-box testing Design philosophyDesign philosophy
Use the control structure described as part of Use the control structure described as part of component-level design to derive test casecomponent-level design to derive test case
What kind of test cases that software engineers What kind of test cases that software engineers can derivecan derive all independent pathsall independent paths Logical decisionLogical decision Loops at the boundary Loops at the boundary Internal structuresInternal structures
13
ExampleExample Path 1:1-11Path 1:1-11 Path 2: 1-2-3-4-5-Path 2: 1-2-3-4-5-
10-1-1110-1-11 Path 3: 1-2-3-6-8-9-Path 3: 1-2-3-6-8-9-
10-1-1110-1-11 Path 4: 1-2-3-6-7-9-Path 4: 1-2-3-6-7-9-
10-1-1110-1-11
1
2
3
64
587
910
11
14
Basis Path Basis Path TestingTestingFirst, we compute the cyclomatic
complexity:number of simple decisions + 1
ornumber of enclosed areas + 1In this case, V(G) = 4
15
Cyclomatic Cyclomatic ComplexityComplexityA number of industry studies have indicatedA number of industry studies have indicated
that the higher V(G), the higher the probability that the higher V(G), the higher the probability or errors.or errors.
V(G)V(G)
modulesmodules
modules in this range are modules in this range are more error pronemore error prone
TECHNICAL DETAIL The cyclomatic complexity of a software module is
calculated from a connected graph of the module (that shows the topology of control flow within the program)
How To Find ? Cyclomatic complexity (CC) = E - N + p where E = the number of edges of the graph N = the number of nodes of the graph p = the number of connected components
17
Basis Path Basis Path TestingTestingNext, we derive the Next, we derive the
independent paths:independent paths:
Since V(G) = 4,Since V(G) = 4,there are four pathsthere are four paths
Path 1: 1,2,3,6,7,8Path 1: 1,2,3,6,7,8Path 2: 1,2,3,5,7,8Path 2: 1,2,3,5,7,8Path 3: 1,2,4,7,8Path 3: 1,2,4,7,8
Path 4: 1,2,4,7,2,4,...7,8Path 4: 1,2,4,7,2,4,...7,8
Finally, we derive testFinally, we derive testcases to exercise these cases to exercise these
paths.paths.
11
22
3344
55 66
77
88
18
Basis Path Testing Basis Path Testing NotesNotes
you don't need a flow chart, you don't need a flow chart, but the picture will help when but the picture will help when you trace program pathsyou trace program paths
count each simple logical test, count each simple logical test, compound tests count as 2 or compound tests count as 2 or
moremore
basis path testing should be basis path testing should be applied to critical modulesapplied to critical modules
19
Graph MatricesGraph Matrices A graph matrix is a square matrix whose size (i.e., A graph matrix is a square matrix whose size (i.e.,
number of rows and columns) is equal to the number of rows and columns) is equal to the number of nodes on a flow graphnumber of nodes on a flow graph
Each row and column corresponds to an identified Each row and column corresponds to an identified node, and matrix entries correspond to connections node, and matrix entries correspond to connections (an edge) between nodes. (an edge) between nodes.
By adding a By adding a link weightlink weight to each matrix entry, the to each matrix entry, the graph matrix can become a powerful tool for graph matrix can become a powerful tool for evaluating program control structure during testingevaluating program control structure during testing Probability that a link will be executedProbability that a link will be executed Process timeProcess time Memory, resourceMemory, resource
20
Control Structure TestingControl Structure Testing Condition testingCondition testing — a test case design method that — a test case design method that
exercises the logical conditions contained in a exercises the logical conditions contained in a program moduleprogram module <, <=, >, >=, not equal, >, or, and, not<, <=, >, >=, not equal, >, or, and, not
Data flow testingData flow testing — selects test paths of a program — selects test paths of a program according to the locations of definitions and uses of according to the locations of definitions and uses of variables in the programvariables in the program DEF(s)={x|statement S contains a definition of X}DEF(s)={x|statement S contains a definition of X} USE(s)={x|statement S contains a use of X}USE(s)={x|statement S contains a use of X} A definition-use (DU) chain of variable X is of the form A definition-use (DU) chain of variable X is of the form
[X,S,S’]. [X,S,S’].
21
Loop TestingLoop Testing
Nested Nested LoopsLoops
ConcatenatedConcatenated Loops Loops Unstructured Unstructured
LoopsLoops
Simple Simple looploop
22
Loop Testing: Simple Loop Testing: Simple LoopsLoops
Minimum conditions—Simple LoopsMinimum conditions—Simple Loops1. skip the loop entirely1. skip the loop entirely
2. only one pass through the loop2. only one pass through the loop3. two passes through the loop3. two passes through the loop
4. m passes through the loop m < n4. m passes through the loop m < n5. (n-1), n, and (n+1) passes through 5. (n-1), n, and (n+1) passes through
the loopthe loopwhere n is the maximum number where n is the maximum number
of allowable passesof allowable passes
23
Loop Testing: Nested Loop Testing: Nested LoopsLoops
Start at the innermost loop. Set all outer loops to their Start at the innermost loop. Set all outer loops to their minimum iteration parameter values.minimum iteration parameter values.
Test the min+1, typical, max-1 and max for the Test the min+1, typical, max-1 and max for the innermost loop, while holding the outer loops at their innermost loop, while holding the outer loops at their
minimum values.minimum values.Move out one loop and set it up as in step 2, holding all Move out one loop and set it up as in step 2, holding all other loops at typical values. Continue this step until other loops at typical values. Continue this step until
the outermost loop has been tested.the outermost loop has been tested.
If the loops are independent of one another If the loops are independent of one another then treat each as a simple loopthen treat each as a simple loop else* treat as nested loopselse* treat as nested loops
endif* endif*
Nested LoopsNested Loops
Concatenated LoopsConcatenated Loops
24
Black-Box TestingBlack-Box Testing
requirementsrequirements
eventseventsinputinput
outputoutput
25
Black-Box TestingBlack-Box Testing Behavioral testingBehavioral testing Focus on the functional requirements of the Focus on the functional requirements of the
softwaresoftware Attempts to find errors in the following categoriesAttempts to find errors in the following categories
Incorrect/miss functionIncorrect/miss function Errors in data structures or external database accessErrors in data structures or external database access Behavior/performance errorsBehavior/performance errors Initialization/termination errorsInitialization/termination errors
26
Black-Box TestingBlack-Box TestingTests are designed to answer the following questionsTests are designed to answer the following questions
How is functional validity tested?How is functional validity tested? How is system behavior and performance tested?How is system behavior and performance tested? What classes of input will make good test cases?What classes of input will make good test cases? Is the system particularly sensitive to certain input Is the system particularly sensitive to certain input
values?values? How are the boundaries of a data class isolated?How are the boundaries of a data class isolated? What data rates and data volume can the system What data rates and data volume can the system
tolerate?tolerate? What effect will specific combinations of data have on What effect will specific combinations of data have on
system operation?system operation?
27
Graph-Based MethodsGraph-Based Methods
newfile
menu select generates(generation time 1.0 sec)
documentwindow
documenttex
t
is represented as
contains
Attributes:
background color: whitetext color: default color
or preferences
(b)
object#1
Directed link(link weight)
object#2
object#3
Undirected link
Parallel links
Node weight(value
)
(a)
allows editingof
To understand To understand the objects that the objects that are modeled in are modeled in software and software and the the relationships relationships that connect that connect these objectsthese objects
In this context, we In this context, we consider the term consider the term “objects” in the “objects” in the broadest possible broadest possible context. It context. It encompasses data encompasses data objects, traditional objects, traditional components components (modules), and (modules), and object-oriented object-oriented elements of elements of computer software.computer software.
28
Equivalence PartitioningEquivalence Partitioning Black-box testingBlack-box testing Divides the input domain of a program into Divides the input domain of a program into
classes of data from which test cases can be classes of data from which test cases can be derived.derived.
One or more tests are derived from each of the One or more tests are derived from each of the class. class.
IdeaIdea If one test fails, most likely all the tests in the domain If one test fails, most likely all the tests in the domain
will failwill fail
29
Equivalence PartitioningEquivalence Partitioning
useruserqueriesqueries mousemouse
pickspicks
outputoutputformatsformats
promptsprompts
FKFKinputinput
datadata
30
Sample Equivalence Sample Equivalence ClassesClasses
user supplied commandsuser supplied commandsresponses to system promptsresponses to system prompts
file namesfile namescomputational datacomputational data
physical parameters physical parameters bounding valuesbounding values initiation valuesinitiation valuesoutput data formattingoutput data formatting
responses to error messagesresponses to error messagesgraphical data (e.g., mouse picks)graphical data (e.g., mouse picks)
data outside bounds of the program data outside bounds of the program physically impossible dataphysically impossible data
proper value supplied in wrong placeproper value supplied in wrong place
Valid dataValid data
Invalid dataInvalid data
31
Boundary Value AnalysisBoundary Value Analysis A complement of equivalence partitioningA complement of equivalence partitioning Select the edges of the classSelect the edges of the class Include Include inputinput and and outputoutput
32
Boundary Value Boundary Value AnalysisAnalysis
useruserqueriesqueries mousemouse
pickspicks
outputoutputformatsformats
promptsprompts
FKFKinputinput
datadata
outputoutputdomaindomaininput domaininput domain
33
Boundary Value AnalysisBoundary Value AnalysisHow do I create BVA test cases?How do I create BVA test cases?
Input:Input: range a and b range a and b a, b, just above and just below a and ba, b, just above and just below a and b
Input:Input: a number of value a number of value Min, max, just above and just below min and maxMin, max, just above and just below min and max
Output:Output: Data structure has boundariesData structure has boundaries
arrayarray
34
Comparison TestingComparison Testing Used only in situations in which the reliability of Used only in situations in which the reliability of
software is absolutely critical (e.g., human-rated software is absolutely critical (e.g., human-rated systems)systems) Separate software engineering teams develop Separate software engineering teams develop
independent versions of an application using the same independent versions of an application using the same specificationspecification
Each version can be tested with the same test data to Each version can be tested with the same test data to ensure that all provide identical output ensure that all provide identical output
Then all versions are executed in parallel with real-time Then all versions are executed in parallel with real-time comparison of results to ensure consistencycomparison of results to ensure consistency
35
Orthogonal Array TestingOrthogonal Array Testing Used when the number of input parameters is
small and the values that each of the parameters may take are clearly bounded
One input item at a time L9 orthogonal array
XY
Z
XY
Z
36
OOT—Test Case OOT—Test Case DesignDesignImplications of OO conceptsImplications of OO concepts
1.1.EncapsulationEncapsulationa)a) Testing operation outside of the class is unproductiveTesting operation outside of the class is unproductiveb)b) Obstacle. Why? Information hideObstacle. Why? Information hide
2.2.InheritanceInheritancea)a) Each mew context of usage requires retesting, even Each mew context of usage requires retesting, even
though reuse has been achieved.though reuse has been achieved.3.3.Polymorphism - take on multiple formsPolymorphism - take on multiple forms
a)a) attribute have more than one set of values and an attribute have more than one set of values and an operation may be implemented by more than one operation may be implemented by more than one method. method.
b)b) Ex –abstract card classEx –abstract card class
37
Testing MethodsTesting Methods Fault-based testingFault-based testing
The tester looks for plausible faults (i.e., aspects of the implementation The tester looks for plausible faults (i.e., aspects of the implementation of the system that may result in defects). To determine whether these of the system that may result in defects). To determine whether these faults exist, test cases are designed to exercise the design or code. faults exist, test cases are designed to exercise the design or code.
Integration testing-three types in the client (Integration testing-three types in the client (not servernot server)) Unexpected resultUnexpected result Wrong operation/message usedWrong operation/message used Incorrect invocationIncorrect invocation
Class Testing and the Class HierarchyClass Testing and the Class Hierarchy Inheritance does not obviate the need for thorough testing of all Inheritance does not obviate the need for thorough testing of all
derived classes. In fact, it can actually complicate the testing process.derived classes. In fact, it can actually complicate the testing process. Scenario-Based Test DesignScenario-Based Test Design
Scenario-based testing concentrates on what the user does, not what Scenario-based testing concentrates on what the user does, not what the product does. This means capturing the tasks (via use-cases) that the product does. This means capturing the tasks (via use-cases) that the user has to perform, then applying them and their variants as tests.the user has to perform, then applying them and their variants as tests.
38
OOT Methods: Random OOT Methods: Random TestingTesting
Random testingRandom testing identify operations applicable to a classidentify operations applicable to a class define constraints on their usedefine constraints on their use identify a minimum test sequenceidentify a minimum test sequence
an operation sequence that defines the minimum life history an operation sequence that defines the minimum life history of the class (object)of the class (object)
generate a variety of random (but valid) test sequencesgenerate a variety of random (but valid) test sequences exercise other (more complex) class instance life historiesexercise other (more complex) class instance life histories
ExampleExample AccountAccount Open->deposit->withdraw->closeOpen->deposit->withdraw->close
39
OOT Methods: Partition OOT Methods: Partition TestingTesting Partition TestingPartition Testing
reduces the number of test cases required to test a class in much reduces the number of test cases required to test a class in much the same way as equivalence partitioning for conventional softwarethe same way as equivalence partitioning for conventional software
state-based partitioningstate-based partitioning categorize and test operations based on their ability to change the state categorize and test operations based on their ability to change the state
of a classof a class AccountAccount
Operations changes the state (withdraw, deposit)Operations changes the state (withdraw, deposit) Operations do not change the state (balance, report)Operations do not change the state (balance, report)
attribute-based partitioningattribute-based partitioning categorize and test operations based on the attributes that they usecategorize and test operations based on the attributes that they use
Balance, credit limitBalance, credit limit category-based partitioningcategory-based partitioning
categorize and test operations based on the generic function each categorize and test operations based on the generic function each performsperforms
AccountAccount Init: open, setupInit: open, setup Computation: deposit, withdrawComputation: deposit, withdraw Query: balance, summarizeQuery: balance, summarize
40
OOT Methods: Inter-Class OOT Methods: Inter-Class TestingTesting Inter-class testingInter-class testing
For each client class, use the list of class operators to For each client class, use the list of class operators to generate a series of random test sequences. The generate a series of random test sequences. The operators will send messages to other server classes.operators will send messages to other server classes.
For each message that is generated, determine the For each message that is generated, determine the collaborator class and the corresponding operator in collaborator class and the corresponding operator in the server object.the server object.
For each operator in the server object (that has been For each operator in the server object (that has been invoked by messages sent from the client object), invoked by messages sent from the client object), determine the messages that it transmits.determine the messages that it transmits.
For each of the messages, determine the next level of For each of the messages, determine the next level of operators that are invoked and incorporate these into operators that are invoked and incorporate these into the test sequencethe test sequence
41
OOT Methods: Behavior TestingOOT Methods: Behavior Testingemptyacctopen setup Accnt
set upacct
deposit(initial)
workingacct
withdrawal(final)
deadacct close
nonworkingacct
deposit
withdrawbalance
creditaccntInfo
Figure 14.3 State diagram for Account class (adapted from [ KIR94])
The tests to be The tests to be designed designed should achieve should achieve all state all state coverage. That coverage. That is, the is, the operation operation sequences sequences should cause should cause the Account the Account class to make class to make transition transition through all through all allowable allowable statesstates
42
Testing Testing PatternsPatterns
Pattern name:Pattern name: pair testing pair testingAbstract: Abstract: A process-oriented pattern, pair testing describes a A process-oriented pattern, pair testing describes a technique that is analogous to pair programming in which two technique that is analogous to pair programming in which two testers work together to design and execute a series of tests that testers work together to design and execute a series of tests that can be applied to unit, integration or validation testing activities.can be applied to unit, integration or validation testing activities.
Pattern name: Pattern name: separate test interface separate test interfaceAbstract: Abstract: There is a need to test every class in an object-oriented There is a need to test every class in an object-oriented system, including “internal classes” (i.e., classes that do not system, including “internal classes” (i.e., classes that do not expose any interface outside of the component that used them). expose any interface outside of the component that used them). The separate test interface pattern describes how to create “a The separate test interface pattern describes how to create “a test interface that can be used to describe specific tests on test interface that can be used to describe specific tests on classes that are visible only internally to a component.” classes that are visible only internally to a component.”
Pattern name: Pattern name: scenario testing scenario testingAbstract: Abstract: Once unit and integration tests have been conducted, Once unit and integration tests have been conducted, there is a need to determine whether the software will perform in there is a need to determine whether the software will perform in a manner that satisfies users. The scenario testing pattern a manner that satisfies users. The scenario testing pattern describes a technique for exercising the software from the user’s describes a technique for exercising the software from the user’s point of view. A failure at this level indicates that the software point of view. A failure at this level indicates that the software has failed to meet a user visible requirement.has failed to meet a user visible requirement.
Question & Review session
For any query feel free to contact For any query feel free to contact fahad@uettaxila.edu.pk
http://web.uettaxila.edu.pk/CMS/seSE2bs/index.asp
Recommended