Object-Oriented Testing
Presented by:
Mr. Apolonio Cabangal Jr.
ObjectivesTo discuss when testing takes place in the life
cycle.
To test the correctness and consistency of OOD and OOA models.
To cover the strategies and tools associated with object oriented testing
Analysis and Design Testing Class Tests Integration Tests Validation Tests System Tests
Why OOT is important?
?design codeanalysis test
Object Oriented Testing ObjectivesFind the greatest possible number of
errors with a manageable amount of effort applied over a realistic time span.
The nature of OO programs changes both testing strategy and testing tactics.
Do we need less efforts for testing because of greater reuse of design patterns?
Answer is no! Binder argues that more testing is needed to obtain high reliability in OO systems, since each reuse is a new context of usage.
Why OO Applications are Different?No sequential procedural
executionsNo functional decomposition No structure charts to design
integration testingIterative O-O development and its
impact on testing and integration strategies
Object-Oriented Testing When should testing begin? Analysis and Design:
Testing begins by evaluating the OOA and OOD models How do we test OOA models (requirements and use
cases)? How do we test OOD models (class and sequence
diagrams)? Structured walk-throughs, prototypes Formal reviews of correctness, completeness and
consistency Programming:
How does OO make testing different from procedural programming?
Concept of a ‘unit’ broadens due to class encapsulation Integration focuses on classes and their execution across
a ‘thread’ or in the context of a use case scenario Validation may still use conventional black box methods
Criteria for Completion of TestingWhen are we done testing? (Are we there yet?)How to answer this question is still a research
question1.One view: testing is never done… the burden simply
shifts from the developer to the customer2.Or: testing is done when you run out of time or
money3.Or use a statistical model:
Assume that errors decay logarithmically with testing time Measure the number of errors in a unit period Fit these measurements to a logarithmic curve Can then say: “with our experimentally valid statistical model
we have done sufficient testing to say that with 95% confidence the probability of 1000 CPU hours of failure free operation is at least 0.995”
Strategic IssuesIssues to address for a successful software testing
strategy: Specify product requirements long before testing
commences For example: portability, maintainability, usabilityDo so in a manner that is unambiguous and quantifiable
Understand the users of the software, with use cases Develop a testing plan that emphasizes “rapid cycle
testing” Get quick feedback from a series of small incremental tests
Build robust software that is designed to test itself Use assertions, exception handling and automated testing tools (Junit).
Conduct formal technical reviews to assess test strategy & test cases “Who watches the watchers?”
Testing OOA and OOD Models (1)
The review of OO analysis and design models is especially useful because the same semantic constructs (e.g., classes, attributes, operations, messages) appear at the analysis, design, and code level.
Therefore, a problem in the definition of class attributes that is uncovered during analysis will circumvent side effects that might occur if the problem were not discovered until design or code (or even the next iteration of analysis).
By fixing the number of attributes of a class during the first iteration of OOA, the following problems may be avoided: Creation of unnecessary subclasses. Incorrect class relationships. Improper behavior of the system or its classes.
If the error is not uncovered during analysis and propagated further more efforts needed during design or coding stages.
Testing OOA and OOD Models (2) Analysis and design models cannot be tested in the conventional
sense, because they cannot be executed. Formal technical review can be used to examine the correctness
and consistency of both analysis and design models. Correctness:
Syntax: Each model is reviewed to ensure that proper modeling conventions have been maintained.
Semantic: Must be judged based on the model’s conformance to the real world problem domain by domain experts.
Consistency: May be judged by considering the relationship among entities in
the model. Each class and its connections to other classes should be
examined. The Class-responsibility-collaboration model and object-
relationship diagram can be used.
Testing ModelsCriteria
CorrectnessCompletenessConsistency
Early informal models are tested informally
The criteria should be interpretive in the context of iterative incremental approach
Model Testing ApproachesTesting by comparison
compares each model to its predecessor or to previous forms of the model
Testing by inspectionuses checklists to make sure that the
model meets certain criteriaTesting by verification
follows certain steps to assure completeness and consistency of one part of the model with another
Examples of Analysis and Design Models to be TestedCRC cardsClass specificationsUse casesState-Transition Models
State transition diagrams for classes, clusters, and subsystems
Object networkMessage sequence between methods in
classes Transaction-Flow Models
Testing the Class Model1.Revisit the Use Cases, CRC cards and UML class
model. Check that all collaborations are properly represented. Inspect the description of each CRC index card to determine if a delegated responsibility is part of the collaborator’s definition
Example: in a point of sale system. A read credit card responsibility of a credit sale class is
accomplished if satisfied by a credit card collaborator2. Invert connections to ensure that each collaborator
asked for a service is receiving requests from a reasonable source
Example: a credit card being asked for a purchase amountHave you tested your analysis and design?
If not, who will do it?
3. These steps are applied iteratively to each class are applied iteratively to each class and through each evolution of the OOA model.and through each evolution of the OOA model.
Testing OO CodeClass/Unit testsClass/Unit tests IntegrationIntegration
teststests
ValidationValidationteststests
SystemSystemteststests
The Structured Testing Pyramid
Requirement Definition
Preliminary Design
Detailed Design
Coding
Integration Testing
Unit Testing
Systems Testing
The OO Testing PyramidUse Case Analysis
Class Hierarchy Design
Method Design
Method Coding
Cluster (Integration) Testing
Method Testing
Use Case Testing
Class TestingTest OOD
Test OOA
Test OOD
Unit TestWhat is a unit?
A single, cohesive function?A function whose code fits on one page?The amount of code that can be written in 4
to 40 hours?Code that is assigned to one person?
We can no longer test a single operation in isolation but rather as part of a class.
In object-oriented programs, a unit is a method within a class.
Smallest testable unit is the encapsulated class
Class Testing Process
test casestest cases
resultsresults
softwaresoftwareengineerengineer
How to test?How to test?
Why a Why a loop?loop?
classclassto beto be
testedtested
Methods for Generating Test CasesFor Unit Testing Statement coverageGraph based
Branch coverageCondition coveragePath coverage
All unit testing methods are also applicable to testing methods within a class.
Class Test Case DesignBerard [BER93,94] proposes the following approach:1. Identify each test case uniquely
- Associate test case explicitly with the class and/or method to be tested
2. State the purpose of the test 3. Each test case should contain:
a. list of specified states for the object that is to be testedb. A list of messages and operations that will be exercised as a
consequence of the testc. A list of exceptions that may occur as the object is testedd. A list of external conditions for setup (i.e., changes in the
environment external to the software that must exist in order to properly conduct the test)
e. Supplementary information that will aid in understanding or implementing the test
Automated unit testing tools facilitate these requirements
Challenges of Class Testing OO class is the target for test case design. Encapsulation:
Difficult to obtain a snapshot of a class without building extra methods which display the classes’ state
Inheritance and polymorphism: Each new context of use (subclass) requires re-testing because
a method may be implemented differently (polymorphism). Other unaltered methods within the subclass may use the
redefined method and need to be tested How is class testing different from conventional testing?
- Conventional testing focuses on input-process-output, whereas class testing focuses on each method, then designing sequences of methods to exercise states of a class
White box tests: Basis path, condition, data flow and loop tests can all apply to
individual methods, but don’t test interactions between methods
Notes on OO Testing 1. Applicability of conventional test case design:
Although white-box testing is useful, the efforts might be redirected to tests at a class level.
Use cases can provide useful input in the design of black-box and state-based tests.
Fault-based testing: (Tests have high likelihood of uncovering faults)Its planning begins with the analysis model.If real faults in an OO system are perceived to be
implausible then this approach is really no better than any random testing technique.
If the analysis and design models can provide insight into what is likely to go wrong, then fault-based testing can find significant numbers of errors with relatively low efforts.
Notes on OO Testing 2.Impact of OO programming on Testing:
When an operation is invoked, it may be hard to tell exactly what code gets exercised.
It can be hard to determine the exact type or class of a parameter.
OO operations are smaller, more time needed for integration. So integration faults become more plausible.
In 3 large scale studies: 50% of methods are less than 2 C++ statements or 4 Smalltalk statements (Law of Demeter connection)
Scenario-based test design:Concentrates on what the user does, not what the product
does.Capturing the tasks via use-cases that the user has to
perform.Testing surface structure and deep structure
Surface structure: externally observable structure (end-user).
Deep structure: internal technical details of an OO program.
OOT Methods: Class Level 1 Random testing
identify operations applicable to a class define constraints on their use identify a minimum test sequence
an operation sequence that defines the minimum life history of the class (object)
generate a variety of random (but valid) test sequences exercise other (more complex) class instance life
histories Example:
Class: Account Operations: open, setup, deposit, withdraw, balance,
summarize, creditlimit, close.1. Open – setup – deposit – withdraw – close2. Open – setup – deposit –[deposit | withdraw | balance |
summarize] * – withdraw – close. Generate random test sequences using this template
OOT Methods: Class Level 2 Partition Testing
reduces the number of test cases required to test a class in much the same way as equivalence partitioning for conventional software
state-based partitioningcategorize and test operations based on their ability to
change the state of a class (e.g.: deposit, withdraw)attribute-based partitioning
categorize and test operations based on the attributes that they use ( e.g.: creditlimit attribute)
category-based partitioningcategorize and test operations based on the generic
function each performs. (e.g.: (Init OP: open , setup) (Comp. OP: deposit, withdraw) (queries: balance, summarize, creditlimit) (Termination OP: close))
What Methods to Test Within ClassesNew methods: defined in the class under
test and not inherited or overloaded by methods in a superclass: Complete testing
Inherited methods: defined in a superclass of the class under test:Retest only if the methods interacts with new or redefined method.
Redefined methods: defined in a superclass of but redefined in the class under test : complete Retest reusing tests from the superclass.
Class Testing TechniquesIn addition to testing methods within a class (either glass box or black box), the following three techniques can be used to perform functional testing for the class:State-Transition testingTransaction-Flow testingException testing
State-Transition TestingA state-transition model describes the
different states and transitions of a class in the context of its position in the inheritance hierarchy.
(OMT dynamic model)The state of an object is the combination of all
the attribute values and objects that the object contains.
An object may transition from a state to state as a result of an event, which may yield an action.
Example
Prospect
Retired-member Life-member
Receive application
Establish-membership
Receive cancellation
Receive cancellation5-years anniversary
Member
State-Transition TestingCreate test cases corresponding to each
transition path that represent a full object life cycle
Make sure each transition is exercised at least once.
[2] Integration TestingOO does not have a hierarchical control
structure so conventional top-down and bottom-up integration tests have little meaning
Integrating operations one at a time into a class is often impossible because of the direct and indirect interactions of the components that make up the class.
Integration applied three different incremental strategies: Thread-based testing: integrates classes required to
respond to one input or event Use-based testing: integrates classes required by one
use case Cluster testing: integrates classes required to
demonstrate one collaboration
- What integration testing strategies will you use?
Cluster (Integration) Testing:Why is it Different?Event-Driven executionNo functional decompositionThe sequence of execution is not knownIntegration testing must be driven by how
object will behave dynamicallyObject composition introduces a new
dimension of integration testing
Random Integration TestingMultiple Class Random Testing
1.For each client class, use the list of class methods to generate a series of random test sequences. Methods will send messages to other server classes.
2.For each message that is generated, determine the collaborating class and the corresponding method in the server object.
3.For each method in the server object (that has been invoked by messages sent from the client object), determine the messages that it transmits
4.For each of the messages, determine the next level of methods that are invoked and incorporate these into the test sequence
Cluster (Integration) TestingA cluster is a collection of classes (possibly from
different systems) cooperating with each other via messaging.
A cluster specification should include methods from each class that will be accessed
Cluster testing focuses on the interaction among the instances of the classes in the cluster
It assumes that each class has been tested individually
Cluster testing is considered a second level of integration testing
Methods for Forming ClustersFunction-based clustering
Based on requirements and use casesDifficult to perform if requirements were not
available during the design phase Subject-based clustering
Based on subject areas that need to test separately
Project Schedule-based clusteringContract-based clustering
Types of Errors Found During Integration TestingMessaging errors:
Failure to meet a requirement, i.e., no method to send or receive a message
Incompatible method and message in sender and receiver
Incorrect event timing between object actionsIncorrect instantiation or destruction of objects
User interface errors:A given sequence of user actions does not have
the expected effect on the component.The timing of events received from the user
results in incorrect functioning of the component
Techniques for Object-Oriented Integration Testing
Message QuiescenceEvent Quiescence
Message QuiescenceA Method/Message path (MM-Path) is a
sequence of method executions linked by messages.
An MM-Path starts with a method and ends when it reaches a method that does not issue a message of its own, i.e., reaches a message Quiescence.
Event QuiescenceAn input port event followed by a set of
MM-Paths, and terminated by an output event is called Atomic System Function (ASF).
meth1
meth3meth2
Class 1
INPUT PORT EVENT A
B
ASF INPUT PORT EVENT
OUTPUT PORT EVENTA
meth1 meth3
meth2
meth2
meth1
B
ASF OUTPUT PORT EVENTClass 2
Class 3
MM-Path
Message
1
2
3
ATM PIN EntryCustomer enters card (event) Screen requesting PIN entry is displayedAn interleaved sequence of digit key
toughs with audible and visual feedbackPossibility of cancellation by customerSystem disposition (valid PIN or card
retained)
BANKCARDSLOT
SECURITY
SCREEN
Keypad
SpecialKeypad
NumKeypad
getKeyEvents
parseKeyEvent
showMessage
pinForPan
checkPin
Screen
memberCard
ValidateCard
CardSlot
Bank
NumKeypadSecurity
Customer inserts card
ASF Starts here
Message is displayed
ASF ends here
Key pushers
[3] Validation TestingAre we building the right product? Validation succeeds when software functions in a
manner that can be reasonably expected by the customer.
Focus on user-visible actions and user-recognizable outputs
Details of class connections disappear at this levelApply:
Use-case scenarios from the software requirements spec Black-box testing to create a deficiency list Acceptance tests through alpha (at developer’s site) and beta
(at customer’s site) testing with actual customersHow will you validate your software product?
[4] System Testing Software may be part of a larger system. This
often leads to “finger pointing” by other system dev teams
Finger pointing defence:1. Design error-handling paths that test external
information2. Conduct a series of tests that simulate bad data3. Record the results of tests to use as evidence
Types of System Testing: Recovery testing: how well and quickly does the system
recover from faults Security testing: verify that protection mechanisms built
into the system will protect from unauthorized access (hackers, disgruntled employees, fraudsters)
Stress testing: place abnormal load on the system Performance testing: investigate the run-time
performance within the context of an integrated system.
System TestingAll rules and methods of traditional
systems testing are also applicable applicable to object-oriented.
A thread can be “re-defined” as a sequence of method executions linked by messages in the object network.
Use cases provide a more precise mechanism for writing test cases.
ExampleUse Case # 1 of the Telephone Banking System:
Task: Making a balance transferUser Class: Current customerFrequency: one per week per customer
User Action System Response
User dials the number System plays greeting and ask for account number
User enters invalid account number System informs user and ask for account number again
User enter a valid account number System asks for PIN#
Automated TestingJunit at Junit.org CPPUnit on SourceForge.net, NUnit for .NetRational Robot
OO Testing Methodology
A Method Unit testing
Message Quiescence Integration
Event Quiescence Integration
Thread testing System
Thread interaction System
Jorgensen and Erickson propose 5 levels:
Stages of Test [RUP] Developer Testing Unit Test Integration Test System Test Acceptance Test
Test Planning [RUP]What iteration you are you in?What stage of test (unit test, integration test,
system test) you are performing?Types of test (functional, stress, volume,
performance, usability, distribution, and so on). Evaluation criteria used (code-based test
coverage, requirements-based test coverage, number of defects, mean time between failure, and so on.)
Testing techniques used (manual and automated)
Completion Criteria [RUP]
Manual/Automated Testing [RUP]
The END