CSE 7314 Software Testing and Reliability Robert Oshana Lecture 20 oshana@airmail.net

CSE 7314Software Testing and Reliability

Robert Oshana

Lecture 20

oshana@airmail.net

Introduction

Chapter 1

Introduction

• Testing - Challenging but understood– Unit, integration, system, acceptance

• Testing OO systems– “the same, only different” ;-)

• We’ll talk about testing OO systems throughout the development cycle

• Testing is the process of uncovering evidence of defects in software systems

Introduction

• Bug; a mistake, misunderstanding, omission, or misguided intent on the part of the developer

• Testing is the process of finding defects

• Testing does NOT include efforts to track down bugs and fix them (e.g. debugging or repair of bugs)

Introduction

• Testing; making sure an application does everything it is supposed to do and nothing that it is not supposed to do

• Guards against time, property, customers, life

What is software?

• Instruction codes and data necessary to accomplish a task on a computer

• Also all representations of those instructions and data– Models created during analysis/design

What is software?

• Testing is not quality assurance

• Testing is a necessary but insufficient part of a quality assurance program

• QA; prevention and well as removal

• Testing will not necessarily improve the quality of a computer program

Testing OO systems

• OO features impact testing– Class inheritance– Interfaces/polymorphism– Language features to support data hiding

impact testing• Operation must be added just to support testing• Can also support more reusable testing software

• Many approaches have counterparts in traditional testing

Testing OO systems

• With an OO approach, a program’s design is structured from the problem, and not from the immediately required solution

• Analysis models map straightforward to design models that map to code

• Can therefore start testing during analysis, and refine during design and implementation

Advantages to testing analysis and design models• Test cases can be identified earlier in

the process• Bugs can be detected early in the

development process• Test cases can be reviewed early in

the process for correctness• Testing can also be used to check

designs for reusability and scalability

Overview of testing approach

• Time and money constraints must be real concern

• Test early, test often, test enough– Analyze a little– Design a little– Code a little– Test what you can

Testing for OO software

• Model testing• Class testing, which replaces unit

testing• Interaction testing, which replaces

integration testing• System (and subsystem) testing• Acceptance testing• Deployment/self testing

Test early

• Start testing at reasonable points during the analysis and design phases of a project

• Can uncover problems when they are easier and cheaper to fix

• Can help scope the size of the effort for system testing

• SW representations are abstract and/or incomplete

Test often

• Incremental or iterative development should require testing at each increment

• After first increment, some testing is in the form of regression testing

Test enough

• Complete testing of every aspect of a software system is infeasible

• Resources should be directed where they have the greatest impact

• Tests should be based in risk analysis, reuse of test cases, and statistical sampling of inputs for test cases

The testing perspective

Testing perspective

• A way of looking at the development product and questioning its validity

• Search for faults both systematic as well as intuitive insight

• Reviews; almost never find something that is missing– Validates what exists– No systematic search to determine things that

should be in are actually in

Testing perspective

• Software must execute according to spec and only that spec– Make sure it does what it is supposed to

do– Does not do what it is not supposed to

do

• Can be performed by same developer or independent view

Review, inspection, execution

• Inspection; examination of SW based on checklist of typical problems– Most based on programming language

semantics or coding conventions• Program variables initialized• Pointers or references set correctly

– Modern compilers can find many of these

Review, inspection, execution

• Review; examination of SW to find faults before the SW is executed• Delves into the meaning of each part of

a program• Missed or misunderstood requirements

or faults in the program’s logic• Whether variables are well chosen• Whether algorithms are as efficient as

they could be

Review, inspection, execution

• Test execution; testing SW in the context of a running program• Tester tries to determine whether it has

the required behavior by giving the program some input and verifying that the resulting output is correct

• Challenge is to identify suitable inputs, determining correct outputs, and how to observe the outputs

Testing perspective

• Skeptical; what proof of quality

• Objective; make no assumptions

• Thorough; don’t miss important areas

• Systematic; searches are reproducible

Object-Oriented Concepts

• Object

• Message

• Interface

• Class

• Inheritance

• Polymorphism

Object

• An operational entity that encapsulates both specific data values and the code that manipulates those values

• Basic computational entity• Objects are created, modified,

accessed and/or destroyed as a result of collaborations

Object

• A representation of some specific entity in a program or in its solution

• Objects are the direct target of the testing process– Behaves according to specification– Interacts appropriately with

collaborating objects

Observations

• Objects encapsulate; makes the complete definition of the object easy to identify; easy to pass around in the system; and easy to manipulate

• Objects hide information; changes to the object sometimes hard to observe, makes checking of test results difficult

Observations

• Object has a state that persists for the life of the object; state can become inconsistent and can be the source of incorrect behavior

• Object has a lifetime; can be examined at various points in the lifetime to determine if in appropriate state based on lifetime (construction too late or destruction too early are common sources of failures)

Message

• A request that an operation be performed by some object

• Includes – Name of the operation– Actual parameters

• Collaboration achieved by sending messages– Sender and receiver

Observations

• Message has a sender; sender determines when to send the message (may make incorrect decision)

• Message has a receiver; may not be ready for the specific message it receives, or receiver may not make correct action when receiving unexpected message

Observations

• Message may include actual parameters; used and/or updated by the receiver while processing the message

• Objects passed as parameters must be in correct states before and after message is processed

• Must implement interfaces expected by the receiver

Object-Oriented concepts

• Interface; aggregation of behavioral declarations

• Grouped to define actions related to a single concept

• A building block for specifications which define to total set of public behaviors for a class

Interface example

public interface Movable {public Point getPosition( );public Velocity getVelocity( );public void setVelocity(Velocity newVelocity);public void tick( );public void move( );public void collideWith(ArcadeGamePiece aPiece, Point aPoint);public void collideWithPaddle(Paddle aPaddle, Point aPoint);public void collideWithPuck(Puck aPuck, Point aPoint);public void reverseY( );public void reverseX( );

}

Observations

• Interfaces encapsulate operation specifications

• These specifications incrementally build the specifications of larger grouping such as classes

• I/F must contain appropriate behaviors or leads to unsatisfactory designs

• I/F has relationships with other I/Fs and classes and may be specified as a parameter type for a behavior

Object-Oriented concepts

• Class; a set of objects that share a common conceptual basis

• A template or “cookie cutter” to create objects

• Classes are the basic elements for defining OO programs while objects are the basic elements for executing OO programs

• Create by instantiation

Object-Oriented concepts

• Class specification; declaration of what each of the objects can do

• Class implementation; definition of how each of the objects in the class do what they can do

Example class specification#ifndef PUCKSUPPLY_H#define PUCKSUPPLY_HClass PuckSupply {Public:

PuckSupply( );~PuckSupply( );

Puck* get( );int size( ) const;

Private:static const int N = 3;int_count;Puck* _store[N];

};#endif

WHAT …

Example class implementation

#include “PuckSupply.h”PuckSupply::PuckSupply( ) : _count(N) {

int i;for ( i=0; i<N; i++) {

_store[ i ] = new Puck;};

}PuckSupply:: int i;

for ( i=0; i<N; i++) {delete _store[ i ];

};}

Puck* PuckSupply::get( ) {return (_count > 0 ? _store[--count] : 0 );

}…..

HOW …

Class specification

• Must be a specification for each operation– Accessor (inspector) operations– Modifier (mutator) operations– Constructor– Destructor

Operation semantics

• Well specified operation semantics are critical to both development and testing– Preconditions– Postconditions– Invariants

Approaches to defining an interface

• Contract approach; emphasizes preconditions and has simpler postconditions

• Defensive programming; emphasizes postconditions and has simpler preconditions (I/F defined in terms of the receiver)– Goal is to define “garbage in” and

eliminate “garbage out”

Class implementation

• A class implementation describes how an object represents its attributes and carries out operations– Set of data values stored in data members– Set of methods– Constructors– Destructors– Private operations

Class testing

• Class testing is important because they define the building blocks for OO programs

• Potential causes of failures– Class spec contains operations to

construct instances that may not properly initialize attributes of new instances

Class testing

– Class relies on collaboration with other classes that may not be implemented correctly and contribute to the failure of the class using the definition

– Class implementation satisfies the spec but the spec may be wrong

– Implementation might not support all required operations or may incorrectly perform operations

Class testing

– Class specifies preconditions to each operation but the class may not provide a way for the precondition to be checked by a sender before sending a message

• The design approach used (contract or defensive) gives rise to different sets of problems

A word on abstraction

• The process of removing detail from a representation

• Allows us to look at a problem or its solution in various levels of detail

• Can leave out any details that are irrelevant to the current level of interest

• OO technologies make extensive use of this

A word on abstraction

• From a testing perspective, layers of abstraction in the development process are paralleled by layers of test analysis

• Start with the highest levels of abstraction and provide a more thorough examination of the development product with more accurate tests

Inheritance

• This is a relationship between classes that allows the definition of a new class based on the definition of an existing class– Reuse of specification and

implementation– Preexisting class does not have to be

modified

Inheritance

• Good OO design uses inheritance for is a (is a kind of) relationship

• Best use is with respect to specification and implementation

Inheritance from a testing perspective

• Provides a mechanism by which bugs can be propagated from a class to its descendents – Testing a class as it is developed eliminates

faults early before they are passed to other classes

• Provides a mechanism to reuse test cases– Can reuse test cases from superclass to test

subclass

Inheritance from a testing perspective

• Models an is a kind of relationship– Inheritance solely for code reuse leads

to maintenance difficulties– Testers can check to make sure

inheritance is used correctly

Polymorphism

• The ability to treat an object as belonging to more than one type

• More typing systems support designs that are flexible and easy to maintain

Inclusion polymorphism

• The occurrence of different forms in the same class

• Provides the ability to substitute an object whose spec matches another object’s spec for the latter object in a request for an operation– Sender can use an object as a

parameter based on its implementation of an I/F rather than its full class

Set diagram for a Brick inheritance hierarchy

Brick

HardBrick PowerBrick

Abstract class

• A class whose purpose is primarily to define an interface that is supported by all of its descendents

• Use of abstractions allows polymorphism to be exploited during design

Set diagram for a Brick class inheritance hierarchy

Brick

HardBrick PowerBrick

PlainBrick

10 minute break

CSE 7314Software Testing and Reliability

Robert Oshana

Lecture 21

oshana@airmail.net

Inclusion polymorphism testing perspective

• Inclusion polymorphism allows systems to be extended incrementally by adding classes rather than modifying existing ones– Unanticipated interactions can occur in

the extensions

Inclusion polymorphism testing perspective

• Inclusion polymorphism allows any operation to have one or more parameters of a polymorphic reference• This increases the number of possible

kinds of actual parameters that should be used

Inclusion polymorphism testing perspective

• Inclusion polymorphism allows an operation to specify replies that are polymorphic references• The actual class of the referent could be

incorrect or unanticipated by the sender

Inclusion polymorphism testing perspective

• The dynamic nature of object-oriented languages places more importance on testing a representative sample of runtime configurations• Static analysis can provide the potential

interactions that might occur, but only runtime configuration can illustrate what actually happens

Parametric polymorphism

• The capability to define a type in terms of one or more parameters

• Templates in C++ provide a compile time capability to instantiate a “new” class– Actual parameter is provided for the

formal parameter in the definition

C++ List class template

template<class ItemType, class Key>class List{public:

void add(ItemType* item);ItemType* retrieve(Key searchValue);

}

Testing perspective

• Parametric polymorphism supports a different type of relationship from inheritance

• If the template works for one instantiation, there is no guarantee that it will work for another!– Template code might assume the correct

implementations of operations such as destructors

– Should be checked during inspection

Development products

Chapter 2

Development products

• Good documentation is critical for successful development and testing

• A collection of work products that represent the system and the requirements

• UML is an example of a conceptual modeling language

• C++ is the programming language

SW end products

• Code and documentation– User manuals– Maintenance documents

• Analysis and design models• Architectural models• Quality requirements

• UML is used to generate these models

Analysis models

Analysis models

• Purpose of analysis is to define the problem to be solved and to determine requirements for a solution to that problem– Domain analysis; focuses on

understanding the problem– Application analysis; focus is on

specific problem and requirements for a solution (more concrete concepts)

Analysis models

• Represents a system from the perspective of what it is supposed to do

• Provides an understanding of the problem and requirements

Use case diagram

• A use case describes a use of the system by an actor to perform some task

• Actors represent roles users play with respect to the system

• Can be expressed at various levels of abstraction

Domain level use cases for arcade games

Name Description

Start a player starts a match

Pause a player pauses a match

Resume a player resumes playing a match

Stop a player stops a match

Application level use cases for Brickles arcade game

Name Description

Start Brickles a player starts playing Brickles by starting the application program under Windows

Pause Brickles a player pauses a Brickles match by pressingthe mouse button

Resume Brickles a player resumes a Brickles match by releasing the mouse button

Move Paddle a player moves the mouse right or left to movethe paddle left or right

Use case hierarchy

• Two hierarchy relationships– Uses; to create more specific use cases– Extends; helps to organize a potentially large

number of use cases

• Use cases represent requirements not software

• Scenario shows a particular application or instantiation of a use case

Use cases for Brickles

Play Brickles

Pause Brickles

Stop Brickles

Break Brick

Move Paddle

Lose Paddle

Win

Lose

<<extends>>

<<extends>>

<<extends>>

<<extends>>

<<extends>>

Class diagrams

• Presents a static view of a set of classes and the relationships between the classes

• Operations, attributes, constraints on relationships shown

Applicationanalysis classdiagramforBrickles

Package diagram

• Groups of classes can be represented as a package

• Allows easy reuse in the future

• Dependencies can be shown

Package diagram

State diagrams

• Describes the behavior of the objects in a class in terms of its observable states and how the object changes states as a result of events that affect the object– State; particular configuration of the

values of data attributes– Transition; a change from one state to

another

State diagrams

– Event; a message arrival– Guard; a condition that must hold

before the transition can be made– Action; specifies processing to be done

while a transition is in progress– Concurrent state diagram can show

groups of states that reflect the behavior of an object from the perspective of two or more independent ways

State diagram for class Timer

Class specification

• An approach to detail the semantics associated with each operation

• Object Constraint Language (OCL) is a specification technique used for this purpose

• Constraints involve– Attributes– Operations– Associations

OCL for operations of PuckSupply

PuckSupplysize >= 0 and size <= 3

PuckSupply:: PuckSupply( )pre: - nonepost: size = 3 AND pucks->forAll( puck: Puck | not puck.inPlay( ) )PuckSupply:: ~PuckSupply( )pre: - nonepost: Puck->size( ) = Puck@pre->size( ) – size@prevoid PuckSupply:: size( ) constpre: - nonepost: result = sizePuck * PuckSupply::get( )pre: count > 0post: result = pucks->asSequence->first AND size = size@pre - 1

Sequence diagram

• Captures the messaging between objects, object creation, and replies from messages

• Illustrate process in domain– How common tasks are carried out through the

interaction of objects in a scenario

• Various levels of abstraction• Can represent concurrency

Sequence diagrams for Brickles

Activity diagram

• Where sequence diagrams capture single traces through a set of object interactions, it is difficult to represent iteration and concurrency

• Activity diagram provides a more comprehensive representation– Flow chart– Petri nets

Activity diagram for move() method in Puck

Activity diagram

Design models

Design models

• Represent how the software meets requirements

• Refinements and extensions of the analysis models

• Means we can reuse and extend test cases developed for analysis models

• Focus on solution rather than the problem

Class diagram

• Depicts the kinds of objects that will be created by the software

• Name, attributes, operations, relationships

• Adds detail to the analysis class diagram

• Maintains most of the structure and adds detail for the solution

Design classdiagram forBrickles

State diagram

• Same as those used in analysis

• Add state diagrams for new classes in the design class diagram and potentially new substates

• More actions and activities may be shown

State diagram

• Should ensure test cases for a class that adequately transitions between states and provides proper processing of messages between states

• Possible to reuse some test cases and test drivers that were developed for testing other classes whose design is based on the same pattern

State diagram for class Timer

Sequence diagrams

• Used in design to describe algorithms• Difference from analysis is the presence

of solution level objects in the design diagram

• Testing perspective; possible errors are violation of contracts, failure to create objects of the correct class, sending of messages for which no navigation is indicated between sender and receiver

Sequence diagram for start-up for a Brickles match

Source code

• Source code is the final representation for software

• Translator makes software executable

• Should be an accurate translation from design models but must be tested

Source code

• Testing actual code is traditional focus

• Test as developed as well as completed product

• Major issues include– Who tests?– What is tested?

Source code

• Major issues include– When testing is done?– How testing is done?– How much testing is done?

Testing and programming language

• Programming language has an impact on testing– Enabling of certain types of error– Strong vs weak typing implies more or

less errors caught by compiler– Pointer and memory problems– Data hiding– Interface testing

End of lecture