Software Testing Practices 1

8/6/2019 Software Testing Practices 1

1/53

# 1 Entity Solutions Limited 2004-05

Software Testing

Practices

By

Prof. V K GargM Tech, CISA, CISM, PMPR

10th & 11th March 2006

Mumbai

Organized by:

Iducate Managers

404, Sai prasad, Road No. 7, MIDC, Near Akruti Trade Centre, Andheri

East Mumbai 400093, phone: 28263207, email: [email protected]

Prasanta Mohanty


2/53


Representing View of Trainer

I am here for representing the view of Prof.

V.K.Garg.

I will lead your kind attention to theconcepts which are new to us and, if

agreed, can be implemented with help of

senior management.


3/53


Why Testing?

HOW CAN IT ADD VALUE?


4/53


Testing

Can be described as a process used for

Twin Terms Testing & Debugging in

software and to establishing that thesoftware has attained a specific degree of

quality with respect to SELECTED

ATTRIBUTES.

Group of procedures carried out to

evaluate some aspect of a piece of

software


5/53


Debugging

Is the process of locating the fault or

defect, repairing the code, and retesting

the code. Debugging process begins AFTER testing

has been carried out and the tester has

noted that the software is not behaving as

specified.


6/53


Software Development Process


7/53


Validation

Is the process of evaluating a software

system or component during, or at the end

of, the Development cycle in order to

determine whether it satisfies specified

requirements

Is usually associated with Traditional

Execution-based Testing that exercisingthe code with Test cases


8/53


Verification

Is the process of evaluating a software

system component to determine whether

the products at a given development

phase satisfy the conditions imposed at

the start of the phase

Is usually associated with activities such

as Inspections and Reviews of Softwaredeliverables.


9/53


The Extended / Modified V Model


10/53


Basic Definitions -1 Errors

An Error is a mistake , misconception ormisunderstanding on the part of a softwaredeveloper

Faults (Defects) A Fault (defect) is introduced into the software as the

result of an error

It is an Anomaly in the software that may cause it toBehave Incorrectly, and not according to its

specifications. Failures

A Failure is the inability of a software system orcomponent to Perform its Required functions withinspecified performance requirements.


11/53


Test Cases

Are procedures that examine thesoftware and produce results that will

lead to Acceptance, Modification orRejections of the Software.

Contains the following information

A set of Test Inputs (Set of Data Items)Execution Condition (Conditions

required for running the test)

Expected Outputs (Specified results to

be produced by the code under test)

Basic Definitions -2


12/53


ExampleTest

case #

Input Description Input Data Expected

Results

Actual

Results

1 Search for a book

by specifying a

valid author name

Author= Post a list of

books written

by this author

2 Search for a book

by specifying a

valid auther name

but non-existent

title

Auther=

Title=

Title name not

found.

List all books

written by this

author

3 Have multiple users

conduct the samequery

simultaneously

Search

Criteria=No.of

Users=

Accurate

results arepresented to all

users, with an

acceptable

response time


13/53


Test

Is a group of Related Test Cases or a group ofrelated Test cases and Test Procedures

(steps needed to carry out a test) Test Bed

Is an Environment that contains all theHardware, and software needs to test asoftware component or a software system

Test Report

Is a Document or piece of software that allowstesters to determine whether test has passedor failed.



14/53


Software Quality Software Quality related to the degree to which a

system, system component, or process meetsspecified requirements

Quality relates to the degree to which a system,system component, or process meets customer oruser needs or expectations.

Reviews

A review is a group meeting (process) whose

purpose is to evaluate a software artifact or a setof software artifacts

Are a type of static testing techniques that can beused to evaluate the quality of a software Artifactsuch as Requirements Document, A Test Plan, A

Design Document or a code component.



15/53


Defects Hypotheses and Tests - 1


16/53


Successful test is one that reveals the presence ofa Hypothesized defects

Role of a Tester is like that of a Doctor, who is inthe process of constructing a diagnosis for an ill

patient. The doctor develops Hypotheses aboutPossible illness using his knowledge of possiblediseases, and the patients symptoms

Test are made in order to make the correctdiagnosis. One could view defective software as the

ill patient. Testers like doctors need to have knowledge about

possible defects (illness in order to develop defecthypotheses)

Defects Hypotheses and Tests - 2


17/53


TEST TYPES

1. User Interface Tests

2. Functional Tests

3. Server side Tests4. Database Tests

5. Help Tests

6. Installation Tests7. Configuration and Compatibility Tests

8. Security Tests

9. Performance Tests


18/53


Software Testing Principles-1

1. Testing is the process of Executing a softwarecomponent using a selected set of test cases with theintent of

1. Revealing defects

2. Evaluating Quality2. A good test case is one that has a high probability ofrevealing a yet undetected defect(s)

3. A test case must contain the expected output or result

4. Test Cases should be developed for both valid and

invalid input conditions.

5. Test results should be inspected meticulously.


19/53


6. Testing should be carried out by agroup that is independent of thedevelopment group.

7. Tests must be repeatable andreusable.

8. Testing should be planned.

9. Testing activities should be integratedinto the software lifecycle.

Software Testing Principles-2


20/53


Testing Challenge

Achieve testing goals by using Finite Number of Test

Cases

Design Effective test cases

Greater probability of detecting defects

More efficient use of organizational resources

Higher probability for Test Reuse

Close adherence to Testing and Project

Schedules and budgets Possibility for delivery of High Quality software

product.


21/53


Smart Testing


22/53


Basic Testing Strategies

Both the strategies should be used to design Test cases


23/53


Black Box Testing - 1

Tester has knowledge of what the system

under test Does. No knowledge of its Inner

Structure, how it works.

Can be used for any size of the softwareunder test such as Simple Module, Member

function, Object cluster, subsystem or a

complete software system

Only considers software Behavior and

Functionality, is often called Functional or

Specification Based Testing


24/53


Black Box Test strategy considers onlyInputs and Outputs as a basis forDesigning Test Cases. Main issue is How

do we choose a suitable set of Inputs fromall possible valid and Invalid Inputs

Highly effective in Testing software with awell defined set of Inputs and Outputs thatare described in specification.

Black Box Testing - 2


25/53


Black Box Testing Methods

Random Testing

Equivalence Class Partitioning

Boundary Value Analysis

Error Guessing


26/53


Random Testing

Every Software module / system has an Input Domain fromwhich Test Input data is selected. If a Tester randomly selectsinputs from the domain, this is called Random Testing

Main Issues:

Are these values adequate

Are there any Input values other than those selected more

likely to reveal defects e.g. values at the beginning or at theEnd of the domain.

Should any values outside the domain be used as Test input

Tools that generate Random Test data are available, theygenerate random test inputs according to a statistics distributionassociated with pattern of usage

Very useful at the system level for stress testing.


27/53


Equivalence Class Partitioning - 1

Equivalence Class partitioning results in a

partitioning of the Input or output domain of

the software under test

The Finite Number of partitions orequivalence classes that result, allow the

tester to select a given number, of an

equivalence class as a Representative of that

class.

A Test value in a particular class is equivalent

to a test value of any other member of that

class.


28/53


The tester must consider both valid

and invalid equivalence classes

Equivalence classes may also beselected for output conditions (not a

common usage)

The derivation of equivalent classesis a heuristic process



29/53


If Input condition is

specified as..

Equivalence Class selection

Range of Values Select one valid equivalence class that covers the allowed range

Select two invalid equivalence classes, one outside each end of

the range

Number of Values Select valid equivalence class that includes the allowed number of

values

Two invalid equivalence classes that are outside each end of the

allowed numbers

Set of valid inputs Select one valid equivalence that combines al the numbers of the

setSelect one invalid equivalence class for any value outside the set

Must be condition Select one valid equivalence class to represent the must be

condition

One invalid class that does not include the must be condition



30/53



The test cases developed based on

Equivalent class partitioning can be

strengthened by use of Boundary ValueAnalysis


31/53


If Input condition is specified

as

Test Case Selection

Range of Values Develop valid test cases for the end of the range

Invalid test cases for possibilities just above and below the ends of the

range

Number of Values Develop valid test cases for the minimum and maximum numbers

Develop invalid test cases that include one lesser and one greater than

the maximum and minimum

Ordered set (e.g. table or

linear list)Develop tests that focus on the first and last element of the test



32/53


White Box Testing Strategy W

hite Box Testing Also called glass box testing

Is a complementary approach to Black box testing

The Testers Goal is to determine if all the logical and dataelements in the software unit are functioning properly

Tester needs knowledge of Internal logic structure of thesoftware under test. This knowledge becomes available tothe tester only during detailed design phase of development.

Follows Black box design

Can be most useful when testing small components, due to

details required for test design Focus is on Structural Elements such as statements and

Branches.


33/53


Coverage Analysis Is important for security point of view

Example - If a Tester sets a goal for a unit specifyingthe Requirement for complete or 100% statementcoverage then the test cases developed must ensurethat all the statements in the unit are executed at least

once. The logical elements most commonly considered for

coverage are:

Program Statements

Decision / Branches Conditions

Combinations of Decisions / Conditions

Paths (Node Sequence)


34/53


Levels of Testing - 1

Execution based Software testing

specially for large systems is usually

carried out at different levels.


35/53


Level What is Tested Goal of Testing

Unit Test A single component Detect functional and structural defects in

the unit

Integration Test Several components

as a group

Component Interaction

System Test System as a whole Performance evaluation such as usability,reliability, and other quality related

attributes

Acceptance Test Custom made s/w for

individual client

Show that s/w meets all the customer

requirements

Alpha Test S/w developed for the

Mass market

Bring potential users to the developers

site to use (test) the s/w

Beta Test S/w developed for the

Mass Market

Send the s/w out to potential user to use it

under real world conditions and report

defects to the developing organization

Levels of Testing - 2


36/53


Test Planning


37/53


IEEE Definition of Test Plan

A document describing the scope,

approach, resources, and schedules of

intended testing activities. It identifies test

items, the features to be tested, the testing

tasks, who will do each task, and any risk

requiring contingency planning


38/53


Test Plan Template (as per IEEE

Standards) Contents1. Introduction

2. Items to be Tested

3. Features to be Tested

4. Approach

5. Pass/Fail Criteria6. Suspension and Resumption Criteria

7. Test Deliverables

8. Testing Tasks

9. The Testing Environment

10. Responsibilities

11. Staffing and Training Needs12. Scheduling

13. Risks and Contingencies

14. Approvals


39/53


INTRODUCTION

Summarize the software items and software features to betested. The need for each item and its history may be included.

References to the following documents, when they exist, are

required in the highest level test plan:

a) Project authorization;

b) Project plan;c) Quality assurance plan;

d) Configuration management plan;

e) Relevant policies;

f) Relevant standards.


40/53


Test Items

Identify the test items including their version/revision level.

Supply references to the following test itemdocumentation, if it exists:

a) Requirements specification;

b) Design specification;

c) Users guide;

d) Operations guide;

e) Installation guide.

Reference any incident reports relating to the testitems.

Items that are to be specifically excluded fromtesting may be identified.


41/53


Features to be tested

Identify all software features and

combinations of software features to be

tested. Identify the test design

specification associated with each feature

and each combination of features.


42/53


Features not to be tested Identify all features and significant

combinations of features that will not be

tested and the reasons.


43/53


Approach

Describe the overall approach to testing. For each major group offeatures or feature combinations

Specify the approach that will ensure that these feature groups areadequately tested.

Specify the major activities, techniques, and tools that are used to testthe designated groups of features.

The approach should be described in sufficient detail to permitidentification of the major testing tasks and estimation of the timerequired to do each one.

Specify the minimum degree of comprehensiveness desired. Identifythe techniques that will be used to judge the comprehensiveness of thetesting effort (e.g., determining which statements have been executedat least once).

Specify any additional completion criteria (e.g., error frequency). Thetechniques to be used to trace requirements should be specified.

Identify significant constraints on testing such as test item availability,testing resource availability, and deadlines.


44/53


Item pass/fail criteria

Specify the criteria to be used to

determine whether each test item has

passed or failed testing.


45/53


Suspension criteria and resumption

requirements

Specify the criteria used to suspend all or

a portion of the testing activity on the test

items associated with this plan.

Specify the testing activities that must be

repeated, when testing is resumed.


46/53


Test deliverables Identify the deliverable documents. The following documents

should be included:

a) Test plan;

b) Test design specifications;

c) Test case specifications;

d) Test procedure specifications;

e) Test item transmittal reports; (item submitted to test eng.)f) Test logs;

g) Test incident reports;

h) Test summary reports.

Test input data and test output data should be identified asdeliverables.

Test tools (e.g., module drivers and stubs) may also beincluded.


47/53


Testing tasks

Identify the set of tasks necessary to

prepare for and perform testing. Identify all

inter-task dependencies and any special

skills required.


48/53


Environmental needs

Specify both the necessary and desired properties of the testenvironment. This specification should contain the physical

characteristics of the facilities including the hardware, the

communications and system software, the mode of usage (e.g.,

stand-alone), and any other software or supplies needed to support

the test.

Specify the level of security that must be provided for the test

facilities, system software, and proprietary components such as

software, data, and hardware.

Identify special test tools needed. Identify any other testing needs

(e.g., publications or office space). Identify the source for all needs

that are not currently available to the test group.


49/53


Responsibilities

Identify the groups responsible for managing,

designing, preparing, executing, witnessing,

checking, and resolving.

Identify the groups responsible for providingthe test items identified and the

environmental needs identified.

These groups may include the developers,

testers, operations staff, user representatives,

technical support staff, data administration

staff, and quality support staff.


50/53


Staffing and training needs

Specify test staffing needs by skill level.

Identify training options for providing

necessary skills.


51/53


Schedule Include test milestones identified in the

software project schedule as well as all

item transmittal events.

Define any additional test milestonesneeded. Estimate the time required to do

each testing task.

Specify the schedule for each testing task

and test milestone. For each testingresource (i.e., facilities, tools, and staff)

specify its periods of use.


52/53


Risks and contingencies

Identify the high-risk assumptions of the

test plan. Specify contingency plans for

each (e.g., delayed delivery of test items

might require increased night shift

scheduling to meet the delivery date).


53/53


Thanks

Documents

Software Testing Practices 1