Testing is a process of evaluating a

system by manual or automatic means

and verify that it satisfies specified

requirements or identify differences

between expected and actual results.

Software Testing

Software Testing is important as it may cause mission failure, impact on operational performance

and reliability if not done properly.

Effective software testing - deliver quality software products - satisfying - user’s requirements, needs

and expectations.

Why Software Testing ?

Quality is defined as meeting the customer’s requirements for the first time

and every time.

Quality is much more than the absence of defects which allows us to meet customers

expectations.

What is Quality ?

Quality is the important Quality is the important factor affecting an factor affecting an organization’s long term organization’s long term performance.performance.

Quality improves Quality improves productivity and productivity and competitiveness in any competitiveness in any organization.organization.

Why Quality ?

Quality assurance is a planned and

systematic set of activities necessary to

provide adequate confidence that products

and services will conform to specified

requirements and meet user needs.

*It is process oriented.

*defect prevention based.

Quality Assurance

Quality control is the process by which

product quality is compared with applicable

standards and the action taken when non

conformance is detected.

* It is product oriented.

* defect detection based.

Quality Control

Quality

Cost

Schedule

Factors Influencing a Project

What is primary role of software testing ?

Determine whether the system Determine whether the system meets specifications(producer meets specifications(producer view)view)

Determine whether the system Determine whether the system meets business and user meets business and user needs(customer view)needs(customer view)

Find Defect, Not Correcting the Defect

Role of a Tester

The purpose of testing is to uncover defects.

A defect is a variance from a desired product attribute.

Two categories of defects are:Variance from product specifications

Variance from customer/user expectation.

What is Defect ?

Wrong Missing Extra

Classification of Defects

Testing Economics And Cost

Traditional Testing Continuous Testing

0 20 10 $10

0 40 15 $25

0 60 18 $42

$480 12 4 $182

$1690 0 0 $582

Requirement

Code

Production

Design

Testing

Testing Policy

A Testing Policy is management’s definition of testing a department.

Quality Policy

A quality policy is again a management definition of providing customer satisfaction for the first

time and every time.

Testing

Static Dynamic

Functional Structural

(Functionality of the program) (Logic of the Program)

Black Box TestingWhite Box Testing

Static Testing

Static testing refers to testing something that’s Static testing refers to testing something that’s

not executing - examining and reviewing it.not executing - examining and reviewing it.

Dynamic Testing Dynamic testing is what you would normally Dynamic testing is what you would normally

think of testing - executing and using the software.think of testing - executing and using the software.

Dynamic Testing

Techniques used are determined Techniques used are determined by the type of by the type of testing that must testing that must be conductedbe conducted

FunctionalFunctional

StructuralStructural

Functional Testing

Structure of the program is not Structure of the program is not considered.considered.

Test cases are decided based on Test cases are decided based on the requirements or specification the requirements or specification of the program or moduleof the program or module

Hence it is often called as “Black Hence it is often called as “Black Box Testing”.Box Testing”.

Structural TestingConcerned with testing ,the Concerned with testing ,the implementation of the program.implementation of the program.

Focus on the internal structure of Focus on the internal structure of the program.the program.

The intent of structural testing is The intent of structural testing is not to exercise all the different not to exercise all the different input or output condition but to input or output condition but to exercise the different programming exercise the different programming structures and data structures in structures and data structures in the program.the program.

Unit Testing

Integration Testing

System Testing

User Acceptance Testing (UAT)

Unit Testing

Test each module Test each module individually.individually.

Follows a white box testing Follows a white box testing (Logic of the program)(Logic of the program)

Integration Testing

Integrate two or more Integrate two or more module.i.e. communicate module.i.e. communicate between the modules.between the modules.

Follow a white box testing Follow a white box testing (Testing the code)(Testing the code)

System Testing

confirms that the system as confirms that the system as a whole delivers the a whole delivers the functionality originally functionality originally required.required.

Follows the black box Follows the black box testingtesting..

UAT

Building the confidence of the client and users is the role of the acceptance test phase.

It is depend on the business scenario.

V-Model

Unit

Integration

System

UAT

LLD

HLD

SRS

Business scenario

White BoxWhite Box

Black BoxBlack Box

IncrementalIncremental

ThreadThread

White Box Testing Techniques

Statement coverage – execute all Statement coverage – execute all statements at least once.statements at least once.

Decision coverage - execute each Decision coverage - execute each decision direction at least once.decision direction at least once.

Condition coverage – execute each Condition coverage – execute each decision with all possible outcomes decision with all possible outcomes at least onceat least once

Black Box Testing Techniques

* Equivalence Partitioning* Equivalence Partitioning

* Boundary Analysis* Boundary Analysis

* Error Guessing* Error Guessing

A subset of data that is representative of a larger classA subset of data that is representative of a larger class

For example, a program which edits For example, a program which edits credit limits within a given range credit limits within a given range ($10,000 - $15,000 would have 3 ($10,000 - $15,000 would have 3 equivalence classes: equivalence classes:

Less than $10,000 (invalid)Less than $10,000 (invalid)

Between $10,000 and $15,000 (valid)Between $10,000 and $15,000 (valid)

Greater than $15,000 (invalid)Greater than $15,000 (invalid)

Equivalence Partitioning

Boundary Analysis

A technique that consists of A technique that consists of developing test cases and data that developing test cases and data that

focus on the input and output focus on the input and output boundaries of a given functionboundaries of a given function

Boundary Analysis Cont.. In the same credit limit example, In the same credit limit example, boundary analysis boundary analysis would test: would test:

Low boundary plus or minus one Low boundary plus or minus one ($9,999 and $10,001)($9,999 and $10,001)

On the boundary ($10,000 and On the boundary ($10,000 and $15,000)$15,000)

Upper boundary plus or minus one Upper boundary plus or minus one ($14,999 and $15,001)($14,999 and $15,001)

Error Guessing

Based on the theory that test cases can be developed based on experience of the Test

Engineer

For example, in an example where one of the inputs is the date, a test engineer might try

February 29,2000 or 9/9/99

Incremental Testing

A disciplined method of testing A disciplined method of testing the interfaces between unit-the interfaces between unit-tested programs as well as tested programs as well as between system componentsbetween system components

Incremental Testing TypesIncremental Testing Types

Top-downTop-down

Bottom-upBottom-up

Top Down

Begins testing from the top of the module hierarchy and works down to the bottom using

interim stubs to simulate lower interfacing modules or programs

Bottom Up

Begins testing from the bottom of the hierarchy and works up to the top

Bottom-up testing requires the development of driver modules which provide the test input, call the module or program being testing, and display test output

Thread Testing

A technique, often used during early integration testing

Demonstrates key functional capabilities by testing a string of units that accomplish a specific function in the application

Various techniques used throughout the testing stages

Stages/ Techniques

White Box

Black Box

Incremental Thread

Unit Testing X

Integration Testing

X X X

System Testing

X

Acceptance Testing

X

Verification

Verification is the process Verification is the process confirming that -software meets its confirming that -software meets its specificationspecification

Use Use – To identify defects in the – To identify defects in the product early in product early in

the life cyclethe life cycle

Validation

Validation is the process confirming Validation is the process confirming that it that it meets the user’s meets the user’s requirements.requirements.

Review types

Inprocess ReviewInprocess Review Phase end/Decision Point/Milestone Phase end/Decision Point/Milestone ReviewReview Post Implementation/Post MortemPost Implementation/Post Mortem

Inprocess

• During a specific period of the development cycle – like design period

• Used to find defects in the work product and the work process

• Catches defects early – where they are less costly to correct.

Decision Point or Phase End

Review of products and processes nearReview of products and processes nearthe completion of each phase ofthe completion of each phase ofdevelopment.development.

Decisions for proceeding with development Decisions for proceeding with development are based on cost, schedule, risk, progress, are based on cost, schedule, risk, progress, readiness for next phase.readiness for next phase.

Also referred to as Milestone ReviewAlso referred to as Milestone Review

Post Implementation

•Also known as “Postmortems”

• Review/evaluation of the product that includes planned vs. actual development results and compliance with requirements

• Used for process improvement of software development

• Can be held up to three to six months after implementation

Classes of Reviews

1. Informal Review (or) Peer review

2. Semiformal Review (or) Walk Through

3. Formal Review (or) Inspection

Informal

• Also called peer-reviews

• Generally one-to-one meeting

•No agenda(scheduling)

• Results are not formally reported

• Occur as needed through out each phase

Semiformal

• Facilitated by the author

• Presentation is made with comment throughout and at the end

•reports are distributed to the participants

• Possible solutions for defects not discussed

• Occur one or more times during a phase

Formal

• Facilitated by moderator

• Assisted by recorder

• Meetings are planned in advance

• Directly dependent on the preparation of participants

• Held at any time

Reviewing Rules

1. The product is reviewed, not the producer

2. Defects and issues are identified, not corrected

3. All members of the reviewing team are responsible for the results of the review

Go

Stop

PrepareReview

Material

Distribute Material To Review

Team Members

Review Meeting

Resolve IssuesRaised in Review

AnotherReview Needed?

YES

No

Review Process

Performance (Load, Stress)

Processing Time

Response Time

Reliability

Availability

Scalability

Usability

Determines how well the user will be able to understand and interact with the system.

This is done prior to the testing levels.

Vendor Validation

This can be conducted jointly This can be conducted jointly by software vendor and the by software vendor and the team.team.

Ensuring that all the requested Ensuring that all the requested functionality has been delivered.functionality has been delivered.

Prior to accepting it & Prior to accepting it & installing it into a production. installing it into a production.

Benefits Realization Test

It is a test or analysis conducted It is a test or analysis conducted after an application is moved into after an application is moved into production.production.

To determine whether the To determine whether the application is likely to deliver the application is likely to deliver the original benefits.original benefits.

This is conducted by the user or This is conducted by the user or client group who requested the client group who requested the project.project.

Configuration Testing

This testing is performed find out the various supporting combinations of hardware and

software.

Recovery Testing

It is nothing but a features built into It is nothing but a features built into the application for handling the application for handling interruptions.interruptions.

Returning to the actual points/Page Returning to the actual points/Page in the application.in the application.

Test plan preparationTest plan preparation

Test case designTest case design

Test execution and Test Log Test execution and Test Log preparationpreparation

Defect TrackingDefect Tracking

Test report preparationTest report preparation

Test PlanTest Plan: A document that : A document that defines the overall testing defines the overall testing approach is called Test Plan.approach is called Test Plan.

Test Case: Test Case: Set of procedures in Set of procedures in order to find out the defects.order to find out the defects.

Test ExecutionTest Execution: Executing the : Executing the test cases.test cases.

Test logTest log: Pass and fail information : Pass and fail information will be stored in this test log.will be stored in this test log.

Defect trackingDefect tracking: All the failed items : All the failed items will come under the defect tracking. will come under the defect tracking. (closing defect)(closing defect)

Test reportTest report: Reports are generated : Reports are generated during the TLC.Final test summary during the TLC.Final test summary report. (generated at the end of report. (generated at the end of testing for making decisions)testing for making decisions)

Test Plan Preparation (manual)

Test ObjectiveTest Objective : A test objective is : A test objective is simply a testing “goal”.It is a simply a testing “goal”.It is a statement of what the tester is statement of what the tester is expected to perform during the expected to perform during the testing activity. testing activity.

Test strategy :Test strategy :

1. Techniques that are going to 1. Techniques that are going to be be used during the used during the testing.testing.

2. What types of tests must be 2. What types of tests must be conducted. conducted.

3. What stages of testing are 3. What stages of testing are required. required.

(e.g.unit,system) (e.g.unit,system)

Test Risk Analysis:

A risk is a condition A risk is a condition that can result in a that can result in a loss.The risk that may loss.The risk that may occur/arise during the occur/arise during the analysis of the testing.analysis of the testing.

* the probability that * the probability that the negative event Will the negative event Will occur.occur.

Risk identified - - -

Unavailability of resources - software

hardware etc .

Migration plan - - -

What are the ways to overcome

Test schedule & Resources

Task Duration Resource

Test plan 3 1

Test design 5 3

Roles and Responsibilities

Test Manager – Manages the entire testing activity (approve).

Test leader - Prepare the test plan, Review test case, monitor defect tracking, provide resources.

Test Engineer – Prepare test case design (test risk, reports).

The communications during The communications during the project will be informed the project will be informed through this through this communication approach.communication approach.

Communications Approach

Test Environment

What are the environments that What are the environments that the person needed to develop a the person needed to develop a testing.testing.

* Software Requirements.* Software Requirements.

* hardware * hardware Requirements.Requirements.

* Tools that are needed.* Tools that are needed.

Automated Test Tools

Identify which tools is used for Identify which tools is used for each each individual testindividual test

Should know the test level Should know the test level criteriacriteria

Test Case Design

Test case IDTest case ID : Sys_xyz_o1 : Sys_xyz_o1Test case descriptionTest case description : a short note : a short note about the testing.about the testing.Test case procedureTest case procedure : Each and every : Each and every step has to be mentioned in test case step has to be mentioned in test case procedure.procedure.Test inputs (or) Test DataTest inputs (or) Test Data :input data :input dataExpected resultExpected result: the Expected out come of the : the Expected out come of the

test case. test case.

Test Execution

Actual result : What we have Actual result : What we have received is the actual result.received is the actual result.

Expected result = Actual Expected result = Actual result then the test is pass result then the test is pass otherwise it is fail.otherwise it is fail.

Test Log

Test Case ID Test Case Description

Test status/Result

Sys_xyz_01 dfgdfg Fail

Sys_xyz_02 hgfhgfh Pass

* Failed test case are going to be the defects

Defect Tracking

All the failed test execution/defects will come under the defect tracking.

Defect TrackingDefect Id : Sys_def_xyz_o1

Test case Id : Sys_xyz_o1

Defect Description : Specify the defect.

Defect Severity : Severity of the Defect

Defect Priority : Major or Minor

Defect Status : Status of the Defect

Identified by : ABC

Module : Database Operations

Project : XYZ Corporation

Dated : 22/9/03

Defect Status

New Open Fixed Rejected Reopen Closed

Defects Severity

CriticalCritical

MajorMajor

MinorMinor

Defects Priority

HighHigh

ModerateModerate

LowLow

What is a Metric

Software Metrics are measure that are used to quantify the software, software development resource and software development process.

Metric – A quantitative measure of the degree to which a system ,component or process possesses a given attribute.

Metric generally classified into 2 types.

- Process Metric

- Product Metric

Metric ?

Metric ?

Process Metric

A metric used to measure the characteristic of the methods,techniques and tools employed in developing, implementing and maintaining the software system.

Product Metric

A metric used to measure the characteristic of

the product.

Testing Metrics

Coverage-MeasureCoverage-Measure

The amount of percentage The amount of percentage covered by testing requirements.covered by testing requirements.

Testing Metrics

Risk -MeasureRisk -Measure

Understand which portion of the Understand which portion of the system functionality represents a system functionality represents a "high risk".."high risk"..

Eg: Prioritizing the testing effortEg: Prioritizing the testing effort

Testing Metrics

Time-MeasureTime-Measure

* Estimate the time needed to * Estimate the time needed to cover the most high-risk functional cover the most high-risk functional areasareas

(for example, if the application (for example, if the application contains 5 major functional areas, contains 5 major functional areas, and 2 of them have been designated and 2 of them have been designated as "high risk" . Majority of QA time as "high risk" . Majority of QA time must focus on those 2 critical areasmust focus on those 2 critical areas

Testing Metrics

Progress MeasureProgress Measure

Measure the number and Measure the number and severity of defects per functional severity of defects per functional area…….area…….

Testing Metrics

Measure the development Measure the development processprocess

Based on the no of defects Based on the no of defects and the fix times and procedures, and the fix times and procedures, grade the development processgrade the development process

Testing Metrics

Measure the testing processMeasure the testing process

Review the average Review the average time for the defect to go through time for the defect to go through the lifecycle. from new to open, to the lifecycle. from new to open, to fixed, to retested, to closed. fixed, to retested, to closed. Measure progress by application Measure progress by application functional area or per each functional area or per each member of the QA team to member of the QA team to estimate team productivityestimate team productivity

Testing Metrics

User ParticipationUser Participation

Use : To find the Involvement of the Use : To find the Involvement of the testertester

User Participation Test TimeUser Participation Test Time Vs Vs Total Test TimeTotal Test TimePath TestedPath Tested

Use : Extent Of TestingUse : Extent Of Testing

No of Paths TestedNo of Paths Tested from from Total Total No of PathsNo of Paths

Testing Metrics

Test CostTest Cost

Use : Resources consumed in the Use : Resources consumed in the testingtesting

Test Cost (Vs) Total Test Cost (Vs) Total System CostSystem Cost

Testing Metrics

Cost To Locate DefectCost To Locate Defect

Use : Resources consumed in the Use : Resources consumed in the testingtesting

* Test Cost * Test Cost

* No of Defects located in the * No of Defects located in the TestingTesting

This metric shows the cost to This metric shows the cost to locate a defectlocate a defect

Reports Functional Testing Status Functional Testing Status 

Functions Working TimelineFunctions Working Timeline

Expected verses Actual Defects Expected verses Actual Defects Uncovered Uncovered Timeline Timeline

Defects Uncovered verses Defects Uncovered verses Corrected Gap Corrected Gap Timeline Timeline

Average Age of Uncorrected Average Age of Uncorrected Defects by Defects by Type Type

Defect DistributionDefect Distribution

FunctionalityFunctionality

UsabilityUsability

Server Side InterfacesServer Side Interfaces

Client side CompatibilityClient side Compatibility

PerformancePerformance

SecuritySecurity

Functionality LinksLinks

Objective is to check for all the links in the Objective is to check for all the links in the website.website.

    All Internal LinksAll Internal Links

    All External LinksAll External Links

    All mail to linksAll mail to links

Check for Broken LinksCheck for Broken Links

Field Level ValidationField Level Validation

Check forCheck for length, special characters, numerical length, special characters, numerical

characters etc.,characters etc.,

Functional ChecksFunctional Checks

Create, modify, view and delete are working.Create, modify, view and delete are working.

Functionality

Error Handling for wrong inputs or actions.

- Check whether Appropriate error messages to be displayed.

Optional and mandatory fields.

- Mandatory field should not be left blank.

Optional should allow the user to skip the field.

Functionality

Data IntegrityData Integrity - - Should not be any missing or wrong data in the Should not be any missing or wrong data in the databasedatabase..

Functionality

Navigation Navigation

Navigation describes the way users navigate Navigation describes the way users navigate within a page, between different user interface within a page, between different user interface controls (buttons, boxes, lists, windows etc.), or controls (buttons, boxes, lists, windows etc.), or between pages via e.g. links. between pages via e.g. links.

Check the following ..Check the following ..

- - Application navigation is proper through Application navigation is proper through tabtab

- - Navigation through MouseNavigation through Mouse

- - Main features accessible from the Main features accessible from the main/home page.main/home page.

- - Any hot keys, control keys to access Any hot keys, control keys to access menus.menus.

Usability

ContentContent

Correctness is whether the information is truthful or Correctness is whether the information is truthful or contains misinformation. The accuracy of the contains misinformation. The accuracy of the information is whether it is without grammatical or information is whether it is without grammatical or spelling errors. Remove irrelevant information from spelling errors. Remove irrelevant information from your site. This may otherwise cause your site. This may otherwise cause misunderstandings or confusion. misunderstandings or confusion.

Usability

Server InterfaceServer Interface

Verify that communication is done correctly, Verify that communication is done correctly,

Web server-Application server, Web server-Application server, Application server-Database server Application server-Database server

and vice versa.and vice versa.

Compatibility of server software, hardware, Compatibility of server software, hardware, network connections. network connections.

Server Side Interfaces

PlatformPlatform

Check for the compatibility of Check for the compatibility of

a. Windows (95, 98, 2000, NT)a. Windows (95, 98, 2000, NT)

b. Unix (different sets)b. Unix (different sets)

c. Macintosh (If applicable)c. Macintosh (If applicable)

d. Linuxd. Linux

e. Solaris (If applicable)e. Solaris (If applicable)

Client Side Compatibility

BrowsersBrowsers

Check for the various combinationsCheck for the various combinations

Internet Explorer (3.X 4.X, 5.X)Internet Explorer (3.X 4.X, 5.X)

Netscape Navigator (3.X, 4.X, 6.X) Netscape Navigator (3.X, 4.X, 6.X)

Frames and Cascade Style sheetsFrames and Cascade Style sheets

Applets, ActiveX controls, DHTML, client side scripting Applets, ActiveX controls, DHTML, client side scripting

HTML specifications.HTML specifications.

Client Side Compatibility