Software Testing Life CycleSoftware Testing Life Cycle

Designed and Compiled by:

Udayakumar SreeSenior Test Engineer

Designed and Compiled by:

Udayakumar SreeSenior Test Engineer

STLC - DefinitionThe course of software being tested in a well-planned way is

known as Software test life cycle.

2

ContractSigning

Requirement Analysis

Test Planning

Test Development

TestExecution

DefectReporting

RetestDefects

ProductDelivery

STLC – Stages InvolvedContract Signing:

Process - The project is signed with client for testing the software

Documents involved: SRS Test Deliverables Test Metrics etc.

3

STLC – Stages Involved

Requirement Analysis: Process: Analyzing software for design and implementation

methods and testable aspects are recorded

Documents involved:

Requirement Specification documents

Functional Specification documents

Design Specification documents (use cases, etc)

Use case Documents

Test Trace-ability Matrix for identifying Test Coverage

4

STLC – Stages InvolvedTest Planning:

Process: To plan, how the testing process should flow Test Process Flow

Test Scope, Test Environment Different Test phase and Test Methodologies Manual and Automation Testing Defect Mgmt, Configuration Mgmt, Risk Mgmt. Etc Evaluation & identification – Test, Defect tracking

tools Documents Involved:

Master Test Plan, Test Scenario, SCM

5

STLC – Stages InvolvedTest Development

Process: Test Traceability Matrix and Test coverage Test Scenarios Identification & Test Case preparation Test data and Test scripts preparation Test case reviews and Approval Base lining under Configuration Management

Documents Involved: Test Plan, RTM Test cases

6

STLC – Stages InvolvedTest Execution:

Process: Executing Test cases Testing Test Scripts Capture, review and analyze Test Results Raising the defects and tracking for its closure

Documents Involved: Test Cases Test Execution report Bug report Requirement traceability matrix

7

STLC – Stages InvolvedDefect Reporting

Process: Defect logging Assigning defect and fixing Retesting Defect closing

Documents involved: Test report Bug Report

8

STLC – Stages InvolvedProduct Delivery

Process: After the product had undergone several tests, the

acceptance test is done by the user/client i.e. UAT, wherein the use cases were executed and the product is accepted to go on live

Test Metrics and process Improvements made Build release Receiving acceptance

Documents involved Test summary reports UAT Test Plan, UAT Test cases

9

Test Plan – What?

Derived from Test Approach, Requirements, Project Plan, Functional Spec., and Design Spec

Details out project-specific Test ApproachLists general (high level) Test Case areasInclude testing Risk AssessmentInclude preliminary Test Schedule Lists Resource requirements

11

Test Plan – Why?

Identify Risks and Assumptions up front to

reduce surprises later.

Communicate objectives to all team

members.

Foundation for Test Spec, Test Cases, and

ultimately the Bugs we find.

Failing to plan = planning to fail.

12

Test Plan – Definition

The test strategy identifies multiple test

levels, which are going to be performed for

the project. Activities at each level must be

planned well in advance and it has to be

formally documented. Based on the individual

plans, the individual test levels are carried

out.

13

Test Plan – Consists of…

Unit Testing Tools Required tool to test at unit level

Priority of Program units Module-wise priority

Naming convention for test cases

Status reporting mechanism

Regression test approach

14

Test Plan – Consists of…ETVX Criteria

Entry means the entry point to that phase. For example, for unit testing, the coding must be complete

and then only one can start unit testing Task is the activity that is performed Validation is the way in which the progress and correctness

and compliance are verified for that phase Exit tells the completion criteria of that phase, after the

validation is done. For example, the exit criterion for unit testing is all unit test

cases must pass

15

Risk Analysis

A risk is a potential for loss or damage to an Organization from materialized threats. Risk Analysis attempts to identify all the risks and then quantify the severity of the risks

Risk Identification Software Risks Business Risks Testing Risks Premature Release Risk Risk Methods

16

Risk Analysis continued…

Software Risks Knowledge of the most common risks

associated with Software development, and the platform you are working on

Business Risks Most common risks associated with

the business using the Software

Testing Risks Knowledge of the most common risks

associated with Software Testing for the platform you are working on, tools being used, and test methods being applied 17

Risk Analysis continued…

Premature Release Risk Ability to determine the risk

associated with releasing unsatisfactory or untested Software Products

Risk Methods Strategies and approaches for

identifying risks or problems associated with implementing and operating information technology, products and process; assessing their likelihood, and initiating strategies to test those risks

18

Test Execution

Software Testing FundamentalsTesting objectives include

Testing is a process of executing a program with the intent of finding an error

A good test case is one that has a high probability of finding an as yet undiscovered error

A successful test is one that uncovers an as yet undiscovered error

20

Testing should systematically uncover different classes of

errors in a minimum amount of time and with a minimum

amount of effort.

A secondary benefit of testing is that it demonstrates that

the software appears to be working as stated in the

specifications

When Testing should start? Testing early in the life cycle reduces the

errors. Test deliverables are associated with every phase of development. The goal of Software Tester is to find bugs, find them as early as possible, and make them sure they are fixed

The number one cause of Software bugs is the Specification

The next largest source of bugs is the Design

21

When to Stop Testing?

Some reasons to stop test are: Deadlines (release deadlines, testing deadlines.) Test cases completed with certain percentages

passed Test budget depleted Coverage of code/functionality/requirements

reaches a specified point The rate at which Bugs can be found is too

small Beta or Alpha Testing period ends The risk in the project is under acceptable limit

22

This can be difficult to determine. Many modern software applications are so complex,

and run in such as interdependent environment, that complete testing can never be done.

Test ExecutionTesting of an application includes:

Unit Testing Integration testing System Testing Acceptance testing

23

These are the functional testing strategies and few other

functional, non-functional, performance and other testing methods

can also be applied on the software.

Test Execution – Unit testingThe unit test plan is the overall plan to carry out the unit test

activities. The lead tester prepares it and it will be distributed to

the individual testers

24

Basic input/output of the units along with their basic functionality will be tested

input units will be tested for the format, alignment, accuracy and the totals

The UTP will clearly give the rules of what data types are present in the system, their format and their boundary conditions

Testing the screens, files, database etc., are to be given in proper sequence

Test Execution – Integration testing

The integration test plan is the overall plan for carrying out the activities in the integration test level

25

This section clearly specifies the kinds of interfaces fall under

the scope of testing internal, external interfaces, with request

and response is to be explained

Two approaches practiced are Top-Down and Bottom-Up

integrations

Given this correctly, the testing activities will lead to the

product, slowly building the product, unit by unit and then

integrating them

Test Execution – System testingThe system test plan is the overall plan carrying

out the system test level activities

26

System testing is based on the

requirements

All requirements are to be verified in the

scope of system testing

The requirements can be grouped in

terms of the functionality

Based on this, there may be priorities

also among the functional groups

Apart from this what special testing is

performed are also stated here

Test Execution – Non-functional testingNon-functional testing includes: Installation testing – Installation environment, practical

obstacles etc.

Compatibility testing – compatibility with other system software

Configuration testing - how well the product works with a broad

range of hardware/peripheral equipment configurations as well as

on different operating systems and software

Security testing - secure from mistaken/accidental users,

hackers, and other malevolent attackers

Recovery testing - how well a system recovers from crashes,

hardware failures, or other catastrophic problems

Usability testing - Testing for 'user-friendliness'

27

Test Execution – Performance testingPerformance testing includes:

Load Testing – Testing with the intent of determining how

well the product handles competition for system resources.

The competition may come in the form of network traffic, CPU

utilization or memory allocation

28

Test Execution – Performance testing Stress testing - Testing with the intent of determining how

well a product performs when a load is placed on the system

resources that nears and then exceeds capacity

29

Bug/Defect Management

BUG LIFE CYCLE - What is a bug?A software bug is an error, flaw, mistake, failure, or fault in a computer program that prevents it from behaving as intended.

31

BUG LIFE CYCLE

In software testing, the

term life cycle refers to

the various stages that a

defect/bug assumes over

its life.

32

What is a Bug Life Cycle?

BUG LIFE CYCLE

The different stages involved in a bug life cycle are as follows:

Finding Bugs

Reporting/ Documentation

Fixing

Retesting

Closing

33

Stages involved in Bug Life Cycle

BUG LIFE CYCLE

34

1. Finding Bugs:

Software Tester finds bug while testing.

It is then logged and assigned to a programmer to be fixed.

2. Reporting/ Documentation:

In software, bugs need to be tracked and managed to

Communicate bug for reproducibility, resolution, and

regression.

Track bug status (open, resolved, closed).

Ensure bug is not forgotten, lost or ignored

Stages explained

BUG LIFE CYCLE3. Fixing:

Once the bug is assigned to the developer, he fixes the

bug.

Once the programmer fixes the code , he assigns it back to

the tester and the bugs enters the resolved state.

4. Retesting:

The tester then performs a regression test to confirm that

the bug is indeed fixed.

5. Closing:

If the bug is fixed, then the tester closes the bug.

Here the bug then enters its final state, the closed state. 35

Stages explained Continued…

Different status of a Bug

36

Description of Various Status of a bugNew: When the bug is posted for the first time, its state will be

“NEW”. This means that the bug is not yet approved.

Open: After a tester has posted a bug, the lead of the tester

approves that the bug is genuine and he changes the state as

“OPEN”.

Assign: Once the lead changes the state as “OPEN”, he assigns

the bug to corresponding developer or developer team. The state of

the bug now is changed to “ASSIGN”.

Test: Once the developer fixes the bug, he assigns the bug to the

testing team for retesting. Before he releases the software with bug

fixed, he changes the state of bug to “TEST”. It specifies that the

bug has been fixed and is released to testing team.37

Description of Various Status of a bug

Deferred: The bug, changed to deferred state means the bug is

expected to be fixed in next releases.

Rejected: If the developer feels that the bug is not genuine, he

rejects the bug. Then the state of the bug is changed to

“REJECTED”.

Duplicate: If the bug is repeated twice or the two bugs mention

the same concept of the bug, then one bug status is changed to

“DUPLICATE”.

38

Description of Various Status of a bugVerified: Once the bug is fixed and the status is changed to

“TEST”, the tester tests the bug. If the bug is not present in the

software, he approves that the bug is fixed and changes the status

to “VERIFIED”.

Reopened: If the bug still exists even after the bug is fixed by the

developer, the tester changes the status to “REOPENED”. The bug

traverses the life cycle once again.

Closed: Once the bug is fixed, it is tested by the tester. If the

tester feels that the bug no longer exists in the software, he changes

the status of the bug to “CLOSED”. This state means that the bug is

fixed, tested and approved. 39

Severity of a Bug It indicates the impact each defect has

on the testing efforts or users and

administrators of the application

under test.

This information is used by developers

and management as the basis for

assigning priority of work on defects.

40

Priority Levels of a Bug Critical :

An item that prevents further testing of the product or function

under test can be classified as Critical Bug. No workaround is

possible for such bugs.

Examples of this include a missing menu option or security

permission required to access a function under test.

Major / High: A defect that does not function as expected/designed or cause

other functionality to fail to meet requirements can be classified

as Major Bug. The workaround can be provided for such bugs.

Examples of this include inaccurate calculations; the wrong field

being updated, etc 41

Priority Levels of a BugAverage / Medium:

The defects which do not conform to standards and

conventions can be classified as Medium Bugs. Easy

workarounds exists to achieve functionality objectives.

Examples include matching visual and text links which lead to

different end points.

Minor / Low: Cosmetic defects which does not affect the functionality of the

system can be classified as Minor Bugs.

42

Various Bug tracking toolsThe various bug tracking tools available are:

Quality Center® – from HP

Bugzilla® - from Mozilla

Dev Track® – from TechExcel

43

Product Delivery

Product Delivery -Test DeliverablesTest Trace-ability Matrix Test Plan Testing Strategy Test Cases (for functional testing) Test Scenarios (for non-functional

testing) Test Scripts Test Data Test Results Test Summary Report Release Notes Tested Build

45

Product Delivery - Test MetricsMeasuring the correctness of the testing process

with measurable is known to be test metrics.

46

Product Delivery - Test MetricsThere are several test metrics identified as

part of the overall testing activity in order to track

and measure the entire testing process.

These test metrics are collected at each phase

of the testing life cycle/SDLC, analyzed and

appropriate process improvements are determined

and implemented. The metrics should be constantly

collected and evaluated as a parallel activity together

with testing, both for manual and automated testing

irrespective of the type of application

47

Product Delivery - Test Metrics - Classification1. Project Related Metrics – such as

Test Size,

# of Test Cases tested per day –Automated (NTTA)

# of Test Cases tested per day –Manual (NTTM)

# of Test Cases created per day – Manual (TCED)

Total number of review defects (RD)

Total number of testing defects (TD) etc.

48

Product Delivery – Test Metrics – Classification

2. Process Related Metrics – such as

Schedule Adherence (SA)

Effort Variance (EV)

Schedule Slippage (SS)

Test Cases and Scripts Rework Effort, etc.

3. Customer related Metrics – such as

Percentage of defects leaked per release (PDLPR)

Percentage of automation per release (PAPR)

Application Stability Index (ASI) etc. 49

Product Delivery – Acceptance testing – UAT

The client at their place performs the acceptance testing. It will

be very similar to the system test performed by the Software

Development Unit

50

There is no specific clue on the way they will carry out the

testing, since the client performs this test

It will not differ much from the system testing

This is just one level of testing done by the client for the overall

product and it includes test cases including the unit and

integration test level details

51