© 2006, Cognizant Technology Solutions. All Rights Reserved.The information contained herein is subject to change without notice.

By

Narendra Ponnuswamy

Narendra.Ponnuswamy@cognizant.com

Performance Testing

Effort :

Estimation or

Guesstimation ?

Agenda

Performance Testing & Estimation

Pain Areas in Estimation

Divide & Conquer Strategy

Estimation Tips

Approach

Summing it up

2

Performance Testing & Estimation

Performance Testing is the process of determining the speed of an application under heavy user load.

It is an automated testing normally done using Industry standard tools.

“Well begun is half done”

A good estimate is a solid foundation for the project.

But some inevitable questions arise during any

Investment.

3

Triangle Model

Optimal resources to

achieve timely

delivery

Quality should not be

compromised

Cost Benefit and Maximum

Value with least investment

Cost

4

Time Quality

Pain Areas in Estimation

NFR’s (Non Functional Requirements) not properly

defined

Less time planned for Performance Testing

Unplanned additional runs due to performance issues

identified

Frequent code changes causing rework

Building a test environment similar to production

5

“There is nothing so easy to learn as experience

and nothing so hard to apply”

- Josh Billings

Divide & Conquer Strategy

“Divide and Conquer “ Strategy was successfully used

by British and Roman empires to rule the world.

Requirements

Analysis

Performance Testing

Effort

Execution

Delivery

Design and

Development

6

Non

Functional

Requirement

(NFRs)

Analysis

Estimate

Development

Tool

Evaluation

Requirements

Signoff

Requirements

Analysis

Design and

Development

Test

Execution

Delivery

•Requirements

Analysis

•Coordination

with other

teams

•Proof of

Concept if any

•Test Strategy

•Test Plan

•Test Design

•Test Scripts

•Smoke Test

Execution

•Test Execution

•Coordination

with other

teams

•Monitoring and

tuning

•Test

Summary

Report

•Executive

Summary

Report

•Review and

Sign Off

Critical Efforts in Various Stages

7

8

Approach

Level 0

• High level Estimate

• Roles defined

• Includes detailed

Requirement Analysis

and Planning effort

• Includes POC effort

• Tool investment if any

• Refined Hour estimate

• Detailed split up of

estimate for various

stages

• Includes detailed

scripting and test

execution effort

• Refined Hour

estimate if

requirements

change

• Includes additional

scripting and test

execution effort

due to additional

complexities

Benefits

Snapshot

Benefit Snapshot

•High level

Estimate

• Proof Of

Concept

effort

•Tool

Investment

if any

• Refined Hour estimate

• Detailed split up of

estimate for various

stages

• Includes detailed

scripting and test

execution effort

•Helps Project

Manager plan for

Performance

Testing effort

•Requirement

Analysis and

Planning effort

well defined

• Informs

Stakeholders with

detailed effort split

up

•Opportunity to

reduce scope (if

there are any

budget

constraints )

•Change in

requirements

is reflected

•Helps avoid

last minute

surprises

8

Level 1 Level 2

Estimation Tool

9

Estimation Tips

10

Which requirements should I consider from the chart below ?

Never be “Penny wise and Pound foolish”

Estimation Tips

11

• Determine application complexity

• Determine the number of Non Functional Requirements

• Identify the number of test cases (Each test case becomes a

script)

• Finalize the number of test runs

• Determine servers that need to be monitored

• Determine Data Setup and Environment Setup effort

Commonly identified risks:

• Frequent application changes impacting initially planned effort

• Unstable environment causing additional effort

I80/20 Rule

12

Tasks that consume more time :

Scripting Activities

Test Execution

Data Setup

Sample display of distribution :

Other Activities

Test Development

Test Execution

Data Setup

13

Performance Testing estimation should be given early in

the project

Listing assumptions in the estimate clearly

communicates the constraints involved.

Bottom Line:

Estimation is not an exact science but it is not an

approximation either.

Lack of a proper technique could make it a

Guesstimation.

Summing it up