Effort Estimation

EFFORT ESTIMATIONMETHODS

Estimation involves the steps and processes involved incalculating the amount of effort required for the execution of aproject. This is absolutely critical from a commercial perspectiveas miscalculations can overrun the financial viability of a project.Its also important from the project management perspective indetermining the team size and schedule of the project.

This white paper explains different methods of effort estimationand emphasizes on using multiple expert opinions, framework/models and interval estimates. Another key aspect discussed isthe benefits of using a framework or a model. It also explainshow we can utilize residual uncertainty in the estimate to ouradvantage - having estimates with different confidence levelsand make use of it strategically.

WHITE PAPER

AJIKUMAR TN

EFFORT ESTIMATIONMETHODS

Page: Table of Contents

Table of Contents

Effort Estimation Methods

INTRODUCTION ..................................................................................................................................................................... 3COLLECTIVE THOUGHT ......................................................................................................................................................... 4DELPHI METHOD ................................................................................................................................................................... 4FRAMEWORK METHOD ....................................................................................................................................................... 5ESTIMATION STATEMENTS ................................................................................................................................................... 6COLLECTIVE THOUGHT - ADVANCED ................................................................................................................................ 8DELPHI- ADVANCED ............................................................................................................................................................ 9FRAMEWORK - ADVANCED .............................................................................................................................................. 10HYBRID METHOD ................................................................................................................................................................ 11CONCLUSION ....................................................................................................................................................................... 12REFERENCES ........................................................................................................................................................................ 13ACKNOWLEDGEMENTS ...................................................................................................................................................... 13ABOUT THE AUTHOR .......................................................................................................................................................... 13ABOUT WIPRO TECHNOLOGIES ......................................................................................................................................... 14WIPRO IN VERIFICATION & VALIDATION ...................................................................................................................... 14

Wipro Technologies Page:


03 of 14

INTRODUCTIONEstimation process is used in project management to estimate the effort required for the execution of a project. This is requiredto do costing of the project for commercial reasons and also in determining the team size and schedule of the project.

This article consolidates different schools of thought about estimation. The technical aspects of estimation are not discussedhere; rather, different methods of estimation and estimation-statements are discussed.

The methods of effort estimation can be classified according to 4 different criteria mentioned below.1) Experts involved in estimation are working together or independently for estimation purpose.2) Whether framework is used for estimation or not.3) Estimators coming up with a single estimate or a triple estimate.4) Whether confidence level is used or not in estimation.

Each of these 4 criteria has two choices each and we will thus have 16 different methods of estimation. It is not practical todiscuss all of them in this article but we will take up important ones. The different methods of estimation discussed in this articleare:

Collective thought Delphi Framework method Collective thought - advanced Delphi - advanced Framework - advanced Hybrid

Table 1 shows these 7 methods of estimation and how they relate to the criteria mentioned above. The terminologies given inthe criteria framework, single estimate, triple estimate and confidence level will be discussed as we proceed.

Table 1: Estimation methods

Experts aretogether orindependent

Whetherframework isused

Single estimate/Triple estimate

Whetherconfidencelevel is usedCriteria

Estimation MethodsEstimation MethodsEstimation MethodsEstimation MethodsEstimation Methods

Collective thought Together No Single No

Delphi Independent No Single No

Framework method Together Yes Single No

Collective Thought - advanced Together Optional Triple Yes

Delphi - advanced Independent Optional Single Yes

Framework - advanced Together Yes Triple Yes

Hybrid Independent Yes Triple Yes



04 of 14

COLLECTIVE THOUGHTIn this method, the project team usually does the estimation. In most of the cases, the full team may not be available to do theestimation at the time of estimation process. It is usually done by some of the senior members in the team. The characteristicsof this exercise are:

The estimation team meets in a discussion room so that they can interact with each other. Some people do join in overteleconference too.

It is expected that all the team members attending the estimation effort meeting would be prepared for a gooddiscussion and would have studied the requirements of the project in advance. For the benefit of those who have beenunable to do so, the moderator would brief the team about all the requirements for the estimation before the processis actually begun.

The team goes through each of the requirements, analyzes all the risks and arrives at an effort based on knowledge,experience and instinct. If team members have different opinions, the moderator helps to reach a consensus on theeffort required.

The effort arrived through consensus is used as the estimated effort.

This method can be used under the following circumstances. Senior team members are available to do estimation at the time of estimation process. They can meet and interact with each other. Team members are able to reach a consensus about the estimate.

DELPHI METHODLike the collective thought method, a moderator would be active in this scenario too, but there are two important differences:

Experts outside the project team could also be asked to participate in the estimation process. Estimation is obtained from the experts in a confidential manner. There would not be any interaction between the

experts involved in the estimation process.

Experts use their experience and gut feel to arrive at the estimate after understanding the requirements. The moderator takes theaverage of the estimates he gets from the different estimators and arrives at the project effort.

As an example,Moderator approaches six independent experts and got the following estimates.210, 215, 219, 215, 224,215

Average estimate AE = (210 + 215 + 219 + 215+ 224 + 215) / 6 = 216.3

Arithmetic mean need not be the only method to arrive at the estimated effort in Delphi. Median or mode could be used as well.

In the example mentioned here, median is 215 and mode is also 215. If you look at the estimates given by the experts above,3 out of the 6 experts have given the estimate as 215. So it makes sense to go by the majority view.

This method is best used in the following circumstances.

There are enough experts available within the organization who can judge over the requirements in hand. The project manager is able to find different experts inside and outside the project team to give estimates.



05 of 14

FRAMEWORK METHODThe estimators use frameworks in order to reduce the variability in estimates between them. Usually estimators use their gut feeland experience from their past projects to arrive at estimates. Frameworks capture data from past similar projects and thusreduce the dependency on experts experience. There are two advantages here. The designer of the framework might have usedall the data available about similar past projects. Usually this information would be more than an expert can bring into anestimate. Also, different estimators will have varied experiences, which may reflect in their estimates. This will bring widevariance in their estimates. Frameworks help to reduce this variance.

This can be explained in another way as follows. There are four main parameters which contribute to overall variance inestimation. They are:

1. Variance when the same expert does the estimation at different times.2. Variance when two different experts do the estimation at the same time.3. Variance when the same expert does estimation for two different requirements.4. Variance when experts interpret risk parameters differently and give different ratings for the same risk.

The variance in estimation due to parameters (3) and (4) are unavoidable. Two different requirements will have two differentestimations anyway. Also, since the risks are something which would occur in future with a probability associated with it, itwould be very subjective. We can expect different estimations in this case too1 .

If an expert does estimation at two different times on the same requirement, nobody expects a variation between the twoestimates. Also, two experts are expected to give the same/close estimates for the same requirement. So, in both these casesvariance should be zero or minimum. This is where using frameworks would help us. It gives the same estimate at different timesfor the same requirement. It also gives close estimates when two people are using it for doing estimation for the samerequirement. This leads to reduced overall variance.

Usually, 3 types of inputs are fed into frameworks: Requirement attributes Framework constants Estimator attributes

Requirement attributes are inputs to the framework depending on the requirement in hand. Examples are no. of features,complexity of the features etc.

Framework constants are usually built into the system based on past data, used at the time of designing the framework. It is truethat these constants would get fine-tuned as more data becomes available over time.

Estimator attributes are those inputs given by the estimator as per his gut feel about different factors, which might occur in thefuture, such as risks, certain team profile etc. For example in a typical testing project, the time taken by a tester to execute a testcase is not known at the time of estimation. This is because the information about who will be in the team is not known at thattime. This will go into the framework as an estimator attribute.

Frameworks use equations to get estimates. Lets take a testing project as an example. The requirements say 10 new featureshave to be tested and the guidelines for the complexity measure tell that on an average the complexity of these features is 2.5.Now these are the inputs to the framework as Requirement attributes.

The data from the past projects reflect that on an average 55 test cases are needed for a feature with a complexity factor of 1.The number 55 is a framework constant, which is already built into the equation in the framework.

1 Subjectivity in risk assessments can be reduced using any standard method of risk identification, rating and prioritization. Some guidelinesalso can be used.



06 of 14

The execution time per test case is to be entered into the framework to get the estimate. But the execution time depends onthe experience level of the project team that would be executing the project. This will go into the framework as an estimatorattribute. If the estimator feels that the expected team profile suggests 20 minutes as the execution time for a test case, the totaltime needed for the project can be calculated as follows.

No. of test cases = no. of features * complexity * test cases = 10 * 2.5 * 55Execution time = 20 * 10 * 2.5 * 55 = 27500 person minutes = 458 person hours.

This method is used under the following circumstances: There exist similar projects executed earlier in the organization and somebody analyzed the past data and has already

come up with a framework. Senior team members are available to do estimation at the time of estimation process.

Breaking off from here, the next section describes different ways used for putting across effort estimates to a client or championof the project. This is required at the present stage as the remaining 4 methods of effort estimation integrate different types ofestimation statements in it. The discussion of estimation methods would resume after this section.

ESTIMATION STATEMENTSThere are 3 ways to state an estimate of the execution effort needed for a project.

Point estimate Interval estimate Upper limit estimate

Point estimatePoint estimatePoint estimatePoint estimatePoint estimateThis is a single value estimate. As an example, the estimate for Project A can be stated as follows. Project A may take 450 personhours of effort. Usually point estimate is the average estimate arrived at through collective thought, Delphi or frameworkmethod.

Interval estimateInterval estimateInterval estimateInterval estimateInterval estimateThis is a double value estimate defining an interval qualified by a confidence level. For example, one can say that Project A maytake 425- 475 person hours of effort with 95% confidence level.

Upper limit estimateUpper limit estimateUpper limit estimateUpper limit estimateUpper limit estimateThis is again a single value estimate; but it specifies an upper limit (rather than an average value) and is qualified by a confidencelevel. As an example, an estimator could state that, Project A will take less than 460 person hours with a confidence level of75%.

Interval estimate is important from the perspective of a project manager, as she/he is more comfortable with an interval ratherthan a point estimate. Here, she/he will get a spread to bank on.

Confidence levelConfidence levelConfidence levelConfidence levelConfidence levelConfidence level is used in Interval and Upper Limit estimates. Confidence level is the probability that the actual effort will bewithin the specified interval of the estimate. In the case of interval estimate, the two numbers in the estimate specify the interval.In the case of upper limit estimate, 0 and the upper limit value specify the interval.



07 of 14

An upper limit estimate qualified by a confidence level is important from business perspective costing and bidding. There are4 main criteria which determine the business criticality of a project. They are:

Existing client - yes/noTerm of association with the client long/medium/short/zeroNature of competition strong/medium/weakNature of the project Fixed Price project (FPP) / Time & Material (T&M)

Combination of these parameters we have in hand determines the confidence level of the estimate that will be used for costingor bidding. The following are the general thumb rules:

If the client is new, it is natural that we bid low; so we will go for a low confidence level estimate to get a new client forus.

If we have already long term association with a client, we can go for a higher level of estimate. If the competition is strong, we bid low. If the project is FPP, the risk is high and we quote for a high confidence level estimate to accommodate for any risk

factors.

A tool has been developed which will help in determining which confidence level of estimate we can use to bid for a project,the Excel version of which may be obtained from the author. This tool is developed based on thumb rules rather than any strictformulas. Depending on the combination of 4 parameters mentioned before, the tool throws out any of the five confidencelevels - 5%, 25%, 50%, 75% or 95%. The effort estimation methods further discussed in the article would help a businessmanager in arriving at an estimate corresponding to any of these confidence levels.

Table 2: Confidence level selection tool for estimation

Business Criticality VsBusiness Criticality VsBusiness Criticality VsBusiness Criticality VsBusiness Criticality Vs

Estimation Confidence LevelEstimation Confidence LevelEstimation Confidence LevelEstimation Confidence LevelEstimation Confidence Level

Existing client Yes

Term of Association Medium

Nature of Competition Medium

Nature of the project FPP

Confidence Level to be used 75%(Upper Limit Estimation)



08 of 14

COLLECTIVE THOUGHT - ADVANCEDIn this case, all the characteristics described for the collective thought method holds true, but the team would come up with 3estimates (triple estimates). The three estimates are termed optimistic, most likely and pessimistic. The use of a framework isoptional and depends on availability of the same.

Optimistic estimate (OE) is defined as the effort wherein everything goes fine for the project. In terms of risks, it can be treatedas an estimate when most of the identified risks do not occur. Most likely estimate (ME) is defined as the effort project may takemost of the time as judged by prior experience. In terms of risks, it can be treated as an estimate after accommodating the risksthat occur frequently. A point to be noted here is that the most likely estimate is not an average estimate; it is the estimate ofeffort the project may take most of the time. Pessimistic estimate (PE) is defined as the effort when everything goes against theproject. In terms of risks, it can be treated as an estimate when most or all of the identified risks do occur.

Now the average estimate and standard deviation are calculated using the following equations.Average estimate AE = (OE + 4*ME + PE) / 6Standard deviation SD = (PE OE) / 6

As an example, if the optimistic effort is 120 person months, most likely effort is 130 person months and pessimistic effort is150 person months; the average estimate is:

AE = (120 + 4*130 + 150) / 6 = 131.7 person monthsSD = (150 120) / 6= 5 person months

With this information we can arrive at the estimates with different confidence levels.Point estimatePoint estimatePoint estimatePoint estimatePoint estimate: Project will take 131.7 person months.

Interval estimateInterval estimateInterval estimateInterval estimateInterval estimate: With 95% confidence level we can say that the project will take 121.9 to 141.5 person months.Upper level estimateUpper level estimateUpper level estimateUpper level estimateUpper level estimate: With 75% confidence level we can say that the project will take less than 135.1 person months.

The tool generates the interval estimate for 75% and 95% confidence levels. It also gives upper limit estimates for 95%, 75%,50%, 25% and 5% confidence levels. Table 2 shows a typical output from the tool for different confidence levels.

Collective thought-advanced method can be used under the following circumstances: Senior team members are available to do estimation at the time of estimation process. Team has enough insight into the project needs and is able to analyze the risk factors so that they can reach on the

optimistic, most likely and pessimistic estimates. Even though the use of framework is optional, it should be used if one is available. Project/business manager needs estimates for different confidence levels.



09 of 14

Table 3: Effort estimation statement (tool output)

DELPHI- ADVANCEDIn this method of estimation, the moderator gets the individual estimates from different experts. Thereafter, the moderatorprocesses these estimates as follows and states his estimate using any of the 3 ways- point, interval or upper limit estimates. Useof framework is optional depends on the availability of the same.

Point estimate:Point estimate:Point estimate:Point estimate:Point estimate: Moderator uses mean, median or mode. For example, let us assume that the individual estimates from expertsare 210, 215, 219, 215, 224 and 215. Then the Mean is 216.3.

Interval estimate:Interval estimate:Interval estimate:Interval estimate:Interval estimate: Moderator finds the mean and standard deviation from the individual estimates. In the above example,Mean = 216.3Standard deviation = 4.7

With this information, one can state with 95% confidence that the effort for the project will be between 207 and 225.

Upper limit estimate:Upper limit estimate:Upper limit estimate:Upper limit estimate:Upper limit estimate: From the above example, it could be stated with 75% confidence that the effort required for the projectwill be less than 219.6 person hours.

Given estimations from experts, using Delphi techniques, the tool automatically calculates the mean estimate and gives effortstatements for different confidence levels. This tool can process inputs from a maximum of 20 experts.

Delphi-Advanced method is used in the following circumstances. There are enough experts available within the organization who can judge over the requirements. The project manager is able to identify different experts inside and outside the project team to provide estimates. Even though the use of framework is optional, it should be used if one is available. The project t different confidence levels for strategic or any other reasons.

Effort Estimation StatementEffort Estimation StatementEffort Estimation StatementEffort Estimation StatementEffort Estimation Statement

Optimistic Effort 120 units

Most likely Effort 130 units

Pessimistic Effort 150 units

Point EstimatePoint EstimatePoint EstimatePoint EstimatePoint Estimate

Project Effort

Lower Limit Upper Limit

75% Confidence Interval

95% Confidence Interval

Estimation by Upper LimitEstimation by Upper LimitEstimation by Upper LimitEstimation by Upper LimitEstimation by Upper Limit

Interval EstimateInterval EstimateInterval EstimateInterval EstimateInterval Estimate

Upper limit Effort

95% probability that effort will be less than





139.9

135.1

131.7

128.3

123.4

131.7

125.9 137.4

121.9 141.5

Lower Limit Upper Limit



10 of 14

FRAMEWORK - ADVANCEDIn this method, a framework is used by an expert to arrive at an estimate. As discussed earlier, there is a category of inputs calledestimator attributes in a framework that adds variability to the estimates. In the Framework-Advanced method of estimation,the expert is allowed to give 3 kinds of estimates - optimistic, most likely and pessimistic. The optimistic estimate is arrived atwith an input of the optimistic values for the estimator attributes in the framework. Similarly for most likely and pessimisticvalues as well.

In the earlier example under Framework Method, if the test case execution time is 17 minutes optimistic, 21 minutes mostlikely and 23 minutes pessimistic; the efforts will be:Execution time optimistic = 17 * 10 * 2.5 * 55 = 23375 person minutes = 390 person hoursExecution time most likely = 21 * 10 * 2.5 * 55 = 28875 person minutes = 481 person hoursExecution time pessimistic = 23 * 10 * 2.5 * 55 = 31625 person minutes = 527 person hours

Mean = (390 + 4* 481 + 527) / 6 = 473.5 person hoursStandard deviation = (527 - 390) /6 = 22.8 person hours

Any one of the estimation statements could be used to state the estimate.Point estimatePoint estimatePoint estimatePoint estimatePoint estimate: The effort required by the project is 473.5 person hoursInterval estimate:Interval estimate:Interval estimate:Interval estimate:Interval estimate: With 95% confidence the effort required by the project will be between 428 and 516 person hoursUpper limit estimate:Upper limit estimate:Upper limit estimate:Upper limit estimate:Upper limit estimate: With 75% confidence the effort required by the project will be less than 489 person hours

The tool introduced in the section of Collective Thought- Advanced method can be used here also to generate estimationstatements easily.

Framework-advanced method is used under the following circumstances: Senior team members are available to do estimation at the time of estimation process. There exist similar projects executed earlier within the organization and analysis has already been done on the past data

and a framework has been created. Team has enough insight into the project needs and is able to analyze the risk factors so that they can reach on the

optimistic, most likely and pessimistic estimates. The project manager needs estimates at different confidence levels.



11 of 14

HYBRID METHODIn this method, participating experts in the Delphi method are asked to use the framework already available and provide 3estimates - optimistic, most likely and pessimistic. This means that all the experts would give 3 estimates each. The followingtables illustrate how this information will be processed to arrive at overall mean and overall standard deviation of the estimates.

OptimisticEffort

Most LikelyEffort

PessimisticEffort

Mean Effort Standard Deviation ofEffort

Expert 1 O1 M1 P1 A1= (O1+4*M1+P1)/6 S1= (P1-O1)/6



Overall Mean Effort (A1 + A2 + A3)/3

Overall SD of Effort Sqrt (S1^2+ S2^2 + S3^2)

Table 4: Processing of triple estimates from Delphi experts

An example is given below

OptimisticEffort

Most LikelyEffort

PessimisticEffort

Mean Effort Standard Deviation ofEffort

Expert 1

Expert 2

Expert 3

400 420 450 421.7 8.3

390 415 430 413.3 6.7

405 425 440 424.2 5.8

Overall Mean Effort 419.7

Overall SD of Effort 12.2

Table 5: Example of processing of triple estimates from Delphi experts

The overall mean and overall standard deviation are used to issue 3 types of estimation statements as explained in the previoussections.

This tool gives interval estimates for 75% and 95% confidence levels. It also gives upper limit estimates for 95%, 75%, 50%,25% and 5% confidence levels. This tool can process inputs from a maximum of 20 experts. A sample of the input data sheetis given below



12 of 14

(Figure 1: Sample Input to the tool)

This method is used in the following circumstances:

There are enough experts available within the organization who can judge over the requirements. The project manager is able to identify different experts inside and outside the project team to give estimates. There are similar projects executed earlier within the organization and analysis has been performed on the past data to

yield a framework. The project manager needs estimates at different confidence levels.

CONCLUSION

Table 6: Estimation methods - Reference table

InputsInputsInputsInputsInputs

Sl. No. Optimistic Effort Most Likely Effort Pessimistic Effort

12345

--------20

420390425435420--------

435420435440430--------

440430445445435--------

Meeting

Triple Estimate

Confidence level Independent Experts

Confidence level

Meeting

Independent Experts

Framework

Framework

Triple Estimate

Confidence level

Independent Experts

Framework

Triple Estimate

Confidence Level

Collective Thought

Delphi Method

Framework Method

Collective Thought

Advanced

Delphi Method

Advanced

Framework Method

Advanced

Hybrid

The project manager, business manager or the moderator is the person who has to decide which method of estimation wouldultimately be used. Table 1 and Table 5 can be used as references to take a decision on which method to use depends on thesituation in hand. The situation is defined by the 4 factors such as experts availability, framework availability, single estimate/triple estimate and nature of project/commercial importance (confidence level).



REFERENCES Levin, Richard I., Rubin, David S. (1999), Statistics for Managers, Prentice-Hall India, 7th Edition. Mulcahy, Rita (2002), PMP Exam Prep: A course in a Book, RMC Publications Inc., 4th Edition

ACKNOWLEDGEMENTSThe author would like to thank Mahesh Venkataraman (Group Head, Wipro Technologies), Pramod Varma (SQA Manger, WiproTechnologies) and Shyam Bhaskaran (Project Manager, Wipro Technologies) for their valuable inputs and support.

ABOUT THE AUTHORAjikumar TN is currently a Technical Manager for the Verification & Validation division of Wipro Technologies. He holds a B Techand an MBA in Software Enterprises Management. Ajitkumar has got over 9 years of experience in the software industry and hasworked for enterprises like BPL Telecom. He may be reached at [email protected].

13 of 14



www.wipro.comeMail: [email protected]

For further information visit us at: http://www.wipro.com/verification&validation

For more whitepapers logon to: http://www.wipro.com/insights

Copyright 2004. Wipro Technologies. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means, electronic, mechanical,photocopying, recording, or otherwise, without express written permission from Wipro Technologies. Specifications subject to change without notice. All other trademarks mentioned herein are theproperty of their respective owners. Specifications subject to change without notice.

14 of 14

ABOUT WIPRO TECHNOLOGIESWipro is the first PCMM Level 5 and SEI CMMi Level 5 certified IT services company globally. Wipro provides comprehensive ITsolutions and services (including systems integration, IS outsourcing, package implementation, software application developmentand maintenance) and research & development services (hardware and software design, development and implementation) tocorporations globally.

Wipros unique value proposition is further delivered through our pioneering Offshore Outsourcing Model and stringentquality processes of SEI and Six Sigma.

WIPRO IN VERIFICATION & VALIDATIONWipros Verification & Validation (V&V) services help software achieve high quality. The V&V service portfolio comprisessoftware life cycle testing, test process consultancy, test automation, and specialized testing. The V&V practice combines yearsof testing capability and a strong industry focus to offer a host of re-useable process frameworks and methodologies to crashtesting cycle times. With industry specific Centers of excellence, knowledge management practices, and above all a highdegree of quality orientation, partnering with Wipro assures a definite improvement in productivity & quality of testing.

Worldwide HQWipro Technologies,Sarjapur Road,Bangalore-560 035,India.

U.S.A. U.K. FranceWipro Technologies Wipro Technologies Wipro Technologies1300 Crittenden Lane, 137 Euston Road, 91 Rue Du Faubourg,Mountain View, CA 94043. London, NW1 2 AA. Saint Honor, 75008 Paris.

Germany Japan U.A.E.Wipro Technologies Wipro Technologies Wipro LimitedHorn Campus, # 911A, Landmark Tower, Office No. 124,Kaistrasses 101, 2-1-1 Minatomirai 2-chome, Building 1, First Floor,Kiel 24114. Nishi-ku, Yokohama 220 8109. Dubai Internet City,

P.O. Box 500119, Dubai.

Documents

Effort Estimation