© Copyright LEXTA GmbH 2006

On the difficulties in determining effort estimation accuracy in practice

MeLLow Workshop

Ursula Löbbert-Passing

October 17-18, 2006

Accuracy_v2.ppt

Exhibit 2

In the IT Management lifecycle, LEXTA covers benchmarking, cost cutting and sourcing as well as professionalising processes and projects

LIFECYCLE IT MANAGEMENT

IT Benchmarking

Professionalising IT processes

IT Sourcing

IT Cost Cutting

ProfessionalisingIT projects

Accuracy_v2.ppt

Exhibit 3

Estimation accuracy is considered as THE quality criterion for an estimation method

ActualActualPlan

Accuracy

• The most common formula for determining effort estimation accuracy (in percent) in practice is

• Positive value = Over-estimation

• Negative value = Under-estimation (budget overrun)

Accuracy_v2.ppt

Exhibit 4

Estimation accuracy is determined by comparing values from different points of time

Project execution

Estimation Measurement of

actual effort

Project lifetime

Unexpected event

(e. g. requirement shift)

Comparison:

Estimation accuracy

t

Accuracy_v2.ppt

Exhibit 5

Four factors “dilute” the effect of the estimation method on estimation accuracy

Information gapFaulty measurement of actual effort

Project management

Unexpected events

Estimation method

Estimation accuracy

Accuracy_v2.ppt

Exhibit 6

All factors contribute jointly to estimation (in)accuracy

Factor Contributes to estimation (in)accuracy, because

Estimation method

• Method takes/does not take all relevant information into account (parameters, number of aspects an expert can consider)

• Method produces (in)correct conclusions from available information (e. g. by model assumptions, extent of expert experience)

Information gap • At time of estimation, available project information (requirements, architecture, team etc.) is incomplete assumptions necessary

• At time of actual effort measurement, more (complete) information is available “Information gap”

Project management

Project management influences actual effort, e. g. by (in)efficient coordination of activities and teams „Parkinson’s Law“

Unexpected events

Unexpected events lead to unexpected effort „Information gap“ between time of estimation and time of actual effort measurement Strictly, estimated and measured effort are not comparable any more

Faulty measure-ment of actual effort

Measured effort is not necessarily the actual effort due to

• Manipulation of measured effort: Incentive exists when employees are evaluated by their estimation accuracy

• Faulty measurement: When actual effort does not “fit” into the measurement scheme (e. g. project phases), it may be booked somewhere non-appropriate Effort measured for one measurement category (e. g. one phase) is not necessarily the actual effort

• No measurement of overtime Measured effort is too low

„Noise“

Accuracy_v2.ppt

Exhibit 7

There are remedies to exclude three of the „noise“ factors

Information gapFaulty measurement of actual effort

Project management

Unexpected events

Estimation method

Estimation accuracy

Document input information for

estimates and update regularly

Update estimates after unexpected

events

Assure quality of effort measurements

• Do not evaluate employees based on their estimation accuracy

• Match structures of effort measurement and estima-tion templates

• Measure overtime

• Ex-post estimates are often recommended but imprac-tical in practice (extra effort)