Hugh Kelly

Cohesion Policy evaluation –

Cost and time estimates of major projects

Warsaw 30 November

Importance of reliable estimates

Estimates are critical:

• Public policy decisions

• Investment decisions

• Accountability

Cost overruns and benefit shortfalls:

• Lead to an inefficient allocation of resources

• Lead to delays and further cost overruns

• Destabilize policy, planning, implementation and project operation1

1. See Megaproject Policy and Planning: Problems, Causes and Cures, Flyvberg, Department of Development and Planning, Denmark

Estimates and evaluation

Ex post evaluation

• What estimates were used in the ex ante CBA?

• How did the outcome compare to estimates?

• What were reasons for differences?

• How did they impact on the effectiveness of the project?

But major projects can go spectacularly over budget

Denver's International Airport

$1.5bn

$5.0bn

Channel Tunnel

£4.7bn

£9.5bn

Boston’s Central Artery “The Big Dig”

$3.7bn

$14.6bn

Sydney Opera House

A$7.2m

A$102m

http://miller-mccune.com/business_economics/bill-of-goods-the-world-s-biggest-boondoggles-601

And the evidence suggests that the problem is not restricted to flagship projects

ERDF 2000-2006 Evaluation (51 projects):• Average cost overrun• Rail +27%• Road +9%• Urban transport +45%• Water +11%• Energy +21%

Flyvberg: 44 Urban rail projects:Average overrun 45%

Traditional explanations

Source: European Commission DG XVI (1998), “Understanding and Monitoring the Cost-Determining Factors of Infrastructure Projects”, A User’s Guide, Brussels

Numerous studies confirm the reasons for over runs

Source: ECORYS et al. (2005), “Ex post evaluation of a sample of projects co-financed by the Cohesion Fund (1993-2002)”, Synthesis Report, Rotterdam

Recent review of ERDF projects

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

Desig

n ch

anges

Legis

latio

n ch

anges

Enviro

nmen

tal i

mpac

t

Autho

rity

delay

s

Work

susp

ensi

ons

Politi

cal

Perm

its/C

onse

nts

Site c

hara

cter

istic

s

Late

com

men

cem

ent

Const

ruct

ion

perio

d

Fundin

g pro

blem

s

Stake

hold

er n

umbe

rs

Techn

ology

Innova

tion d

egre

e

Contra

ctor

spec

ific

Desig

n co

mpl

exity

Contra

ct c

omple

xity

Proje

ct m

anag

emen

t

Infla

tion

Site a

cces

s

Public

rela

tions

Force

maj

ure

Poor p

lanni

ng

Supplie

r dis

putes

Busin

ess

inad

equac

yOth

er

Econom

ic

Exchan

ge ra

tes

Sc

ore

Cost overrun score Time delay score

Traditional approach to dealing with the risk

• Risk management

– PPP

• Contingency funding

• More and more detailed budgets

Traditional solution – better planning and contingency

Source: Flyvbjerg, Bent in association with COWI (2004), “Procedures for dealing with Optimism Bias in Transport Planning: Guidance Document”, The British Department for Transport, London, June.

But if we understand the reasons for overrun why do we keep getting it wrong ?

Original bid cost

£2.3bn

£9.3bn

Current estimate

Alternative approach: Flyvberg: Delusion and Deception in large Capital Projects1

• Delusions Planning fallacy

Human optimism Anchoring and adjustment

Tendency to ‘stick’ to initial estimate Deception

Principal-Agent ProblemSources of strategic deception

Delusions and Deception in Large Infrastructure Projects, Bent Flyvberg, Massimo Garbuio, Dan Lovallo California Management Review Vol 51 No 2 Winter 2009

Principal agent issues

Sources of Strategic Deception

• Actors self interests

• Asymmetric information

• Different risk preferences

– Different time horizons

– Diffuse accountability

Different types of risks for different projects

Solutions – (a) risk sharing

• Pass risk of over runs to:– Member states– Local Governments– Contractors

Solutions – (b) reference class projects

• Select reference class• Assess distribution of outcomes• Make an intuitive prediction of project’s position in the

distribution• Assess the reliability of prediction• Correct intuitive estimates