View
250
Download
5
Category
Preview:
Citation preview
Why Large Projects Fail More OftenMegaproject Failures: Understanding the Effects of Size
Edward W. Merrow, President & CEOIndependent Project Analysis, Inc.
Joint Meeting of the AACEI National Capital Section and AmericanSociety of Mechanical Engineers Section
Wednesday, April 20, 2011
CONFIDENTIAL 2 INDEPENDENT PROJECT ANALYSIS
Framing the Issues
• Large projects in the Process Industries (oil, chemicals, minerals, power, pharmaceuticals) have substantially poorer outcomes than smaller projects
• By the time a project achieves megaproject status, over about $1 billion, the failure rate is two-thirds!
• Many more large projects are being developed than ever before in every industrial sector
• Our market forecast for this decade is for more rapid growth in megaprojects in history
• These projects represent a major source of financial risk to companies
CONFIDENTIAL 3 INDEPENDENT PROJECT ANALYSIS
The Key Questions
• Large projects are more difficult on almost every dimension of project management
• Do large projects fail more often simply because they are more difficult—or…
– Do we alter our practices?
– Do the projects not respond to what are good practices for smaller projects?
– Are practices not appropriate for the problems at hand?
CONFIDENTIAL 4 INDEPENDENT PROJECT ANALYSIS
Outline
• Databases
• Outcomes Change with Size
• Larger Projects Are More Sensitive to Practices
• Practices Vary with Project Size
• Implications and Recommendations
CONFIDENTIAL 5 INDEPENDENT PROJECT ANALYSIS
IPA’s Vantage Point on the Industry
• IPA evaluates capital projects for the process industries around the world
• In a typical year, we evaluate 700-800 projects
• Our evaluations usually occur twice on the front-end and twice at the end of the project, at the end of startup and after one year of operation
• As a result, we have developed very large global databases of capital projects
CONFIDENTIAL 6 INDEPENDENT PROJECT ANALYSIS
Depth and Quantitative Nature of Practice and Outcome Measurements
• Up to 3,500 questions per project; 13,000+ total projects, 318 megaprojects over 22 years– Number and type of questions/data depend on project
scope, industry sector, type of project> Greenfield, revamp, turnaround, new technology, pipelines
vs. plant facilities
> Categories are based on quantitative, statistically-validated evaluations of practices and their outcomes
> Data on practices – mixture of quantitative, yes/no, dates, text, likert scales; all projects include lengthy face-to-face interview with project team
• Key practices include clarity of business & project objectives, front end loading, and team integration
CONFIDENTIAL 7 INDEPENDENT PROJECT ANALYSIS
$3,250$197 (mean)
$58 (median)Project Cost (in millions 2011 US$)
Megaprojects Database
Process Facilities Database
Number of Owners Represented
Percent of Projects with Any New Technology
Average Execution Duration (months)
Average Authorization Year
Number of Projects
340
16%
22 (mean)
21 (median)
2000
4500
77
36%
43
2005
318
Research Database Description
Characteristics
CONFIDENTIAL 8 INDEPENDENT PROJECT ANALYSIS
Industries Represented
Process Facilities DatabaseRefining Refining
27%27%
Chemicals Chemicals 38%38%
MMM MMM 8%8%
PharmsPharms8%8%
Consumer Consumer ChemChem& Pulp/Paper & Pulp/Paper
7%7%
Distribution Distribution 5%5%
Onshore E&P Onshore E&P 4%4%
OtherOther3%3%
Refining Refining 21%21%
Chemicals Chemicals 10%10%
MMM MMM 15%15%
LNG LNG 8%8%
Power Power Generation Generation
3%3%
Distribution Distribution 2%2%
Other Other 2%2% Oil & Gas Production Oil & Gas Production
41%41%
Megaprojects Database
CONFIDENTIAL 9 INDEPENDENT PROJECT ANALYSIS
United States Process: 48%
Mega: 13%
EuropeProcess: 19%
Mega: 15%
AsiaProcess: 8%
Mega: 9%
OceaniaProcess: 4%
Mega: 9%
AfricaProcess: 3%
Mega: 9%
South AmericaProcess: 7% Mega: 19%
Geographical DistributionProcess Projects and Megaprojects
CanadaProcess: 5%
Mega: 7% Central Asia Mega: 4%
Middle EastProcess: 4% Mega: 15%
Caribbean Process: 1%
Mexico Process: 1%
CONFIDENTIAL 10 INDEPENDENT PROJECT ANALYSIS
Outline
• Database
• Outcomes Change with Size
• Larger Projects Are More Sensitive to Practices
• Practices Vary with Project Size
• Implications and Recommendations
CONFIDENTIAL 11 INDEPENDENT PROJECT ANALYSIS
Most Key Outcomes Degrade with Size
0.02DeclinesOperability
0.001
0.03
Decreases up to $650 million and then increases
Schedule Slip
0.0001Gets worseCost Competitiveness
0.001IncreasesCost Growth
Probability OutcomeAs Project Size Increases
CONFIDENTIAL 12 INDEPENDENT PROJECT ANALYSIS
Outcomes Diverge
• We mostly maintain control of projects up to about $600-750 million
• Over $750 million, project outcomes start to degrade very rapidly
• A wide chasm develops between good and poor projects:
– Good projects are genuinely excellent
– Poor projects tend to be poor on most every outcome
CONFIDENTIAL 13 INDEPENDENT PROJECT ANALYSIS
Costs grew (real) 25% +
Schedule Slipped 25% +
Execution time (Absolute Measure) 50% +
Severe and Continuing Operational Problems into Year 2 after startup
Yes
Defining Success and Failure
• We deem a project to be a failure if one or more of the following occurred:
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Megaprojects
Failure Rate
Overspent (Absolute Measure) 25% +
Projects <$500 MM
CONFIDENTIAL 14 INDEPENDENT PROJECT ANALYSIS
80%
COST INDEX – (Competitiveness)
PRODUCTION(Predictability)
EXECUTION SCHEDULE INDEX
(Competitiveness)
EXECUTIONSCHEDULE SLIP
(Predictability)
COST GROWTH(Predictability)
1.3
1.0
1.2
1.0
1.3
1.2
1.1
0%10%
30%
20%
0%
Failed Projects
Successful Projects (35 percent)
40%
100%
60%
1.1
10%
20%30%
Source: Merrow, Edward W. (2011). Industrial Megaprojects : Concepts, Strategies, and Practices for Success. Hoboken, New Jersey: John Wiley & Sons, Inc.
Megaprojects Split Into Radically Different Groups
CONFIDENTIAL 15 INDEPENDENT PROJECT ANALYSIS
80%
COST INDEX
PRODUCTION
EXECUTION SCHEDULE INDEX
EXECUTIONSCHEDULE SLIP
COST GROWTH
1.3
1.0
1.2
1.0
1.3
1.2
1.1
0%10%
30%
20%
0%
Failed Projects
Successful Projects (63 percent)
40%
100%
60%
1.1
10%
20%30%
Non-Megaprojects Separate Very Differently
CONFIDENTIAL 16 INDEPENDENT PROJECT ANALYSIS
Outline
• Database
• Outcomes Change with Size
• Larger Projects Are More Sensitive to Practices
• Practices Vary with Project Size
• Implications and Recommendations
CONFIDENTIAL 17 INDEPENDENT PROJECT ANALYSIS
-0%
-2%
-4%
-6%
-8%
-10%
-12%
-14%
$10MM -$25MM
$25MM -$100MM
$100MM -$250MM
$250MM -$500MM
Megaprojects
Effe
ct o
f Tea
m U
nder
stan
ding
Bus
ines
s O
bjec
tives
on
Cos
t Ind
ex
Clear Objectives Are Much MoreImportant for Megaprojects
CONFIDENTIAL 18 INDEPENDENT PROJECT ANALYSIS
Project ObjectivesProject ObjectivesProject Objectives Team IntegrationTeam IntegrationTeam Integration Roles &Responsibilities
Roles &Roles &ResponsibilitiesResponsibilities
ProjectImplementation
Process
ProjectProjectImplementationImplementation
ProcessProcess
• Specific project objectives have been developed
• Objectives have been clearly communicated to and are understood by all project team members
• All functions that can influence the project’s outcomes are represented on project team
• Team is adequately staffed; no TBD on org chart
• Roles, responsibilities, and expectations have been clearly defined
• Responsibilities and tasks have been agreed on
• Project team is aligned
• Problem areas have been identified
• Common work process is in place for developing and executing the project
• Process is well understood by project team
Components of Team DevelopmentPB4
Slide 18
PB4 Added this slide
For same reason we present clearly understood objectives, we present Team Integration,
And I want to show a bit more on the basis
Didn't shorten bullets on this, not as many as for FELPaul Bizot, 4/18/2011
CONFIDENTIAL 19 INDEPENDENT PROJECT ANALYSIS
Team Integration Is Important for AllBut Crucial for Larger Projects
-30%
-25%
-20%
-15%
-10%
-5%
0%$10MM -$25MM
$25MM -$100MM
$100MM -$250MM
$250MM -$500MM
Megaprojects
Effe
ct o
f Tea
m In
tegr
atio
n on
Cos
t Ind
ex
CONFIDENTIAL 20 INDEPENDENT PROJECT ANALYSIS
Front End Loading is a Key Practice
SiteFactors
SiteFactors
EngineeringDefinition
EngineeringDefinition
ProjectExecution
Plan
ProjectExecution
PlanFELFEL
• Equipment layout
• Soils data
• Environmental requirements
• Health & Safety requirements
• Local labor quality and availability
• Local materials availability
• Engineering tasks– Detailed scope– Feedstock/product
properties– Program Definition– Drawings which are
basis for design– Major equipment specs– Cost estimate
• Participation/buy-in of all who can influence project
• Contracting strategy• Team participants & roles• Integrated schedule• Plans for all phases and
resources• Cost/schedule controls
PB3
Slide 20
PB3 Added this slide
Again this audience probably has very weak knowledge of IPA, and minimal "process" knowledge, though quite possibly all of them areinvolved in projects, some of them very large projects
Consolidated the details of this slide somewhat.
Am ambivalent about sharing the details; again this goes back to sharing something of the robustness and comprehensiveness of the measurements. Paul Bizot, 4/18/2011
CONFIDENTIAL 21 INDEPENDENT PROJECT ANALYSIS
FEL is Most Important for Megaprojects
0%
10%
20%
30%
40%
Best Practical Good Fair Poor Screening
Effe
ct o
f FEL
on
the
Cos
t Ind
ex
FEL Index
Megaprojects
MISERABLEPOORFAIRGOODBEST
250 - 500
100 - 250
<100MM
Note that FEL is also crucialfor even “largish” projects
CONFIDENTIAL 22 INDEPENDENT PROJECT ANALYSIS
Turnover of the Project Leader Always HurtsBut It Destroys Megaprojects
-40%
-35%
-30%
-25%
-20%
-15%
-10%
-5%
0%$10MM -$25MM
$25MM -$100MM
$100MM -$250MM
$250MM -$500MM
Megaprojects
Effe
ct o
f PM
Tur
nove
r on
Pro
ject
Suc
cess
Rat
e
CONFIDENTIAL 23 INDEPENDENT PROJECT ANALYSIS
The Effects of Practices Increase with Project Size
• Large projects are very sensitive to practices because they are more complex
• Much more difficult to “work around” surprises because so much more has to be coordinated
• All of this is well known
• Therefore, logic would suggest that larger projects systematically follow better practices than smaller ones
CONFIDENTIAL 24 INDEPENDENT PROJECT ANALYSIS
Outline
• Database
• Outcomes Change with Size
• Larger Projects Are More Sensitive to Practices
• Practices Vary with Project Size
• Implications and Recommendations
CONFIDENTIAL 25 INDEPENDENT PROJECT ANALYSIS
Practices Actually Degrade with Size
• Team integration is much poorer for megaprojects (55 percent) than projects under $750 million (74 percent)
• Turnover increases
• One megaproject team in five admits it does not understand the business objectives versus one in ten for smaller projects
• Megaproject teams are much more likely to report being confused about trade-offs among outcomes
• FEL completeness actually declines as projects get large
CONFIDENTIAL 26 INDEPENDENT PROJECT ANALYSIS
25%
30%
35%
40%
45%
50%
10 50 100 150 200 250 300 350 400 450 500
Cost 2011$ (million)
Perc
ent o
f Pro
ject
s th
at E
xper
ienc
e Pr
ojec
t Man
ager
Tur
nove
r
Larger Projects Experience More Frequent Project Manager Turnovers
Pr < 0.001
Note: Controlling for Execution Schedule Index
Turnover rate in Megaprojects
Pr < 0.005
CONFIDENTIAL 27 INDEPENDENT PROJECT ANALYSIS
Most FEL Elements Degrade as Size Increases
Tends to degradeEngineeringTends to improveExecution PlanningDegradesOperational Health and SafetyDegradesPermittingNo changeSoils DefinitionDegradePlot Plans and ArrangementsDegradeLocal Material CostsDegradesLabor Cost and ProductivityDegradesLabor AvailabilityNo changeLocationChange with SizeElement of Definition (FEL)
CONFIDENTIAL 28 INDEPENDENT PROJECT ANALYSIS
0%
10%
20%
30%
40%
50%
60%
70%
Perc
enta
ge o
f Pro
ject
s
FEL Index
MISERABLEPOORFAIRGOODBEST
How Well Are Megaprojects Defined at Authorization?
Source: Industrial Megaprojects, op. cit.
CONFIDENTIAL 29 INDEPENDENT PROJECT ANALYSIS
What Difference Does it Make?
0%
10%
20%
30%
40%
50%
60%
70%
Percent in Group Percent Successful in Group
Perc
enta
ge o
f Pro
ject
s
FEL Index
MISERABLEPOORFAIRGOODBEST
CONFIDENTIAL 30 INDEPENDENT PROJECT ANALYSIS
As Projects Get Large…
• They are more likely to have multiple, sometimes competing, objectives
• Business sponsors tend to be less certain and clear about what they need for a successful venture
• It becomes much more likely that there are JV partners and otherstakeholders to align
• Governments take more interest
• We are more likely to require new technology
• Even if the technology is conventional, we are less likely to have complete basic data
• Permits are more important for large projects, but we are much less likely to have them in-hand at authorization
• The business role is to shape the opportunity in a way that makes project management practicable
CONFIDENTIAL 31 INDEPENDENT PROJECT ANALYSIS
Megaproject Development Is More Complex
FEL 2 Scope Development FEL 3 FEED
Venture Shaping Process
• Allocate value • Align stakeholders• Close the business deal• Stabilize the project environment
Project FEL Process
Basic Data Development
Project Timeline
Closure
CONFIDENTIAL 32 INDEPENDENT PROJECT ANALYSIS
Excellent FEL Completes a Complex Process
Clarity of Trade-offsAmong Outcomes
Clarity of theBusiness Objectives
ExcellentFEL
Best Practical FEL Doubles Chances of Success
TeamIntegration
CONFIDENTIAL 33 INDEPENDENT PROJECT ANALYSIS
0
10
20
30
40
50
60
70
Very Fairly Somewhat Unclear
Under $500MM Megaprojects
Perc
ent o
f Pro
ject
sOnly a Slight Difference in Clarity of Business
Objectives Between Megas and Non-Megas
Team Understanding of Project Business Objectives
CONFIDENTIAL 34 INDEPENDENT PROJECT ANALYSIS
0
10
20
30
40
50
60
70
80
Very Fairly Somewhat Unclear
Under $500MM Megaprojects
Perc
ent o
f Suc
cess
fulP
roje
cts
But Differences in Effects Are Huge
Team Understanding of Project Business Objectives
CONFIDENTIAL 35 INDEPENDENT PROJECT ANALYSIS
Causal Path Runs Through Team Integration and Trade-offs to FEL
Clarity of Trade-offsAmong Outcomes
Pr>|t|< .0001
Clarity of theBusiness Objectives
ExcellentFEL
Best Practical FEL Doubles Chances of Success
TeamIntegration
Pr>|X2|< .0001
Pr>|t|< .0001
Pr>|z|< .0001
Pr>|X2|< .0001
CONFIDENTIAL 36 INDEPENDENT PROJECT ANALYSIS
Team Integration and Clarity of Objectives Go Together
0
10
20
30
40
50
60
70
80
Very Fairly Somewhat Unclear
Integrated Not Integrated
Data are for megaprojects only, but are similar for all projects
Perc
ent o
f Pro
ject
s
Team Understanding of Project Business Objectives
Pr>|X2|< .0001
CONFIDENTIAL 37 INDEPENDENT PROJECT ANALYSIS
Causal Path Runs Through Integration and Trade-offs to FEL
Clarity of Trade-offsAmong OutcomesPr>|X2|< .0001
Clarity of theBusiness Objectives
ExcellentFEL
Best Practical FEL Doubles Chances of Success
TeamIntegration
Pr>|t|< .0001
Pr>|t|< .0001
Pr>|z|< .0001
Pr>|X2|< .0001
CONFIDENTIAL 38 INDEPENDENT PROJECT ANALYSIS
Trade-offs Are Less Clear for Megaprojects
0
5
10
15
20
25
30
35
40
45
Very Fairly Somewhat Unclear
<$500MM Megas
Perc
ent o
f Pro
ject
s
Team Understanding of Cost/Schedule/Operability Trade-offs
CONFIDENTIAL 39 INDEPENDENT PROJECT ANALYSIS
Team Integration Is the Critical Enabler
Very Clear Fairly Clear SomewhatClear
Not Very Clear Not Clear At All
FEL
Inde
x
Clarity of the Cost/Schedule/Operability Trade-offs
Fair
Bes
tM
iser
able
Goo
dPo
or
Pr < 0.0001
Pr < 0.91
Source: Industrial Megaprojects, op. cit.
Team Is Not IntegratedTeam Is Integrated
CONFIDENTIAL 40 INDEPENDENT PROJECT ANALYSIS
Causal Path Runs Through Integration and Trade-offs to FEL
Clarity of Trade-offsAmong OutcomesPr>|X2|< .0001
Clarity of theBusiness Objectives
ExcellentFEL
Best Practical FEL Doubles Chances of Success
+Team
Integration
Pr>|t|< .0001
Pr>|t|< .0001
Pr>|z|< .0001
Pr>|X2|< .0001
All of the relationships on this graph are among the strongest statistical results in project research
CONFIDENTIAL 41 INDEPENDENT PROJECT ANALYSIS
What Is Going On?
• As size increases, we see difficulty accelerating in the face ofdegrading practices
• The practices are harder as projects get bigger
– More resources are required to develop
– Those resources are in short supply
• If the project is far from home, personnel are more expensive
• If the project is remote, we do not (or cannot) staff it
• As the demographic shift plays out, experienced people are becoming scarce
• Worst of all, as projects get bigger, the businesses are less sure of what they want and the chances of non-alignment with stakeholders increase
CONFIDENTIAL 42 INDEPENDENT PROJECT ANALYSIS
Outline
• Database
• Outcomes Change with Size
• Larger Projects Are More Sensitive to Practices
• Practices Vary with Project Size
• Implications and Recommendations
CONFIDENTIAL 43 INDEPENDENT PROJECT ANALYSIS
Answering the Questions Posed
• Very large projects do not fail simply because of the difficulties stemming from size
• Not only do megaprojects respond well to good practices, they respond better than smaller projects
• Conversely, they degrade very rapidly with practices that are merely mediocre for smaller projects
• But we are actually much less likely (for organizational reasons) to follow those good practices for our megaprojects
• The only practice that is almost unique for megaprojects is Venture Shaping and most companies lack a coherent approach to this process
CONFIDENTIAL 44 INDEPENDENT PROJECT ANALYSIS
Some Implications
• Very large project outcomes are so much worse than those of smaller projects that we are systematically over-estimating economies of scale
• As an industry, we are headed for trouble in this decade that will dwarf the difficulties of the previous decade– We have more megaprojects coming in this decade than
ever before—literally hundreds in the next 5 years
– Neither owners nor contractors have the staff needed to properly resource these projects
– Businesses seem no more sure-footed today than in days past, maybe less so
– Businesses are even less project-savvy today than yesterday
CONFIDENTIAL 45 INDEPENDENT PROJECT ANALYSIS
Recommendations
• Doing real portfolio management of projects will be more important than ever
• Marginal projects simply must be dropped or there will be little chance of doing anything well
• Resource constraints must figure into the project selection decisions
• Disadvantages of megaprojects in the portfolio need to be recognized and risks hedged
• Nothing will help, however, unless we can achieve stronger business work process for megaprojects
CONFIDENTIAL 46 INDEPENDENT PROJECT ANALYSIS
Megaprojects Require a StrongBusiness-Led Shaping Process
• In the past, we have developed each megaproject as a unique event with no systematic business work process
• Business professionals need to adopt a much more disciplined approach to large project development
• That work process must:
– Frame the business opportunity
– Systematically align the stakeholders by distributing the project’s value
– Create a stable platform from which the project can be executed without excessive turbulence in its environment
• Shaping process must be integrated with the FEL process and come to closure at the end of FEL 2 (Scope Development)
CONFIDENTIAL 47 INDEPENDENT PROJECT ANALYSIS
Implications for Public Sector Projects
• There is widely held belief that large public sector projects tend to overrun because the estimates are deliberately low-balled
• Our analysis of large private sector projects suggests that no Machiavellian explanation is required
• Large projects have a dismal track record because we have not adjusted our practices to fit the difficulty that the projects present
Thank you!Contact Information:
IPA, Inc.44426 Atwater Drive
Ashburn, Virginia USA 20147+1 703.729.8300
www.ipaglobal.com
Recommended