Benchmarkingin the Projects Arena
By Larry DysertAACE International Cascade Section
AACE International – January 2004 – Portland, OR
Agenda
Definitions Benchmarking Process in General Benchmarking Process for Projects Project Benchmarking Organizations Practices vs. Metrics Findings & Trends
AACE International – January 2004 – Portland, OR
Benchmarking
AACE International – January 2004 – Portland, OR
Benchmark
A surveyor’s mark made on apermanent landmark of known positionand altitude - Webster’s New World Dictionary
A standard point of reference inmeasuring or judging quality, value, etc.- Webster’s New World Dictionary
AACE International – January 2004 – Portland, OR
Benchmarking
The practice of being humble enough toadmit that someone else is better atsomething, and being wise enough totry to learn how to match and evensurpass them at it - InternationalBenchmarking Clearinghouse
AACE International – January 2004 – Portland, OR
Benchmarking
The search for industry best practicesthat lead to superior performance -Robert Camp
The process of identifying and learningfrom best practices anywhere in theworld: A powerful tool in the quest forcontinuous improvement - IndependentProject Analysis
AACE International – January 2004 – Portland, OR
Benchmarking
The search for industry best practicesthat lead to superior performance -Robert Camp
The process of identifying and learningfrom best practices anywhere in theworld: A powerful tool in the quest forcontinuous improvement - IndependentProject Analysis
AACE International – January 2004 – Portland, OR
Benchmarking
Benchmarking can be applied to allfacets of a business.
Benchmarking implies measurement: Business function metrics Business practices
Benchmarking goes beyond traditionalcompetitive analysis
AACE International – January 2004 – Portland, OR
Benchmarking
Benchmarking challenges the currentway of doing business by bringing innew ideas and practices
Benchmarking is an objective-settingprocess
Effective benchmarking is a continuousprocess
AACE International – January 2004 – Portland, OR
Benchmarking
The origins of benchmarking are rootedin the basic competitive analysis andtotal quality management practiceswidespread in business
Benchmarking has passed the test oftime as a useful process and costimprovement technique
AACE International – January 2004 – Portland, OR
Approaches to Benchmarking
Competitive benchmarking
Functional benchmarking
Internal benchmarking
AACE International – January 2004 – Portland, OR
Competitive Benchmarking
Studies product designs, processcapabilities, or administrative methodsused by business competitors
Competitors may not employ best-in-class practices
Competitors can be reluctant partnersfor benchmarking
AACE International – January 2004 – Portland, OR
Functional Benchmarking
Benchmarking studies performed withnon-competitors
Attempts to find the secrets of anindustry leader’s success
Functional benchmarking relies oncooperation from best-in-class leaders
AACE International – January 2004 – Portland, OR
Internal Benchmarking
Attempts to find study partners withinsame organization
Fewer barriers to establish acooperative atmosphere
AACE International – January 2004 – Portland, OR
Benchmarking
Basic steps to benchmarking Know your operation Study the industry leaders and competitors Incorporate the best Gain superiority
AACE International – January 2004 – Portland, OR
Benchmarking Code of Conduct(International Benchmarking Clearinghouse)
Keep it legal Demonstrate willingness to share same level
of information you are requesting Respect confidentiality Don’t refer without permission - utilize
company benchmarking representativeswhen possible
Be prepared at all contacts withbenchmarking partners
AACE International – January 2004 – Portland, OR
Project Benchmarking Several studies in early 1980’s identified
similar trends in project execution: Reduction in project execution capabilities Lack of formal project execution training
programs Lack of business expertise in project teams Engineering practices and project
management practices not being applied
AACE International – January 2004 – Portland, OR
Project Benchmarking
Project benchmarking organizations: Construction Industry Institute Independent Project Analysis, Inc.
AACE International – January 2004 – Portland, OR
Construction Industry Institute 92 Companies 45 Owner Companies 47 Contractors
Variety of Industries Heavy Industrial Light IndustrialManufacturing Buildings Infrastructure
AACE International – January 2004 – Portland, OR
Construction Industry Institute
Objectives: Provide industry with performance metricsMeasure use of “best practices” on projects Quantify the value of utilizing “best
practices” Educate the industry in benchmarking
practices and opportunities forimprovement
AACE International – January 2004 – Portland, OR
Construction Industry Institute Best Practices Pre-Project Planning Team Building Constructability Safety Design/Information Technology Project Change Management Strategic Alliances
AACE International – January 2004 – Portland, OR
Construction Industry Institute
Performance Metrics Cost Schedule Safety Changes Rework
AACE International – January 2004 – Portland, OR
Independent Project Analysis
Industry Benchmarking Conference
Voluntary association of owner firms in theprocess industry
Divided into Upstream & Downstreamprocess groups
AACE International – January 2004 – Portland, OR
Independent Project Analysis
ObjectivesMeasure and compare project performance Identify and share practices that drive
excellence Conduct research into new project
practices
AACE International – January 2004 – Portland, OR
Independent Project Analysis
Best Practices Front-End Loading (FEL) Use of New Technology Use of Value Improving Practices Integrated Teams Safety
AACE International – January 2004 – Portland, OR
Independent Project Analysis
Performance Metrics Cost Schedule (Cycle Time) Operability Safety
AACE International – January 2004 – Portland, OR
Practices vs. Metrics
Benchmarking is the understanding ofpractices.
Metrics are used to quantify the effect ofpractices.
AACE International – January 2004 – Portland, OR
Practices vs. Metrics
Metrics have value beyond the processof benchmarking
Project cost and schedule metrics canimprove asset evaluation and conceptdevelopment
AACE International – January 2004 – Portland, OR
Practices vs. Metrics Metrics collection can be used for: Validation tool in estimating Strategic tool in estimating Setting project goals (“should” costs, target
schedules) Assess internal metrics versus industry
norms Support calibration of internal company
tools and databases Improve understanding of cost or schedule
drivers for value engineering analysis
AACE International – January 2004 – Portland, OR
Practices vs. Metrics
To use benchmarking effectively, it isimportant to stress practices in additionto metrics
Ensure that “best practices” areidentified, and utilized. Performancegains (as measured by metrics) shouldfollow
Sample Metrics
Sample Metrics
Sample Metrics
The Search for Best Practices
AACE International – January 2004 – Portland, OR
Findings & Trends
Cost risk is highly correlated with projectdefinition Good project definition results in smaller
cost deviations
Good project definition results insignificantly less variability
Good project definition results in more costeffective projects
Results of GoodProject Definition
0.90
0.95
1.00
1.05
1.10
1.15
1.20
3 3.5 4 4.5 5 5.5 6 6.5 7 7.5 8 8.5 9
FEL Index
Perf
orm
ance
Inde
x
SCREENINGPOORFAIRGOODBEST
Cost
Schedule
Best Practices for Project Control
Achieving Cost/ScheduleEffectiveness
Achieving Cost/SchedulePredictability
Project Control Best Practices Owner cost specialist validates cost
estimate Detail physical progressing Frequent and detailed progress status
reporting Actual cost data captured in an owner
database Benchmarking/Metric Analysis systems
in place
AACE International – January 2004 – Portland, OR
Findings & Trends
Overall, project costs are becomingmore predictable Absolute cost deviation is decreasing
Median % deviation is close to zero
Variability is still relatively high
AACE International – January 2004 – Portland, OR
Findings & Trends
Cost effectiveness is improving veryslightly Cost predictability does not yield cost
effectiveness
Progress in developmental projects islimited
Cost engineering is not playing a largeenough role in project selection
AACE International – January 2004 – Portland, OR
Findings & Trends
Schedules are becoming moreimportant Schedules are getting faster
Schedule slip is declining
Contributes to lack of improvement in costeffectiveness
Reasons forNon-Competitive Results
Non-competitive target setting Corporate culture requires underruns Lack of benchmarks/metrics Previous projects overran cost or schedule
Turnover in key personnel Lack of team continuity Lose contact with contractors Team stretched too thin
Reasons forNon-Competitive Results
Business Issues Changes in product characteristics Cash flow delays
Contractor Issues Inexperience at site/company Poor change management Unqualified contractors Lack of contractor cooperation
Reasons forNon-Competitive Results
Practices Are Not As Good As TheyAppear Funding with inadequate project definition VIP’s not effective Thorough schedule analysis not done Links to other projects not incorporated into
plan Emphasis on saying we’re the best instead
of being the best
AACE International – January 2004 – Portland, OR
Benchmarking Reality Check
“The government are very keen on amassingstatistics. They collect them, add them, raisethem to the nth power, take the cube root,and prepare wonderful diagrams. But youmust never forget that every one of thesefigures comes in the first instance from thevillage watchman who just puts down what hedamn pleases.” - Sir Josiah Stamp (1869-1919)
AACE International – January 2004 – Portland, OR
Summary Benchmarking is a positive, pro-active
approach to change project processes toachieve superior performance
Benchmarking, by its nature, challenges thecurrent methods of doing business
Benchmarking is an objective-setting process
Benchmarking should be a continual, long-term process (and requires managementcommitment)