48
PMI College of Scheduling ‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc. May 9, 2006 Quantitative Cost and Schedule Risk Assessment and Management for Large Infrastructure Projects RT-05 Bill Roberds, Sc.D. Travis McGrath, Ph.D., P.E. Golder Associates, Inc. Redmond, WA

Cost and Schedule Risk Analysis for Infrastructure ...microplanning.com.au/wp-content/uploads/downloads/... · Schedule Estimates Develop Risk Register Assess Risk Factors Quantify

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

  • PMI College of Scheduling‘’PMI’ is a registered trade and service

    mark of the Project Management institute, Inc.

    May 9, 2006

    Quantitative Cost and Schedule

    Risk Assessment and Management

    for Large Infrastructure Projects

    RT-05

    Bill Roberds, Sc.D.

    Travis McGrath, Ph.D., P.E.

    Golder Associates, Inc.

    Redmond, WA

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    2May 9, 2006

    Introduction

    Cost and Schedule overruns are common on large infrastructure projects. Historically1:

    •90% of estimates for transportation infrastructure projects have been low

    •On average, cost estimates for road projects have been 20% short of final costs

    •Estimates for some projects have been off by 100% or more (e.g., the “Big Dig”)

    1. Flyvbjerg et.al., Journal of the American Planning Association, Summer, 2002.

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    3May 9, 2006

    Project Cost and Schedule Uncertainty

    Poor cost and schedule estimates lead to:

    •Cost and schedule over-runs or under-runs

    •Poor decisions (i.e., among alternatives)

    •Resource competition among projects

    •Media attention

    •Public mistrust (and therefore funding problems)

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    4May 9, 2006

    Probabilistic Cost and Schedule Risk Assessment and Risk Management

    (RA/RM)

    Mitigate these problems by:

    •Quantifying project cost and schedule uncertainty

    • Identifying and prioritizing critical risks and key opportunities

    •Quantifying potential benefits of risk-management strategies

    • Improving project understanding and communication

    • Increasing confidence in estimates and budgets

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    5May 9, 2006

    Outline

    •Key Concepts

    •Process and Examples

    •Performance to Date

    •Challenges for Better Implementation

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    6May 9, 2006

    Key Concepts

    •Comprehensive look at the project at appropriate level of detail

    •Quantify uncertainty in key project assumptions

    •Utilize independent perspective (for validation)

    •Employ a collaborative, team approach

    •Achieve consensus (when possible)

    •Focus attention on the key issues

    •Update as the project changes significantly

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    7May 9, 2006

    Key Concepts, cont’d

    •Quantify uncertainty in project cost and schedule, which is a function of:

    – Sequence of all significant project activities

    – “Base” activity costs and durations

    – Risks, Opportunities, and other uncertainties

    •Prioritize critical activities, risks, and opportunities

    • Identify and evaluate strategies to improve project performance

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    8May 9, 2006

    Describing Uncertainty

    0%

    25%

    50%

    75%

    100%

    Cumulative Distribution Function (CDF)

    Cu

    mu

    lati

    ve

    Pro

    bab

    ilit

    yValue

    Range in values

    Rel

    ativ

    e L

    ikel

    ihood

    Probability Density Function (PDF)

    Standard

    Deviation

    Mean

    (Expected)

    Value

    Percentiles

    10th 90th

    80% confidence interval

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    9May 9, 2006

    “Base + Risk” Approach versus Traditional Single-Valued Approach

    Cost0

    Rel

    ati

    ve

    Lik

    elih

    oo

    d

    Aggregate Risk and Opportunity

    Base (with uncertainties)

    “Base + Risk”

    Traditional

    Contingency

    “below the line”

    Traditional

    Conservative

    “above the line”

    Traditional

    Single-valued

    (Deterministic) Estimate

    + =

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    10May 9, 2006

    Outline

    •Key Concepts

    •Process and Examples

    •Performance to Date

    •Challenges for Better Implementation

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    11May 9, 2006

    Steps in Quantitative Cost and Schedule Risk Assessment and Risk Management

    Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions

    Review Cost and Schedule Estimates

    Develop Risk Register

    Assess Risk Factors

    Quantify Uncertainty and Sensitivity in Cost,

    Schedule, and Other Measures

    Develop Cost and Schedule

    Uncertainty Model

    Update Periodically or at Key Milestones

    Identify and Evaluate Risk-Management Strategies

    Develop Base Factors

    Project team

    implements

    mitigation

    strategy and

    monitors

    performance

    Workshop setting

    Collaborative effort

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    12May 9, 2006

    Steps: Structuring the RA/RM

    Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions

    • Project team presents current plan

    • Clarify project scope and delivery strategy

    • List all significant project assumptions

    • Develop project “flow chart” for each alternative:

    – Sequence of major project activities (all precedence requirements)

    – Based on team’s strategy and schedule

    – Reasonableness / accuracy check on schedule logic

    – Appropriate level of detail

    – Basis for integrated cost and schedule model

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    13May 9, 2006

    Example Project “Flow Chart”

    14

    Widen Existing

    Embankment

    15

    Construct New

    Embankment

    16

    Remaining Bridge

    Construction

    19

    Demo Old

    Bridge

    22

    Remove Old

    Alignment

    23

    Close-out

    Items

    24

    Complete

    25

    Open to

    Traffic

    21

    Bridge Approaches

    17

    In-Water

    Bridge

    Construction

    18

    S + 1 month

    Paving

    20

    Develop

    Alternatives

    2Design

    and ROW

    Funding

    1Env Resource

    Inventory

    3

    Design to 10%

    4

    Prepare

    Resource

    Reports

    5

    Environmental

    Doc + Approval

    6

    Design to 60%

    8

    Finalize Design

    and PS&E

    9

    Acquire

    ROW

    11

    Permitting

    12

    Ad / Bid /

    Award / NTP

    14

    Construction

    Funding

    13

    14

    F-1 month

    Relocate

    Utilities

    10Design to 30% +

    Design Approval

    7

    S+50%

    S+50%Programmed:

    1/1/2005

    Programmed:

    1/1/2007

    Pre-Construction Activities

    Construction Activities

    14

    Widen Existing

    Embankment

    15

    Construct New

    Embankment

    16

    Remaining Bridge

    Construction

    19

    Demo Old

    Bridge

    22

    Remove Old

    Alignment

    23

    Close-out

    Items

    24

    Complete

    25

    Open to

    Traffic

    21

    Bridge Approaches

    17

    In-Water

    Bridge

    Construction

    18

    S + 1 month

    Paving

    20

    Develop

    Alternatives

    2Design

    and ROW

    Funding

    1Env Resource

    Inventory

    3

    Design to 10%

    4

    Prepare

    Resource

    Reports

    5

    Environmental

    Doc + Approval

    6

    Design to 60%

    8

    Finalize Design

    and PS&E

    9

    Acquire

    ROW

    11

    Permitting

    12

    Ad / Bid /

    Award / NTP

    14

    Construction

    Funding

    13

    14

    F-1 month

    Relocate

    Utilities

    10Design to 30% +

    Design Approval

    7

    S+50%

    S+50%Programmed:

    1/1/2005

    Programmed:

    1/1/2007

    Pre-Construction Activities

    Develop

    Alternatives

    2Design

    and ROW

    Funding

    1Env Resource

    Inventory

    3

    Design to 10%

    4

    Prepare

    Resource

    Reports

    5

    Environmental

    Doc + Approval

    6

    Design to 60%

    8

    Finalize Design

    and PS&E

    9

    Acquire

    ROW

    11

    Permitting

    12

    Ad / Bid /

    Award / NTP

    14

    Construction

    Funding

    13

    14

    F-1 month

    Relocate

    Utilities

    10Design to 30% +

    Design Approval

    7

    S+50%

    S+50%Programmed:

    1/1/2005

    Programmed:

    1/1/2007

    Pre-Construction Activities

    Construction Activities

    Critical path

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    14May 9, 2006

    Steps: Reviewing Project Estimates

    Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions

    Review Cost and Schedule Estimates

    • Collaborative effort

    • Confirm that cost matches scope

    • Confirm that schedule matches scope

    • Check reasonableness of design and strategy

    • Identify and remove contingency and conservatism

    • Note: Estimate is for the ultimate cost and schedule

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    15May 9, 2006

    Steps: Defining the “Base”

    Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions

    Review Cost and Schedule Estimates

    Develop Base Factors

    • The “Base”

    – The planned scope and strategy, with no problems

    – Unbiased, excluding contingency and/or float

    • “Map” the “base” estimates to flow chart activities

    • For each flow chart activity, develop:

    – Base cost (current $)

    – Base duration (calendar time)

    – Cost-escalation rate

    • Include significant uncertainties and correlations

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    16May 9, 2006

    Example Base Factors (excerpt)

    Flowchart Number

    Project Activity Base Cost 1

    (2005 $M)

    Base Duration 1

    (months)

    Average Escalation

    Rate 1 (%/yr) 0 Costs to Date 0.11 -

    1 Design and ROW Funding milestone -

    2 Develop Alternatives 0.02 1.0 3.0

    3 Env. Resource Inventory 0.02 0.5 3.0 4 Design to 10% 0.30 6.0 3.0

    5 Prepare Resource Reports 0.10 3.0 3.0 6 Environmental Doc + Approval 0.05 2.0 3.0

    7 Design to 30% + Approval 0.30 4.0 3.0

    … … … … … 14 Ad / Bid / Award / NTP 0.05 2.5 3.0

    15 Widen Existing Embankment 1.30 2.0 3.0 16 Construct New Embankment 1.85 3.5 3.0

    17 Bridge Approaches 1.75 3.0 3.0 18 In-Water Bridge Construction

    0.37 0.75 3.0

    19 Remaining Bridge Construction

    2.13 3.0 3.0

    … … … … … 24 Close-out Items 0.57 1.0 3.0 25 Complete milestone -

    Note 1. May be uncertain and described with probability distributions; only the expected values are shown here.

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    17May 9, 2006

    Steps: Developing the Risk Register

    Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions

    Review Cost and Schedule Estimates

    Develop Risk Register

    Develop Base Factors

    • Identify all potential events - risks and opportunities -outside the “base” assumptions

    • Comprehensive and non-overlapping

    • Appropriate level of detail

    • Include significant relationships among risks

    • Consider possible causes and effects

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    18May 9, 2006

    Steps: Quantifying the Risks

    Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions

    Review Cost and Schedule Estimates

    Develop Risk Register

    Assess Risk FactorsDevelop Base Factors

    For each risk and opportunity, quantify:

    • Consequences if event occurs

    – Changes in costs and durations to affected activities

    – Other measures?

    • Likelihood of occurrence, consistent with consequences

    • Include significant uncertainties and correlations

    • Ensure consistent with “base”

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    19May 9, 2006

    Steps: Quantifying the Risks, cont’d

    Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions

    Review Cost and Schedule Estimates

    Develop Risk Register

    Assess Risk FactorsDevelop Base Factors

    Risk-factor assessments:

    • Based on objective analysis when data are available

    • Based on subjective assessments (from the experience and judgment of SMEs) if data are limited

    •When properly elicited, subjective assessments have been shown to be accurate, defensible, and ‘the best we can do’

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    20May 9, 2006

    Example Event Tree for Structuring Risks and Opportunities

    Vertical

    Concept &

    Location

    1. Tunnel

    P1

    3. Bridge /

    Aerial

    P3 = 1-P1-P2

    2. At-grade

    P2

    Structure

    Type A. Cut-and-Cover

    PA|1

    B. Bored / Mined

    PB|1 = 1- PA|1

    C. Concrete Box

    PC|3

    D. Steel Plate Girder

    PD|3 = 1 - PC|3

    Note: Probability of alternative concepts may consider factors other than cost or duration

    Probability ∆∆∆∆$ ($M) ∆∆∆∆Months

    PA|1P1 0 (“base”) 0

    PB|1P1 +5 +2

    P2 -4 -3

    PC|3P3 -2 -1

    PD|3P3 -3 -1

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    21May 9, 2006

    Example Risk Register and Risk Factors (excerpt)

    Item Risk or Opportunity Affected Project

    Activities

    Probability of

    Occurrence

    Cost Change (2005 $M)

    Duration Change

    (months)

    D8

    Project alignment different than assumed Includes related design, ROW, and construction impacts Potential mutually-exclusive outcomes include:

    A. EIS alignment (“base”), which passes through town on existing roadway alignment

    B. Alternative alignment #1, which bypasses town to north on new alignment

    C. Alternative alignment #2, which passes through town on new alignment to east of EIS alignment

    60% Design and

    ROW Acquisition

    Discrete distribution: A. 40%

    (Base) B. 40% C. 20% Cost change perfectly correlated to duration change

    A. 0 (Base) B. 1.0 to

    Design; -5 to ROW C. 1.0 to

    Design; Uniform

    (2, 10) to ROW

    A. 0 (Base) B. 6 to

    Design; -3 to ROW C. 6 to

    Design; Uniform

    (3, 9) to ROW

    P4

    Delays in required transportation agency approvals Several agencies have review and approval authority over the design. Delays in approval are possible from delays in review cycle.

    60% Design

    Discrete distribution: A. 10% (Base) B. 60% C. 30%

    Additional escalation costs are simulated as a function of duration change

    Discrete distribution: A. 0 (base) B. 3 C. 6

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    22May 9, 2006

    Steps: Developing the Integrated Cost and Schedule Model

    Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions

    Review Cost and Schedule Estimates

    Develop Risk Register

    Assess Risk FactorsDevelop Base Factors

    Develop Cost and Schedule

    Uncertainty Model

    • Probabilistic computer model

    • Can include

    – Shut-down “windows”

    – Decisions and other “triggered”actions

    – Funding or other resource constraints

    – Uncertainty in critical path

    – Delay-related costs

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    23May 9, 2006

    Steps: Quantifying Uncertainty in Project Cost and Schedule

    Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions

    Review Cost and Schedule Estimates

    Develop Risk Register

    Assess Risk FactorsDevelop Base Factors

    Quantify Uncertainty and Sensitivity in Cost,

    Schedule, and Other Measures

    Develop Cost and Schedule

    Uncertainty Model

    • Combine base and risk factors in the model

    • Simulate results of interest

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    24May 9, 2006

    Integrated Cost and Schedule Model

    Activity “Base” Costs (and Uncertainties)

    Activity “Base” Durations (and Uncertainties)

    Activity A

    Activity C

    Activity B

    Start End

    Uncertainty in

    Total Escalated

    Project Cost

    and Duration, plus

    Sensitivities

    $

    T

    7 56

    12111 0

    8 4

    21

    9 3

    _

    7 56

    1 21 11 0

    8 4

    21

    9 3

    _

    Risk Events (likelihood of occurrence, and uncertainty

    in activity cost and duration changes if the event occurs)

    Event Y

    Event X

    time

    escalation

    cash flow

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    25May 9, 2006

    Example Monte Carlo Sample Set Showing Cost and Schedule Correlation

    Dec-11

    Dec-12

    Dec-13

    Dec-14

    Dec-15

    Dec-16

    Dec-17

    Dec-18

    Dec-19

    Dec-20

    Dec-21

    0 200 400 600 800

    Total Project Cost ($M)

    4th

    Bo

    re O

    pen

    ing

    Da

    te

    unescalated

    escalated

    Dec 2021

    Dec 2020

    Dec 2019

    Dec 2018

    Dec 2017

    Dec 2016

    Dec 2015

    Dec 2014

    Dec 2013

    Dec 2012

    Dec 2011

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    26May 9, 2006

    Example CDF for Project Cost Showing Contributing Factors

    0%

    10%

    20%

    30%

    40%

    50%

    60%

    70%

    80%

    90%

    100%

    2,4

    00

    2,5

    00

    2,6

    00

    2,7

    00

    2,8

    00

    2,9

    00

    3,0

    00

    3,1

    00

    3,2

    00

    3,3

    00

    3,4

    00

    3,5

    00

    3,6

    00

    3,7

    00

    3,8

    00

    3,9

    00

    4,0

    00

    4,1

    00

    4,2

    00

    4,3

    00

    4,4

    00

    4,5

    00

    Total Project Cost (2005 $Million or YOE $Million)

    Cu

    mu

    lati

    ve

    Pro

    ba

    bil

    ity

    (Per

    cen

    tile

    )Base (with no risk or opportunity; in 2005 $)

    With Risk and Opportunity, but Without Escalation (2005 $)

    With Risk and Opportunity and Escalation (Year-of-Expenditure $)

    With Risk and Opportunity and $400M/year Funding Constraint (YOE $)

    Mean risk and

    opportunity

    Mean cost

    escalation

    Mean

    funding

    constraint

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    27May 9, 2006

    Example CDF for Project Schedule Showing Contributing Factors

    0%

    10%

    20%

    30%

    40%

    50%

    60%

    70%

    80%

    90%

    100%Ja

    n 2

    01

    5

    Jan

    20

    16

    Jan

    20

    17

    Jan

    20

    18

    Jan

    20

    19

    Jan

    20

    20

    Jan

    20

    21

    Jan

    20

    22

    Jan

    20

    23

    Jan

    20

    24

    Overall Project Completion Date

    Cu

    mu

    lati

    ve

    Pro

    ba

    bil

    ity

    (Per

    cen

    tile

    )Base (with no risk or opportunity)

    With Risk and Opportunity but Without Funding Constraint

    With Risk and Opportunity and $400M/year Funding Constraint

    Mean risk and

    opportunity

    Mean impact of funding

    constraint

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    28May 9, 2006

    Example Probability of Meeting Specified Target

    0%

    10%

    20%

    30%

    40%

    50%

    60%

    70%

    80%

    90%

    100%

    2,4

    00

    2,5

    00

    2,6

    00

    2,7

    00

    2,8

    00

    2,9

    00

    3,0

    00

    3,1

    00

    3,2

    00

    3,3

    00

    3,4

    00

    3,5

    00

    3,6

    00

    3,7

    00

    3,8

    00

    3,9

    00

    4,0

    00

    4,1

    00

    4,2

    00

    4,3

    00

    4,4

    00

    4,5

    00

    Total Project Cost (2005 $Million or YOE $Million)

    Cu

    mu

    lati

    ve

    Pro

    ba

    bil

    ity

    (Per

    cen

    tile

    )Base (with no risk or opportunity; in 2005 $)

    With Risk and Opportunity, but Without Escalation (2005 $)

    With Risk and Opportunity and Escalation (Year-of-Expenditure $)

    With Risk and Opportunity and $400M/year Funding Constraint (YOE $)

    Probability of

    meeting $3.7B

    budget = 85%

    Probability of

    meeting $3.7B

    budget = 37%

    Alternatively,

    could identify required

    budget for desired

    confidence level

    (e.g., 80th percentile)

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    29May 9, 2006

    0

    500

    1,000

    1,500

    2,000

    2,500

    3,000

    3,500

    4,000

    2005 2007 2009 2011 2013 2015 2017 2019 2021

    Year

    Pro

    ject

    Ca

    sh F

    low

    Wit

    ho

    ut

    $4

    00

    M/y

    ear

    Fu

    nd

    ing

    Co

    nst

    rain

    t (Y

    OE

    $M

    )

    Cumulative - 90th Percentile

    Cumulative - Mean

    Cumulative - 10th Percentile

    Annual - 90th Percentile

    Annual - Mean

    Annual - 10th Percentile

    Example Uncertainty in Cash Flow of Expenditures

    (Unconstrained Funding Scenario)

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    30May 9, 2006

    Example Impact of Scenario Assumptions on Uncertainty in Cash Flow

    0

    100

    200

    300

    400

    500

    600

    700

    800

    2005 2007 2009 2011 2013 2015 2017 2019 2021 2023 2025

    Year

    An

    nu

    al

    Ca

    shfl

    ow

    (Y

    OE

    $M

    )

    No Funding Constraint - 90th Percentile

    No Funding Constraint - Mean

    No Funding Constraint - 10th Percentile

    Funding Constraint - 90th Percentile

    Funding Constraint - Mean

    Funding Constraint - 10th Percentile

    Constraint delays

    peak expenditure

    by 5 years

    Constraint

    reduces peak

    expenditure

    by $270M

    at the mean However, constraint

    increases total cost

    significantly

    (increased schedule

    and thus escalation)

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    31May 9, 2006

    Steps: Quantifying Sensitivity in Project Cost and Schedule

    Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions

    Review Cost and Schedule Estimates

    Develop Risk Register

    Assess Risk FactorsDevelop Base Factors

    Quantify Uncertainty and Sensitivity in Cost,

    Schedule, and Other Measures

    Develop Cost and Schedule

    Uncertainty Model

    • Evaluate sensitivity of cost and schedule to the uncertain “base”and “risk” factors:

    – Mean

    – Standard Deviation

    – Percentile (combination)

    • Prioritize critical activities and risks in costs and time

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    32May 9, 2006

    Example Sensitivity of 80th Percentile of Project Cost to Uncertain Inputs

    34.7

    23.2

    16.3

    11.8

    11.7

    7.8

    6.7

    6.5

    6

    5.2

    4.9

    3.9

    3.8

    3.6

    2.9

    2.4

    2

    1.9

    1.8

    1.7

    1.6

    1.6

    1.3

    1.2

    1.1

    1.1

    1

    1

    0 5 10 15 20 25 30 35 40

    B.3. Limited Number of Qualified Bidders

    D.26. Extended Pre-Construction Admin Costs

    C.35. Extended Construction Admin Costs

    Identified Minor Risks (aggregate)

    Unidentified Risks (aggregate)

    E.1. Challenge to ROD/FONSI due to public opposition

    C.25. Unanticipated Mitigation of Construction Traffic

    B.11. Non-Responsive Low Bid

    D.7. Unanticipated Mitigation for Impacts Outside of ...

    C.14. Valid Differing Site Condit ions (DSC) (e.g., ...

    C.7. Inexperienced Contractor (Separate from C.5 ...

    D.5. More Fire Protection / Ventilat ion Required

    D.14. Conversion from Metric to English Units

    C.5. Contractor Unfamiliarity with CalTrans Admin...

    W Tunnel

    C.15. Tunnel Face Instability / Excessive Inflow

    E Tunnel

    C.29. Unanticipated Work Stoppage due to Adverse...

    E.5. Unexpected / Late Number of Public Comments ...

    B.6. Poor Material Prices and Availability

    C.20. Slope Instability / Retaining Wall Failure

    B.5. Opposition from Local Contractors / Business...

    D.16. Problems in Design Coordination (Including ...

    D.17. Design Changes / Breakage (Separate from ...

    E13. Additional Impact Analysis

    August 2005$ / non-ROW escalat ion rate

    C.16. Tunnel Collapse During Construction

    D.4. Change in Design Standards (e.g., Seismic) ...

    Uncertain Factors (base

    uncertainty or

    risk/opportunity)

    Contribution ($M) to 80th Percentile

    of Escalated Project Cost

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    33May 9, 2006

    Steps: Managing Risk

    Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions

    Review Cost and Schedule Estimates

    Develop Risk Register

    Assess Risk Factors

    Quantify Uncertainty and Sensitivity in Cost,

    Schedule, and Other Measures

    Develop Cost and Schedule

    Uncertainty Model

    Identify and Evaluate Risk-Management Strategies

    Develop Base Factors

    Project team

    implements

    mitigation

    strategy and

    monitors

    performance

    • Identify potential risk-management actions (e.g., prevention; mitigation)

    • Quantify likely implementation costs

    • Quantify likely benefits

    • Simulate net effect

    • Ties in to Value Engineering

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    34May 9, 2006

    Risk Management

    Performance (e.g., cost)

    Relative

    Likelihood Total with

    Risk Management

    Risks and Opportunities with

    Risk Management

    Note: Base costs include mitigation implementation costs

    Base (with uncertainties; now

    including RM costs)

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    35May 9, 2006

    Example Risk Management: Mitigated Cost Uncertainty (if Mitigation Succeeds)

    0

    0.05

    0.1

    0.15

    0.2

    32

    0

    33

    0

    34

    0

    35

    0

    36

    0

    37

    0

    38

    0

    39

    0

    Total Project Cost (Future $M)

    Pro

    ba

    bil

    ity

    Mitigated (Workshop Plan)

    Unmitigated (July 19 2003)

    The mitigation planwould save$10M at the owner’s desired confidence level (80th percentile)

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    36May 9, 2006

    Example Risk Management: Mitigated Schedule Uncertainty

    0%10%20%

    30%40%50%60%70%

    80%90%

    100%

    Jun

    20

    08

    Sep

    20

    08

    Dec

    20

    08

    Mar

    20

    09

    Jul

    20

    09

    Oct

    20

    09

    Jan

    20

    10

    May

    20

    10

    Revenue Service Date

    Cu

    mu

    lati

    ve

    Pro

    ba

    bil

    ity

    (Per

    cen

    tile

    )

    July 2004 Mitigated

    July 2004 Unmitigated

    The mitigation planwould save7-8 months at the owner’s desired confidence level (80th percentile)

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    37May 9, 2006

    Outline

    •Key Concepts

    •Process and Examples

    •Performance to Date

    •Challenges for Better Implementation

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    38May 9, 2006

    Performance

    • Not many projects evaluated using this approach have been completed

    • Can’t fully evaluate performance (yet)

    • As an interim measure, compared results from risk assessments to published costs for completed projects

    • Favorable reaction from public (i.e., public “ok”with ranges expressing uncertainty)

    • Improved project team understanding

    • Focused risk management

    • Improved confidence in cost and schedule estimates, however…

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    39May 9, 2006

    Performance: Bias Comparisons

    •Flyvbjerg et. al. (2002) calculated inaccuracy (bias) in cost estimates for completed projects as the percent difference between actual (ultimate) project cost and estimated cost at the

    time of decision to build

    •As an interim measure, the authors calculated the “apparent bias” in project team estimates relative to the mean RA/RM results for projects still in design as follows:

    [mean unescalated cost from risk assessment – team cost estimate (unescalated)]

    team cost estimate (unescalated)

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    40May 9, 2006

    Performance: Results from Risk Assessments vs. Published Results for Completed Projects

    0.0

    0.1

    0.2

    0.3

    0.4

    0.5

    0.6

    -40 -20 0 20 40 60 80 100 120 140 160 180 200 220 240

    Bias in Cost Estimate (%) 1

    (numbers are upper bounds for bins)

    Fre

    qu

    ency

    From 62 Risk Assessments (2002 to 2005) on

    Northwestern US Road Projects

    After Flyvbjerg et. al. (2002) for 167

    Completed Road Projects (Worldwide)

    Mean = 20%

    Standard deviation = 30%

    Mean = 14%

    Standard deviation = 24%

    0.0

    0.1

    0.2

    0.3

    0.4

    0.5

    0.6

    -40 -20 0 20 40 60 80 100 120 140 160 180 200 220 240

    Bias in Cost Estimate (%) 1

    (numbers are upper bounds for bins)

    Fre

    qu

    ency

    From 62 Risk Assessments (2002 to 2005) on

    Northwestern US Road Projects

    After Flyvbjerg et. al. (2002) for 167

    Completed Road Projects (Worldwide)

    Mean = 20%

    Standard deviation = 30%

    Mean = 14%

    Standard deviation = 24%

    Note: Larger variability in completed projects likely reflects larger data set and variability among agencies

    Similar partial validation of approach

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    41May 9, 2006

    Performance: Apparent Bias in Cost Estimate versus Design Level

    -40

    -20

    0

    20

    40

    60

    80

    100

    Design Level at Time of Risk Assessment

    Bia

    s in

    Co

    st E

    stim

    ate

    (%

    ) 1

    From 62 Risk Assessments on

    Northwestern US Road Projects

    Conceptual Preliminary Final

    Mean

    Mean plus one standard deviation

    Mean minus one standard deviation-40

    -20

    0

    20

    40

    60

    80

    100

    Design Level at Time of Risk Assessment

    Bia

    s in

    Co

    st E

    stim

    ate

    (%

    ) 1

    From 62 Risk Assessments on

    Northwestern US Road Projects

    Conceptual Preliminary Final

    Mean

    Mean plus one standard deviation

    Mean minus one standard deviation

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    42May 9, 2006

    Outline

    •Key Concepts

    •Process and Examples

    •Performance to Date

    •Challenges for Better Implementation

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    43May 9, 2006

    Challenges for Better Implementation

    • If properly implemented and averaged over many projects, risk-based approaches appear to “get closer” to the ultimate project cost than does deterministic estimating,

    •However, RA/RM is not perfect:

    – Bids for some recent projects evaluated using probabilistic RA/RM (by the authors and by others) have fallen outside the RA/RM ranges

    – Effort for RA/RM for some projects has been excessive

    •This implies that probabilistic RA/RM still needs improvement

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    44May 9, 2006

    Challenges for Better Implementation

    1. Convincing the owner that the RA should include all significant uncertainties

    • Excluding major uncertainties makes the results:

    – Conditional on key assumptions (often forgotten)

    – Underestimate the true range of uncertainty, and perhaps inaccurate as well

    • Examples of major uncertainties that are often excluded for political or other reasons:

    – Uncertainty in the amount or timing of funding

    – Potential delays due to process and management inefficiencies

    – Uncertainty in key policies within stakeholder orgs

    – Uncertainty in project scope (i.e., evaluating a particular scope instead of all possible project scopes)

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    45May 9, 2006

    Challenges for Better Implementation

    2. Removing optimism and/or “management bias” (e.g., hidden agendas) from assessments

    • Consensus is desirable, but should not be achieved at the expense of accuracy

    • If necessary, risk assessors should infuse their experience from other projects to de-bias and/or expand ranges of critical assessments to ensure reasonable results

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    46May 9, 2006

    Challenges for Better Implementation

    3. Making results of RA/RM understandable and useful to the owner

    • Make sure the RA addresses the owner’s specific needs

    • Educate the owner on how to interpret and use the results

    • Don’t let the owner assign more certainty or precision to the results than is justifiable

    • Qualify any limitations to avoid misunderstanding

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    47May 9, 2006

    Summary

    •Probabilistic risk assessment and risk management for large infrastructure projects can cost-effectively provide

    – Better understanding and communication of project uncertainties

    – More confidence in estimates

    – Better project management and decision making

    •However, careful (and better) implementation is required to ensure accurate and defensible results

  • PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.

    48May 9, 2006

    Dr. Travis McGrath, P.E.

    Golder Associates, Inc.

    18300 NE Union Hill Rd, Suite 200

    Redmond, WA 98052

    (425) 883-0777

    [email protected]

    Contact Information