Transportation Decision Making Principles of Project Evaluation and Programming

Chapter 18Evaluation of Transportation Projects and Programs Using Multiple Criteria

Kumares C. Sinha and Samuel Labi

Decision criteria can have multiple dimensionsDollarsNumber of crashesAcres of land, etc.

All criteria are not of equal importance

For a given criterion, different stakeholders may have different weights.

Typical Steps in Multi-Criteria Decision Making

1. Establish Transportation Alternatives

3. Establish Criteria Weights

4. Establish Scale to be Used for Measuring Levels of Each Criterion

5. Using Scale, Quantify Level (Impact) of Each Criterion for Each

Alternative

2. Establish Evaluation Criteria

6. Determine Combined Impact of all Weighted Criteria for Each Alternative

Weighting

Amalgamation

Scaling

11. Determine the Best Alternative

Establishing Weights

Weights reflect the relative importance attached by decision makers to various criteria

In some cases, the decision maker refers to the agency as well as the facility user. In those cases, the weight used for each criterion is a weighted average of the weights from these two parties.

Weighting Techniques1. Equal Weights2. Direct Weighting3. Derived Weights4. Delphi Technique5. Gamble Method6. Pair-wise comparison: AHP7. Value Swinging

Equal Weights - Example

Project Cost 33.3%

Travel Time Saving 33.3%

VOC Saving 33.3%

Direct Weighting

1. Point Allocation – A number of points are allocated among the criteria according to their importance.

2. Ranking – Simple ordering by decreasing importance.

Point allocation is preferred because unlike ranking, it yields a cardinal rather that an ordinal scale of importance.

Point Allocation (0-100) Ranking(Cardinal) (Ordinal)

Project Cost 70 1

TT Saving 50 3

VOC Saving 60 2

Regression-Based Observer-Derived Weighting

1. Survey respondents assign scores of overall “benefit” or “desirability” for a given combination of criteria levels achieved by each alternative

2. Weights are then the resulting regression coefficients

( )2

2

i

i j ji ij

Minimize

TV w V

ε

ε= +

∑∑

i = alternativej = CriterionTV = score or desirability

Regression

7 Respondents21 Data Points

TV = wcost* Cost + wtime * Time

wcost = 0.214

wtime = 0.786

R2 = 0.98

Delphi Technique

Individual responses aggregated

Effect of assessment of other respondents

Consensus building

Iterative, generally 2 rounds to achieve stable values

Scaling Methods

GAMBLE METHOD

1. Carry out an initial ranking of all criteria in order of decreasing importance. set the first criterion at its most desirable level and all other criteria at their lest desirable levels

2. Compare between the following two outcomes:Sure thing: The outcome is that the criterion in question is at its most desirable level while all other criteria at their least desirable levelsGamble: In this outcome, all criteria attained their most desirable levels p% of the time, their least desirable levels (1-p)% of the time

3. At a certain level of ‘p’ the two situations (sure thing and gamble) are equally desirable. At that level, the value of ‘p’ represents the weight for the criterion in question

Example:

Bus Route AssessmentHeadway (from 5 to 15 minutes)Population Served (from 5,000 to 10,000)

Solution:1. Sure Thing: Bus headway is 5 minutes and population served is 5,0002. Gamble: Two outcomes:

a. A p% chance of an outcome that headway is 5 minutes and population is 10,000

b. A (1-p)% chance of an outcome that headway is 15 minutes and population is 5,000.

Suppose the respondent were found to be indifferent between surething and gamble at p = 60%, then, the relative weight for bus headway is 0.6.

Pair Wire ComparisonAnalytical Hierarchy Process (AHP)

12 1

12 2

1 2

1 ...................1/ 1 ...................... .... .... .... .... .... .......1/ 1/ ..... ... .... 1

n

n

n n

a aa a

a a

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

= relative importance of two criteria I and j on the basis of a scale of 1 to 9

=

ija

/i jw w

Table 18.1: Ratios for Pair wise Comparison Matrix

Value Swinging Method

1. Consider a hypothetical situation where all criteria at their worst values

2. Determine the criterion for which it is most preferred to “swing” from its worst value to best value, all other criteria remaining at their worst values.

3. Repeat steps 1 and 2 for all criteria.4. Assign the most important criterion the highest weight

in a selected weighting range (100 for 1-100 scale) and then assign weights to the remaining criteria in proportion to their rank of importance.

Scaling of Performance Criteria

Certainty - Value Function

Risk - Utility Function

Uncertainty - Scenario Analysis

Value Function

a. Direct Rating Method – direct assignment of value to various levels of a criterion

b. Mid Value Splitting Technique – based on “indifference” between changes in levels of criterion.

c. Regression – based on data from direct rating

Discrete Value Function

Discrete Dis-Utility Function for Performance Measure of Impact on Natural, Socio-Economic, Historical &

Cultural Resources

-100

-80

-60

-40

-20

0

No

Impact

Minor

Impact

ModerateIm

pact

Major

Impact

Huge

Impact

Extreme

Impact

Utility

Continuous Value Function

Dis-Utility Function for Single Performance Measure of Emissions

-100

-80

-60

-40

-20

00 20 40 60 80 100 120

Percentage Increase in Emissions

Utility

Developed Value Functions

Utility Function

Direct Questioning Using the Gamble Approach

Guaranteed prospect of an outcome vs. risky prospect of a more favorable outcome.

Example 18.7

Utility Functions for agency cost, ecological damage, and vulnerable population served.

Solution:For Agency Cost: Ucost ($30 Million) = 0 (Worst)

Ucost ($ 0 Million) = 1 (Best)

Sure Thing: The outcome is that agency cost is guaranteed to be $20 Million

Gamble: There is a 50% chance that cost is 0 and 50% change it is $30 Million

X50 = $20 Million is the Certainty Equivalent because the expected utility is 0.5

0

20

40

60

80

100

120

0 5 10 15 20 25 30 35 40 45

Criteria Level

Util

ity

0

20

40

60

80

100

120

0 5 10 15 20 25 30 35 40 45

Criteria Level

Util

ity

Cost (in $millions)

Wetland lost in acres (in tens)

Population served (in thousands)

Combination of Performance Criteria

Pareto Optimality

Difference ApproachNet Utility = U(B) – U(C)NPV = PV (B) – PV(C)

Ratio ApproachUtility Ratio = U(B)/U(C)BCR = PV(B) / PV(C)

Cost Effectiveness

Costs and Benefits are not necessarily expressed in the same metrics

Indifference CurvesTradeoffs – marginal rates of substitution between criteria

TV = 2*TTR + PCC

Indifference Curves Using Mathematical Form of Utility/Value Function for Combined Performance Measures

Ranking and Rating Method

i i j ijj

Score P w O For each i= ∑

Impact Index Method

1 2

1max( , ,........, )

tan

( 0.5 0.5)

i j j ij j j j ijj

jj

jj

jj j nj

j

I R S X e R S X

wR relativeweight for criterion jw

S scaling factor of measurement X of criterion j X X X

e RN drawn fromarec gular distribution

e

= +

= =

= =

=

− ≤ ≤+

∑

∑

Table E18.10.1: Performance of Alternatives

Figure E18.10: Plot of Confidence Intervals