Robustness in assessment of strategic transport projects The 21st International Conference on Multiple Criteria Decision Making Jyväskylä June 13-17. 2011

Robustness in assessment of strategic transport projectsThe 21st International Conference on Multiple Criteria Decision MakingJyväskyläJune 13-17. 2011

Anders Vestergaard JensenDepartment of TransportTechnical University of Denmark

DTU Transport. Technical University of Denmark Anders Vestergaard Jensen - MCDM 20112 15.06.2011

Introduction• The paper is developed as part of the Ph.D study: Appraisal of Transport

Projects: Assessing Robustness in Decision Making

• The focus is on:– Dealing with subjectivity in assessment of larger transport projects– Transparent decision making process– Consensus– Communicative issues

• The background can be seen from the following statements– Policy-making has entered an era in which societal benefits of

governmental actions are increasingly questioned.

– Multi criteria analysis (MCA) is of growing importance due to inclusion of non-economic factors, MCA criteria weights are in this respect the decisive factors in decision-making.

DTU Transport. Technical University of Denmark Anders Vestergaard Jensen - MCDM 20113

Strategic decision making• Recognizing that formal economic evaluation analyses tend to be

inadequate – because objectives often are broader than pure economic or market concerns

• Focus is on transparent decision making processes

• For public decision making - decisions need to be justifiable

• There is a need to research the subjective part of the decision processes and the role of the decision support

15.06.2011


Appraisal approach

15.06.2011

• Assumption: Finite set of alternatives Idealized preference structure (participants are assumed to avoid

strategic behavior)

• Methodology Pair wise comparisons Multiplicative AHP (REMBRANDT) Rank order weights (ROD)

• Decision conference Stakeholder preferences Tailored to specific decision problem

• Robustness analysis• Weight stability interval

Group processes Technical


Framework

15.06.2011

Decision Conference

Definition of MCA Criteria

Pair wise comparisons using

REMBRANDT

Weighting of criteria using rank

order

Set of alternatives

Cost Benefit Analysis

Multi Criteria Analysis

Robustness Analysis

Input to the decision making process


Robustness Analysis• Robustness of the assessment of alternatives is in this context explored

by focusing on the uncertainties in the different model components

• Specifically the uncertainty relating to the criteria weights is examined

• This examination is carried out by modeling the influence that the weights have on the preference order of the alternatives

• The purpose is to create understanding and consensus of the decision making outcome

15.06.2011


Weight stability intervals (WSI)Investigate how the desirability of the alternatives changes when all the

weights of the criteria are kept constant except for one criterion.

We change 1 criterion (Ci):

All other weights are adjusted so only the importance of Ci relative to the other criteria is modified:

and are related and constrained as follows:

and

15.06.2011

ii ww )1('

jj ww '

i

i

w

w

1

)1(1

i

i

w

w

11

iw

1

10


Case: Elsinore-Helsingborg Connection• Alternatives (all tunnels):

15.06.2011

Alternatives

A1 Passenger train – Public transport

A2 Passenger train and car (dual tunnel)

A3 Passenger and goods train. car (dual tunnel)

A4 Passenger and goods train. car (single tunnel)

All alternatives have been found to be socio-economic feasible


Appraisal• Criteria:

– Construction cost, maintenance, scrap value, time savings, operating

costs, emissions, revenue (tickets)

– Impact on towns and land-use

– Regional economy

– Transport network and accessibility

– Impact on greening of transport

– Impact on flexibility in logistics

15.06.2011


Pair wise comparison

Criterion: Socio-economic robustness

Alt1 Alt2 Alt3 Alt4 GeoMean

Alt1 0 -6 -5 -7 0.044

Alt2 6 0 1 -1 2.828

Alt3 5 -1 0 -2 1.414

Alt4 7 1 2 0 5.657

15.06.2011


Overall score

Criteria Criterion weight

The alternative score within criterion

Alt1 Alt2 Alt3 Alt4

Socio-economic robustness 0.297 0.044 2.828 1.414 5.657

Town and land-use 0.139 0.022 1.414 5.657 5.657

Regional economics 0.188 0.074 0.841 6.727 2.378

Transport network and accessibility

0.241 0.074 0.841 6.727 2.378

Greening of transport 0.091 0.595 0.037 8.000 5.657

Flexibility on logistics 0.045 0.022 0.707 8.000 8.000

Weighted score: 0.062 0.968 4.233 3.960

15.06.2011


Weights assessed by the DMs

15.06.2011

Socio-economic robustness

Town and land-use

Regional economics

Transport network and accessibility

Greening of transport

Flexibility on logistics

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35

GroupEDCBA

Criteria Weights


Weight intervals

15.06.2011

Socio

-eco

nom

ic ro

bustne

ss

Town

and

land

-use

Regiona

l eco

nom

ics

Tran

spor

t net

work an

d ac

cess

ibilit

y

Green

ing

of tr

ansp

ort

Flexib

ility on

logistics

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Stated weight intervals(min. and max. of all individual weights)

We

igh

t


Weight stability

15.06.2011

0

0.05

932

0.11

864

0.17

796

0.23

728

0.29

66

0.35

592

0.41

524

0.47

456

0.53

388

0.59

32

0.65

252

0.71

184

0.77

116

0.83

048

0.88

98

0.94

912

0

2

4

6

8

Criterion: Socio-economic robustness

Alt1Alt2Alt3Alt4

Tota

l score

Weight assessed by the group

Upp

er

and

low

er

weig

hts

Weight


Weight Stability Interval• In the interval defined by the lower and upper the preference order does

not change

15.06.2011

Criteria Group weight

Lower stable

Upper stable

Socio-economic robustness 0.297 0 0.329

Town and land-use 0.139 0 1

Regional economics 0.188 0.132 1

Transport network and accessibility 0.241 0.188 1

Greening of transport 0.091 0 0.545

Flexibility on logistics 0.045 0 1

Top ranked alternatives

shift preference





Socio

-eco

nom

ic ro

bustne

ss

Town

and

land

-use

Regiona

l eco

nom

ics

Tran

spor

t net

work an

d ac

cess

ibilit

y

Green

ing

of tr

ansp

ort

Flexib

ility on

logistics

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Stated weight intervals(min. and max. of all individual weights)

We

igh

t

Weight stability intervals


Framework

15.06.2011

Decision Conference

Definition of MCA Criteria

Pair wise comparisons using

REMBRANDT

Weighting of criteria using rank

order

Set of alternatives

Cost Benefit Analysis

Multi Criteria Analysis

Robustness Analysis

Input to the decision making process


Conclusions (I)Case:• A small increase on the weight on the CBA will result in a change of

preference order– Within the stated weight range from the DMs

• A minor decrease in the weights for Regional economic and Transport network and accessibility will change the preference order

– Within the stated weight range from the DMs

• Variations in 3 out of 6 criteria weights will not alter the preference order of the top ranked alternatives

• Only changes in 2 criteria weights can influence on the top preferred alternatives

15.06.2011


Conclusions (II)Methodology:• Possible to determine if changes in the criteria weights are critical for the

preference order of alternatives

• Transparent analysis, which can be communicated to DMs

• Assessing the impact of the subjective weights

Perspectives:• A ‘full scale’ decision conference is to be held with experts in October

• Modification of methodology when tested on ‘real’ decision makers

15.06.2011


The work presented in these slides is part of the Øresund EcoMobility project and co-funded by the EU / Interreg IV A ØKS Programme

http://www.oresundecomobility.org15.06.2011

http://www.oresundecomobility.org/


Thank you…

15.06.2011


Socio-economic robustness criterion

15.06.2011

Alternative B/C rate Socio-economic robustness

Alt1 1.23 27%

Alt2 2.69 77%

Alt3 2.25 69%

Alt4 3.10 82%

DTU Transport. Technical University of Denmark MCDM 2011, Jyväskylä27

REMBRANDT calculations

• A1, A2, A3 and A4, are compared in a pair wise way under the five criteria

• The final scores are calculated using the multiplicative model and normalised:A1: 0.710.19 . 4.000.06 . 1.000.27 . 4.000.12 . 0.500.36 = 0.93 ~ 0.12A2: 0.040.19 . 4.000.06 . 0.030.27 . 16.000.12 . 0.030.36 = 0.09 ~ 0.01A3: 11.310.19 . 0.250.06 . 16.000.27 . 0.060.12 . 4.000.36 = 3.69 ~ 0.47A4: 2.830.19 . 0.250.06 . 2.000.27 . 0.250.12 . 16.000.36 = 3.12 ~ 0.40

13-06-2011

Pair wise comparisons (δjk) Transformations (γ = 0.7)Geo.mean

A1 A2 A3 A4 A1 A2 A3 A4 ScoreA1 0 4 -4 -2 1 16 0.0625 0.25 0.71A2 -4 0 -8 -6 0.0625 1 0.0039 0.0156 0.04A3 4 8 0 2 16 256 1 4 11.31A4 2 6 -2 0 4 64 0.25 1 2.83

Documents

Robustness in assessment of strategic transport projects The 21st International Conference on Multiple Criteria Decision Making Jyväskylä June 13-17. 2011