1 INTRODUCTION 3

2 CBA 3

2.1 THE STRENGTHS OF CBA 4 2.2 THE WEAKNESSES OF CBA 5

3 MCA 7

3.1 THE STRENGTHS OF MCA 8 3.2 THE WEAKNESSES OF MCA 8

4 WHEN TO USE CBA AND WHEN MCA? 9

5 APPLICATION OF MCA IN THE CLG PROJECT 11

5.1 MCA SCHOOLS 11 5.2 FORMULATION OF SHORT-LIST 12 5.3 PRESENTATION OF A PRACTICAL CASE 12

6 THE DANISH ROAD DIRECTORATE METHOD (VD) 14

6.1 DESCRIPTION 14 6.2 NUMERICAL EXAMPLE 14 6.3 CRITICISM 15 6.4 METHOD PERSPECTIVE IN CLG 15

7 THE ANALYTIC HIERARCHY PROCESS (AHP) 15

7.1 DESCRIPTION 15 7.2 NUMERICAL EXAMPLE 16 7.3 CRITICISM 18 7.4 METHOD PERSPECTIVE IN CLG 19

8 THE SIMPLE MULTIATTRIBUTE RATING TECHNIQUE (SMART) 19

8.1 DESCRIPTION 19 8.2 NUMERICAL EXAMPLE 20 8.3 CRITICISM 21 8.4 METHOD PERSPECTIVE IN CLG 22

9 COMPOSITE METHOD FOR ASSESSMENT (COSIMA) 22

9.1 DESCRIPTION 22 9.2 NUMERICAL EXAMPLE 23

2

9.3 CRITICISM 24 9.4 METHOD PERSPECTIVE IN CLG 24

10 FLEXIBLE METHOD FOR EFFICIENT USE OF INFORMATION IN DECISION

AID (FEIDA) 25

10.1 DESCRIPTION 25 10.2 NUMERICAL EXAMPLE 26 10.3 CRITICISM 27 10.4 METHOD PERSPECTIVE IN CLG 27

11 WEIGHTING METHODS 28

11.1 EQUAL WEIGHTS 28 11.2 USER DERIVED WEIGHTS 28 11.3 DIRECT WEIGHTING 28 11.4 THE ANALYTIC HIERARCHY PROCESS 29 11.5 SWING WEIGHTS 29 11.6 INDIFFERENCE TRADEOFF WEIGHTS 29 11.7 GAMBLE METHOD 29 11.8 CONCLUDING REMARKS 30

12 COMPARISON AND CHOICE OF METHOD 31

3

Cost-Benefit Analysis (CBA) and alternative approaches from the Centre for Logistics and Goods

(CLG) study of evaluation techniques, 2004

By

Stine Gissel Goldbach and Steen Leleur

1 Introduction In the debate on evaluation methods the discussion often tends to be one of cost-benefit analysis (CBA) against multi-criteria analysis (MCA), where proponents of one method fail to recognize the possible strengths of the other method – on the contrary they see the other method much as a “competitor” to “their own” method. In this paper, we will try to build bridge between the different views represented in the different evaluation methods. We do not necessarily consider CBA and MCA as competing alternative evaluation methods. Rather we recognize their individual strengths as well as their weaknesses, and we see the methods more as complements than as substitutes.

First we discuss the strengths and weaknesses of each of the methods as well as their applicability in different decision situations. Next, we focus on different types of multi-criteria methods to illustrate to a certain extent the variety of MCA methods. Finally, in a practical application of the different MCA methods as well as the CBA method we will try to derive some conclusions on the applicability of the presented methods.

2 CBA The basic principle underlying the CBA is that of maximizing the net socio-economic benefit of the project, which may be seen as society’s welfare gain. As society in some sense consists of the sum of its individuals, it is natural to see the social change in welfare from a given investment as the aggregate value of the individual utility gains and losses. With this point of view, constructing the measure of social welfare used in CBA can in principle be broken into two steps (Kopp et al., 1997). In the first step, the attempt is to develop measures of well-being for individuals in the society. In the second step, one aggregates the measures of individual welfare into one of social welfare. Individual measures of well-being are premised on a fundamental economic assumption: that the satisfaction of individual preferences gives rise to individual well-being (Kopp et al., 1997). This assumption states that given the choice set available to the individual, he will always make choices in a way that maximises his own welfare (or

4

utility). That is, the individual knows what is good for him, his preferences for actions and outcomes reflect this knowledge, and he will act in a manner consistent with these preferences in a desire to increase his well-being. Economists take the assumption of such a “rational consumer” as a matter of faith, and it underlies most if not all of economic theory. However, others (especially psychologists) question this “preference satisfaction” as it does not leave appropriate room for taking more altruistic concerns into consideration. The debate will probably continue, and for now it should just be noted that the assumption is of crucial importance to the theoretical foundation of CBA. If the “preference satisfaction” assumption is accepted, it is natural to look at people’s choices as reflections of their well-being – hence using such observations of trade-offs between alternative choices as guides to defining and quantifying economic values (as in the economic valuation methods known as revealed preferences, or RP, and stated

preferences, also SP). The second step is to aggregate the measures of individual welfare to one of social welfare. The traditional approach in CBA is to express the social welfare as an un-weighted sum of the individual welfare (or utility) gains and losses. Such a welfare function is called a utilitarian welfare function and it rests on the Kaldor-Hicks criterion from welfare theory, which says that an alternative can be supported provided the welfare gainers could in principle compensate the welfare losers – even if they do not1. As in this way individuals are treated anonymously, this means that a policy, which increases the welfare of rich people more than it decreases the welfare of poor people, would be considered a social welfare improvement. This example shows one of the criticisms of CBA – that the method ignores important considerations on equity and distribution. This will be further discussed below.

2.1 The strengths of CBA The appealing features of the CBA are quite convincing and well-known. Hence, they are only briefly mentioned here. They may be categorized according to the following overall bullets:

• Transparency

• Comparability / consistency

• Ignorance revelation (through systematic collection of information) Firstly, the CBA converts all social implications into an absolute monetary measure of the social profitability. It is desirable to be able to sum up all aspects of the decision problem in one simple value. Secondly, the CBA provides a methodological tool for comparing projects and/or alternatives, which makes it a powerful decision support tool in the planning process; The values on cost and benefit elements are consistent between investments and over

1 Thus this criterion is less restrictive than the Pareto criterion, which states that an alternative

leads to a welfare improvement if at least one person gains utility (i.e. welfare) and nobody loses utility (welfare).

5

time. This means that the social profitability of projects or policies can be compared across sectors and at different points in time. Thirdly, the CBA requires the collection of detailed information of financial as well as social costs and benefits. This gathering of information improves the basis on which the decision is made and may give valuable insight into the level of ignorance regarding important aspects of the evaluated project or policy.

2.2 The weaknesses of CBA There are of course also problems associated with the CBA method. In short these may be categorized according to the following bullets:

• “False” transparency

• Practical measurement problems

• Inter-generational equity (sustainability)

• Social equity Firstly, it is difficult to maintain consistency between the theoretical assumptions of the CBA method and the practical application of it, due to the fact that there may be problems involved when estimating unit prices for non-marketed impacts such as travel time savings, emissions, safety, etc. In practice, therefore, compromises are often made on the valuation of such non-marketed impacts, implying that the resulting unit prices are inherently of a subjective nature – without such subjectivities being visible in the evaluation. This is a problem with the CBA method since the presentation of a single evaluation measure thus implies a “false air of objectivity”. Also, what is seen by most economists as one of the great advantages of CBA, namely its great transparency, is argued by others as the exact opposite: All financial, environmental and social considerations are reduced to a single number – thereby shielding the results behind a technical mystique. This disagreement could be argued as being a matter of taste, but it is a real problem if the general public perceives the evaluation method as some kind of “black box”. Secondly, there are impacts which can hardly be quantified or for which it is difficult or even impossible to estimate unit prices. These are especially impacts of a more long-term and/or strategic nature – as for example many environmental impacts. Thirdly, an important philosophical and moral problem in the evaluation of (long term) impacts is that of the present generation valuing an impact, which they may not live to experience. This means that they are valuing such impacts on behalf of the future generation(s). The Brundtland Commission defines sustainable development as: “development that meets the needs of the present without compromising the ability of future generations to meet their own needs” (Beder, 2000). With this definition (environmental) sustainability is an issue of equity between generations.

6

However, the discounting of costs and benefits, which is a fundamental part of the CBA, disregards the desires and needs of future generations, hence compromising inter-generational equity. Costs that are more than thirty years away become almost valueless when discounting at normal rates. Hence, long-term costs, such as e.g. environmental resource depletion may be effectively ignored in a CBA. Discounting therefore discriminates against future generations by saying that future costs are worth less than present costs, and that present benefits are worth more than future benefits. As noted by Beder (2000) the logic behind discounting derives from the logic of money – that a person would prefer to receive money now than the same amount in the future (the time preference rate is positive). This is because:

1. money obtained now can be invested and earn interest 2. people tend to be impatient (they want to enjoy benefits sooner and costs later) 3. the person might die before he or she gets the money 4. one cannot be sure of getting the money in the future 5. people in the future will probably be better off; money will not be worth as

much then Seen from society’s point of view, it is more the number and types of individuals receiving a given benefit, which matters, and not whether it is a specific person. Hence, the idea that someone would like to consume now rather than in the future is not applicable to public goods, which can be enjoyed now and in the future. Also, the risk of one person dying before he or she gets the benefit is of no relevance if this person is just “exchanged” by another (as will be the case for a number of costs or benefit elements accruing over time). Any positive discount rate devalues future costs or benefits and this disadvantages future generations with respect to today’s decisions. The logic of money – and in this respect the logic of discounting – may thus seem inappropriate when evaluating certain types of costs and benefits. This is especially the case for (long term) environmental impacts. The final problem with CBA to be mentioned here is that of social equity. This can be divided into three separate questions: The first critique relates to the individual welfare measurements: When valuing costs and benefits often methods based on individuals’ willingness to pay are used. As people’s willingness to pay, whether measured directly or inferred in some way, will be intimately linked with their ability to pay, the market can be seen as a system which advantages those most able to pay. Hence (as noted by Beder, 2000) using the market, whether an actual market or a contrived one, tends to produce values that reflect the existing distribution of income. This can be argued as an equity problem. The second critique relates to the aggregation of individual welfare measures into one of social welfare: In its conventional form CBA is about aggregated (and un-weighted) costs and benefits and does not deal with the issue of how they are distributed – although this is of prime concern when considering equity. As long as the sum of

7

benefits outweighs the sum of costs (no matter who or how few people get the benefits and who or how many people suffer the costs) the society as a whole is assumed to be better off. Some argue, that in principle the CBA does not presuppose that individuals are treated anonymously – that is with equal weight in the aggregation of individual welfare into a measure of social welfare. In theory, one could aggregate individual welfare measures in a way (i.e. with weights) reflecting relevant equity concerns. However, as there is no established “right” to equity in the distribution of individual welfare, where would a decision maker get the needed weights? No unique set of “equity weights” exists, and therefore anonymous aggregation has become the default in CBA. The third critique is that although the method rests on the aggregation of individuals’ willingness to pay, no actual payment takes place and no actual redistribution of money results. Hence, the socio-economic optimum resulting from the CBA could be argued on equity grounds as being somewhat hypothetical.

3 MCA In MCA the relative values of different criteria are explicitly subjective – as opposed to the CBA where unit prices reflect some sort of objectivity. Hence, MCA methods presuppose a preference structure giving preferences on the different criteria. It is emphasised that with an assumption of such a preference structure the methods depend very much on the personality of the decision maker and the circumstances in which the decision process takes place. The purpose of a MCA is therefore not searching for some kind of hidden truth – but rather to assist the decision maker in mastering the (often complex) data involved and advance towards a solution (Gissel, 1999). Therefore, we prefer to refer to MCA as a decision aiding tool. Multi-criteria decision problems can be naturally divided into problems concerning multiple objectives and problems concerning multiple attributes. In this report the focus is on decision problems of the second kind – that is, decision problems with given alternatives described by their multiple criteria (or characteristics). In general, the essence of decision analysis is to break down complicated decisions into smaller pieces that can be dealt with individually and then recombined in a logical way. For the MCA methods there are basically three such distinct pieces: the set of possible alternatives, their characteristics (represented by a set of criteria), and the preference structure of the decision maker(s) – reflected in criteria weights. Generally, the alternatives and their criteria represent the objective part of the decision process, whereas the subjective part of the decision process lies in the preference structure. However, in the case where a given criterion cannot be quantified in an obvious way, the decision maker and the analyst may be forced to make subjective assessments of the criteria scores, or they will have to find a surrogate measure which can function as a good proxy for the criteria. As discussed in Gissel (1999) the use of

8

proxies should be preferred to subjective scores whenever possible. The principal argument for this is to restrict the subjectivity in the decision process to elements for which a constructive exchange of (political) opinion can take place. There exists a wide variety of MCA methods representing a corresponding variety in methodological approach. However, the fundamental structure of the methods is generally the same – namely the threefold structure described above.

3.1 The strengths of MCA The strengths of the MCA may be summarized in the following:

• MCA overcomes most measurement problems

• Participation of the decision maker(s)

• MCA can be accommodated to address equity concerns Firstly, MCA overcomes the difficulty of translating all values into monetary units by using subjective weights. Also, both qualitative and quantitative indicators can be used depending on the criteria, and the time and resources available. That is, if an impact cannot be quantified (be it because of scarcity of time or resources or not) it may instead be represented by some sort of indicator (a proxy or a subjective score). Secondly, stakeholders can be involved throughout the decision making process to determine the alternatives and criteria, the criteria weights and to score and determine the best solution. The technique offers – in fact it often requires – a more participatory approach as it takes decisions out of the hands of analysts and puts it with those stakeholders involved. Thirdly, MCA can address equity concerns by incorporating equity criteria into the analysis, and it allows each individual the same representation, unlike the market (i.e. CBA) where those with greater money exert greater influence. MCA may be seen as an extension of the CBA. In practice, the CBA represents only a part of the decision making basis; other non-monetised impacts represent another, and the final choice is based on a weighing of these different parts. Using only CBA this weighing may be completely opaque. The MCA methods offer a tool for approaching the subjectivities in the decision process and for reaching a decision in a methodological and transparent way. In MCA both the monetised impacts of the CBA as well as more strategic impacts can be accommodated in one approach.

3.2 The weaknesses of MCA Of course, there are also problems associated with the use of MCA. In short these may be presented as the following:

• The method can give no “absolute” measure of “goodness” – it is a tool for comparative evaluation only

• The participatory nature of the MCA makes it both time and resource intensive

9

• Difficulties in deriving criteria weights Firstly, the method is a tool for comparative evaluation only, as it gives no absolute measure of the “goodness” of the project or policy as does the CBA with its socio-economic profitability. This makes MCA a tool for deciding between options and not for a “go/no-go” decision. Secondly, the active involvement of the decision maker(s) makes the MCA both time and resource intensive as it requires much of decision makers. However, in our opinion, this should not necessarily be seen as a drawback of the method. Decisions are seldom objective, and if subjective judgments are present these should be transparent – at least to the decision maker himself. MCA offers a tool for obtaining this insight. Thirdly, there can be problems with the elicitation of criteria weights. The analyst should be very aware that the derivation of weights is a fundamental and critical step in the MCA. Different MCA methods may require different types of weights derived in different ways (see also the chapter on weights later in this paper). In the weights derivation process the decision maker should be helped to understand the meaning and importance of his stated weights to increase the understanding and acceptance of the MCA with the decision maker. This understanding and acceptance is crucial for the applicability of the method.

4 When to use CBA and when MCA? CBA is a partial model and in that sense it does not include all possible types of effects that might occur as a consequence of a project or policy. This can be perfectly reasonable in many cases where the costs and benefits measured in the CBA are a reasonable measure of the overall effect. In other cases there may be substantial “wider economic impacts” which are wider effects on economic activity than those traditionally measured in a CBA. These include effects on employment, prices, and economic growth at the local, regional, national or international level, see ECMT(2001) for a thorough discussion on this. Although CBA may be extended to accommodate for some of these effects, the CBA may be stretched to a degree that makes this partial model so complex that one might just as well prefer to use a general equilibrium model. Also, there may be other strategic (or “wider”) effects than the economic ones. Especially many environmental effects will fall into this category, and these are often very hard to measure on a monetary scale. The conclusions from the ECMT report on these types of effects in evaluation are the following:

• Network effects derived from macroeconomic analyses can be added to CBAs, but the uncertainties when including these are considerable.

10

• Effects, which are hard to express in monetary terms, should not be included in CBAs, but be presented separately according to existing MCA yardsticks.

Hence, the question on when to use CBA, when to use MCA – and when, possibly, a combination of the two – depends on the nature of the relevant effects as well as the extent to which these effects can be quantified and assigned a monetary value. From our point of view, the CBA method is most appropriate in cases where there are no important distributional effects and where there are no other difficult political questions of a principle nature. Furthermore, for CBA to be “enough”, there should be no important effects, which cannot be given an economic value. When this is not the case, CBA should be supplemented with qualitative (and possibly quantitative) analyses addressing these special effects. When adding such supplementary analyses to the CBA, the thought of combining the CBA with some sort of MCA does not seem far. With the widely accepted CBA as a solid base for project assessment, there is to us nothing odious to the thought of extending the assessment with MCA data. To us, this seems as a clear step towards more comprehensive analyses. The applicability of the two methods in different planning situations can thus be illustrated by the following table.

11

Few effects can be quantified Most important effects can be quantified

No important effects of a strategic/political nature

MCA CBA

One or more important effects of a strategic/political nature

MCA CBA + MCA

Table 1: Applicability of CBA and MCA under different conditions

When considering especially transport infrastructure projects, the existence of strategic effects and/or effects of a principle political nature depends to a large degree on the size of the project. Furthermore, the availability of quantified data is much dependent on the planning phase – in early stages of planning, only limited amounts of data exist, and “quantifications” will be more of the nature of indicators. Hence, Table 1may be “translated” to the following:

Time of evaluation Size of project

Early stages of planning More advanced (later) stages of planning

Small infrastructure project MCA CBA

Larger infrastructure project MCA CBA + MCA

Table 2: Applicability of CBA and MCA in different transport planning situations

In this way we see the role of MCA as two-fold. On the one hand as a screening tool in early phases of planning, where quantifications are not always present. And on the other hand in combination with a solid CBA base in the assessment of (larger) projects with some strategic and/or politically important effects.

5 Application of MCA in the CLG project 5.1 MCA schools The existing variety of MCA methods can be categorized into two overall schools; The American school and the French school. In the American school the view of the decision maker is disaggregate in the sense that the decision maker is assumed to have a complete preference system. This preference system enables him to express his preferences on all aspects of the decision problem, and it may be derived through asking the decision maker relevant questions. Lootsma (1999) refers to this approach as the normative approach. Typical examples of this approach are the multi-attribute value theory (MAVT) and the multi-attribute utility theory (MAUT). The Simple Multi-Attribute Rating Technique (SMART) is a variant of

12

the MAVT. This method will be explored further below as will the Analytic Hierarchy Process (AHP). The French school covers the family of outranking methods. In the French school the existence of a well-ordered preference system is questioned, the view is more that of the decision maker as a rational economic man (Gissel, 1999). The basic assumption is that the decision maker explores the assertation that “alternative i is at least as good as alternative k”, and the only pre-existing preferences he has is an idea of the relative importance of the criteria. Lootsma (1999) refers to this approach as the constructive

approach. The most well known outranking method is that of ELECTRE, which has been developed in a number of versions. The ELECTRE method will be discussed further in Section 6. The SMART method and the AHP method initially require the performances of the alternatives to be measured for each of the criteria. In SMART the performance of the alternatives under the respective criteria, evaluated via a direct-rating procedure, is expressed in grades on a numerical scale (Lootsma, 1999). In AHP the alternatives are considered in pairs. Their relative performance can equivalently be expressed as a ratio of subjective values (Multiplicative AHP) or as difference of grades (Additive AHP, or SMART with pairwise comparisons) (Lootsma, 1999). When the alternatives have been measured with respect to each of the criteria, the next step is to assign weights to the different criteria. The weights are used to calculate weighted means of the performance measures. In the case of SMART the aggregation rule is that of arithmetic mean, whereas the AHP uses the geometric mean aggregation rule. This will be demonstrated further in later chapters. In Section 7 the concept of weights is discussed further.

5.2 Formulation of short-list The methods that will be considered in this comparison represent the variety in the spectrum of MCA methods. The Danish Road Directorate Method (VD) is in fact a CBA method and is included to represent and illustrate the CBA evaluations as compared to the results of the MCA methods. The CBA evaluation is expanded with multicriteria elements in COSIMA where the “softer” criteria are weighted against the CBA of the “harder” criteria. The analytic hierarchy process (AHP) and the simple multi-attribute rating technique (SMART) represent the American MCA-school, whereas the French MCA-school is represented by FEIDA (Flexible method for Efficient use of Information in Decision Aid) which is in fact a hybrid method that links the American and the French MCA schools.

5.3 Presentation of a practical case When describing the different methods it is useful to illustrate the concepts of the methods by use of a numerical example. The numerical example that will be used here concerns the assessment of seven by-pass alternatives for relieving Høng on Western Zealand for through traffic.

13

The seven alternatives are shown in figure 1. There are three western alternatives (p1, p2 and p3) and four eastern alternatives (p4, p5, p6 and p7)

Figure 1: The seven by-pass alternatives in the Høng case

In the following chapters, each of the methods mentioned above will be illustrated by use of the Høng case. Table presents a gross list of criteria to be considered in the applications in the following sections. First-year-benefit, FYB, describes the sum of CBA benefit elements in the first year after the completion of the by-pass. Total benefit is the sum of CBA benefit elements over a 30-year period – discounted to the base year. Construction costs is obvious. The B/C rate is the rate between Total benefits and construction costs. Network accessibility, Landscape and Urban planning are strategic effects. Each of these strategic effects have been carefully judged on a scale from –5 to 5, see Steffensen & Testmann (2000).

14

Alternative FYB

(kDkr)

Total benefit (kDkr)

Construction

costs (kDkr)

B/C

rate

Network

accessibility

Landscape Urban

planning

p1 1375 29033 16700 1.74 2 -2 1

p2 881 18603 15300 1.22 1 4 1

p3 465 9819 16900 0.58 -1 2 3

p4 2900 61234 18000 3.40 4 2 -2

p5 2476 52281 17200 3.04 3 3 2

p6 1839 38831 20800 1.87 -3 0 4

p7 2349 49600 19900 2.49 3 3 1

Table 3: Criteria to be considered in the subsequent illustrations of the different MCA methods on

the short-list

6 The Danish Road Directorate method (VD)

6.1 Description The evaluation framework used for many years by the Danish Road Directorate (Vejdirektoratet, 1992) concerns national road network projects. The framework was initially developed around 1980 but has been continually maintained and updated. The basic principle is cost-benefit analysis, and the evaluation principle is that of “first year rate of return”. The framework consists of “traditional” benefit and cost components like construction costs, system operating and maintenance costs, vehicle operating costs, travel time savings, accidents, noise, air pollution and severance.

6.2 Numerical example The Høng case serves as the numerical example. For each of the relevant cost and benefit elements the difference between the reference alternative and each of the seven by-pass alternatives is calculated. These quantitative measures are converted to monetary ones by the use of standard unit prices as used by the Danish Road Directorate. These calculations are standard and also an explicit element of the COSIMA model to be described later. For the Høng case COSIMA thus gives the first-year-benefits (FYBs) as indicated in Table 4. The table also presents the construction costs and the resulting first year rate-of-returns.

15

Alternative FYB (kDkr) Construction cost (kDkr) First year rate-of-return

p1 1375 16700 0.08

p2 881 15300 0.06

p3 465 16900 0.03

p4 2900 18000 0.16

p5 2476 17200 0.14

p6 1839 20800 0.09

p7 2349 19900 0.12

Table 4: First-year-benefit, construction cost and resulting first year rate-of-return for the seven

alternatives in the Høng case

6.3 Criticism Pros and cons of the method mainly relate to the pros and cons of CBA in general – as discussed earlier.

6.4 Method perspective in CLG The VD method used by the Danish Road Directorate represents the starting point of transport project appraisal in Denmark. With its solid CBA kernel the VD method forms a good basis for developments into a multicriteria setting.

7 The Analytic Hierarchy Process (AHP)

7.1 Description The Analytic Hierarchy Process (AHP) was developed by Saaty around 1980, and the method has proved to be one of the more widely applied MCA methods. The basic concept in the AHP is that of pairwise comparisons, which means that the decision maker answers series of questions of the type “How important is A compared to B?” This concept may be used to establish, within AHP, both weights for criteria and performance scores for options on the different criteria (see also Section 11 on weighting methods). When deriving weights the procedure is that of pairwise comparisons of criteria, and when performance scores are derived the options (for a given criteria) are being compared. AHP can be applied directly on the full set of criteria or it can be used in a hierarchical clustering of the criteria. In its hierarchical form, the AHP engages decision makers in breaking down a decision into smaller parts, proceeding from the overall goal to the criteria and down the hierarchy to levels of subcriteria and the set of alternatives.

16

Decision makers then make simple pairwise comparisons throughout the hierarchy to arrive at overall priorities for the alternatives. Since the original AHP by Saaty, there have been some developments of the method. These mainly relate to the scale, which quantifies the verbal comparative judgements. The Additive AHP and the Multiplicative AHP are such developments. For a decision maker there is no difference between the original AHP and the Additive or the Multiplicative AHP as far as input of the judgemental statements is concerned – the difference is in the subsequent quantifications of the answers. The scales of the different AHP methods are presented in Table 5 (from Lootsma (1999)).

Comparative preferential judgement of Aj

with respect to Ak

Original

AHP

Additive

AHP

Multiplicative

AHP

Very strong (absolute) preference for Aj 9 8 256

Strong preference for Aj 7 6 64

Strict (definite) preference for Aj 5 4 16

Weak preference for Aj 3 2 4

Indifference between Aj and Ak 1 0 1

Weak preference for Ak 1/3 -2 ¼

Strict (definite) preference for Ak 1/5 -4 1/16

Strong preference for Ak 1/7 -6 1/64

Very strong (absolute) preference for Ak 1/9 -8 1/256

Table 5: Numerical scales to quantify comparative preferential judgement in the original AHP, the

Additive and the Multiplicative AHP

The Additive AHP uses a scale based on differences of grades whereas the Multiplicative AHP – like the original AHP uses a scale with estimated ratios of subjective values. On the basis of the pairwise comparison matrix obtained by the set of pairwise judgements (as given by the corresponding scales) the weights (or alternatively the scores) that are most consistent with the relativities expressed in the matrix are derived. In the original AHP this is done by use of the Perron-Frobenius eigenvector method. However, later contributions showed that when retrieving weights from an inconsistent pairwise comparison matrix, a more justifiable method was the geometric mean aggregation rule (see e.g. Gissel (1999)). For a variant which is based upon difference information, i.e. the Additive AHP, the arithmetic mean aggregation rule is the appropriate rule (see Lootsma (1999)).

7.2 Numerical example The seven bypass alternatives for Høng are evaluated on the basis of the B/C rate and the three more strategic criteria: network accessibility, landscape and urban planning.

17

The AHP approach is convenient in the case where alternatives under a given criterion are not measured on some natural scale. In the case of the Høng example, the criteria network accessibility, landscape and urban planning are such qualitative criteria. The importance of the different “stimuli” of a given strategic effect might then be judged indirectly through a set of pairwise comparisons of stimuli. Consider for example the criterion Network accessibility. The importance of each of the (seven) stimuli of this criterion may be derived after judging the relative importance of pairs of stimuli. All in all there are 7*6/2=21 pairwise comparisons that can be made. It is unlikely that the decision maker is able to maintain consistency throughout these comparisons, and through mathematical approaches the set of weights is found which minimises the inconsistency of the judgements. As mentioned above this step may be approached either by the Perron-Frobenius eigenvector method or by use of the geometric mean aggregation rule. However, if the decision maker is fully consistent in his relative judgements, then he needs not make more than 6 pairwise comparisons for the set of weights to be derived (given that the weights sum to one this gives seven equations with seven unknowns). For simplicity we assume that the judgements are consistent, and that the following six pairwise comparisons are made:

Alternative Preference relation Alternative Resulting AHP rating

(original AHP)

p1 is equally preferred to p2 1

p2 is equally to weakly preferred to p3 2

p4 is weakly to strictly preferred to p3 4

p4 is equally preferred to p5 1

p5 is strictly preferred to p6 5

p7 is strictly preferred to p6 5

Table 5: Comparative preferential judgement of alternatives in the Høng case – with respect to

stimuli on the criteria Network accessibility

These statements indicate that with respect to Network accessibility the score of p1 is equal to that of p2, which is again twice the score of p3. The score of p4 is four times that of p3, and so on. The scores corresponding to these relative judgements may thus be calculated through solving the seven equations with seven unknowns. This gives the scores as presented in Table 6. The scores resulting from (hypothesised) relative judgements on the other two strategic criteria as well as the normalised B/C rates are also shown in this table. (It is assumed that the quantitative B/C rates are directly transferable to the preference scale, i.e. that the ratio of two B/C rates represents the relative preference between these two alternatives).

18

Alternative B/C rate Network

accessibility

Landscape Urban planning

p1 0.121 0.112 0.058 0.095

p2 0.085 0.112 0.290 0.095

p3 0.040 0.056 0.145 0.190

p4 0.237 0.225 0.145 0.048

p5 0.212 0.225 0.145 0.143

p6 0.130 0.045 0.072 0.286

p7 0.174 0.225 0.145 0.143

Table 6: (Normalised) impact scores for the criteria in the Høng case

Next, the criteria weights are derived. These can be determined through comparative preferential judgement as for the impacts scores. However, here it is assumed that the decision maker directly sets the weight profile as in Table 7.

Criterion B/C rate Network accessibility

Landscape Urban planning

Weight 70% 15% 7.5% 7.5%

Table 7: Overall weight profile in the Høng case

The total AHP score of a given alternative is calculated as the weighted sum of impact scores. Hence, for alternative p1 the score, Sp1, is calculated as:

Sp1 = 0.7*0.121 + 0.15*0.112 + 0.075*0.058 + 0.075*0.095 ≅ 0.114 In a similar way, the scores of the remaining alternatives can be calculated.

7.3 Criticism Pros: Users often find the pairwise comparison form of data input straightforward and convenient, and the method is especially useful when input information is more of a descriptive nature (as for the three strategic criteria in the numerical example) rather than given as “excact” measurements. The decision maker in some cases makes judgments, which may seem inconsistent but which are nevertheless perfectly reasonable for the decision maker himself. As an example he may prefer Red to Yellow and Yellow to Blue – and then prefer Blue to Red. Such judgments conflict with the standard requirement of transitivity in preferences – but they may make sense to the decision maker, and they can be handled in the AHP. Cons:

19

There are doubts about the theoretical foundations of the AHP, as the link between the verbal descriptions of preference and the numerical AHP scale (whichever is chosen) has no theoretical foundation. An example of a problem with a predefined scale like the original 1-9 scale is illustrated by the following example: The decision maker finds alternative A five times better than alternative B, which he again finds five times better than alternative C. But then, if he is consistent, he should find A around 25 times better than C. This judgment cannot be represented by the original AHP scale. In principle one could construct similar examples with the other predefined AHP scales. This raises the question of why a predetermined scale is needed. Perhaps it would be better to let the user decide directly upon a rate of tradeoff? The rank reversal problem: the possibility that when a new alternative is added to the set of alternatives being evaluated, the ranking of two existing alternatives (not related to the new one) can be reversed. This problem arises from a failure to relate measurement scales to the associated weights, and thus, relates to the discussion of weights in Section 11.

7.4 Method perspective in CLG AHP has been widely used over the past decades. The method thus represents a family of MCA methods, which the authors feel cannot be excluded from consideration – especially because of the apparent convenience of the method felt by users of it, and its appealing nature when judgments of qualitative impacts need to be made.

8 The Simple Multiattribute Rating Technique (SMART)

8.1 Description SMART is a linear additive model. This means that an overall value of a given alternative is calculated as the total sum of the performance score (value) of each criterion (attribute) multiplied with the weight of that criterion. In SMART the performances of the alternatives are assessed directly, in the natural scales of the criteria – where such are available. The different scales of the criteria are converted to a common scale (from 0 to 1) by means of value functions. A value function, VA, describes a person’s preferences with regard to different levels of an attribute, A. The simplest choice of a value function is a linear function, however, for some criteria it might be more appropriate to use non-linear functions to account for non-linearities in the perception of the criteria outcomes. The value functions of the criteria may be derived through asking the decision maker series of questions on his preferences towards different changes in the attribute level. In some cases it might be more relevant to ask specialists – for example a biologist is

20

better at judging the value function for wetland impacts, and an economist would scale the monetary benefits. For non-quantitative criteria the value functions can be assessed by categorising the different stimuli and assigning these categories values between 0 and 1. (In this case, the value function is not a continous function, and the concept of tradeoffs – as discussed in a later chapter on weighting methods – does not make much sense). When value functions have been assessed for all attributes, and when weights have been derived, the overall value – the decision score – of each of the alternatives can be calculated as:

Vj = Σi wi Vi(Aij)

Where Vj is the total value (or “decision score”) of alternative j and wi is the weight of attribute i. The alternatives can then be ranked according to their decision score – the alternative with the highest score being the (seemingly) best alternative. It should be noted, that the definition of the end points of the value functions plays a role and should be considered carefully. This especially applies if weights are not derived as swing weights or tradeoff weights (see the Section 11 on weighting methods). SMART with swing weights is also referred to as SMARTS2.

8.2 Numerical example The seven bypass alternatives for Høng are evaluated on the basis of the criteria in the traditional CBA (of the Danish Road Directorate) – as represented by the criteria Total benefits and Construction costs – as well as the three more strategic criteria Network accessibility, Landscape and Urban planning. The criteria are quantified as indicated in Table 8. For the criteria Total benefit and Construction costs the end points of the value functions have been defined as rounded values of the minimum and maximum values of the ranges shown in this table. For the strategic criteria – which are measured on a –5 to 5 scale – the end points are defined as –5 and 5 respectively.

2 Another version of SMART is SMARTER (SMART Exploiting Ranks) which is less demanding

in its information input requirements from the decision maker (see Dodgson et al. (2001)).

21

Alternative Total

benefit

(kDkr)

Construction

costs (kDkr)

Network

accessibility

Landscape Urban

planning

p1 29033 16700 2 -2 1

p2 18603 15300 1 4 1

p3 9819 16900 -1 2 3

p4 61234 18000 4 2 -2

p5 52281 17200 3 3 2

p6 38831 20800 -3 0 4

p7 49600 19900 3 3 1

Table 8: Quantified impacts in the Høng case

The weights have been derived as tradeoff weights. The two criteria Construction costs and Total benefits (measured on the same monetary scale) have been traded of one to one. The strategic impacts have been traded of as follows: A change from –5 to 5 on Network accessibility is traded off with 1.300.000 kr. A change from –5 to 5 on Landscape is traded off with 650.000 kr. A change from –5 to 5 on Urban planning is traded off with 650.000 kr. These tradeoffs imply that the weights on the overall criteria CBA, Network accessibility, Landscape and Urban planning are given as in Table 9.

Criterion Weight

CBA 69.4%

Network accessibility 15.3%

Landscape 7.6%

Urban planning 7.6%

Table9: Weights on overall criteria as implied by the tradeoffs

Given the value functions and the weights, the decision score of each of the alternatives can be calculated.

8.3 Criticism Pros: The structure of the SMART method is similar to that of the traditional CBA in that the total “value” is calculated as a weighted sum of the impact scores. In the CBA the unit prices act as weights and the “impacts scores” are the quantified (not normalized) CBA impacts. This close relationship to the well-accepted CBA method is appealing and makes the method easier to grasp for the decision maker.

22

The method is independent of the alternatives, which means that “rank reversal” is not a problem in SMART as it may be in AHP. Cons: In a screening phase where some poorly performing alternatives are rejected leaving a subset of alternatives to be considered in more detail the SMART method is not always the right choice. This is because, as noted by Hobbs & Meier (2000), SMART tends to oversimplify the problem if used as a screening method as the top few alternatives are often very similar. Rather different weight profiles should be used and alternatives that perform well under each different weight profile should be picked out for further analysis. This also helps identify the most “robust” alternatives. The method has rather high demands on the level of detail in input data. Value functions need to be assessed for each of the lowest-level attributes, and weights should be given as trade-off weights. It may be difficult to account for criteria, which can only be described qualitatively.

8.4 Method perspective in CLG SMART – as a linear additive model – has a straightforward intuitive appeal and transparency that should ensure it a central role in any discussion of MCA (Dodgson et al., 2001). Its close similarity with the traditional CBA method implies that the method may be more acceptable to decision makers than perhaps other more complex MCA methods. Together with the AHP method, SMART thus makes a fine representative for the well-established American school of MCA methods.

9 Composite Method for Assessment (COSIMA)

9.1 Description The basic concept of COSIMA (or COSIMA STIP which stands for composite model for the assessment of strategic transport infrastructure projects) is that of comparing the B/C rate (the benefit-cost rate) with MCA-type impacts. The traditional CBA part of COSIMA builds on the same models and unit prices as the VD method. However, the result of the CBA is given as the B/C rate (with the impacts discounted over a 30 year period) rather than the first year rate of return. The traditional CBA impacts are linked to more strategic impacts through a MCA module where the B/C rate acts as one impact in a weighted sum of all MCA impacts – giving an overall B/C rate. These weights – which are referred to as unit prices – are estimated so that the contribution to the total B/C rate from a given strategic impact corresponds to the overall weight of that impact as set by the user (see also the example below).

23

9.2 Numerical example The seven bypass alternatives for Høng are evaluated on the basis of the following four criteria: First year benefit (from the CBA module in COSIMA), network accessibility, landscape and urban planning. The first year effects of each of the CBA impacts are calculated and summed giving the total first year benefit (FYB) for each of the seven alternatives. On the basis of this first year benefit, the cost of the alternative, as well as factors describing the development of the impact over a 30-year period, the B/C rate for each alternative can be calculated. For each of the alternatives, the remaining MCA impacts are given a grade on a scale running from –5 to 5 as also mentioned in the introduction. Table 10 below reproduces the input data.

Alternative FYB (kDkr) Network

accessibility

Landscape Urban planning

p1 1375 2 -2 1

p2 881 1 4 1

p3 465 -1 2 3

p4 2900 4 2 -2

p5 2476 3 3 2

p6 1839 -3 0 4

p7 2349 3 3 1

Table 10: Quantified MCA impacts for the seven Høng alternatives

The B/C rate of a given impact may be expressed as: B/C = discount factor * unit price * effect (grade) / cost Suppose that the decision maker decides that the overall weight scheme should be that of Table 11. He first chooses one or more anchoring alternatives (here p4) and with this reference he then adjusts the MCA unit prices (manually but guided by the program) until the CBA-based part of the benefits for p4 corresponds to 70% of the total amount of benefits for p4, and the benefit share for Network accessibility to 15% and Landscape and Urban Planning both to 7.5%. The percentage calculation takes the negative value for Urban Planning into account by the use of numerical values.

Impact CBA Network access. Landscape Urban planning

Weight 70% 15% 7.5% 7.5%

Table 11: Weights for the four MCA impacts in the Høng case

24

With alternative 4 as the “anchoring” alternative, this gives B/C rates as indicated in Table 12.

Alternative B/C CBA B/C Netw. access. B/C Landscape B/C Urban plan. Total B/C rate

p1 1.74 0.36 -0.36 0.18 1.92

p2 1.22 0.18 0.73 0.18 2.31

p3 0.58 -0.18 0.36 0.55 1.31

p4 3.40 0.73 0.36 -0.36 4.13

p5 3.04 0.55 0.55 0.36 4.50

p6 1.87 -0.55 0.00 0.73 2.05

p7 2.49 0.55 0.55 0.18 3.77

Table12: B/C rates in the Høng case with alternative 4 as the anchoring alternative

This numerical example illustrates that in the “pure” CBA, alternative 4 is the highest-ranking alternative, whereas alternative 5 ranks higher when MCA impacts are included with weights as indicated in the weights scenario in Table 11 above. COSIMA may also be used to address uncertainties in information. Thus, Monte Carlo simulations may be conducted by use of the @RISK add-in to the Excel based calculations. This means that data may be input as distributions rather than specific numbers.

9.3 Criticism Pros: The COSIMA model is straightforward in its design and application as it simply “adds to” (and does not hide or change) CBA information. Through its incorporation of relevant MCA criteria COSIMA may be seen as an MCA extension of the traditional CBA and may therefore by administrative units be perceived as less of a “black box” than other types of current MCA decision aid tools. Cons: COSIMA depends on an interpretation of the unit prices of the MCA criteria. What does it mean that given MCA impacts must contribute with a certain percentage of the total amount of benefits associated with the reference project(s) – which indirect trade-offs lie behind such percentages? The method assumes that the users can express their preferences by the percentage measures as described above.

9.4 Method perspective in CLG

25

The relevance of the COSIMA method in CLG relates to its nature as a CBA method incorporating MCA elements rather than an MCA method covering also the CBA result. This makes the method an innovative and very interesting MCA approach.

10 Flexible Method for Efficient Use of Information in Decision Aid (FEIDA)

10.1 Description FEIDA rests on the integration of two principally different MCA methods: the ELECTRE method and the multiattribute value method, MAVT, which in fact covers SMART, which was described earlier. The ELECTRE method belongs to the family of outranking methods, which rest on the idea that only the “sure” part of the decision maker’s preferences should be modelled. The general idea of an outranking relation is a binary relation defined on a set of alternatives, such that A is preferred to B if there are enough arguments to decide that alternative A is at least as good as alternative B – while there are no essential reasons to refute that statement. The multiattribute value method (or SMART) is based on the assumption that the decision maker has an a priori complete preference system, which enables him to express his preferences on all aspects of the decision problem. In the ELECTRE method (and in outranking methods in general) the decision maker is assumed only to have an idea of the relative importance of the attributes describing the decision problem. These different views on the decision maker imply that the methods are characterised by different requirements on the information contained in the attribute weights. SMART requires much (detailed) information on both the attribute measures and the preferences of the decision maker. This makes it difficult to apply the method in early phases of planning. Furthermore, in practice it can prove a very difficult task to derive weights from the decision maker at the detailed level required by the method. In the ELECTRE method on the other hand, the (lack of) interpretation of the weights is an aspect, which is sometimes criticised. These weights are not based on any axiomatic requirements and merely reflect some “general notion of relative importance” of the attributes. Such “importance weights” seem especially inappropriate for the weighing of specific quantitative attributes, as such weights are not related to the attribute scales (see also Section 11 chapter on weighting methods). The basic idea underlying FEIDA is thus a maximal use of the available information (on attributes as well as preferences). The general idea of the method arises from the clustering of attributes into a hierarchy. The overall attributes (main categories of attributes) define the upper-level of the attribute hierarchy and these are then specified down the hierarchy to specific quantified measures (or indices) at the lover-level in the hierarchy. The aim of the FEIDA method is to make use of the possible detail in

26

information on low-level attributes as well as the ability of the decision maker (or experts) to express his preferences on attributes within certain attribute clusters. At the same time FEIDA recognises that often preferences between upper-level attributes can only be expressed by some notion of relative importance. Thus the SMART method is applied within each branch of the attribute hierarchy – from the bottom of the hierarchy and as high up the hierarchy as possible. At this point, the method transforms into the ELECTRE method, which makes use of the derived value functions from SMART as input (they function as attribute measures) and which requires only “importance weights” (or “direct weights” – see also Section 11 on weighting methods below) from the decision maker. The overall comparisons of alternatives are then based on the so-called outranking principle which states that alternative A is preferred to alternative B if there are enough arguments to decide that alternative A is at least as good as alternative B – while there are no essential reasons to refute that statement (Gissel, 1999). In FEIDA this outranking relation is a variant of ELECTRE I (see Gissel (1999)) which measures the degree to which alternative A outranks alternative B by both the concordance index and the discordance index.

10.2 Numerical example The Høng case again serves as the numerical example. The information contained in the CBA measure – obtained through the ability to weight CBA attributes by use of unit prices – is acknowledged and preserved. Hence, the B/C rate together with the three MCA impacts act as criteria in the ELECTRE method. For all pairs of alternatives A and B it is determined for which criteria A is better than (or at least as good as) alternative B. The weights associated with such criteria are then summed to give the concordance index cAB. With the same weight profile as before (70;15;7.5;7.5) the concordance index cp1p2 can be calculated as cp1p2 = 70+15+7.5=92.5 because p1 is as least as good as p2 on the B/C rate, Network accessibility and Urban planning. On the basis of such concordance indeces, the net concordance dominance value of each of the alternatives can be calculated. This is given (for alternative p1) as:

cp1 = Σi cp1i - Σi cip1 = -200

The net concordance dominance value measures the degree to which the total dominance of alternative p1 exceeds the degree to which it is itself dominated by other alternatives. That is, the higher this value, the higher chance of accepting alternative p1. Among the criteria for which alternative A is not as good as alternative B there might be some that will shed doubt upon whether A generally outranks B. The discordance index thus relates to the “opposite” aspect of the concordance index: it measures the degree to which alternative A is evaluated worse than B.

27

Mathematically, the discordance index is stated as:

DAB = maxi∈D wi |xAi – xBi| / max wi |xAi – xBi|

Where D is the set of criteria for which alternative A is not as good as B, and xij is the normalized impact value of alternative j with respect to criteria i. Thus, the discordance index indicates how much alternative A is evaluated worse than B in the “worst possible criteria” This means that, if there is a criterion which is strongly against the assertion that alternative A is better than alternative B, then the discordance index is relatively high. Hence, the index increases if the preference of B over A becomes very large for at least one criteria, and it rules out the possibility that a major disadvantage on one criterion could be compensated for by a large number of advantages on other criteria – as represented by the concordance index. On the basis of such discordance indexes, the net discordance dominance value of each of the alternatives can be calculated. This is given (for alternative A) as:

dA = Σi dAi - Σi diA

The lower this value, the higher chance of accepting alternative A. In the version of ELECTRE considered in the FEIDA method (see Gissel, 1999) the alternatives are ranked according to both the concordance index and the discordance index, and the average of these two rankings is then taken to represent an alternative’s overall ranking.

10.3 Criticism Pros: The very thought behind FEIDA is also its greatest asset, namely the efficient use of available information that lies in the combined use of ELECTRE and SMART. The use of the outranking ELECTRE method on higher levels in the attribute hierarchy implies that more general weights are used for overall criteria. Such weights may in some sense be recorded as (“constant”) general political values, which can be used for different evaluations at different points in time (see also Lootsma (1999)). Cons: The ELECTRE method that forms one part of the FEIDA method has a rather complex structure. Despite the immediate appeal of the outranking principle, the calculations (of the concordance and discordance indexes) in ELECTRE are rather tedious and troublesome. This may entail that the ELECTRE method – and thus the FEIDA method – will be perceived by decision makers as some sort of “black box method”.

10.4 Method perspective in CLG

28

The FEIDA method with its integration of SMART and ELECTRE links the American and the French MCA schools. It thus has an innovative character, which together with its fundamental principle of efficient use of information, makes it a very interesting method believed by the authors to be worthwhile considering in more depth.

11 Weighting methods The AHP method, SMART, FEIDA and other MCA methods require weights. This task should be approached very carefully as the weight profile can make a large difference in the resulting ranking of alternatives. Hobbs & Meier (2000) provide a very thorough overview of the different types of weighting methods and the following will be a brief extract of this description.

11.1 Equal weights The simplest approach is to apply equal weights across criteria. However, the user should be aware that this does not mean that they are equally important. It depends on the scales of the criteria: If a change from worst stimulus to best stimulus represents only a trivial change in criterion A but a significant change in criterion B, then, implicitly, a higher importance is assigned to A. This is because a large improvement in B is required to compensate for a small worsening in A.

11.2 User derived weights This method uses regression analysis on past decisions in order to determine which weights best predict these (unaided) evaluations. However, as noted by Hobbs & Meier (2000), this method simulates judgements where the purpose of MCA decision aid should be to improve judgements.

11.3 Direct weighting As the name indicates such methods ask decision makers to specify numerical values for weights directly. Examples of direct weighting methods include point allocation (allocate 100 points among attributes in proportion to their “importance”), categorization (assign attributes to different categories of importance, each carrying a different weight), ranking (the least important attribute receives a weight of 1, the next lowest a weight of 2, etc.), ratio questioning (which asks for ratios of the importance of attributes, two at a time), and rating (each attributes importance is rated on a scale of, say, 0 to 10). However, direct weighting methods often fail to yield weights that correspond to tradeoffs people are willing to make: the reason is that subjective notions of “importance” may have little to do with the rate at which a person is willing to trade off the scales of two attributes.

29

The direct weighting methods may be applied in a hierarchical way if there are many criteria that can be grouped into general categories. The hierarchical approach may be applied either top-down or bottom-up.

11.4 The Analytic Hierarchy Process A popular version of ratio questioning is the Analytic Hierarchy Process, AHP. This method was described in an earlier chapter. Briefly, the AHP asks decision makers to compare every possible pair of criteria, stating the ratio of their importance. The criteria weights are then calculated as the weights that are “most consistent” with these ratios. AHP suffers the same problem as rating and most other direct weighting methods; with no explicit reference to the ranges of the attributes, assessors may apply some notion of importance other than their willingness to trade off attributes. To combat this, AHP may be combined with swing weighting. This method, as will be explained next, attempts to ensure that users have ranges of attributes in mind when they choose weights.

11.5 Swing weights A final variant of direct weighting is swing weighting. The user is asked to consider a hypothetical alternative where all attributes are at their worst value. The user is then asked: “which of the attributes would you prefer to swing from its worst value up to its best value?” Then the user is asked which attribute he would swing up second, and so on. After ranking the attributes in this manner, the user is asked to give the most important attribute a weight of 100, and then assign weights to the other attributes in proportion to the importance of their ranges, while being sure that the most highly ranked attributes receive the highest weight.

11.6 Indifference tradeoff weights In this method decision makers are asked to make tradeoffs between the attributes where after the implied weights may be derived. This makes the tradeoff method an implicit weighting technique. As Hobbs & Meier (2000) note, this method is preferred by many decision analysts because the derived weights will not be contaminated by incorrect notions of attribute importance. However, the method is difficult compared to direct weighting. Hobbs & Meier (2000) mention that while physicists and economists like the method, others struggle with it. People are used to rating items on a scale from 1 to 10, but stating how much of one valued item they are willing to sacrifice for a given amount of another is very difficult unless they have very definite preferences or are used to making such choices.

11.7 Gamble method

30

In the approach the decision maker is asked to compare a “sure thing” and a “gamble”. He then adjusts the probability in the gamble until he finds the two options to be equally attractive (or equally unattractive). From such comparisons, one weight at a time may be derived.

11.8 Concluding remarks No matter which method is used the resulting weights should have some desired properties. Firstly, if attribute A is twice as “important” as attribute B, then wA should be twice as large as wB. Secondly, the weights should represent the rate at which the user is willing to trade off one criterion for another. The recommendations made by Hobbs & Meier (2000) is that whenever the decision makers feel comfortable in answering tradeoff questions, and just one method is to be used, the indifference tradeoff weights are to be preferred. Because of the difficulty of this method, the consistency of the tradeoffs should be checked by asking more than the minimum number of questions needed to determine weights. This also aids the learning process by encouraging decision makers to reflect on their values and the problem. On the other hand, as Hobbs & Meier (2000) note, practical applications have shown that many decision makers are very uncomfortable with tradeoff questions. As a result the answers may not be meaningful and the decision makers have little confidence in the results. In such cases, direct weighting methods should be applied instead. Hobbs & Meier (2000) recommend that some precautions be taken if direct methods are used. First, the decision makers should be informed about the desired properties of the weights. Second, the decision makers should firmly keep in mind the ranges of the attributes. Third, some tradeoff questions should be asked as checks upon the weights. Furthermore, they recommend that more than one weighting method be used, if time allows. Hobbs & Meier focus on the ability of the decision maker to state criteria weights. However, the multicriteria method, which is used, also has an influence on the way weights should be derived. In Choo et al. (1999) the different possible interpretations of criteria weights in various multicriteria methods are analysed. With respect to AHP and SMART, the paper concludes that the weights in these two methods may be interpreted in much the same way – as marginal contribution, trade-offs, criteria importance, and other. In ELECTRE however, these interpretations do not apply – here the only interpretation of the weights is as measures of criteria importance. These differences in interpretation underline the importance of carefully considering the demands on weights before one starts to elicit them.

31

12 Comparison and choice of method The different decision support methods considered here all have both advantages and drawbacks. These have been discussed in the preceding chapters and are summarised in Table 13. No method is without drawbacks of some sort. However, it is believed by the authors that the current methods represent decision aid tools where the advantages are so convincing that the methods cannot be left out of consideration. At the same time the methods represent a broad spectrum of MCA methods. Method Pros Cons

VD method (CBA in general)

• Well-established CBA method

• The general merits of CBA (consistent values over time and between investments, detailed data gathering, one absolute measure of social profitability)

• Lack of unit prices for all impacts

• Hypothetical socio-economic optimum (no actual redistribution of money)

AHP • Pairwise comparisons are perceived by users as straight-forward and convenient

• Very useful when criteria are of a qualitative nature

• Inconsistencies in relative judgments can be handled

• Weak theoretical foundation (the link between the verbal descriptions and the corre-sponding numerical scale)

• The rank reversal problem

SMART • A structure similar to CBA makes it appealing to decision makers

• Method is independent of alternatives (which means that there cannot be rank reversals)

• Not very useful as a screening method

• High demands on detail in information (both on weights and criteria/attributes)

COSIMA • Straightforward in its design: Retains the CBA structure by including MCA elements in a traditional CBA

• - this also makes it more appealing to decision makers

• Lack of interpretation of the unit prices of the MCA elements

FEIDA • Efficient use of available information

• The possibility of using “constant” political values on overall (aggregate) criteria

• Complexity of the ELECTRE method may result in the FEIDA method being perceived by decision makers as a “black box” method

Table 13: Pros and cons of the different decision aid methods to be considered in ASTRA

32

The application of the different methods on the Høng case resulted in the rankings of the seven by-pass alternatives as summarised in Table 14. It should be remembered that the weighting step in the MCA methods is always subjective in nature, and that the rankings in the numerical applications are merely examples on how the decision maker’s preferences might express themselves. However, it was aimed at using the same weight profile in each application. Thus, it is interesting that the four MCA methods (AHP, SMART, COSIMA and FEIDA) rank the alternatives rather similar, although the COSIMA ranking deviates a little from the three other rankings. The total picture suggests that the four Eastern alternatives (p4, p5, p6 and p7) are preferred to the Western alternatives. Furthermore, alternative p4 and p5 present themselves as the most attractive by-pass solutions.

Ranking Alternative

VD AHP SMART COSIMA FEIDA

P1 5 5 5 6 4

P2 6 6 6 4 5

P3 7 7 7 7 7

P4 1 1 1 2 1

P5 2 2 2 1 1

P6 4 4 4 5 5

P7 3 3 3 3 3

Table 14: Ranking of the seven alternatives in the Høng case as given by the different methods

This overview has mainly served to introduce some of the characteristics of the examined candidate methods. On the basis of this review and based on the decision to apply the Øresund Fixed Link as a method case in Task 9 the study team decided to use the COSIMA approach in the further modelling work.

33

References Beder, S. (2000) Costing the Earth: Equity, sustainable development and environ-mental economics, New Zealand Journal of Environmental Law, 4, 2000, pp. 227-243. Dodgson, J., M. Spackman, A. Pearman and L. Phillips (2001) Multi Criteria Analysis:

A Manual, UK Department for Environment, Food & Rural Affairs. ECMT (2001) Assessing the benefits of transport, European Conference of Ministers of Transport, OECD Gissel, S. (1999) Decision aid methods in rail infrastructure planning, Ph.D. thesis, Report 4, Technical University of Denmark, Department of Planning. Grant-Muller, S. M., P. Mackie, J. Nellthorp and A. Pearman (2001) Economic appraisal of European transport projects: the state-of-the-art revisited, Transport

Reviews, 21, 237-261. Hobbs, B. F. and P. Meier (2000) Energy decisions and the environment – a guide to

the use of multicriteria methods, Kluwer Academic Publishers Kopp, R. J., Krupnick, A. J. and Toman, M. (1997) Cost-benefit analysis and regulatory

reform: An assessment of the science and the art, Resources for the Future, Discussion Paper 97-19 Leleur, S. (2000) Road Infrastructure Planning – A Decision-Oriented Approach, 2nd edition, Polyteknisk Forlag, Lyngby, Denmark. Leleur, S. (2001) Transport infrastructure planning: Modelling of socio-economic feasibility risks, Proceedings of EUROSIM 2001. Lootsma, F. A. (1999) Multi-criteria decision analysis via ratio and difference

judgement, Kluwer Academic Publishers Steffensen, M. & Testmann, L. (2000). Vurdering af Omfartsveje – Belyst gennem alternative projekter for Høng, Eksamensprojekt, CTT-DTU. Vejdirektoratet (1992) Undersøgelse af større hovedlandevejsarbejder. Metode for

effektberegninger og økonomisk vurdering, Vejdirektoratet, Økonomisk-Statistisk Afdeling