Upload
phungtruc
View
217
Download
0
Embed Size (px)
Citation preview
Mat-2.108 – Independent Research Project in Applied Mathematics
Hierarchical Preference Elicitation inRobust Portfolio Modeling
Antti Toppila 60719S
February 2, 2007
Helsinki University of TechnologyDepartment of Engineering Physics and MathematicsSystems Analysis Laboratory
1 INTRODUCTION
Contents
1 Introduction 1
2 Robust Portfolio Modeling 32.1 Multi-Criteria Project Portfolio selection . . . . . . . . . . . . 32.2 Modeling Incomplete Information . . . . . . . . . . . . . . . . 5
3 Value Trees 73.1 Structure of Value Trees . . . . . . . . . . . . . . . . . . . . . 73.2 Mathematical Treatment . . . . . . . . . . . . . . . . . . . . . 8
4 Software Implementation 12
5 Illustrative Example 13
6 Conclusions 18
1 Introduction
The selection of a project portfolio is a combinatorial problem where the
decision-maker (DM) has to choose the best projects from a set of candidate
projects. The projects have different strengths and weaknesses, and therefore
each project is evaluated with regard to two or more criteria (or attributes).
Possible portfolios are restricted by constraints such as a budget limitation,
for instance. Ideally, the selected portfolio performs in some sense best with
regard to all criteria. The final decision depends on the DM’s preferences.
As with all combinatorial problems, the solution space may become very
large, which results in difficulties if all possible portfolios are to be evaluated
separately. Another problem is that all necessary information does not exist
or cannot be produced in time or at a reasonable cost. This means that
it may be helpful to work with incomplete information. In addition, many
interdependencies among the projects can make it hard to limit the decision-
making process to feasible portfolios only. Yet, the DM should be able to
make a decision which can be subjected to critical inspection.
1
1 INTRODUCTION
There are numerous approaches to this problem. Golabi et al. (1981) de-
veloped an interactive procedure to assist in the selection of an R&D portfolio
of solar photovoltaic applications experiments posing some of the criteria as
minimum level constraints. This method is suitable when nonmonetary crite-
ria are accounted for. For instance, a strategic approach to allocating capital
in healthcare systems (Kleinmuntz et al., 1999) would make such considera-
tions necessary when dealing with soft values such as patient satisfaction or
quality of care.
Robust Portfolio Modeling (RPM, Liesio et al., 2006) supports project
portfolio selection in view of multiple criteria, incomplete information and
limited resources. It is developed for large scale problems with support for
incomplete information. This is necessary, as the decision-maker may not be
able to give complete preference information due to the very large number
of projects.
The method is based on finding non-dominated project portfolios: a port-
folio is non-dominated if no other portfolio is better for all possible realiza-
tions of feasible parameters. Projects that are included in all non-dominated
portfolios should always be selected (core projects). Conversely, projects
that are not included in any non-dominated portfolio should not be selected
(exterior projects). This allows the DM to focus on the remaining borderline
projects and to find new core and exterior projects by giving additional infor-
mation about the borderline projects, since new information can influence the
status of borderline projects only. This saves resources and enables quicker
results, as candidate projects can be completely eliminated each round.
Value trees (Keeney and Raiffa, 1976) are a common method for problem
structuring in multiple criteria decision-making by decomposing the problem
into smaller parts which can be evaluated separately. Also, in these methods
the DM must structure her objectives as a hierarchy of attributes (Belton
and Stewart, 2001). The effect which the parts have on the final decision can
be modeled with different types of weighting methods. In this context the
weighting process can be called elicitation of preference information.
2
2 ROBUST PORTFOLIO MODELING
Value trees, however, are not supported by RPM. In this paper we will
extend the RPM-framework to be compatible with value trees and implement
the RICH hierarchical preference elicitation method (Salo and Punkka, 2005)
as a part of the RPM-Decisions software. The graphical elicitation tool
created provides a novel and illustrative drag-and-drop interface for eliciting
incomplete ordinal preference information. It also performs the necessary
computations required to convert the information to a RPM model.
The reminder of this study is structured as follows: Section 2 presents
the RPM-method for the modelling of incomplete information in hierarchi-
cal value trees. Section 3 introduces value trees and gives a mathematical
treatment of thei structure. Section 4 describes a software for giving ordinal
preference information and section 5 demonstrates the preference elicitation
process through an example. Section 6 summarizes the main results in this
paper.
2 Robust Portfolio Modeling
2.1 Multi-Criteria Project Portfolio selection
Assume that m projects X = {x1, . . . , xm} are evaluated with regard to n
criteria. Let the consequence of choosing project xj with regard to the ith
criteria be xji . We define that x∗i and x0
i is the best and the worst possible
consequence for the ith criterion, respectively. Under certain conditions, there
exists an additive value function (see e.g., Keeney and Raiffa, 1976)
V (xj) =n∑
i=1
Vi(xji ) , (1)
where Vi is a cardinal value function of criterion i. The value function Vi is
unique up to positive affine transformations V ′i (.) = αVi(.)+β, α > 0, β ∈ R
3
2 ROBUST PORTFOLIO MODELING
so we can set Vi(x0i ) = 0 without loss of generality. From (1) we get
V (xj) =n∑
i=1
[Vi(x
ji )− Vi(x
0i )
]=
n∑i=1
Vi(x∗i )− Vi(x
0i )
Vi(x∗i )− Vi(x0i )
[Vi(x
ji )− Vi(x
0i )
]=
n∑i=1
[Vi(x
∗i )− Vi(x
0i )
]︸ ︷︷ ︸=:wi
Vi(xji )− Vi(x
0i )
Vi(x∗i )− Vi(x0i )︸ ︷︷ ︸
=:vji
⇐⇒ V (xj) =n∑
i=1
wivji . (2)
The numbers vji are called performance scores of project j with regard
to criterion i; they form the score vector vj = [vj1, . . . , v
jn], j = 1, . . . m. The
weights w = [w1, . . . , wn]T are scaled so that
w ∈ S0w = {w ∈ Rn|wi ≥ 0,
n∑i=1
wi = 1} .
The interpretation is that when the score changes from the worst possible
value to the best possible value, the impact on the overall value V (xj) is
described by the weights wi. If the score vector is not scaled as above, then
this interpretation cannot be done. Nevertheless, the RPM-method can be
applied, using equation (2) where the weights and scores are given directly
with some other interpretation. However, in this paper it is assumed that
the weights have the above interpretation, as otherwise the weight elicitation
in a hierarchical value tree would be unintuitive.
A project portfolio p ⊆ X is a subset of all projects and the set of all
possible portfolios is P := 2X , the power set of X. The overall value of a
portfolio is given by
V (p, w, v) :=∑xj∈p
n∑i=1
wivji ,
4
2 ROBUST PORTFOLIO MODELING
which is the sum of the constituent projects’ overall values (for details, see
Golabi, 1987). A portfolio p1 is preferred to portfolio p2 if and only if its
overall value is greater, i.e. V (p1, w, v) ≥ V (p2, w, v).
Choices are often limited as it may be infeasible to select certain portfo-
lios. For instance, a budget or logical constraint could prevent the selection
of a particular set of projects into a portfolio. We limit us to constraints that
can be described as a set of linear equations and define the set of feasible
portfolios as
PF := {p ∈ P |Az(p) ≤ B} , (3)
where the inequality holds component-wise, z : P → {0, 1}m is a bijection
such that zj(p) = 1 if xj ∈ p and zj(p) = 0 if xj /∈ p, A ∈ Rq×m and
B = [b1, . . . , bq]T ∈ Rq.
The problem has now been reduced to that of finding the feasible port-
folio with the highest overall value: This corresponds to the integer linear
programming problemmaxz(p)
z(p)T vw
s.t. Az(p) ≤ B
z(p) ∈ {0, 1}m
(4)
2.2 Modeling Incomplete Information
Use of incomplete information may be warranted, as the DM may not be
able to give precise estimates. For instance, the task of predicting the future
sales of a firm in a new country could be overwhelming. In addition, if the
DM is forced to give more accurate information than she is comfortable with,
she may distrust the results of the analysis and not be committed to them.
Also, for considerations of sensitivity analysis, useful qualitative results can
be obtained by evaluating the sensitivity of the result to parameter changes
(Rios Insua and French, 1991). Salo and Hamalainen (2004) suggest the use
of the term ”Preference Programming” in this context. The term means
modeling techniques where i) incomplete information is modeled by sets of
5
2 ROBUST PORTFOLIO MODELING
feasible parameters instead of point estimates and ii) dominance structures
and decision rules are used to make decision recommendations.
Elicitation of incomplete information about criterion weights can be clas-
sified into five categories (Kim and Ahn, 1999):
(i) weak order (wi ≥ wj)
(ii) strong order (wi − wj ≥ αij)
(iii) ranking with multiples (wi ≥ αijwj)
(iv) interval form (αi ≤ wi ≤ αi + εi)
(v) ranking of differences (wi − wj ≥ wk − wl when j 6= k 6= l)
where α., ε. ≥ 0∀i. We will use form (i) used in the RICH-methodology.
In RPM, incomplete information is modeled through the analysis based
on the set of feasible parameters Sw consistent with the DM’s preference
statements. Preference information on criterion weights is given by demand-
ing that w ∈ Sw ⊆ S0w and it is required that Sw is a polyhedron. A set Sw
is a polyhedron if it can be expressed with a set of linear inequalities, i.e.
Sw = {w ∈ S0w|Aww ≤ Bw} , (5)
where Aw ∈ Rm×n and Bw ∈ Rm.
In the RPM method it is possible to give incomplete score information by
requiring that v ∈ Sv = {v ∈ Rm×n|v ≤ v ≤ v}, where the upper and lower
bounds for each project and criteria are assessed by the DM (other types of
sources may also be used, for instance measurement data).
In general, problem (4) does not have a unique solution with incomplete
information, therefore an asymmetric, irreflexive and transitive dominance
relation �S in used to compare portfolios.
We define that for any p, p′ ∈ P portfolio p dominates p′ with regard to
the information set S := Sw × Sv, denoted by p �S p′, if and only if
6
3 VALUE TREES
(i) V (p, w, v) ≥ V (p′, w, v) for all (w, v) ∈ S and
(ii) V (p, w, v) > V (p′, w, v) for some (w, v) ∈ S.
In other words, a project portfolio dominates another if and only if its overall
value is at least as good with all feasible parameter values and strictly better
with some feasible parameters.
This property, however, is not needed in this paper, and we refer to Liesio
et al. (2005). The main idea is to find the set of non-dominated portfolios
PN(S) := {p ∈ PF |p′ �S p ∀p′ ∈ PF} (6)
with regard to the information set S. Only the portfolios in the set of non-
dominated portfolios needs further consideration, because no other portfolio
has higher overall value for all parameter values. It may be, that a non-
dominated portfolio is never the portfolio with the highest overall value for
any feasible parameter value, but it may provide a robust solution in the
sense, that it yields a good overall value for most feasible parameter values.
3 Value Trees
3.1 Structure of Value Trees
In practice it may be difficult to elicit scores and weights. Score vectors usu-
ally have some quantifiable measures, but weight vectors are more difficult.
A concrete and illustrative weight elicitation process is the use of a hierar-
chical value tree. For instance, the overall value of a meal could be divided
into taste and health. Health could be divided even further to vitamins and
cholesterol, etc.. Value trees help the decision-maker (DM) to structure her
decision in terms of entities, that can be elicited and/or weighted indepen-
dently of each other. A basic hierarchical value tree with attributes as nodes
is presented in Figure 1. In hierarchical weight elicitation a∗ is often called
7
3 VALUE TREES
the overall goal, which is evaluated through its components related to it ac-
cording to the figure. The attributes lowest in the hierarchy (a2, a3 and a4)
are called twig level attributes.
The value tree in Figure 1 could have been the result of structuring the
portfolio choosing problem of opening new stores in new cities. Then the
attributes could be such that the overall goal is to select the best possible
sites to open stores, a1 the market, a2 the price, a3 the competition and a4
the customers. Obviously the decision could be structured even further by
dividing price into fixed costs and risk or taking stand to the effect on the
whole company like public relations (PR) value, for example.
The hierarchy explicitly accounts the factors influencing the decision,
when deciding the value of a certain property. It should be noted, how-
ever, that the additive value function describes the DM’s preferences if and
only if the attributes are preferentially independent (see Keeney and Raiffa,
1976). Also a value tree should be
(i) Minimal (it contains only relevant attributes)
(ii) Operational (attributes are illustrative and meaningful, i.e. there are
differences between the alternatives in these criteria)
(iii) Complete (all necessary attributes are included)
(iv) Decomposable (attributes can be evaluated individually)
(v) Non-redundant (all attributes appear only once) .
3.2 Mathematical Treatment
This section presents a mathematical framework of value trees suitable for
developing a procedure for obtaining the set Sw. Consider an undirected graph
G = (A, E) consisting of a set of attributes A = {a0, a1, . . . , ap} acting as
nodes and a set of arcs E connecting the attributes (Bertsimas and Tsitsiklis,
8
3 VALUE TREES
GFED@ABCa∗
������
����
����
����
���
AAA
AAAA
AA
GFED@ABCa1
~~||||
||||
|
BBB
BBBB
BB
GFED@ABCa2 GFED@ABCa3 GFED@ABCa4
Figure 1: An example of a simple value tree. Using the notation in sec-tion 3.2 we have S(a∗) = {∅}, S(a1) = S(a2) = {a∗}, S(a3) = S(a4) = {a1},S−1(a∗) = {a1, a2}, S−1(a1) = {a3, a4}, S−1(a2) = S−1(a3) = S−1(a4) = {∅}.
1997). The arcs are unordered pairs {ai, aj} of distinct attributes. Therefore
circular arcs like {ai, ai} are excluded and {ai, aj} = {aj, ai}.
A path from the attribute ai1 to attribute ait in an undirected graph is
defined as a finite sequence ai1 , ai2 , . . . , ait such that {aik , aik+1} ∈ E , k =
1, . . . , t − 1 and all attributes in the sequence are distinct. A cycle is a
sequence ai1 , . . . , ait , ait+1 where ai1 , . . . , ait is a path, {ait , ait+1} ∈ E , ai1 =
ait+1 and t− 1 ≥ 3. An undirected graph is connected, if there exists a path
between every two distinct nodes in the graph.
Definition 1 An undirected graph is a tree if it is connected and contains
no cycles.
A directed graph G = (A, E) where A is set of attributes and E is a set
of ordered pairs (i, j) of distinct attributes. Again circular arcs (ai, ai) are
forbidden, but (ai, aj) and (aj, ai) are different elements in E . For an arc
(ai, aj) we say that there is an arc from ai to aj.
Definition 2 A directed graph G defines a tree if the undirected graph that
is formed when ignoring the direction of the arcs is a tree.
We define a function S : A → 2A, where 2A is the power set of A, such
that
S(ai) = {aj ∈ A|∃(aj, ai) ∈ E} . (7)
9
3 VALUE TREES
The set S(ai) includes the attributes having an arc from them to the node
ai. The inverse function S−1 : A → 2A is defined as
S−1(ai) = {aj ∈ A|ai ∈ S(aj)} . (8)
The set S−1(ai) includes all attributes which ai has an arc going to. The
functions S and S−1 defines the overall goal and twig level attributes.
Definition 3 An attribute ai is an overall goal if S(ai) = {∅}.
Definition 4 An attribute ai is a twig level attribute if S−1(ai) = {∅}.
Definition 5 A directed graph G = (A, E) is a hierarchical value tree if
(i) G defines a tree,
(ii) G has exactly one overall goal a∗ and
(iii) |S(ai)| = 1 ∀ai ∈ A, where the number of elements in a set X is denoted
by |X| and |{∅}| = 1.
Assume that G is a hierarchical value tree with the attributes A = {a∗}∪{ai|i = 1, . . . p}, a∗ is the overall goal and the set At := {ai ∈ A|S−1(ai) =
∅} contains all the twig level attributes. The twig level attribute function
L : A → 2At is defined recursively as
L(ai) =
{ai} ai ∈ At⋃
aj∈S−1(ai)
L(aj), otherwise (9)
In other words, the set L(ai) is the set of twig level attributes that are below
ai in the hierarchy. It is also clear by construction, that L(a∗) = At.
Each attribute ai is assigned a weight wi ∈ [0, 1] such that for twig level
attributes ∑ai∈At
wi = 1, wi ≥ 0
10
3 VALUE TREES
and for all other attributes
wi =∑
aj∈L(ai)
wj . (10)
Thus, the weights of twig level attributes {wi|ai ∈ At} are sufficient to define
all the weights.
The DM’s preferences are modeled through linear constraints on the
weights. For instance, assume that attribute a1 is more important than
a2, implying that a change from the worst to the best performance level in
a1 has more influence on the overall value than the corresponding change in
a2. Then the weight w1 must be larger than w2. Any such statement on
attributes A′ ∈ A can be expressed in the form∑ai∈A′
ciwi ≤ b, A′ ⊆ A . (11)
These restrictions can be presented through weighted sums of twig level at-
tribute weights. By substituting (10) to (11) we get∑ai∈A′
ci
∑aj∈L(ai)
wj ≤ b ⇐⇒∑
ak∈At
ckwk ≤ b
ck =∑j∈Jk
cj, Jk = {j ∈ N|aj ∈ A′ ∧ ak ∈ L(aj)} .(12)
Thus any preference statement at any level of the value tree can be expressed
with constraints on the twig level attributes. These types of restrictions are
supported by RPM, so we have established a new method for giving prefer-
ence information compatible with RPM. The restrictions can be presented
in matrix form Aww ≤ bw, giving us the information set
Sw = {w ∈ S0w|Aww ≤ bw} ,
which is compatible with (5). In fact, we have reduced the tree structure
to a one level tree which means taking a weighted average of the twig level
attribute scores. If we want to ensure that hierarchy is respected, i.e. only
11
4 SOFTWARE IMPLEMENTATION
attributes with the same attribute on the next higher level of the hierarchy
are compared, we demand that A′ ⊆ S−1(ai) for some ai ∈ A.
Let us continue our earlier example involving Figure 1 by giving ordinal
preference information about the attributes. Assume that when deciding the
overall goal, price (a2) is more important than the market (a1) and when
evaluating the market, customers (a4) are more important than competition
(a3). To have compatible information, the associated weights wi would have
to fulfil the constraintsw2 ≥ w1
w3 ≥ w4
.
Expressed with twig level attribute weights using (12) this would correspond
tow2 ≥ w3 + w4
w3 ≥ w4
.
4 Software Implementation
The value tree framework presented in this study is implemented in a decision
support software RPM-Decisions. The software is a Javatm-program with a
graphical user interface (GUI). Incomplete information on weights can be
given by defining the weight constraints (i.e. the elements of the matrix
Aw and the vector bw) using spreadsheets. However, it is quite hard to
produce the elements resulting from transforming statements in a value tree
without computational support. Therefore a preference elicitation tool was
implemented, which automatically generates these constraints as the user
defines her preferences.
The value tree tool includes an interface for building the tree, illustrated
in Figure 2. Each attribute in the tree, except the twig level attributes, can
be assigned preference information about the attributes one level lower in the
hierarchy. By double-clicking on an attribute statements of the form (11) can
be given.
12
5 ILLUSTRATIVE EXAMPLE
Figure 2: Screen shot of the value tree at start and after structuring.
Statements of the form (11) also need some context to be understandable.
Therefore a tool for giving incomplete ordinal preference information was
implemented (see e.g., Salo and Punkka, 2002). In this tool attributes are
divided into importance classes with a drag and drop interface explained in
Figure 3. Importance classes can easily be transformed to matrix form.
5 Illustrative Example
Let us study a case where a business is expanding to several new cities after
developing a concept in one city. The problem is to choose the best set of
cities {x1, . . . , xm} where new offices are started. In other words, the DM
has to decide not only in which cities to start new offices, but also how many
offices to start. This type of problem typically has budget and capacity
constraints, for instance.
The DM has chosen six twig level criteria, which are in Table 1, divided
into three groups: ‘Market’, ‘Customers’ and ‘Cost’. These criteria form
a hierarchical value tree presented in Figure 2. The DM gives preference
information about the criteria using the drag and drop interface. All three
13
5 ILLUSTRATIVE EXAMPLE
a1
_ _ _ _ _ _ _ _ _ _ _ _����������������
����������������_ _ _ _ _ _ _ _ _ _ _ _
1b76540123``
a1
765401231a
22
a2 a3
a4
a5
a1
_ _ _ _ _ _ _ _ _ _ _ _����������������
����������������_ _ _ _ _ _ _ _ _ _ _ _
a2 a3
/.-,()*+2rra4 a3
a5
a1
_ _ _ _ _ _ _ _ _ _ _ _����������������
����������������_ _ _ _ _ _ _ _ _ _ _ _
a2
a5 a4 a3
a5
765401233a
22
3b76540123~~
Figure 3: An example presenting a drag and drop interface for giving ordi-nal preference information. The attributes ai, i = 1, . . . , 4 are divided intoimportance classes. In 1a the attribute a1 is reassigned to an another impor-tance class that does not yet exist, thus a new importance class is generated(1b). In 2 attribute a3 is moved to an existing importance class. The originclass remains as attribute a2 is still in it. In 3a attribute a5 is moved toanother importance class. The importance class (3b) will vanish as there areno attributes left in that class.
Table 1: Twig level criteria in example casei ai Group1 Competitors Market (a7)2 Market share Market3 Customer wealth Customers(a8)4 City population Customers5 Fixed costs Cost(a9)6 Risk Cost
groups have two criteria, meaning that only statements where one criterion
is more important than the other one can be given.
When deciding the importance of the criteria, the DM has to consider
trade-offs. For instance, cost is a strong candidate for the most important
criterion, but if the difference between the alternatives are minor with re-
gard to price, other criteria might become more important. Assume that the
DM has considered the trade-off property and decided that ‘Market share’
is more important than ‘Competitors’, ‘City population’ is more important
than ‘Customer wealth’ and ‘Fixed costs’ is more important than ‘Risk’. Fur-
14
5 ILLUSTRATIVE EXAMPLE
thermore the DM assesses ‘Market’ as the most important criterion under
the overall goal but cannot decide which one of the remaining criteria ‘Cus-
tomers’ and ‘Cost’ is more important. This information corresponds to the
linear constraintsw2 ≥ w1
w4 ≥ w3
w5 ≥ w6
w7 ≥ w8
w7 ≥ w9 .
Using equation (10), we get additionally the linear equality constraints
w7 = w1 + w2
w8 = w3 + w4
w9 = w5 + w6 .
In terms of twig level criteria weights, these constraints are equal to
Aww =
1 −1 0 0 0 00 0 1 −1 0 00 0 0 0 −1 1−1 −1 1 1 0 0−1 −1 0 0 1 1
w1
w2
w3
w4
w5
w6
≤
000000
= bw .
For comparison, assume that the DM would not want to structure her
decision into a multilevel tree, but would like to compare the twig level
attributes directly (see Figure 4). The information she gives would be such
that ‘Market share’ is the most important, ‘Competitors’ second most impor-
tant, ‘City population’ and ‘Fixed costs’ third most important and ‘Customer
wealth’ and ‘Risk’ the least important (see Figure 5). This would correspond
15
5 ILLUSTRATIVE EXAMPLE
to the set of constraints
Aww =
−1 1 0 0 0 00 −1 0 1 0 00 −1 0 0 1 00 0 1 −1 0 00 0 0 −1 0 10 0 1 0 −1 00 0 0 0 −1 1
w1
w2
w3
w4
w5
w6
≤
0000000
= bw .
16
5 ILLUSTRATIVE EXAMPLE
Figure 4: Screen shot of a one level value tree.
Figure 5: Screen shot of the drag and drop interface for giving ordinal pref-erence information.
17
6 CONCLUSIONS
6 Conclusions
In this paper, we have extended the RPM framework to allow the use of hier-
archical value trees in weight elicitation. We have developed a mathematical
definition of value trees and a mathematical formulation suitable for software
implementation. In addition we have discussed the use of ordinal preference
information for modeling the decision-maker’s preferences in different levels
of the value tree and illustrated the use of this method by an example.
The use of software as decision-making tool is very important: it illus-
trates the situation to the DM giving her a transparent process to follow
without the need of performing computations. A clear software structure
keeps the user on track of all the implied assumptions.
Value trees are a natural way of organizing decisions. Therefore it is
important for RPM to have support for value trees including the appropriate
software. In this paper it was assumed that the value functions were scaled
from one to zero. Future research could focus on providing support for value
trees when value functions are not scaled, which is the general case supported
by RPM.
18
REFERENCES
References
[1] Belton, V. and Stewart, T.J. (2001). Multiple criteria decision analysis:
An integrated approach. Kluwer Academic Publishers, Boston.
[2] Bertsimas, D. and Tsitsiklis J. (1997). Introduction to Linear Optimiza-
tion. Athena Scientific, Belmont.
[3] Golabi, K. (1987). Selecting a Group of Dissimilar Projects for Funding.
IEEE Transactions on Engineering Management, vol. 34, pp. 138-145.
[4] Golabi, K., Kirkwood, W.C. and Sicherman, A. (1981). Selecting a port-
folio of solar energy projects using multiattribute preference theory Man-
agement Science, vol. 27, pp. 174-189.
[5] Keeney, R.L. and Raiffa, H. (1976). Decisions with multiple objectives:
Preferences and value tradeoffs. John Wiley & Sons, New York.
[6] Kim, S.H. and Ahn, B.S. (1999) Interactive group decision making pro-
cedure under incomplete information. European Journal of Operational
Research, vol. 116, pp. 498-507.
[7] Kleinmuntz, C.E., Kleinmuntz, D.N., (1999). Strategic Approaches for
Allocating Capital in Healthcare Organizations, Healthcare Financial
Management, Vol. 53, pp. 52-58.
[8] Liesio, J., Mild, P., Salo, A. (2004) Preference programming for robust
portfolio modeling and project selection. European Journal of Opera-
tional Research (to appear)
[9] Liesio, J., Mild, P., Salo, A. (2006). Robust Portfolio Modeling with
Project Interdependencies and Balance Constraints, Helsinki University
of Technology. European Journal of Operational Research (to appear).
[10] Punkka, A. (2002). Use of RICH in Multi-level Value Trees. Independent
Research Project in Applied Mathematics. Systems Analysis Laboratory,
19
REFERENCES
Helsinki University of Technology. Available at
http://www.sal.tkk.fi/Opinnot/Mat-2.108/pdf-files/epun02b.pdf.
[11] Rios Insua, D. and French, S. (1991). A framework for sensitivity anal-
ysis in discrete multi-objective decision making. European Journal of
Operational Research, vol. 54, pp. 176-190.
[12] Saaty, T.L. (1980). The Analytical Hierarchy Process. McGraw-Hill,
New York.
[13] Salo, A. and Hamalainen, R.P. (2004). Preference programming. Sys-
tems Analysis Laboratory, Helsinki University of Technology. Available
at http://www.sal.hut.fi/Publications/pdf-files/msal03b.pdf.
[14] Salo, A. and Punkka, A. (2005) Rank Inclusion in Criteria Hierarchies,
European Journal of Operational Research, Vol. 163, pp. 338-356.
20