Contracting cleaning services in a European public underground transportation company with the aid of a DSS

43 (2007) 1485–1498www.elsevier.com/locate/dss

Decision Support Systems

Contracting cleaning services in a European public undergroundtransportation company with the aid of a DSS

Antonio Jiménez ⁎, Alfonso Mateos, Sixto Ríos-Insua, Luis Carlos Rodríguez

Department of Artificial Intelligence, School of Computer Science, Technical University of Madrid,Campus de Montegancedo s/n, 28660 Boadilla del Monte, Madrid, Spain

Available online 10 August 2006

Abstract

This paper deals with the selection of a supplier for cleaning services in a European public underground transportation companyas established in the European Community directives, where several conflicting criteria, such as improving service levels andreducing total service costs, must be taken into account simultaneously. The problem is analyzed in depth using the decisionanalysis methodology, and a decision support system, the Generic Multi-Attribute Analysis system, is used to allay the operationaldifficulties involved. This system can deal with incomplete information about decision-maker preferences, accounts for uncertaintyabout offer performance, and uses so-called decision-making with partial information to identify the best offer, taking advantage ofimprecise inputs.© 2006 Elsevier B.V. All rights reserved.

Keywords: Cleaning services; Decision analysis methodology; Decision support system; Incomplete information

1. Introduction

Cleaning is a part of a major service provided to un-derground transportation users.When donewell, it is oftentaken for granted; when done poorly, it is immediatelynoticed. Cleanliness is directly tied to user satisfaction, aswell as to performance quality levels for the public trans-portation services.

The appearance of the underground buildings, trainsand public spaces, as measured by overall cleanliness, isone of the most significant criteria, in conjunction withreliability and safety, for assessing the effectiveness ofunderground management.

From a cost management perspective, the cleaningservice amounts to an important part of the operating costs

⁎ Corresponding author.E-mail address: [email protected] (A. Jiménez).

0167-9236/$ - see front matter © 2006 Elsevier B.V. All rights reserved.doi:10.1016/j.dss.2006.06.010

of an underground transportation service, accounting, inthis case, for close to 10% of total annual operatingexpenses.

In the past, service contracting with both publicagencies and private companies alike has suffered fromsystemic problems (i.e., cost overruns, delays and poorperformance). The problems arise due to two funda-mental premises: the intangibility and heterogeneity ofthe service and the idea that production and consump-tion are inseparable. Inherent service intangibility,heterogeneity and inseparability greatly constrain ser-vice acquisition and management, see Ref. [21].

No comprehensive view has been taken of availablecleaning services measurement or of systems that can beeasily used at the work place to inspect cleaningactivities and provide an objective view of the resultsachieved, making an objective assessment of contractorperformance difficult.

mailto:[email protected]

http://dx.doi.org/10.1016/j.dss.2006.06.010

1486 A. Jiménez et al. / Decision Support Systems 43 (2007) 1485–1498

To improve cleaning services contracting, we workedon different initiatives throughout the acquisitionprocess to improve the service level and to significantlyreduce total service cost. In addition, as a public service,this acquisition process had been carried out accordingto the public procurement policies and legislation for thenational and European public organizations, as estab-lished in directives 98/4/EC of the European Parliamentand of the Council of 16 February 1998 amendingDirective 93/38/EEC coordinating the procurementprocedures of entities operating in the water, energy,transport and telecommunications sectors, and Directive97/52/EC of the European Parliament and of the Councilof 13 October 1997 amending Directives 92/50/EEC,93/36/EEC and 93/37/EEC concerning the coordinationof procedures for the award of public service contracts,public supply contracts and public works contracts,respectively.1

For the purpose of determining the most economic-ally advantageous suppliers as regards the problemunder consideration, several conflicting criteria weretaken into account simultaneously to provide the mostrelevant information about what other factors, inconjunction with prices, to apply during the award ofcontract process, like delivery conditions and humanresources, technical merit and resources, quality controlprocedures, etc. These criteria were compiled as a resultof a team effort, including procurement and technicalexperts from the organizational areas responsible forcleaning services, finance and other departments orrepresentatives, such as legal affairs and customerservice. They were all integrated into a cross-functionalteam responsible for preparing the process and gatheringall the information necessary to achieve the desiredoutcomes. From now on, we will denote this group ofexperts and representatives as decision-makers (DMs).

Preference trade-offs between differing levels ofachievement of one criterion and another must be takeninto account simultaneously. Moreover, it may be farfrom straightforward to specify how each offer willperform in the end. This requires formal analysis,because it is very difficult to deal with the abovecomplexities informally in the mind.

The goal of Decision Analysis (DA) is to structureand simplify the task of making hard decisions, see Refs.[3], [8] and [9]. It is developed on the assumption thatthe alternatives (offers from suppliers) will appeal to theDMs depending on how each offer is likely to performand what DM preferences are. The DA cycle can be

1 http://europa.eu.int/comm/internal_market/publicprocurement/legislation_en.htm.

divided into four steps: (1) structuring the problem(which includes specifying objectives, building a valuehierarchy and establishing attributes for the lowest-levelobjectives); (2) identifying the feasible offers, theirperformances and uncertainty (if necessary); (3) quanti-fying preferences (which includes the assessment of thecomponent attribute utilities, as well as the value trade-offs); (4) evaluating offers and performing sensitivityanalysis (SA).

The Generic Multi-Attribute Analysis (GMAA)System, a user-friendly PC-based decision supportsystem founded on an additive multi-attribute utilitymodel (http://www.dia.fi.upm.es/∼ajimenez/GMAA),will be used to allay many of the operational difficultiesinvolved in the DA cycle, see, e.g., Refs. [5], [7] and[17].

Note that the group of DMs were guided throughoutthe decision-making process by experts in DA metho-dology, specifically, the team that developed andimplemented the GMAA system. Decision-theory con-cepts and tools included in the GMAA system wereclarified at several meetings.

We have divided the paper, according to DA stages,into five sections, where we describe the cleaningservice contracting problem and how it is modelledin the GMAA system and, finally, provide someconclusions.

2. Structuring the problem

An objective hierarchy including all the relevantaspects related to the problem under consideration wasbuilt, aimed at improving DM understanding of thedecision. For our complex decision-making problem, itdescribes how offers have to be evaluated in terms oftechnical merits and costs, see Fig. 1.

There are five main top-level objectives: Deliveryconditions and human resources (Delivery Con), whichaccounts for how consistent and coherent the humanresources allocated to the services are; Technical meritand resources (Tech Resourc), which it is an importantefficiency factor leading to a significant reduction inlabor cost; Price (Price), which represents the lowestprice offered by suppliers; Quality control procedures(Quality Mean), which accounts for accredited qualitycertifications and how quality systems and proceduresare deployed; and Graffiti prevention and cleanup(Graffiti), which is one of the most common incidentsdetracting from the appearance of the undergroundbuildings.

The objective Delivery conditions and humanresources is split into three sub-objectives, Workload,

http://www.dia.fi.upm.es/~ajimenez/GMAA

http://europa.eu.int/comm/internal_market/publicprocurement/legislation_en.htm

http://europa.eu.int/comm/internal_market/publicprocurement/legislation_en.htm

Fig. 1. Objective hierarchy.

1487A. Jiménez et al. / Decision Support Systems 43 (2007) 1485–1498

Workload allocation and Cost optimization program.Workload (Workload) quantifies how much effort isdeployed in terms of resources to achieve serviceperformance objectives. Workload allocation (Wrk.Allocat) measures the coherence and consistency ofthe allocation of the resources throughout the under-ground facilities and buildings. Cost optimizationprogram (Cost optimiz) measures the suitability ofsupplier cost reduction initiatives using four additionalsub-objectives: Understanding of client's needs andissues (Understand), in which suppliers will be rewardedin as much as their offers show a good understanding ofthe cleaning needs regarding underground operationsand public transportation services; Cost reductionapproach (Cost Reduct.), in which case suppliers willbe rewarded in as much as their offers develop astructured solution for minimizing service costs withinthe contract period and providing details on how otherclients have successfully implemented cost reduction

programs; Suitability of cost reduction initiatives (SuitReduct), which quantifies how applicable and realisticthe cost reduction initiatives are as things now stand andhow they will ensure significant cost savings; and Costreduction target setting and performance monitoring(Target Set), in which suppliers will be rewarded in asmuch as their offers include procedures and measuresand key performance indicators, setting aggressivetargets to drive for change. These metrics and measure-ments procedures will form a model for successfullybudgeting for and tracking cost savings.

Technical merit and resources (Techn. Resources) isalso split into two sub-objectives, Quantity of technicalmeans and Technical resources suitability. Suppliers willbe rewarded in as much as their offers include numerousand extensive use of technical resources by Quantity oftechnical means (Techn. Means). Technical resourcessuitability (Res. Suitab.) is measured taking into accountthree additional sub-objectives: Equipment suitability to

Table 1Attribute names, units and ranges

Unit Range

Workload Absolute percentageerror (APE)

[0, 50]

Wrk. allocat Mean absolute percentageerror (MAPE)

[0, 100]

Understand Subjective scale [0, 1]Cost reduct. Subjective scale [0, 1]Suit reduct Subjective scale [0, 1]Target set Subjective scale [0, 1]Techn. means Number of pieces

of equipment[0, 90]

S. buildings Subjective scale [0, 1]Tech. docum. Subjective scale [0, 1]Clean. doc. Subjective scale [0, 1]Price Monetary units [15,000, 30,000]Qual. certif Discrete values 0, 1, 2 and 3Control capa Subjective scale [0, 1]As required Subjective scale [0, 1]Correc. act. Subjective scale [0, 1]Training Subjective scale [0, 1]P. monitoring Subjective scale [0, 1]Experience Discrete values 0, 1, 2, 3 and 4Suit. proced Subjective scale [0, 1]Tech. solut Subjective scale [0, 1]Suit. produc Subjective scale [0, 1]


client's buildings (S. Buildings), which quantifies itsfitness for underground facilities (accessibility, buildinglayout, types of surfaces, etc.) and possible adverseenvironmental impacts; Equipment technical documenta-tion (Tech. Docum.), related to how well technicaldocumentation describes the equipment providing agood specification and understanding of equipmentperformance; and Cleaning product technical documen-tation (Clean. Doc.), which accounts for how welltechnical documentation describes the cleaning products,providing a good specification and understanding ofproduct performance.

The objective Quality control procedures is mea-sured in terms of two sub-objectives, Number of qualitycertifications and Quality measurement systems andprocedures. For Number of quality certifications (Qual.Certif), suppliers will be rewarded for having qualitycertifications accredited by national or European qualitycommittees and institutions. Quality measurementsystems, documentation and procedures (Sys/Doc/Proc) accounts for how quality systems and proceduresare deployed. This sub-objective is further split into fivesub-objectives: Supplier quality control capabilities(Control Capa.), which quantifies current suppliercapabilities in terms of the supplier's quality assuranceorganization, its structure, number of resources, currentskills and attested competencies; Quality systems, plansand procedures described as required (As Required), inwhich suppliers will be rewarded in as much as theiroffer contents have been developed according to thetender guidelines; Poor performance corrective actions(Correc. Act.), which measures the proposed proceduresand action plans to be put in place to remedy poorperformance and incidents in cleaning services deliver-ing; Change management approach and training(Training), which accounts for the communicationsand training actions to be adopted to ensure that allpeople (employees, contractors, users) have the requiredknowledge and meet the operational and businessrequirements for delivering the services; and Qualityperformance monitoring (P. Monitoring), whichaccounts for the measurement system to be implemen-ted, i.e., what will be inspected, how the inspection willbe carried out and the measurement criteria to be usedduring inspection are evaluated to verify that theyguarantee greater objectivity, completeness and highquality services.

Finally, Graffiti prevention and cleanup takes intoaccount two sub-objectives, past experiences and graffiticleanup procedures and products. Past experiences ingraffiti prevention and cleanup (Experience) verifiessupplier competence, assessing details on how the

supplier has successfully implemented these specificservices for other clients. Graffiti cleanup proceduresand product features (Proc and Prods) is measured bytaking into account three additional sub-objectives:Suitability of graffiti cleanup procedures (Suit. Proced),which measures their fitness for underground facilities(building layout, types of surfaces, types of graffiti, etc.)and possible adverse environmental impacts, Technicalsolution for prevention and cleanup (Tech. Solut) andSuitability of graffiti cleaning products (Suit. Produc),which quantify how well the technical documentationdescribes the equipment, leading to a good specificationand understanding of the equipment and productperformance, respectively.

Once the objective hierarchy has been built,attributes must be established for the lowest-levelobjectives to indicate to what extent they are achievedby the respective offers. Table 1 shows the attributenames, units and ranges for cleaning service contracting.

Note that Experience and Qual. Certif are assigneddiscrete attribute values. For Experience, 0 means Noreferences, 1—one reference, 2—two or three refer-ences, 3—two or three similar problems and 4—three ormore similar problems; while for Qual. Certif, 0 meansno certificate, 1—quality control proc./prog., 2—European certificates and 3—EN ISO 9001/ISO 9002.On the other hand, Workload, Wrk. Allocat, Techn.


Means and Price are straightforward in the sense thatthey are directly related to their associated objectives,and a continuous scale is used.

The units for Workload and Wrk. Allocat, i.e., APEand MAPE, represent the following:

• If the number of offers is odd and higher than 3, theyare ordered from the highest to the lowest value, anda mean (M) is assessed by discarding the highest andlowest values.

• If the number of offers is even and higher than 2, themean is assessed considering all values. Then, the offerwhose difference with respect to the mean is thehighest, positive or negative, is discarded. A newmean(M) is then assessed taking the remaining values.

• If the number of offers is 2 or 3, the mean (M) isassessed considering all values.

Then, the absolute percentage error and the meanabsolute percentage error for the j-th offer, APEj andMAPEj, are:

APEj ¼ ABSðM � valuejÞM

MAPEj ¼ 1N�XN

i¼1

ðM � valuejÞ � 100

M

where N is the number of offers. Consequently, the bestvalue for attributes is 0, while the worst values are 50and 100, respectively.

Finally, the remaining attributes are subjective, whichmeans that there is no objective quantity, and a subjectiveindex has been constructed associated with a scale from 0(worst) to 1 (best).

3. Quantifying decision-maker preferences

Following the European Community directives, stat-ing that criteria should be set and at least ordered byimportance before examining the offers, stages 2 and 3 inthe DA methodology must be switched around, i.e., DMpreferences must be quantified before identifying offers.

Quantifying preferences involves assessing the singleDM attribute utilities, which represent DM preferencesconcerning possible offer performances, and weights,which indicate the relative importance of criteria in theobjective hierarchy. Both will be used later to evaluatethe offers by means of an additive multi-attribute utilityfunction.

Preferences were assessed as a result of a teameffort, including procurement and technical expertsfrom the organizational areas responsible for cleaningservices, finance and other departments or representa-tives such as legal affairs and customer service.Procedures for preference assessment provided by theGMAA system were used, which can deal withincomplete information through value intervals asresponses to the probability questions the DM isasked, see Ref. [17] or [7]. This is less stressful onexperts, see Refs. [13], [19] and [20]. Moreover, itmakes the system suitable for group decision support,because individual conflicting views or judgments in agroup of DMs can be captured through impreciseresponses, see, e.g., [10].

For attributes with a continuous scale (Price, Work-load, Workload allocation and Quantity of technicalmeans) precise linear piecewise utility functions wereassessed. For attributes for which a set of discreteattribute values was defined (Number of qualitycertifications and Past experiences in graffiti preventionand cleanup), an imprecise utility for each discreteattribute value was provided. No preferences have to beassessed for the remaining attributes, all of which have asubjective scale. In this case, the subjective offerperformances between 0 (worst value) and 1 (bestvalue) shown in the next section were used directly asutilities.

Figs. 2 and 3 show the precise piecewise utilityfunctions and imprecise utilities for discrete attributevalues examined, respectively.

Note that utility functions are decreasing for Work-load, Wrk. Allocat and Price, and increasing for Tech-nical Means.

Additionally, attribute weights were hierarchicallyelicited, i.e., local objective weights along each branchand level of the hierarchy representing their relativeimportance were elicited in ascending order and thenattribute weights were assessed by multiplying the localweights of the objectives along the path from the overallobjective to each attribute.

Again, weighting procedures provided by theGMAA were used, a direct assignment and a methodbased on trade-offs, [8]. As in the utility assessment,imprecision concerning DM responses was allowed bymeans of ranges of responses. Then, a normalizationprocess was automatically performed on the DMresponses, leading to an average normalized weightand a normalized weight interval for each objectiveunder consideration, see Refs. [5] and [7].

For instance, Table 2 shows the DM responses fortop-level objectives weight elicitation by direct assignment

Fig. 2. Piecewise linear utility functions.


and the respective average normalized weights andnormalized weight intervals.

Fig. 4 shows the local average normalized weights inthe objectives hierarchy, and Fig. 5 illustrates attributeweights, calculated by multiplying local averagenormalized weights and the endpoints of the normalizedweight intervals of the objectives along the path fromthe overall objective to the respective attribute.

Looking at the above figure, we can see that the mostimportant attribute in the decision is Price, with anaverage normalized weight of 0.513, followed farbehind by Workload (Workload), Quantity of Tech.Means (Techn. Means), N. quality certification (Qual.Certif), Experience with Graffiti (Experience) andWorkload Allocation (Wrk. Allocat), with averagenormalized weights of 0.125, 0.082, 0.052, 0.052 and0.036, respectively.

Fig. 3. Imprecise utilities f

4. Alternatives identification

Next, the feasible supplier offers must be identifiedand measured in terms of attributes. Table 3 shows thescores or performances of the different attributes for thesix offers considered.

Note that although the table includes precise perfor-mances, uncertainty about some of them was taken intoaccount by means of percentage deviations. Thesepercentage deviations represent tolerances or correctionfactors for offers where information provided by thebidders was somewhat ambiguous or was not clear enough.

Specifically, a 20% deviation was introduced inWorkload for offers 5 and 6, and 15% for offer 4; a 10%deviation was introduced in Wrk. Allocat for offers 3and 6, and 20% for offer 2; a 30% and a 15% deviationwas introduced in Techn. Means for offers 3 and 5,

or discrete attribute.

Table 2DM responses and respective weights for top-level objectives

Responses Avg. norm.weight

Norm. weightinterval

Delivery con [0.15, 0.20] 0.179 [0.154, 0.205]Tech resourc [0.10, 0.10] 0.103 [0.103, 0.103]Price [0.50, 0.50] 0.513 [0.513, 0.513]Quality mean [0.05, 0.15] 0.103 [0.051, 0.154]Graffiti [0.05, 0.15] 0.103 [0.051, 0.154]


respectively. In all these attributes the original offersincluded some overlapping information, e.g. resourcesand equipment overlaps.

Additionally, a 10% deviation was introduced in S.Buildings for offers 2 and 5, and 5% for offer 3; and a10% deviation was introduced in Control Capa foroffers 3 and 5, and in Suit. Proced for offers 3, 4, 5 and6. In these cases, where a subjective scale has been used,the percentage deviations aimed to account for missingtechnical knowledge, e.g. the fitness of certain equip-ment and graffiti cleanup procedures for undergroundfacilities and possible adverse environmental impacts,

Fig. 4. Local average no

and, additionally, to match up different evaluations bymembers of the cross-functional team who used theirown approach assessment based on their experience andknowledge, e.g. how a quality assurance organizationhas to be structured to be more effective.

5. Evaluating offers

This step involves evaluating each offer to helpidentify the best one by means of an additive multi-attribute utility model, which takes the form

uðOffer jÞ ¼Xn

i¼1

wiuiðx ji Þ

¼ w1u1ðx j1Þ þ …þ wnunðx j

nÞ;

where xji is the consequence of the j-th offer for the

attribute Xi, ui are the single utilities assigned to therespective attribute Xi and wi is the i-th attributeweight.

For the reasons described in Refs. [12] and [18], weconsider the additive form to be a valid approach.

rmalized weights.

Fig. 5. The attribute weights.


As we have considered imprecision concerningsingle utilities and weights and uncertainty about theoffer performances, the additive model is used to assess,on the one hand, average overall utilities, on which theranking of offers is based and, on the other hand,minimum and maximum overall utilities, which givefurther insight into the robustness of this ranking, seeFig. 6.

Offers 3 and 6 are the best and worst ranked ones,with average overall utilities of 0.837 and 0.351,respectively. Looking at the utility intervals, we candiscard offer 6, because 13 its maximum utility is lower

Table 3Offer performances

Offer 1 Offer 2 Offer

Workload 5.62 11.85 14Wrk. allocat 13.63 30.66 19Understand 0.46 0.49 0Cost reduct. 0.35 0.64 0Suit reduct 0.55 0.63 0Target set 0.60 0.57 0Techn. means 0.23 0.19 0S. buildings 0.52 0.88 0Tech. docum. 0.86 0.2 0Clean. doc. 0.54 0.35 0Price 18,088 19,329 17,083Qual. certif 3 3 3Control capa 0.76 0.84 1As required 0.85 0.57 0Correc. act. 0.84 0.65 0Training 0.88 0.54 0P. monitoring 0.74 0.60 1Experience 1 3 4Suit. proced 0.12 0.51 0Tech. solut 0.33 0.48 0Suit. produc 0.35 0.58 0

than the minimum utility of offer 2. Consequently, wecan conclude that it is dominated. However, althoughoffer 3 appears to be the most highly recommended, theoverlapped utility intervals (ranking robustness) shouldbe examined in more detail through SA.

Also another intermediate objective can be selectedto rank by. Fig. 7 shows the ranking of offers for De-livery conditions and human resources, where offer 1 isnow the best-ranked and dominates the others, whileoffer 3 is the second best-ranked.

As shown in Fig. 6, the GMAA system providesdifferent displays of ranking results, [16], such as a

3 Offer 4 Offer 5 Offer 6

.61 29.21 33.31 36.195.78 16.27 26.6

.74 0.21 0.34 0.33

.20 0.83 0.24 0.36

.75 0.54 0.64 0.24

.75 0.58 0.21 0.28

.63 0.31 0.8 0.24

.99 0.74 0.33 0.71

.84 0.40 0.21 0.58

.31 0.87 0.58 0.4917,994 17,549 21,993

3 2 10.86 0.71 0.22

.81 0.85 0.73 0.51

.84 0.96 0.86 0.51

.48 0.75 0.84 0.271 0.59 0.334 4 4

.88 0.71 0.62 0.84

.97 0.80 0.49 0.78

.76 0.88 0.90 0.89

Fig. 6. Offer evaluation.


Stacked Bar Ranking (provides more detail on how thealternative's average utilities for the attributes affect theaverage utility of the overall objective, see Fig. 8), aMeasure of Utilities for Alternatives (shows perfor-mance of a single offer for the attributes, taking intoaccount average consequences and individual utilities,and where the width of an attribute is proportional to itsweight), or a Compare Alternatives Graph (provides adetailed comparison of the differences between twoalternatives, see Fig. 9).

Note that, according to attribute weights in thedecision, a good performance for the Price attributeleads to higher utilities in the overall objective, as thisattribute makes the biggest contribution.

Fig. 7. Ranking of offers for Delivery c

Looking at the Compare Alternatives Graph we cansee which of the two selected offers have the higheraverage utility associated with the different attributes.The length of the horizontal bars is proportional to thedifference in utility among the two. Note that offer 3 isbetter than offer 4 for Price, Workload and Quantity ofTech. Means (their average weight in the decision addsup to 0.72).

6. Sensitivity analysis (SA)

DA is typically an iterative process. Once the modelhas been built, SA is performed. This step should beconsidered as a means of encouraging DMs to think

onditions and human resources.

Fig. 8. Stacked bar ranking.


about the problem in more depth and can give furtherinsight into the robustness of the recommendations.Refs. [14] and [15] introduce a framework for SA inmulti-objective decision-making.

The GMAA system provides several types of SA, see[16]. Classical SA and the assessment of stability weightintervals can be considered as pure SA. Classical SAessentially involves examining changes in the ranking ofoffers as a function of the input parameters.

Fig. 9. Comparing o

Any offer performance, component utility or weightcan be changed and the system takes charge of how thischange is propagated through the objective hierarchyand automatically recalculates the overall utilities foreach offer and the resulting ranking.

Stability weight intervals, i.e., the interval where theaverage normalized weight for any objective at anylevel in the hierarchy can vary without the best-rankedoffer changing, can also be assessed. Stability weights

ffers 3 and 4.

Fig. 10. Weight stability interval.


interval for all objectives throughout the hierarchy is[0, 1], which implies that whatever their relativeimportance offer 3 remains the best-ranked offer,except for Delivery conditions and human resources(Delivery Con), see Fig. 10, and Quality controlprocedures (Quality Mean) with stability weightintervals [0, 0.598] and [0, 0.748], respectively.Consequently, if the average normalized weight foreither of these nodes is modified and the new value ishigher than 0.598 and 0.748, respectively, offer 3 willnot be the best-ranked offer.

Taking into account that the narrower a stabilityweight interval is the more sensitive the offers rankingis, we can conclude that the offers ranking is robust withrespect to the elicited weights.

The system also includes so-called decision-makingwith partial information, [14], which intends to takeadvantage of the imprecise information collected duringthe assignment of single utilities and weights and thealternative performances entered under uncertainty by

Fig. 11. Non-dominance and

computing non-dominated and potentially optimaloffers and using Monte Carlo simulation techniques.

The assessment of non-dominated and potentiallyoptimal offers can reduce the set of offers of interest,[14,15], mainly by discarding dominated and/or non-potentially optimal offers. Note that offer i is dominatedby offer j when for all the possible combinations ofimprecise input parameters (performances, componentutilities and weights), the ranking for offer i is betterthan for offer j, while an offer is potentially optimalwhen it is the best-ranked offer for at least one com-bination of imprecise input parameters. This involvesdealing with non-linear optimization problems that canbe transformed into linear problems and solved usingthe Simplex Method, see Ref. [11].

Fig. 11 shows that only offer 3 is non-dominated andpotentially optimal, so we can definitively conclude thatit is the best offer.

On the other hand, the system runs simulationtechniques for SA. This kind of SA uses Monte Carlo

potential optimality.

Fig. 12. Partial attribute rank.


simulation, [2,4,6], allows simultaneous changes toattribute weights and generates results that can be easilyanalyzed statistically to provide more insight into themulti-attribute model recommendations. We proposeselecting the weights at random using a computersimulation program so that the results of manycombinations of weights can be explored efficiently.The system uses a multiplicative linear congruentialgenerator based on Schrage's method, see Ref. [1].

Three general classes of simulation can be used:Random weights, weights for the attributes are generatedcompletely at random, which implies no knowledgewhatsoever of the relative importance of the attributes;Rank order weights, weights are randomly generatedpreserving a total or partial attribute rank, which mayprovide more meaningful results; and Response dis-

Fig. 13. Multipl

tribution weights, attribute weights are now randomlyassigned values taking into account weight intervalsprovided by the DM in the weight elicitation methods.

Once the simulation has been run, the systemcomputes several statistics about the rankings of eachoffer, like minimum, maximum, mean… This informa-tion can be useful for discarding some available offers,aided by a multiple boxplot display.

For our specific problem it is pointless to useResponsedistribution weights, because, as shown earlier, theimprecise elicited attribute weights (Fig. 5) lead to justone non-dominated and potentially optimal offer; offer 3.Consequently, this offer would always be best-ranked andno further information would be gained. However, itwould be interesting to use Rank order weights to analyzethe robustness of this recommendation. In this case, on thebasis of the information in Fig. 5, the partial attribute rankshown in Fig. 12 would be used.

Fig. 12 reflects that the most important attribute inthe decision is Price, followed by Workload, Quantityof technical means, Number of quality certifications,Past experiences in graffiti prevention and cleanup andWorkload allocation, respectively. The remaining attri-butes are less important than the above, but there is noinformation about their relative importance.

Fig. 13 shows the multiple boxplot for this type ofsimulation, whereas the associated statistic measures areshown in Fig. 14.

Offers 1, 3 and 4 are the only offers that are best-ranked across all 10,000 simulations, and their worstranking is fourth, fourth and fifth, respectively. Lookingat mean classifications, however, the best value is foroffer 3: 1.382. Consequently, we can conclude that therecommendation to select offer 3 is robust.

It is important to note that the DMs were completelysatisfied with this recommendation. SA tools played a

e boxplot.

Fig. 14. Associated statistics measures.


key role in this respect. The results achieved in theassessment of non-dominated and potentially optimaloffers and the simulation techniques with Rank orderweights were conclusive.

7. Conclusions

The selection of a supplier for cleaning services inunderground facilities is a complex decision-makingproblem where several conflicting criteria must be takeninto account simultaneously, including prices, deliveryconditions and human resources, technical merit andresources, or quality control procedures.

Throughout the paper, the problem has been analyzedin depth on the basis of the decision analysismethodology. An objectives hierarchy including all therelevant aspects related to the problem under considera-tion has been built, decision-makers' preferences havebeen quantified and offers have been identified as wellas their performances in terms of the attributesestablished for the lowest-level objectives. Finally, anadditive multi-attribute utility model has been used toidentify the best one offer, leading to imprecise overallutilities and an offer ranking on the basis of averageoverall utilities. As a result, offer 3 appears to be the bestoffer for selection.

Moreover, taking advantage of the sensitivity analysistools provided by the Generic Multi-Attribute Analysissystem, we have reached the conclusion that it is a robustrecommendation. Specifically, stability weight intervalsare wide, so the recommendation is not sensitive to theelicited objective weights; offer 3 is the only non-dominated and potentially optimal offer; and the use of aranking of attributes by partial importance attribute rankaccording to elicited attribute weights in simulationtechniques confirms the above results.

Consequently, the GMAA system has been proven tobe an adequate tool for modeling the problem under

consideration on the basis of the decision analysismethodology and for achieving a better understandingof the recommendation and its robustness.

Acknowledgments

This paper was supported by the Spanish Ministryof Education and Science Project TSI2004-06801-C04-04 and the Madrid regional government project S-505/TIC/02.

References

[1] P. Bratley, B.L. Fox, E.L. Schrage, A Guide to Simulation,Springer-Verlag, New York, 1983.

[2] J. Butler, J. Jia, J. Dyer, Simulation techniques for the sensitivityanalysis of multi-criteria decision models, European Journal ofOperational Research 103 (1997) 531–546.

[3] R.T. Clemen, Making Hard Decisions: An Introduction toDecision Analysis, Duxbury Press, Beltmon, 1996.

[4] J.S. Dyer, T. Edmunds, J.C. Butler, J. Jia, A multiattribute utilityanalysis of alternatives for the disposition of surplus weapons-grade plutonium, Operations Research 46 (6) (1998) 749–762.

[5] A. Jiménez, S. Ríos-Insua, A. Mateos, A decision support systemfor multiattribute utility evaluation based on imprecise assign-ments, Decision Support Systems 36 (1) (2003) 65–79.

[6] A. Jiménez, S. Ríos-Insua, A. Mateos, MonteCarlo simulationtechniques in a multi-attribute decision support system, Proceed-ings of the 12th IASTED International Conference on AppliedSimulation and Modelling, 2003, pp. 85–90.

[7] A. Jiménez, S. Ríos-Insua, A. Mateos, A generic multi-attributeanalysis system, Computers and Operations Research (2004) (toappear).

[8] R.L. Keeney, H. Raiffa, Decision with Multiple Objectives:Preferences and Value-Tradeoffs, Wiley, New York, 1976.

[9] C.W.Kirkwood, StrategicDecisionMaking.MultiobjectiveDecisionAnalysis with Spreadsheets, Duxbury Press, Belmont, 1997.

[10] A. Mateos, A. Jiménez, S. Ríos-Insua, Modelling individual andglobal comparisons for multi-attribute preferences, Journal ofMulticriteria Decision Analysis 13 (2003) 1–14.

[11] A. Mateos, A. Jiménez, S. Ríos-Insua, Solving dominance andpotential optimality in imprecise multi-attribute additive models,Reliability Engineering and System Safety 79 (2) (2003) 253–262.

1498 A. Jiménez et al. / Decision Support Systems 43 (2007) 1485

[12] H. Raiffa, The Art and Science of Negotiation, Harvard UniversityPress, Cambridge, Mass., 1982

[13] S. Ríos, S. Ríos-Insua, D. Ríos Insua, J.G. Pachon, Experimentsin robust decision making, in: S. Ríos (Ed.), Decision Theory andDecision Analysis: Trends and Challenges, Kluwer, Boston,1994, pp. 233–242.

[14] D. Ríos Insua, Sensitivity Analysis in Multiobjective DecisionMaking, Springer, Berlin, 1990 LNEMS 347.

[15] D. Ríos Insua, S. French, A framework for sensitivity analysis indiscrete multi-objective decision-making, European Journal ofOperational Research 54 (1991) 176–190.

[16] S. Ríos-Insua, A. Jiménez, A. Mateos, Sensitivity analysis in ageneric multi-attribute decision support system, in: K.J. Engemann,G.E. Lasker (Eds.), Advances in Decision Technology andIntelligent Information Systems, The International Institute forAdvanced Studies in Systems Research and Cybernetics, Ontario,2003, pp. 31–35.

[17] S. Ríos-Insua, A. Jiménez, A. Mateos, T. Prieto, A multiattributedecision support system with imprecise input and time-dependentmodelling, International Journal of Technology, Policy andManagement 3 (3) (2003) 230–250.

[18] T.J. Stewart, Robustness of additive value function method inMCDM, Journal of Multi-Criteria Decision Analysis 5 (1996)301–309.

[19] R. von Nitzsch, M. Weber, Utility function assessment on amicro-computer: an interactive procedure, Annals of OperationResearch 16 (1998) 149–160.

[20] M. Weber, Decision making with incomplete information,European Journal of Operational Research 28 (1987) 44–57.

[21] V.A. Zeithaml, L.L. Berry, A. Parasuraman, Delivering QualityService: Balancing Customer Expectations and Perceptions, FreePress, Detroit, 1990.

Antonio Jiménez Martín obtained a Comput-ing Science degree from Madrid TechnicalUniversity. He is currently an AssociateProfessor of Operations Research and Simu-lation at School of Computer Science. Hisresearch interest is decision analysis and isinvolved in the development and implementa-tion of decision support systems based onmulti-attribute utility theory. His research hasappeared mainly in Computers and Opera-tions Research, DSS, EJOR, J. of Multicriteria

Decision Analysis, GDN, JORS, Ecological Modelling, … and has co-authored three books.

Alfonso Mateos Caballero is an Associate
Professor of Statistics and Operations Re-search at School of Computer Science(Madrid Technical University). His researchinterest is decision analysis and is currentlyinvolved in the development intelligent deci-sion support systems based on influencediagrams and multi-attribute utility theorywith applications to environment, medicineand e-democracy. His articles have appearedin various academic journals including:
–1498

Decision Support Systems, European Journal of Operational Research,J. Opl Res. Soc., Annals of Operations Research, ComputationalOptimization and Applications, OPSEARCH, JORS, EcologicalModelling, GDN and Journal of Multicriteria Decision Analysis.

Sixto Ríos-Insua is a Professor of Statisticsand Operations Research at Madrid TechnicalUniversity. His research interest is decisionanalysis and is at present involved in thedevelopment intelligent decision support sys-tems based on influence diagrams and multi-attribute utility theory with applications toenvironment and medicine. His articles haveappeared in various academic journals includ-ing: Computers and Operations Research,Theory and Decision, EJOR, JORS, Annals

of Operations Research, Computational Optimization and Applica-tions, OPSEARCH and J. of Multicriteria Decision Analysis.

Luis Carlos Rodríguez obtained a ComputingScience degree from Valencia TechnicalUniversity (UNITEC). He is currently aSenior Manager in Supply Chain ServicesLine in Accenture with extensive experiencein procurement strategies and procurementtransformation programs. Luis Carlos recentlyled several procurement transformation pro-grams in the natural gas, automotive andtransportation industries. Focused on sourcingstrategies and procurement processes, the

programs produced substantial savings. His research interest isdecision analysis and is involved in the development and implementa-tion of decision support systems based on multi-attribute utility theory.

Documents

Contracting cleaning services in a European public underground transportation company with the aid of a DSS