Ethical issues in modeling: Some reflections

Perspectives for Practice

Ethical issues in modeling: Some re¯ections

J.P.C. Kleijnen *

Department of Information Systems (BIK)/CentER for Economic Research (CentER), School of Economics and BusinessAdministration

(FEW), Tilburg University (KUB), Postbox 90153, 5000 LE Tilburg, The Netherlands

Received 11 August 1999; accepted 24 November 1999

Abstract

Ethics are involved in a modelÕs purposes (for example, the purpose might be to increase a heroin dealerÕs pro®ts).

These purposes imply consideration of the various stakeholders (modelers, users, public) and their values. Ethics also

concern professional standards of conduct for the modelers. These standards require that the modelers validate the

model assumptions. Hence, modelers should provide model documentation. Validation, however, is virtually impossible

when the model represents unique events, such as nuclear accidents; credibility is then the maximum attainable.

Anyhow, modelers should try to develop ÔrobustÕ models; that is, models that are not very sensitive to their assump-

tions. This article pays special attention to the use of models in crime, war, and nuclear applications, which might be

controversial applications indeed. It also discusses freedom of science. Hopefully, this article will stimulate further

discussion in the academic community! Ó 2001 Elsevier Science B.V. All rights reserved.

Keywords: Philosophy; Education; Modelling

1. Introduction

Should operations researchers worry aboutethics? Let me give some preliminary answers:1. Operations Research (OR) societies have no

formal codes of ethics, but other societies dohave them!

2. Recently, some prominent OR academics havediscussed ethics, at important fora.

3. Modelers are human beings, and all humansshould face moral issues!More precisely, neither the European OR so-

cieties nor the international Institute for Opera-tions Research and the Management Sciences(INFORMS) have codes of ethics. There are onlythe old guidelines of the Operations Research So-ciety of America (ORSA), drafted by Caywoodet al. (1971) and recently criticized by Taket (1994).

However, in 1999 the American Statistical As-sociation (ASA) published detailed ethical guide-lines; see ASA (1999) and its web page (http://www.amstat.org/profession/ethicalstatistics.html).I ®nd that these guidelines can be applied nearly adverbatim to OR; after all, statistics is one of the

European Journal of Operational Research 130 (2001) 223±230www.elsevier.com/locate/dsw

* Corresponding author. Tel.: +3113-466-2029; fax: +3113-

466-3377; http://center.kub.nl/sta�/kleijnen.

E-mail address: [email protected] (J.P.C. Kleijnen).

0377-2217/01/$ - see front matter Ó 2001 Elsevier Science B.V. All rights reserved.

PII: S 0 3 7 7 - 2 2 1 7 ( 0 0 ) 0 0 0 2 4 - 2

OR techniques that is most often applied.Throughout its guidelines, ASA emphasizes vali-dation. The guidelines also refer to internationalorganizations (such as the United Nations) andother professional organizations (not further de-tailed).

There are more professional organizations thatdo have ethics codes. The American PsychologicalAssociation (APA) has such a code (and so does,for example, the Netherlands Institute of Psy-chologists). Indeed, psychologists are expected toadhere to a strict code of conduct (and so aremedical doctors, lawyers, and journalists).Whereas classical OR only simulates human be-ings, social scientists work with real people; alsosee again ASA (1999). Some OR studies, however,use gaming so that real people are involved(teaching business ethics with management andmarketing games is described by Wolfe andFritzsche (1998); Experimental Economics alsouses simple games with real monetary payments tostudy egoistic versus altruistic behavior of people).The British school of Ôsoft ORÕ is another exampleof a type of OR that involves users±and modelers±more intensely; see Lane and Oiva (1998) andagain Taket (1994a,b).

Further, the Association for Computing Ma-chinery (ACM) has a code of ethics; see Anderson(1992). A draft code for Software Engineering hasbeen proposed by ACM and the Institute forElectrical and Electronic Engineering-ComputerSociety (IEEE-CS); see their web page (http://www.computer.org/tab/seprof/code.htm).

An example of a prominent OR academicdiscussing ethics, is professor Howard of Stan-ford; he delivered the Distinguished PlenaryLecture on ÔThe ethical OR/MS professionalÕ atthe May 1999 INFORMS meeting in Cincinnati(see Howard, 1999). Another OR example is thesymposium on Èthics in modeling' reported byWallace (1994). There was also a discussion onethics in the British Journal of the OperationalResearch Society; see Taket (1994) once more.Gass (1994) discusses the code of ethics for theOR modeler as a member of the academic com-munity: relationships as a professor with students,relationships as an author and referee, plagiarism,etc. (also see ASA, 1999) Wilson (1997) elaborates

principles of ethical conduct in science ± espe-cially in simulation; he emphasizes validation, andalso reports on his experience as a departmentaleditor of Management Science (I add references tothe websites of MIS Quarterly and Elsevier,which provide examples of editorial feedback:http://www.misq.org/archivist/editor.html andhttp://www.elsevier.nl/oasis/). Below I shall returnto these publications.

In this article I discuss some ethical issues basedon my personal experience as a modeler. But ®rst:what is meant by the term ethics? Webster's NewWorld Dictionary (1984 edition) de®nes ÔethicalÕ asÔ(1) having to do with ethics; of or conforming tomoral standards; (2) conforming to professionalstandards of conductÕ. I claim that a mathematicalmodel itself has no morals (neither does it have-say-color); a model is an abstract, mathematicalentity that belongs to the immaterial world. Thepurpose of a model, however, does certainly haveethical implications; for example, a model meantto increase the pro®ts of a heroin dealer has moralaspects. Note that I use the term ÔpurposeÕ to referto the problem that the model is supposed to helpsolve.

I further claim that WebsterÕs second meaning(Ôprofessional standards of conductÕ) has to dowith the use of a model by the modelers and theirclients. More speci®cally, any model is based onparticular assumptions (for example, it may assumelinear equations or Poisson processes with speci®cparameter values). Hence, the model results applyif those assumptions hold. But, what happenswhen these assumptions do not hold? This is oftennot known-because the modelers did not investi-gate this issue ± or is not emphasized enough ±because the users did not want to be bothered byÔall these technicalitiesÕ. Yet I think that this sec-ond meaning of ethical is of great practical im-portance!

My experience suggests that the interest in thevalidation of model assumptions is more articu-lated in the public domain, especially the militarydomain; in private business, proprietary aspects(con®dentiality) dominate. Details on validationcan be found in Kleijnen (1999, 1995b), includingmany more references. Below I shall also return tothe validation issue.

224 J.P.C. Kleijnen / European Journal of Operational Research 130 (2001) 223±230

The rest of this article is organized as follows.Section 2 discusses two earlier workshops on ethicsin modeling, and also addresses the related issuesof model validation and robustness. Section 3 ex-amines ethical aspects of the purposes of modelsused in crime, war and peace. Section 4 studiesethical professional conduct, including threats tothe freedom of scientists. Section 5 gives someconclusions. The articles ®nishes with thirty-eightreferences for further study.

2. Literature on ethics, validation, and robustness

Most mathematical models are only subsystemsof a decision support system (DSS). And like manyother tools (hammers, knives, etc.), these DSSs canbe used in good or in bad ways, by users or de-velopers, consciously or not. And passers-by maybe hit by the chips produced accidentally by thosehammers, etc.

DSSs include models that might be used ac-tively by clients themselves to answer Ôwhat ifÕquestions, whereas traditional OR models are runby modelers, not clients. Such model usage directlyby the clients themselves may be dangerous, as weshall see below.

Personally I got interested in the topic of ethicswhen I was invited to participate in an interna-tional videoconference on ÔEthics in modelingÕ on28 October 1994. This conference connected foursites, which had the following discussion leaders:William Wallace at Rensselaer Polytechnic Insti-tute, Warren Walker at Rand Europe, John Littleat MIT, and Saul Gass at the University ofMaryland. This conference emphasized that usersmay not understand the reasoning that is built intoa particular model (I would say that it is like givinga bazooka to kids ± or to adults, but withoutsupplying them with an instruction manual).Therefore I wish to point out that the documen-tation of a model should explain the modelÕs un-derlying reasoning, especially its performancemeasures (responses, outputs) and its assumptionswith their validation. For example, it makes adi�erence whether a model on drug usage is meantto maximize the dealerÕs pro®ts or to minimize theusersÕ consumption. Considering assumptions, I

tried to explicitly state all assumptions in, for ex-ample, my critical analysis of IBM's inventorypackage `ÌMPACTÕÕ in Kleijnen and Rens (1978).

When testing the validity of a simulationmodel, Ôauxiliary assumptionsÕ are introduced; forexample, normality of the simulation responses isoften assumed (also see Wilson (1997)Õs discussionof the so-called Duhem±Quine problem). Actually,OR modelers are brainwashed into assumingGaussian distributions so they often forget distri-bution-free (non-parametric) tests and computer-driven statistical techniques such as jackkni®ngand bootstrapping. So the assumptions of anystatistical techniques used for testing the validityof the simulation model, should also be docu-mented. More details on model documentationcan be found in Gass (1984) and ASA (1999) (e.g.,multiple tests increase the probability of falselyrejecting a valid model: type I error or modelerÕsrisk).

Earlier ± in 1989 ± another workshop on thesame issue was held at the Rensselaer PolytechnicInstitute. That workshop is reported by Wallace(1994). He emphasizes that model documentationis necessary in order to enable other researchers toreplicate the outcomes of the model; such replica-tion is a basic principle of science (also see Wilson,1997)!

Further, Wallace (1994) emphasizes the role ofvalues; that is, the values of clients, modelers, andother stake holders (for example, the public af-fected by the clientsÕ decisions). I think that afascinating example is the simulation of livertransplants, especially the great many policies forthe matching of donors and patients, as explainedby Pritsker (1998a,b, 1999) in an article titled ÔLife& death decisionsÕ in OR/MS Today. Another ex-ample of stakeholders (in which I was personallyinvolved as chairman of a steering committee) isthe simulation model that computes the ®nancialconsequences of changes in certain social securitylaws-for both the national government and theindividual laborers in the Netherlands; see Boschet al. (1994). Note that values are related to thepurposes of the model.

Next I consider various types of models. Unlikesimulation models, expert systems try to explaintheir reasoning; that is, they perform not only

J.P.C. Kleijnen / European Journal of Operational Research 130 (2001) 223±230 225

what±if analysis, but also state the ÔwhyÕ of theiroutcome (validation of the underlying expert rulesis another issue). Both simulation and expert sys-tems do not optimize, whereas mathematical pro-gramming models (e.g., linear programming) doprovide the optimal solution ± if the modelÕs as-sumptions hold and the correct response type isselected! My own experience in simulation is thatinstead of telling the users which decision to make,the simulationists should present the users with aset of non-dominated solutions. Depending on theusersÕ values, the users decide. In the private do-main, managers are paid so well because they mustmake such decisions ± and live with the conse-quences!

Spreadsheets can be a type of simulation. Manyusers, however, do not realize that a particularspreadsheet is indeed a simulation model! Conse-quently, these users may not be aware of the gar-bage-in-garbage-out (GIGO) characteristic ofmodels (a dramatic example is the bombing of theChinese embassy in Belgrade during the Kosovowar: wrong city map used!). Personally I rememberthat many years ago I was contacted by a mort-gage broker who o�ered me a mortgage that wasÔidealÕ for me. When I voiced some doubts, hementioned that his advice was based on a com-puterized spreadsheet ± and the computer cannotbe wrong! ASA (1999), however, says: ÔThe factthat a procedure is automated does not ensure itscorrectness ...Õ Note that most spreadsheet soft-ware complicates the validation of the underlyingmodel, since that model is not explicitly formu-lated in terms of equations and inequalities. Alsosee Whittaker (1999).

I think that it is a challenge to develop on-linedocumentation on the modelÕs purposes and as-sumptions and their validation. This documenta-tion should be accessible through a help button, asis now the case for modern software. Indeed,nowadays many discrete-event simulation modelsdo provide part of their documentation throughanimation, which explains±in user terms±the sys-tem being simulated. (Animation, however, can bea misleading validation technique, since it usesvery short simulation runs.)

Below I shall show that simulation models areoften used in uncertainty analysis or risk analysis:

they quantify the probability of a ÔdisasterÕ, such asa nuclear accident, an ecological collapse, or a ®-nancial mis-investment. I emphasize that thesedisasters are unique events, whereas (say) a su-permarket queueing model concerns repetitiveevents (e.g., customer waiting times). Conse-quently, validation in risk analysis is very di�cult;see Jansen and DeVries (1998). A better term maythen be credibility; see Fossett et al. (1991), andHodges (1991).

The dangers of wrong usage of a model becomemuch smaller if that model is robust; that is, themodelÕs output is not very sensitive to the exactvalues of the modelÕs parameters and inputs.Taguchi has emphasized the importance of ro-bustness, but he limited himself to physical prod-ucts such as cars (not abstract products such asmodels). An example of the study on model ro-bustness is the paper on pull production-planningsystems ± such as the Japanese Kanban systems ±by Kleijnen and Gaury (1999). In that paper,various types of pull systems are ®rst optimizedassuming a speci®c, most likely scenario for theenvironment. Usually modelers then select theoptimized system. In practice, however, the actualenvironment always di�ers from the assumed sce-nario. Even then, the system should not result indisastrous performance! Therefore Kleijnen andGaury quantify this disaster probability, applyingMonte Carlo sampling followed by bootstrapping(a statistical technique). So in general, we shouldconsider a population of scenarios, which impliesan average scenario and a worst-case scenario.Also see Rosenhead (1989).

These issues become even more important whenthe modelers do not know who the users will be!Let me compare a model with a car. A modelwithout documentation is like a car without aninstruction booklet! If the model is used respectingthe documentation, then the users are entitled to aÔwarrantyÕ: the modelers have to pay for wrongmodel conclusions. If, however, the clients areusing the model outside its validity range, thenthese clients are to be blamed. While ÕdrivingÕ themodel, red warning lights may switch on wheninputs are entered into the model that violate itsvalidity range (see Zeigler (1976)Õs ÔexperimentalframeÕ). A car is periodically returned to the


garage for maintenance; similarly a model may bereturned to its builders, for updating. With othersoftware it is well-known that maintenance is acrucial ± and expensive ± part of the life cycle!Note that illegal copies of software ± includingmodels ± may be fought through hardware keys orby selling compiled versions only (instead ofsource code).

Another metaphor is the instructions that comewith most medicines: these instructions warnagainst all kinds of undesirable side-e�ects. Like-wise, the documentation of a model should warnagainst improper usage. And likewise, this docu-mentation should be updated continually. Suchupdating is standard in software: new versionskeep appearing, repairing ÔbugsÕ discovered duringusage.

3. Ethical model purposes?

The example of the heroin dealer given above,is an example that most modelers would ®nd notconforming to their moral standards. How manymodels have been developed at the request of or-ganizations that the government classi®es ascriminal organizations? I have no idea at all ± butwho has? These organizations themselves have noreason to publish such information; I do not knowof any publications by the government on modelusage by criminals. I do know of a few publica-tions on the use of models by the authorities to®ght crime. For example, Van Meel (1993) dis-cusses two case studies within the Amsterdammunicipal police force. The RAND Corporationdeveloped a gaming model to study the USA'sdrug problem; see Caulkins (1995).

In practice, it is not always clear what consti-tutes a crime: is abortion a crime, even in case ofrape (see again Howard, 1999)? In this context, Ialso mention my technical article on Gitlow'smethodology for designing abortion clinics: Git-low (1976), Kleijnen (1979).

Besides Ôlaw and orderÕ inside the nation, thereare the essential and eternal problems of interna-tional war and peace. Not all scientists are pre-pared to work for the military establishment (yet,the origin of OR is the development of military

models during World War II). Personally I do notbelieve in the good nature of mankind, so I thinkthat modeling for military defense is morally ac-ceptable. But what is acceptable weaponry? De-fensive weapons have been de®ned as thoseweapons that our country owns, whereas o�ensiveweapons are by de®nition in the hands of ournational enemies ± whoever they are. One exampleof a military model that I was personally involvedin and that has been published, is the use of sonarto search for mines on the sea bottom (my con-tribution was the use of statistical techniques forthe validation of this simulation model; see Kleij-nen, 1995a).

Modern weaponry takes us inevitably to nu-clear weapons; for many scientists a moral dilem-ma, for sure! (My own visit to the nuclear bombmuseum in Albuquerque ± New Mexico, USA ±shed some new light on this issue, for myself.) Butnuclear processes also play a role in modernmedicine! And this takes me to another problemthat I was involved in: the deposit of nuclearwaste.

The USA must dispose of its radioactive waste(for example, contaminated garments). One prac-tical solution to this problem is the disposal un-derground. For that purpose the Waste IsolationPilot Plant (WIPP) has been built near Carlsbad,New Mexico, at 2000 feet below the surface. Be-fore this plant will start òperation' (that is, storageof nuclear waste), it must obtain permission fromthe Environmental Protection Agency (EPA) ofthe Department of Energy (DOE). Since there islittle practical experience with nuclear waste dis-posal, the EPAÕs permission depends heavily onsimulation of the WIPP. Part of this simulationconsists of nonlinear partial di�erential equationswith constraint equations, initial conditions, andboundary conditions. (One simulation run takesone to ®ve hours of computer time on a VAXAlpha with VMS; 1800 runs were executed andanalyzed as part of the uncertainty analysis.) Theseequations form a deterministic simulation model(since the underlying physical and chemical pro-cesses are modeled deterministically; other simu-lation WIPP submodels, however, include randomelements such as human activities that may lead tointrusions into the WIPP). Many parameters of


this deterministic model, however, are unknown sothey are sampled from statistical distributionfunctions ± by means of Monte Carlo sampling(especially, a re®ned sampling technique calledLatin Hypercube Sampling or LHS). This ap-proach is known as uncertainty analysis or riskanalysis. For details on WIPP and its uncertaintyand sensitivity analyses I refer to Kleijnen andHelton (1999); also see Howard (1999)Õs commentson nuclear waste disposal.

4. Ethical professional conduct?

Recently, K�obben and Tromp (1999) ± twoDutch social scientists ± wrote a book on thethreats that scientists may have to face when theyreport results that their ÔbossesÕ do not like. Theseauthors present the case of a Dutch physicist (atKEMA; see http://www.kema.nl) who changed hisposition in the debate on nuclear energy from proto contra; subsequently he had to change jobs!Another case is that of an expert at the NationalInstitute for Fishery Research (Rijksinstituut voorVisserij-onderzoek, RIVO). This expert publiclycriticized the reasoning behind the new ®shingquota imposed by the Ministry. His managementconsidered this public criticism to be ÔdisloyalÕ, asthe Ministry was the biggest customer of the In-stitute. (Also see Mentzel et al., 1995).

K�obben also participated in the discussion onethical questions in the social sciences that wasorganized by the Social Sciences Council (SWR inDutch) of the Royal Netherlands Academy ofSciences (KNAW). This interesting discussion isreported ± in Dutch ± by Mentzel et al. (1995).

In 1999, Dutch parliament members raisedquestions about the permission to Amsterdamairport (Schiphol) for its expansion plans: oneof the employees at the National Institute forPublic Health and Environmental Protection(Rijksinstituut voor Volksgezondheid en Milieu,RIVM) claimed that this permission was based ona wrong model, not on real-world measurementsof tra�c noise and pollution. I do not wish todiscuss here the advantages and disadvantages ofmodeling versus real-world measurements; insteadI do wish to repeat the need for validation of

models, and the related issues of sensitivity anduncertainty analyses (see the WIPP problemabove).

A practical problem with both ethical andtheoretical implications is: who pays the bill? ASA(1999) and Samuelson (1999) discuss this issue; Iadd that when more than a single party bene®ts,game theory may be applied to obtain an equitableanswer.

An issue related to professional conduct is theprotection of whistle blowers (Ôbell ringersÕ): em-ployees who warn against fraud; see ASA (1999).

Finally, recent books on professional conductare Kucßuradi (1999), Lawrence (1999), andMaclagan (1998).

5. Conclusion

Ethical issues in modeling are essential issuesfor all modelers, since there is life outside the of-®ce: all modelers are human beings, and humansare the only ÔanimalsÕ facing moral problems!

Nevertheless ± to the best of my knowledge ±these issues are not part of the standard academicOR curriculum. Exceptions that I am familiarwith, are the courses by Howard at StanfordUniversity (see Howard, 1999) and Walker atDelft University.

Occasionally these issues arise in the popularpress (such as newspapers), but these issues arethen not discussed in a scienti®c manner. I mustadmit that I myself have seldom stopped to thinkat much length about these problems. Therefore ishas been a challenge to force myself to re¯ect somemore on this problem when writing this article.

Since there seem to be so few specialists in this®eld, I hope that my article is a worthwhile con-tribution to the ®eld, and that it will stimulatefurther discussion on the issues of ethics in ORmodeling!

Acknowledgements

The ®rst version of this paper was prepared forthe symposium titled Ethical Issues in Modelling


and Simulation, organized at the occasion of theretirement of professor Maurice Elzas in Wage-ningen (the Netherlands) on 2 July 1999. I thankthe participants of that symposium for theircomments, and also Saul Gass (University ofMaryland), Jon Helton (Sandia), Alan Pritsker(Symix Systems), Doug Samuelson (Infologix),Karel Soudijn (Tilburg University), Ann Taket(South Bank University), Eric Van Damme (Til-burg University), Warren Walker (RAND Cor-poration), and Jim Wilson (North Carolina StateUniversity) for their useful comments on variousversions of this paper.

References

Anderson, R.E., 1992. ACM code of ethics and professional

conduct. Communications of the ACM 35 (5), 94±99.

ASA, 1999. ASA issues ethical guidelines. Amstat News 269,

pp. 9±15.

Bosch, L., Smit, L., Elsendoorn, G., Verhees, P., 1994.

Loonelementen, daglonen en uitkeringslasten. Sociale Ver-

zekeringsraad, Zoetermeer Netherlands.

Caulkins, J.P., 1995. Dealing with the country's drug problem.

OR/MS Today 21 (1), 32±40.

Caywood, T.E., et al., 1971. Guidelines for the practice of

operations research. Operations Research 19, 1123±1148.

Fossett, C.A., Harrison, D., Weintrob, H., Gass, S.I., 1991. An

assessment procedure for simulation models: A case study.

Operations Research 39, 710±723.

Gass, S.I., 1984. Documenting a computer-based model.

Interfaces 14 (3), 84±93.

Gass, S.I., 1994. Model world: Ethics in the not so real MS/OR

world. Interfaces 24 (6), 74±78.

Gitlow, H.S., 1976. A methodology for determining the optimal

design of a free standing abortion clinic. Management

Science 22 (12), 1289±1298.

Hodges, J.S., 1991. Six (or so) things you can do with a bad

model. Operations Research 39 (3), 355±365.

Howard, R.A., 1999. The ethical OR/MS professional. Work-

ing paper, Department of Engineering ± Economic Systems

and Operations Research, School of Engineering, Stanford

University.

Jansen, M., DeVries, B., 1998. Global modelling: Managing

uncertainty, complexity and incomplete information. In:

van Dijkum, C., de Tombe, D., van Kuijk, E. (Eds.),

Validation of Simulation Models. SISWO, Amsterdam.

Kleijnen, J.P.C., 1979. An extension of Gitlow's methodology

for designing abortion clinics. Management Science 25 (4),

408.

Kleijnen, J.P.C., 1995a. Case study: Statistical validation of

simulation models. European Journal of Operational Re-

search 87 (1), 21±34.

Kleijnen, J.P.C., 1995b. Veri®cation and validation of simula-

tion models. European Journal of Operational Research 82

(1), 145±162.

Kleijnen, J.P.C., 1999. Validation of models: Statistical tech-

niques and data availability. In: Farrington, P.A., Nemb-

hard, H.B., Sturrock, D.T., Evans, G.W. (Eds.),

Proceedings of the 1999 Winter Simulation Conference,

pp. 647±654.

Kleijnen, J.P.C., Gaury, E., 1999. Short-term robustness of pull

production management systems. Working Paper (preprint-

ed as CentER Discussion Paper, no. 9870).

Kleijnen, J.P.C., Rens, P.J., 1978. IMPACT revisited: A critical

analysis of IBM's inventory package IMPACT. Production

and Inventory Management Journal of the American

Production and Inventory Control Society 19 (1), 71±90.

Kleijnen, J.P.C., Helton, J., 1999. Statistical analyses of scatter

plots to identify important factors in large-scale simulations.

(In two parts) Reliability Engineering and Systems Safety,

65, no. 2, August 1999, pp. 147±185, 187±197 (Also

published as Sandia Report SAND98-2202, April 1999).

K�obben and Tromp, 1999. De onwelkome boodschap±Of hoe

de vrijheid van wetenschap bedreigd wordt. Mets, Nether-

lands.

Kucßuradi, I. (Ed.), 1999. The Ethics of the Professions:

Medicine Business, Media, Law. Springer, Berlin.

Lane, D.C., Oiva, R., 1998. The greater whole: Towards

a synthesis of system dynamics and soft systems method-

ology. European Journal of Operational Research 107, 214±

235.

Lawrence, J., 1999. Argument for Action: Ethics and Profes-

sional Conduct. Ashgate, Abingdon, United Kingdom.

Maclagan, P., 1998. Management and Morality: A Develop-

ment Perspective. Sage, London.

Mentzel, M.A., Kohnstamm, G.A., Becker, H.A. (Eds.), 1995.

Ethische vragen bij sociaal-wetenschappelijk onderzoek.

Van Gorcum, Assen, Netherlands.

Pritsker, A.A., 1998a. Life and death decisions. OR/MS Today

25 (4), 22±28.

Pritsker, A.A., 1998b. Life and death decisions. OR/MS Today

25 (5), 8±9.

Pritsker, A.A., 1999. Life and death decisions. OR/MS Today

26 (3), 16±18.

Rosenhead, J. (Ed.), 1989. Rational Analysis for a Problematic

World: Problem Structuring Methods for Complexity,

Uncertainty and Con¯ict. Wiley, Chichester.

Samuelson, D.A., 1999. Simple ethics? OR/MS Today 26 (3),

14.

Taket, A., 1994a. Undercover agency?±Ethics responsibility

and the practice of OR. Journal of the Operational

Research Society 45 (2), 123±132.

Taket, A., 1994b. Undercover agency? ± Ethics responsibility

and the practice of OR. Journal of the Operational

Research Society 45 (8), 965±969.

Van Meel, J.W., 1993. The dynamics of business engineering:

Re¯ections on two case studies within the Amsterdam

municipal police force. Ph. D. dissertation, Technical

University Delft.


Wallace, W.A. (Ed.), 1994. Ethics in Modeling. Pergamon,

Oxford.

Whittaker, D., 1999. Spreadsheet errors and techniques for

®nding them. Management Accounting 77 (9), 505±551.

Wilson, J.R., 1997. Doctoral colloquium keynote address:

Conduct, misconduct, and cargo cult science. In: Andra-

dottir, S., Healy, K.J., Withers, D.H., Nelson, B.L. (Eds.),

Proceedings of the 1997 Winter Simulation Conference, pp.

1405±1413.

Wolfe, J., Fritzsche, D.J., 1998. Teaching business ethics with

management and marketing games. Simulation and Gaming

29 (1), 44.

Zeigler, B., 1976. Theory of Modelling and Simulation. Wiley-

Interscience, New York.


Documents

Ethical issues in modeling: Some reflections