This article was downloaded by: [University of Auckland Library]On: 05 December 2014, At: 00:26Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Theoretical Issues in ErgonomicsSciencePublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/ttie20

Leveraging intelligence for highperformance in complex dynamicsystems requires balanced goalsF. Elga Graduate School for Human Machine Interaction , CognitiveEngineering, IDA, Linköpings Universitet , 58183 Linköping,Sweden E-mail:Published online: 23 Feb 2007.

To cite this article: F. Elg (2005) Leveraging intelligence for high performance in complexdynamic systems requires balanced goals, Theoretical Issues in Ergonomics Science, 6:1, 63-72,DOI: 10.1080/14639220512331311571

To link to this article: http://dx.doi.org/10.1080/14639220512331311571

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoeveras to the accuracy, completeness, or suitability for any purpose of the Content. Anyopinions and views expressed in this publication are the opinions and views of theauthors, and are not the views of or endorsed by Taylor & Francis. The accuracyof the Content should not be relied upon and should be independently verifiedwith primary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages, and otherliabilities whatsoever or howsoever caused arising directly or indirectly in connectionwith, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden.Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Leveraging intelligence for high performance in complex dynamic

systems requires balanced goals

F. ELG*

Graduate School for Human Machine Interaction, Cognitive Engineering,IDA, Linkopings Universitet, 58183 Linkoping, Sweden

Psychometric intelligence is weakly correlated with control performance inmicro-worlds. Previous attempts at explaining these low correlations have focusedon reliability problems of micro-worlds and/or the need for skills and capabilitiesnot captured in static problem-solving tests. Meta cognitive factors are hypo-thesized to influence the efficacy of problem-solving capability in managing amicro-world control task, thus explaining some of the low correlations betweenperformance and psychometric intelligence. Specifically, goal level, implicitly thecontrol problem difficulty, is proposed to interact with psychometric intelligencein determining control performance. Forty-six participants managed the micro-world Moro under opaque test conditions. Three-way ANOVA analysis shows aninteraction between goal level and intelligence (Raven’s APM) for several perfor-mance variables over time, indicating the importance of meta-cognitive processesfor leveraging psychometric intelligence.

Keywords: Psychometric intelligence; Micro-world; Meta-cognition problemsolving; Control performance; Goal-setting.

1. Introduction

Intelligence is expected to have an important influence on problem-solving and theestablishment of appropriate processes for achieving and maintaining control ofcomplex dynamic systems (e.g. Gottfredson 1997). Experimental decision-makingenvironments called micro-worlds have been used to explore the relationshipbetween individual characteristics, such as intelligence and behaviour of decision-makers and system characteristics of such control problems. These micro-worlds aretypically complex, dynamic and opaque (Brehmer 1992):

1. Complex, from multiple inter-connected variables;2. Dynamic, from tightly coupled interactions between these variables; and3. Opaque, from little task structuring and limited availability of information.

The resulting task has multiple inter-related goals and evolves over time, auton-omously and as a consequence of decisions. The task is furthermore complicated bydelays, balancing and reinforcing effects from actions taken and changes in theimpact of decisions, where knowledge must be developed in real time by hypothesistesting.

Theoretical Issues in Ergonomics ScienceVol. 6, No. 1, January–February 2005, 63–72

Theoretical Issues in Ergonomics ScienceISSN 1463–922X print/ISSN 1464–536X online # 2005 Taylor & Francis Ltd

http://www.tandf.co.uk/journalsDOI 10.1080/14639220512331311571

*Email: freel@gmx.net

Dow

nloa

ded

by [

Uni

vers

ity o

f A

uckl

and

Lib

rary

] at

00:

26 0

5 D

ecem

ber

2014

Results from research based on micro-worlds, however, indicate a low cor-

relation between problem-solving capability, usually measured by psychometric

intelligence tests such as Raven’s advanced progressive matrices and problem-solving

performance in these micro-worlds (Putz-Osterloh and Luer 1981, Dorner et al. 1983,

Staudel 1987, Strohschneider 1991). A review of these can be found in Kluwe et al.

(1991).

These results, thus, provide little evidence of a positive influence of problem-

solving capability and explain a very small proportion of problem-solving perfor-

mance. Four hypotheses have been offered to explain these low correlations (Rigas

2000).

1. Other demands. Complex dynamic systems require additional capabilities

beyond problem-solving capability (Dorner 1986, Putz-Osterloh 1993).

2. Low test-reliability. Low reliability in measuring performance in micro-

worlds, due to the large number of strategies that can generate the same result

and the random effects generated by micro-world opacity, making it difficult

to differentiate between results (Buchner 1995, Funke 1995).

3. Task novelty. Micro-world decision situations are completely new decision-

making environments where problem-solving in these contexts is of little

importance for determining outcome (Raaheim 1988).

4. Design reliability, or the complexity continuum hypothesis (Snow et al. 1984).

The degree of complexity and problem variability in a problem-solving test is

higher, whereas the micro-world provides a much smaller number of measures

(few scenarios).

These hypotheses have been thoroughly reviewed in recent research (Rigas

2000) where all four received support, helping to explain the low correlation

between problem-solving in complex dynamic micro-worlds and psychometric test

intelligence.

However, all four of the hypotheses described above build on some assumption

about the nature of the test situation as an explanation of low correlations. The basis

is assumed either to depend on additional problem-solving demands not captured in

traditional problem solving tests, or an insufficient number of variations of the task.

Specific aspects of the interaction between decision-maker and micro-world are,

thus, not considered as keys for explaining the low correlations. In fact, none of the

hypotheses above provide insights on or quantification of what specific aspect of the

interaction with a micro-world that offset the effect of problem-solving capability on

performance.

Looking at the process of dynamic decision-making, one can develop some

insight into possible intervening processes: The successful establishment of a process

for control of a complex dynamic system requires continuous new learning about a

partly unknown and opaque dynamic problem-solving environment. By comparing

how dynamic decision-making is differentiated from traditional static problem solv-

ing, one can provide some insight on process requirements for control of complex,

dynamic and opaque micro-worlds (Brehmer 1992):

1. Real time. Decisions must be made in real time, the problem will not wait for

decision-maker input

2. Multiple decisions are required per task. Several decisions or interventions are

required over time.

64 F. Elg

Dow

nloa

ded

by [

Uni

vers

ity o

f A

uckl

and

Lib

rary

] at

00:

26 0

5 D

ecem

ber

2014

3. Decisions are interdependent. Decisions and interventions are not independentfrom each other, but previous decisions influence the conditions for laterdecisions

4. Problems are changing. The decision-making situation changes, both as aconsequence of endogenously driven dynamics and as a consequence of deci-sions made.

The control problem, thus, requires the decision-maker to not only rapidly learnabout the decision-making situation, but requires the decision-maker to establishrobust decision-making policies while learning about the system to control.

Under these conditions, there is a conflict between an effective allocation ofexisting knowledge for reaching goals and the need to acquire new knowledgethat, in the longer term, is required for ensuring reliable and high performance.The fundamental requirements of this process have been described by Dorner andWearing (1995) as a general recursive synthetic constellation amplification process(GRASCAM).

The challenge for decision-makers is, however, not only in establishing the rightprocess or methods for building control and knowledge, but also in developing theright hypotheses. The opacity of micro-worlds makes it difficult to identify priorities,the complexity makes for multiple alternative hypotheses and dynamics makeshypothesis testing difficult. Add the difficulties people have in correctly identifyingcomplex dynamic systems (Sterman 1989a, b, Senge and Sterman 1992, Paich andSterman 1993) and integration of flows over time into stocks (Booth Sweeny andSterman 2000) and it is not difficult to understand the high degree of failure that wasnoted in early studies of dynamic decision-making using complex dynamic opaquemicro-worlds.

In this context, clearly it can be no drawback to have a high problem-solvingability. Indeed, recent research suggests stable but low correlations between psycho-metric intelligence and decision-making performance in micro-worlds (Rigas 2000).However, a high problem-solving capability may co-vary or interact with otheraspects of decision-makers and there can be circumstances where decision-makerswith a high psychometric intelligence is not performing better than decision-makerswith a lower intelligence. Specifically, one can expect decision-makers with highgoals in complex dynamic systems to increase their exposure to risk and, therefore,their likelihood for failure. In these cases, one cannot expect the higher problem-solving capability of a decision-maker to compensate for the increased level ofdifficulty in managing a task away from failure.

Correspondingly, in a study based on a micro-world characterized by a highdegree of complexity, dynamics and opacity, one can expect participants with highproblem-solving capability in combinationwith amoderate goal level to be less subjectto failure than participants with low problem-solving capabilities and high goals.

2. Hypothesis

One can expect a positive interaction between psychometric intelligence and goal levelin determining control performance on key performance indicators (hypothesis).A high goal in combination with high problem-solving capability is, thus, expectedto lead to better performance than either a high or low goal in combination withlow problem-solving capability or a low goal and high problem-solving capability.

Leveraging intelligence 65

Dow

nloa

ded

by [

Uni

vers

ity o

f A

uckl

and

Lib

rary

] at

00:

26 0

5 D

ecem

ber

2014

3. Method

The micro-world WinMoro, version 7, for Windows (Victor and Brehmer 1993),based on ‘Moro’ version 17.5.88 (Dorner et al. 1986), was used. A total of 46participants were recruited among students of the Universities in Uppsala, Stock-holm and Linkoping. Results from one participant were not used due to discovery ofprevious experience from the Moro micro-world. Participants were compensatedwith two cinema tickets or 150 SEK. Data from 23 of participants were used in apre-study presented at ‘The International conference on system dynamics’, Istanbul,Turkey, 1997 (Rigas and Elg 1997).

Participants were screened for epistemic competence in terms of previous experi-ence from Moro. All interaction between Moro and participants was mediatedthrough an experimenter. Two experimenters were used. After a brief oral intro-duction to the study participants were presented with Raven’s APM set I and II at10 and 40min time restriction, respectively. A comprehensive instruction was thenpresented in written form after which participants were requested to answer anumber of essay questions designed to enable evaluation of participants’ mentalmodels and aspirations (MMQ) and multiple choice questions designed to tap intothe current mood.

Two independent raters assessed goal level and the other MMQ measures fromthe essays (only goal level is used in this study). Goal level is defined as the combina-tion of two goal level components. These two components were, the desire to expandthe Moro population and resource base that can be leveraged for production and,aspiration to conserve financial and non-renewable resources in achieving the desiredgrowth. Goal level was measured based on the essay questions completed prior toimplementation of the first policies in Moro at year 0 in the simulation.

A four-level ordinal scale was used to assess the expansion goal component.Three points were assigned if the participants mentioned an aspiration to increasethe amount of cattle held, an increase in the population, use of TseTse eradicationpolicies and development of a high level of medical standard. Two points wereassigned if some but not all of these expansion-driving policies were articulated.One point was assigned if no clear goals were defined and 0 points were assignedif the goals were deliberately set at a low level, mentioning slow or deliberategrowth of cattle or other resources, where words relating to caution was used askey inclusion criteria.

Similarly, a four-level ordinal scale was used to assess the resource usage goallevel. Three points were assigned if the participant explicitly mentioned the conser-vation of financial and non-renewable resources, e.g. the ground-water reservoir.Two points were assigned if the dynamics of the ground-water level or a generalaspiration of resource conservation was explicitly stated. One point was assigned ifthere were no specific conservation goals stated. Zero points were assigned if theparticipant explicitly identified the ground water reservoir and the capital base forleveraging growth in the Moro region. Participants were then split by the medianvalue of the combined goal levels to generate a low goal level and a high goal levelgroup.

Validity and influence of the study design on behaviour and performance vari-ables was made on a pilot sample of 22 participants. No systematic performanceeffect of the essay or multiple choice questions could be identified compared witha sample from a study without additional tasks beyond the Moro interaction

66 F. Elg

Dow

nloa

ded

by [

Uni

vers

ity o

f A

uckl

and

Lib

rary

] at

00:

26 0

5 D

ecem

ber

2014

(see Rigas and Elg 1997). Questions were repeated at year 4 and at the end of

the interaction with Moro at year 30. Mood questions were also presented at the

occurrence of catastrophes within Moro. The time allocated for interaction with the

Moro system was 3 h, excluding time for instructions, questions, etc. The experiment

duration for each participant was approximately 6 h.

Performance was measured using CRITERIA and individual key performance

indicators within the Moro system. See table 1 for the definitions of criteria groups.

These criteria represent the standard performance measure for Moro, used as proxy

for performance since Moro’s conception with Strohschneider in 1986.

A recent study (Elg 2002) using system dynamics and systems thinking analysis

methodology showed that the key dynamic difficulty in establishing growth in

the Moro system is in managing the tight balance between grazing pastures and

cattle growth, where too many cattle at an early stage triggers over-grazing and

fundamentally limit the ability to manage growth in Moro. CATTLE and GRASS

are, therefore, suggested as continuous proxy variables for performance, allowing

statistical analysis of participant performances.

4. Results

Results on intelligence alone show, as expected, a small but positive correlation

between performance on CRITERIA (see table 1) over the 30 year period APM

score, as well as APM Group (Results on APM divided into two groups of high and

low scores) of 0.21 and 0.06, respectively, but not statistically significant at the 5%

level. When using the continuous performance indicators for CATTLE and available

GRASS, respectively, these results become clearer, with higher positive correlations

identified for the three performance-indicators (see table 2).

Leveraging intelligence 67

Table 1. Criteria for evaluation of participant performance.

Criteria group

Variable 6 5 4 3 2 1

Ground water level 99% 95% 90% 80% — —Capital 750 000 0 �100 000 — — —Starvation deaths per year 0 0 0 0 0 —Grazing pastures (km2) 300 300 300 100 10 —Annual grain harvest (kg) 15 000 15 000 15 000 or 15 000 or — —Number of cattle 5000 5000 5000 5000 — —

Table 2. Correlations between intelligence (APM) and performance indicators.

APM APM high vs low

Cattle, year 0–30 0.31 0.01Grass, year 0–30 0.15 0.19Cattle�Grass, year 0–30 0.20 0.22

Dow

nloa

ded

by [

Uni

vers

ity o

f A

uckl

and

Lib

rary

] at

00:

26 0

5 D

ecem

ber

2014

These results are what can be expected from a single scenario interaction with

a complex dynamic system such as Moro (e.g. Rigas 2000). These results also align

with classic German results (Staudel 1987, Strohschneider 1991).

4.1. Hypothesis testing

According to the hypothesis, an interaction can be expected between goal-level

(GOAL), problem-solving capability (APM) and performance measures, including

criteria group (CRITERIA) and key performance indicators (CATTLE and

GRASS). These performances were measured over five periods, years 0–5, 6–11,

12–18, 19–25 and 26–30.

To test the relationship between psychometric intelligence and goal setting in

influencing performance in control of Moro, a three-way ANOVA analysis was

performed where participants were divided into two groups split by the median

APM value and, similarly, two groups based on the initial combined goal levels

(GOAL).

Results using average criteria group evaluation, CRITERIA, show a high impact

of goal level on performance. Whilst for the group with low goals there is little

impact from problem-solving capability on performance, there is a big impact on

performance for the group with high goals. High goals for the group with low

problem-solving leads to worse performance, whereas high goals for the group

with high problem-solving capability leads to better performance (see figure 1).

A detailed analysis of the interaction between goal level and psychometric

intelligence for the continuous key performance variables CATTLE and GRASS

show similar results with a consistent support in the direction of the hypothesis.

The level of statistical significance is, however, low, p-levels of p<0.093,

p<0.099 and p<0.103, respectively. (See figures 2, 3 and 4.)

68 F. Elg

Low APM

High APMLow Goals

Ave

rage

Cri

teri

a gr

oup

2.5

3

3.5

4

4.5

5

5.5

6

1-5 6-11 12-18 19-24 25-30 1-5 6-11 12-18 19-24 25-30

High Goals

Figure 1. Average performance criteria by APM and interaction over time (criteria groupdata only). Higher criteria group means higher performance.

Dow

nloa

ded

by [

Uni

vers

ity o

f A

uckl

and

Lib

rary

] at

00:

26 0

5 D

ecem

ber

2014

5. Discussion

Intelligence seems to be beneficial in solving complex dynamic problems with a weakbut positive correlation between intelligence and control of complex dynamicsystems. The hypothesis tested in this study elaborates on this weak correlationand tries to explain some of the dynamics of otherwise intelligent participants failingcatastrophically at an early stage.

The proposed answer is novel, providing new depth to the problem of articulat-ing the contributing factors for low correlations between micro-world performance

Leveraging intelligence 69

Low APM

High APMLow Goals

Gra

ss a

vaila

ble

per

year

200000

250000

300000

350000

400000

450000

500000

550000

600000

650000

700000

750000

1-5 6-11 12-18 19-24 25-30

High Goals

1-5 6-11 12-18 19-24 25-30

Figure 3. Average amount of grass available for grazing per year by interaction of APMand goal level at year 0, F(4, 140)¼ 1.94; p<0.106.

Low APM

High APM

Low Goal

Num

ber

of c

attle

1000

2000

3000

4000

5000

6000

7000

8000

1-5 6-11 12-18 19-24 25-30

High Goal

1-5 6-11 12-18 19-24 25-30

Figure 2. Average number of cattle by interaction of APM and goal level at year 0,F(4, 140)¼ 1.99; p<0.099.

Dow

nloa

ded

by [

Uni

vers

ity o

f A

uckl

and

Lib

rary

] at

00:

26 0

5 D

ecem

ber

2014

and traditional psychometric intelligence measures such as APM. Furthermore, the

tested hypothesis is intuitively appealing in that it provides support for ‘grandmother

rules’ such as ‘one who reaches for a lot often loses all’.

Specifically, it could be shown how high goals among decision-making with low

problem-solving capability lead to disaster at an early stage in interacting with

Moro, relative to participants with lower goals or higher problem-solving capability.

The analysis of these results shows support for the hypothesis with all three key

performance indicators statistically significant or nearly significant at the 10% level,

despite grouping goal levels and APM performances into just two groups each for

analysis purposes. In fact, this support has a similar magnitude of statistical effects as

the effect of APM on its own.

In summary, the null-hypothesis could not be discarded at the 5% level, which

could indicate that other factors influence performance other than the interaction

between goal level and intelligence. However, given the consistent results across

performance indicators and the expectation of low effect size due to the reliability

provided by the single scenario micro-world context, one can be confident in the

results and should keep the hypothesis that meta cognitive processes interact with

cognitive capabilities in determining decision-making performance in complex

dynamic and opaque decision-making environments such as the Moro micro-world.

Future research should focus on clarifying the interaction between meta cognitive

components and cognitive capabilities in influencing problem-solving/control

behaviour and performances. Replicating the current study with a larger sample

and alleviating some of the difficulties that are generated by the nature of micro-

worlds in terms of reliability by adding several micro-world tasks and/or repeated

scenarios would be useful to facilitate the acceptance of these important findings

in the scientific community. Additionally, the objective of such research should be to

establish clarity on what specific aspects of the interaction between decision-makers

70 F. Elg

Low APM

High APMLow Goals

Inde

x C

attle

*Gra

ss

0

1000

2000

3000

4000

5000

6000

7000

1-5 6-11 12-18 19-24 25-30

High Goals

1-5 6-11 12-18 19-24 25-30

Figure 4. Average combined performance on Cattle�Available Grass (Index) per year byinteraction of APM and goal level at year 0, F(4, 140)¼ 2.03; p<0.093.

Dow

nloa

ded

by [

Uni

vers

ity o

f A

uckl

and

Lib

rary

] at

00:

26 0

5 D

ecem

ber

2014

and complex dynamic systems that generate dysfunctional behaviour and perfor-mances and how to support decision-makers so that catastrophes are prevented fromoccurring.

Also, in this study, the ability of decision-makers to readjust their goal levelsbased on emerging feedback from interacting with a micro-world was not examined.Such a study would constitute an opportunity to understand the difference betweenpeople that manage to capture feedback for realignment of goal levels and those whodon’t, providing valuable guidelines for decision-maker selection and support fortasks where inability to realign goals may cause disaster.

The implication of the findings presented here are profound in that it is moreimportant for decision-makers to develop relevant goal levels than it is important todevelop the right problem-solving skills in interaction with complex dynamicsystems. Training for many real life decision-making tasks need to change dramati-cally.

References

BOOTH SWEENY, L. and STERMAN, J.D., 2000, Bathtub dynamics: initial results of a systemsthinking inventory. System Dynamics Review, 16, 249–286.

BREHMER, B., 1992, Dynamic decision making: human control of complex systems. ActaPsychologica, 81, 211–241.

BUCHNER, A., 1995, Basic topics and approaches to the study of complex problem solving. InComplex Problem Solving, The European Perspective, edited by P.A. Frensch andJ. Funke, pp. 27–63 (Hillsdale, NJ: Lawrence Erlbaum Associates).

DORNER, D., 1986, Diagnostik der operativen Intelligenz [Diagnosis of operative intelligence].Diagnostica, 32, 290–308.

DORNER, D. and WEARING, A.J., 1995, Complex problem solving: toward a (computer simu-lated) theory, In Complex Problem Solving. The European Perspective, edited by P.A.Frensch and J. Funke, pp. 65–102 (Hillsdale, NJ: Lawrence Erlbaum Associates).

DORNER, D., KREUZIG, H.W., REITHER, F. and STAUDEL, T., 1983, Lohhausen, Vom UmgangMit Unbestimmtheit und Komplexitat (Hans Huber: Bern, Switzerland).

DORNER, D., STAUDEL, T. and STROHSCHNEIDER, S., 1986, Moro Programmdokumentation,Memorandum 23 (Universitat Bamberg, Lehrstuhl Psychologie II: Bamberg, Germany).

ELG, F., 2002, Ett Dynamiskt Perspektiv pa Individuella Sillnader av Heuristisk Kompetens,Intelligens, Mentala Modeller, mal och Konfidens i Kontroll av Mikrovarlden Moro(Institute of Technology Linkopings Universitet: Linkoping).

FUNKE, J., 1995, Experimental research on complex problem solving. In Complex ProblemSolving, The European Perspective, edited by P.A. Frensch and J. Funke, pp. 244–268(Hillsdale, NJ: Lawrence Erlbaum Associates).

GOTTFREDSON, L.S., 1997, Why g matters: the complexity of everyday life. Intelligence, 24, 79–132.

KLUWE, R.H., SHILDE, A., FISHER, C. and OELLERER, N., 1991, ProblemloseleistungenbeimUmgang mit komplexen Systemen und Intelligenz (Problemsolving performance wheninteracting with complex systems and intelligence). Diagnostica, 37, 291–313.

PAICH, M. and STERMAN, J.D., 1993, Boom, bust, and failure to learn in experimental markets.Management Science, 39, 1439–1458.

PUTZ-OSTERLOH, W., 1993, Complex problem solving as a diagnostic tool, In PersonnelSelection and Assessment: Individual and Organizational Perspectives, edited byH. Schuler, J.L. Farr and M. Smith, pp. 289–301 (Hillsdale, NJ: Lawrence ErlbaumAssociates).

PUTZ-OSTERLOH, W. and LUER, G., 1981, Uber die Vorhersagbarkeit komplexerProblemloseleistun-gen durch Ergebnisse in einem Intelligenztest [On whether resultsfrom a test of intelligence can predict problem solving performance]. Zeitschrift furExperimentelle und Angewandte Psychologie, 28, 309–334.

Leveraging intelligence 71

Dow

nloa

ded

by [

Uni

vers

ity o

f A

uckl

and

Lib

rary

] at

00:

26 0

5 D

ecem

ber

2014

RAAHEIM, K., 1988, Intelligence and task novelty, In Advances in the Psyhology of HumanIntelligence, edited by R.J. Sternberg, Vol. 4, pp. 73–97 (Hillsdale, NJ: Erlbaum).

RIGAS, G., 2000, On the Relationship Between Psychometric Intelligence and Decision Making inDynamic Systems. 98 (Uppsala, Sweden: Uppsala University).

RIGAS, G. and ELG, F., 1997, Confidence, and performance in a complex dynamic decisionmaking environment. In Proceedings of the International System Dynamics Conference,19–22 August, Istanbul. Available online at: http://ieiris.ef.boun.edu.tr/isdc97/virtuals.html.

SENGE, P. and STERMAN, J.D., 1992, Systems thinking and organizational learning: actinglocally and thinking globally in the organization of the future. European Journal ofOperational Research, 59, 137–160.

SNOW, R.E., KYLLONEN, P.C. and MARSHALEK, B., 1984, The topography of ability and learn-ing correlations, In Advances in the Psychology of Human Intelligence, edited by R.J.Sternberg, Vol. 2, pp. 47–103 (Hillsdale, NJ: Erlbaum).

STAUDEL, T., 1987, Problemlosen, Emotionen und Kompetenz [Problem solving, emotion, andcompetence] (Regensburg, Germany: Roderer).

STERMAN, J.D., 1989a, Modeling managerial behavior: misperceptions of feedback in adynamic decision making experiment. Management Science, 35, 321–339.

STERMAN, J., 1989b, Misperception of feedback in dynamic decision making. OrganizationalBehavior and Human Decision Processes, 43, 301–335.

STROHSCHNEIDER, S., 1991, Problemlosen und Intelligenz: Uber die Effekte derKonkretisierung komplexer Probleme [Problem solving and intelligence: the effects ofproblem concreteness]. Diagnostica, 37, 353–371.

VICTOR, T. and BREHMER, B., 1993, WinMoro ver. 7 (Department of Psychology, UppsalaUniversity: Uppsala).

About the author

Fredrik Elg received a college degree in Industrial and Mechanical Engineering fromForsmarks Skola, Sweden, in 1991. In 1996, he received the MSc degree in Psychology atUppsala University, in 2001 a MSc in Business management from Stockholm School ofBusiness and in 2002 the PhD Lic degree (Fil Lic) in Cognitive Engineering at LinkopingInstitute of Technology. He has been a visiting Scholar at MIT Sloan School of Management,IIM Calcutta and IFI Bergen. He is currently completing his PhD in Cognitive Ergonomics,focusing on modelling, analysis and prediction of human control behaviour and performancein complex dynamic systems. Additionally, he is working as a strategy consultant, helpingleading businesses articulate sustainable strategies in an increasingly complex, dynamic andconsumer driven business environment. He is a member of the ACM, IEEE, HFES, theMarketing Society, the Market research society and the International System DynamicsSociety.

72 F. Elg

Dow

nloa

ded

by [

Uni

vers

ity o

f A

uckl

and

Lib

rary

] at

00:

26 0

5 D

ecem

ber

2014