Creating engaging courseware using system dynamics

Pergamon Computers in Human Behavior, Vol. 13, No. 2, pp. 127-155, 1997

Published by Elsevier Science Ltd Printed in Great Britain

0747-5632/97 $17.00 + 0.00

PII: S0747-5632(96)00003-4

Creating Engaging Courseware Using System Dynamics

J. Michael Spector

Instructional Systems Research, Armstrong Laboratory

I. Davidsen

Department of Information Science, University of Bergen

Abstract - - There is much discussion in the instructional technology literature concerning the importance for engaging courseware, especially in contrast to page-turning courseware. While we believe that there is a useful place for simple, page-turning courseware (e.g., in tutorials accompanying software products, for overview introductions to a topic, etc.), we agree that for more sophisticated and complex learning situations the key to a successful learning environment is the degree to which learners are cognitively engaged with the subject matter. System dynamics has been shown to be an effective tool in managing (representing, modeling, and comprehending) the complexities of domains that involve complex structures, especially those characterized by feedback loops; delays, and uncertainty (Forrester, 1961, 1985, 1992; Senge, 1990). In this paper, we shall suggest a framework for using system dynamic tools and technologies as the basis for constructing highly engaging learning environments. Published by Elsevier Science Ltd

Requests for reprints should be addressed to J. Michael Spector, Instructional Systems Research, Armstrong Laboratory, 7406 Slippery Elm Street, San Antonio, TX 78240, USA.

127

128 Spector and Davidsen

This paper is an outgrowth of work initiated by the Grimstad Group (1995a,b,c). 1 An earlier version of this paper was posted on ITFORUM for discussion in the instructional technology community in October 1995. We have incorporated many of the ideas that emerged from that discussion into this version of the paper. More specifically, we have attempted to capture the sense of the ITFORUM discussion in the last two sections of this paper. One of the themes of this section of Computers in Human Behavior is that public forums on the Internet such as ITFORUM and PSYGAME can provide for a very fluid and timely exchange of ideas. Such exchanges may eventually serve to advance exploration and understanding in a variety of domains.

We have two primary goals in this paper: (a) to provide an appreciation for the system dynamics (SD) perspective of the relationships between structure and behavior in complex systems; and (b) to provide a basis for the use of SD as a basic building block in the construction of highly engaging, interactive learning environments. In order to accomplish these goals, we shall first provide a brief overview of SD. Then we will be able to indicate some possibilities for the use of SD in the creation of effective courseware. The possibilities will include sample building blocks for engaging courseware.

We shall make a number of assumptions with regard to the nature of reality, learning, learning environments, learning effectiveness, and instruction. We do not wish to defend these assumptions in the course of this discussion, although we do realize that many or all of these assumptions may be questioned. Our aim is to provoke a discussion about what we believe to be a very powerful way to design learning environments.

For the purposes of this discussion, learning refers to intentional learning and involves persisting, stable, and observable changes in an individual. Learning environments can be described in terms of the following attributes: (a) a variety of actors/agents (e.g., learners, instructors, computer-mediated delivery mechanisms, etc.); (b) appropriate and reasonably well-defined roles for those actors (not necessarily fixed or predetermined); (c) a setting in which learning occurs (this may involve multiple settings as in distance learning situations or variable settings as in laboratory-based courses); (d) a knowledge base pertinent to a subject matter area, possibly with links to other knowledge bases (e.g., a collection of learning materials or an instructional database); (e) learning goals, objectives, and desired outcomes (these may be negotiable); and (f) a set of possible relationships and interactions among the actors, roles, knowledge base, and settings which facilitate or contribute to accomplishment of goals and objectives. Learning effectiveness refers to the efficacy of an environment in the attainment of those goals. Instructional design (ID) refers to a structuring of the learning environment for the purpose of facilitating learning or improving learning effectiveness. We shall further assume that there are external realities,

Creating engaging courseware 129

meaningfully described in terms of causal relationships, which often form the subject matter for many learning situations (cf. Simon, 1982 and Towne, 1995).

SYSTEM DYNAMICS AND LEARNING ENVIRONMENTS

First, we shall provide an overview of a generic SD perspective (Forrester, 1961, 1985, 1992; Richmond, 1993; Sterman, 1988, in press). SD suggests that a complex system can be described in terms of stocks (containers) of things or substances, the quantity of which may change over time. Such quantities are called stock levels. The concept "thing" is very broad and covers, in this context, items such as commodities in an inventory, money in an account, animals in a population, molecules in a container, persons infected by a virus, tasks to be accomplished, levels of understanding, and even mental states. "Substance" refers to a single item of continuous nature such as a population, a liquid or a gas often considered to be an aggregate of things in which single items are considered indistinguishable (for the purpose of the model). In SD, stocks may be visually represented as containers (usually rectangular boxes, Figure 1).

Stocks can be distinguished in two different ways. On the one hand, we distinguish stocks of different kinds of things. In this case, stocks represent a categorization of things (e.g., commodities, money, animals, molecules, or persons). On the other hand, we distinguish stocks of the same kind of things in different states, viz., a categorization of states that characterize things. For instance, commodities exist in different states along a production and delivery chain, or understanding exists in different states as, for example, stages of maturity or expertise. Each of these states or stages may be represented by a stock and the amount of commodities in that state or understanding at that stage by the level of the stock.

System dynamicists often distinguish between real, perceived, and desired levels of stocks (Morecroft & Sterman, 1994; Senge, 1990; Sterman, in press). Frequently, the emphasis in a dynamic system is placed on the perceived and desired levels, as it is often people's perceptions and goals that give rise to dynamic behavior. These distinctions make it possible to create representations of and speak meaningfully about mental models. We shall suggest that

Model Model boundary boundary

Stock Inflow_rate Outflow_rate

Figure 1. Generic system dynamics model.


a particularly powerful aspect of SD is that it provides learners with tools and techniques that allow them to represent and explore the implications of their own mental models.

The levels of all stocks in a system at a particular point in time constitute the state of the system at that point in time. As already suggested, we may refer not only to material but also to mental states by stock levels. We may represent a level of aggression, a level of frustration, and so on. In such cases, it is extremely important to agree upon an operational definition underlying the term, although the definition may involve some fuzziness or uncertainty. The state is but one of two fundamental aspects of dynamic systems; the other one being the change of state. Since the state is represented by stock levels, the change of state may be conveniently represented by flows, and the rates of change by flow rates. Flows accumulate in stocks, and the rate at which a stock level changes is determined by the net rate of the flows accumulating in the stock. If a stock level is characterized in terms of units, the associated rates of flows are all characterized in terms of units per time.

For instance, a population (stock) is increased by an inflow of individuals being born and (simultaneously) decreased by an outflow of individuals dying per year. We call these the birth rate and the death rate, respectively. It is often possible to specify rates which at least partially determine how fast the things or substances in a stock increase or decrease in quantity, such as order and sales rates, production and delivery rates, birth and mortality rates, infection rates, and so on. These rates can be thought of as the magnitudes of flows into or out of a stock and are usually depicted as arrows into or out of a rectangular box with a control valve to represent the notion of a variable flow rate (Figure 1), as in a faucet controlling the flow of water into a bath tub or water basin - - the faucet represents a variable flow rate, in the familiar sense that opening the valve more allows more water to flow into the basin (i.e., creates a higher flow rate). The tub or basin can be thought of as the stock in this case, and the drain might be thought of as a flow out of the stock. The drain also has an associated flow whose rate depends largely on the amount of water in the basin and the size of the drain pipe.

Figure 1 depicts the basic components of an SD model: stocks and flows. Notice that there are boundaries to the model represented by clouds. These boundaries represent things (stocks and flows) outside the model which are not explicitly modeled. It is important to realize that there are always limits to what is being represented, and making explicit one's assumptions about those limits and their implications for the model is a significant aspect of a learning process. We say this because testing one's assumptions is an important learning activity in our estimation. Recall that stocks need not be thought of as discrete items - - system dynamicists are often interested in modeling desires, expectations, perceptions, and so on. Finally, since the


state of a model is determined by the levels of the stocks at a point in time, it is fair to say that system dynamicists tend to view natural processes as accumulation or integration processes.

In addition to stocks and flows, in understanding complex systems we also need to take into account a variety of variables which ultimately influence the flow rates associated with the stocks in the system. For example, resistance to drought may be thought of as a variable which may influence the mortality rate of a particular species in an ecosystem, or perceived wealth may be a factor which may be thought of as influencing the investment rate in an economic model. In SD, variables represent causal relationships and are depicted as circles with links or curved arrows to indicate the lines of influence. The totality of all the causal relationships among stocks, flow rates, and variables in a system is called the structure of the system.

At this point, we want to make an important comment. We have only introduced a small portion of the basics of SD, but what we have discussed has a very strong visual component (rectangles, circles, arrows, etc.). We have not discussed the notions of delay, nonlinearity, and uncertainty which are central notions in SD - - indeed, it is these notions which distinguish SD from other modeling and simulation techniques.

In addition to constructing stock-and-flow diagrams to represent complex systems, system dynamicists often construct causal loop diagrams to make explicit causal relationships among the various components of a system (Figure 4). Of particular interest in such causal loop diagrams are positive and negative feedback loops. An example of a positive feedback loop might consist of a savings account (a stock of money), an earnings rate on our savings (the rate at which that stock of money increases, partly determined by what might be a constant interest rate and partly by the balance in the savings account), and a savings factor (which represents how much of our total earnings we can redirect back into savings). If we make a number of simplifying assumptions (such as zero inflation, constant other earnings, constant expenditures, etc.), we might say that as the earnings rate on savings increases, our savings increases, which has a positive influence on total earnings, which has a positive influence on how much can be diverted back to savings, which means that the earnings on savings will continue to increase.

Figure 2 depicts a simplified version of just such a system dominated by a positive growth loop. The interest rate in this example is a constant (indicated by the diamond shape) and happens to be quite generous (1% per month - - recall that we are writing from Norway!). The savings outflow has been set to zero, as is the investment factor constant; this reflects the simplifying assumption that savings are never spent or drained. As a result, the attached graph indicates that the quantity of our savings experiences a period of exponential growth. The real value of this example is that it reflects how a


simple SD model may be used to make one's mental model of a complex situation explicit.

We might all like to believe that our savings could enjoy such sustained growth, but we know that there are moderating or balancing influences which make it very unlikely that we shall enjoy such periods of expansion in our savings. These moderating influences might be represented as negative feedback loops, causal loops that have one (or an odd number of) negative link(s). For example, as our savings grow, we might spend more on consumable goods (making the assumption about constant expenditures no longer applicable), or we might make more risky investments, some of which will result in a loss of income (making the assumption about constant nonsavings earnings invalid), or we might be taxed at a higher rate (at least in some countries), and so on. In short, these and many other factors have a very real and balancing effect on our earnings.

Figure 3 depicts a slightly modified version of the savings example in which we no longer hold onto the assumption that our savings are never drained or spent. Since we know we are unable to sustain the savings growth depicted in Figure 2, we are in a position to gradually introduce each of the factors which detract from our idealized savings growth. In Figure 3 we have chosen to introduce a simple tax paid on our assets in a savings account. The tax rate is based on a table of rates (i.e., it is variable and not constant). The graph that depicts the behavior of the modified system now shows that our savings grow for a while and then taper off (suggesting that the tax table represents a progressive tax).

SD makes it possible to make all of these relationships explicit and then to study their interactions over time. The change in the levels of the stocks over time constitutes the behavior of a system. With complex systems, it is often less than obvious how the structure of the system (the various stocks, flows, and variables) determines the behavior of the system (changes in the levels over time). SD can be thought of as a systematic approach to understanding the complexities of dynamic behavior (e.g., in systems with a dominant

~nvestment_factor < ~

"lav,nJs nteres~_earnings Savings_outflow

Interest_rate

7,000+ ~ e,oo~ /

s0oo÷ / 4,ooot- / ~ z,ooo-F / 1,000 ~ ; i i

'16o 200 Months

I I

"1-" Savings

Figure 2. Simple savings example with positive growth loop.


Savings_assets

ings w

Interest_rate '~',~_J" TaxRate

3,000

5o0 /I . = i = I 1

0 50 100 150 200 Months

-'1- Savings_assets

Figure 3. Savings example with an outflow.

positive or growth loop which, as time passes, becomes less influential, while a negative or balancing loop gradually becomes more influential).

In the process of coming to understand the behavior of complex systems, it is often useful to construct causal loop diagrams, as already mentioned. Figure 4 depicts a very simple causal loop diagram of our savings model, with the tax outflow from savings depicted as a negative loop. This diagram indicates that as the interest rate increases, there is a positive change in the earnings rate which also increases savings (from what they would have been without positive changes in the interest rate and interest earnings). Likewise, the more there is in our savings account, the more we earn in interest (so there is a positive link to interest earnings). The positive link between savings and the tax rate indicates that there is a progressive tax which increases as our savings increase. An increase in the tax rate causes an increase in our taxes, which in this model are being paid out of our savings assets (the outflow from savings in our simple model) - - hence the negative link.

SD also makes it possible to explicitly represent factors outside the system (exogenous variables) which influence system behavior - - some of these factors represent things which might have been empirically established while others are established by consensus or assumption. Since one can never model everything, it is important to make simplifying assumptions and to

• Savings Tax ~ assets + r ,e

~ + / \ Interest Taxes ".J ~ __earnings paid J ~ ~ - q

+) 'nZres* Figure 4. Simple causal loop diagram.


make those assumptions explicit. We believe that the activities of making assumptions explicit, exploring the dynamic implications of assumptions, and examining alternative assumptions are critically important activities in many learning environments.

We began with a monetary example because we thought most readers would be familiar with the concepts of earnings, savings, and interest and, as a consequence, would be able to follow the discussion. This is an example of following Reigeluth's (1983) elaboration principle of using epitomizing examples and Gagnr's (1985) third instructional event of stimulating recall of prior knowledge. However, we expect that most readers are more interested in improving the quality of instruction and learning than in bank accounts. In order to push our discussion of SD into the domain of learning, we will consider a simplified ecosystem as the subject of a learning environment that we want to construct for a group of learners. We might start with a rather vague learning goal, such as understanding how the population of a particular species is effected by a predator species in that environment. In other words, our task is to design a learning environment to facilitate learning about, for example, how a prey population fluctuates in relationship to a predator population. To make this exercise more personally meaningful and motivating, we leave it to readers to supply their favorite prey-predator pairs (e.g., rabbits and cougars, rats and cats, little fish and big fish, etc.). Now, imagine an area where these two species exist in reasonable abundance. It will be especially helpful if this area is near a human population which will develop a preference for more or fewer of one of the species (e.g., rabbits may become a bothersome pest to farmers' crops, rats may begin to spread diseases, there are too few big fish to make for worthwhile fishing excursions, etc.).

In designing courseware about this subject we will have to address the following kinds of issues (Goel & Pirolli, 1989; Nelson, Magliaro, & Sherman, 1988; Rowland, 1992; Spector, 1994; Spector, Polson, & Muraida, 1993; Tennyson, 1994): (a) Learners: Who are the learners? What are they like? What do they like? What do they already know? etc.; (b) Subject matter: What is the subject matter? What kinds of things are to be learned? What links exist to related subjects? etc.; (c) Historical value: Why do people study this subject? Who has studied this subject? What kinds of things have they said? Why? etc.; (d) Learning support: What alternative learning settings exist? How can learning about this subject for the indicated purposes best be supported? What roles can be supported for learners, peer groups, tutors, teachers, computers? etc.; and (e) Learning effectiveness: How can progress in this learning situation be evaluated? How can students learn to be reasonably accurate in self-assessment?


At this point, we can make explicit some of the ways in which we believe that SD can be used to support the creation of effective learning environments. Since space is limited and our purpose is to stimulate a discussion, we shall take a rather abbreviated approach to indicating how SD might be linked to a set of principles for designing effective learning environments. Let us accept for the time being a small set of more or less well-established ID principles (e.g., initiate the learning experience with an epitomizing and motivating example, provide the means to elaborate each aspect of the epitome, provide a variety of learning support mechanisms such as visual representations of key relationships, provide meaningful feedback with regard to learner performance, encourage and support self-assessment, etc.). The problem then arises how best to implement such principles. Our discussion of SD should suggest a strong possibility of supporting such ID principles with interactive models and simulations as the basic building blocks of interactive courseware. We believe that these SD building blocks are especially promising for these reasons, among others: (a) they actively engage learners in the process of making explicit their own mental models of complex systems; (b) they encourage learners to identify assumptions; (c) they provide means for exploring implications of assumptions; and (d) they provide means for examining alternative assumptions (Davidsen, 1993, 1994; Gonzalez & Davidsen, 1995; Grimstad Group, 1995a,b,c). Learners can create and modify SD models, and these models will reflect changes and growth in their knowledge about a subject.

Suppose, for example, that in our imaginary learning environment we want to initiate learning about predator/prey dynamics with an epitomizing and motivating example. We may have a ready-made SD model concerning some particular predator/prey situation that developed counter-intuitively. This model could be presented to learners, identifying key components of the model (not all of the model components and certainly not all of its complexity). Learners might then be asked to predict the behavior of this model over time. Then, we run the simulation based on the underlying model to show how the model actually behaves over time. If the outcomes of the simulation are significantly different from expected outcomes (the learner has already been asked to predict the behavior), and especially if the particular example has some recency or currency in terms of general topics of concern, then we will have implemented the important ID principle of gaining attention using a motivational epitome using SD as the underlying mechanism. The model can then be left in a library of relevant materials that may be revisited later or which learners may decide to explore in further detail at their convenience.

Now, let us tackle the heart of our planned learning experience. Our nominal problem is to understand how the populations of two animal


populations are related with the purpose being to explain and perhaps control the spread of some disease ostensibly spread by one of the animal populations. Students will be asked to construct an initial model of the situation, and the courseware will provide two kinds of support: tools for building the model (such as those available in PowerSim ~ or STELLAr) , and hints for using the tools for the specific purpose at hand; such hints can be provided under learner or system control and might even form the basis for an adaptive interface, automatically adjusting to the perceived level of learner understanding.

Figure 5 is intended to suggest some of the potential for using SD models and simulations in a systematic manner for constructing learning environments. The model in Figure 5 might serve as an early epitomizing example to introduce students to the problem situation. In general, this model shows two populations with associated birth and death rates. The model reflects some simplifying assumptions, such as constant fertility rates and a fixed area for the prey population. The variables in the model can be used to activate learner reasoning about the various relationships that might exist. For example, learners can be asked about the likely relationship between prey density and prey killed per predator (the model contains a causal link and there is a graph symbol indicating that some relationship is being used in this model). Learners might quite reasonably answer that as prey density increases, predators will be more successful in killing more prey which causes an increase in the death rate for the prey. The model now contains a link between prey density and predator mortality and learners might be asked about that link as well. It would not be unreasonable for learners to say that predator mortality is more directly dependent on the prey killed as that reflects their consumption of food. In such a case, learners would be advised

Prey-fertilitY'v ~ ~ Prey_deathrate ~ ~ 70,000 T

Prey_b~_rate( ~prey~opulation~ X " ~ "~ 5°'0~0-0t~ I/ / I t l / I f l ' , ,~r - - ' ~ ' Y " ~ ;;:;;~'~ // 1/ I/ // 1/ / Prey Area~'~ __~Prey_densl~,.~r / ) } >~ 20000.[ V V V V V ~, -1-

- " ~ / / == 1 0 1 0 0 0 / / ~ Z Prey_~.~lality / 0. o l , , ,

Pred ~ "~.~,~Pre~__killed_per_~l~-=d 0 200 400 600

Pred dean r a t e ~ , Pred_fertility

Figure 5. A predator/prey model for instructional purposes.


to make changes to the model and then run the simulation to see the effects. Such changes are extremely easy to make with most SD modeling and simulation tools. In addition, learners could be asked about the indicated fluctuations in the prey population (e.g., the fluctuations in Figure 5 are much too regular and, therefore, unlikely to reflect the actual situation). They might also be asked to make predictions about the predator population based on the indicated fluctuations in the prey population. The simulation could then be run for comparison purposes. Finally, when the model is reasonably robust (believed to reflect something close to actual fluctuations and relationships), learners could be asked to devise a hunting or eradication policy with the purpose of limiting the population of one of the species (perhaps to control the spread of a disease). Observing the effects on the other species then becomes a very interesting and meaningful exercise.

This process of building an initial model can be collaborative and staged (consistent with ID principles concerning chunking and graduated complexity) so as to deal with the following questions: (a) What factors contribute to the level or state of each animal population at any given time (e.g., birth and mortality rates)?; (b) What is a reasonable estimate of the specific influence (e.g, each individual animal lives 77 years)?; (c) What simplifying assumptions are reasonable with regard to factors within the model (e.g., the predators become less effective as the prey population decreases)?; (d) What simplifying assumptions are reasonable to make with regard to outside influences (e.g., no more than three natural disasters per work day - - similar to an ordinary university working environment)?; and (e) What can be done to defend these initial assumptions (e.g., consult other references). The value of SD in this staging process (the process of constructing an explicit model of a complex situation) is that it requires learners to make assumptions explicit, to simplify the problem initially, to consider evidence for initial biases, and so on (Gonzalez & Davidsen, 1995; Gonzalez & Vavik, 1994; Grimstad Group, 1995a,b,c).

Next, let us assume that learners have progressed to a point where they have constructed reasonable models (i.e., models which, when run over a period of time, appear to learners to produce approximations of actual fluctuations in the pertinent animal populations). Our learning goal involved the possibility of introducing a new factor into the model - - a policy aimed at controlling one of these populations so as to moderate or eliminate the spread of a disease, for example. Learners are now in a position to modify their models by introducing additional factors of influence to see how they might effect the behavior of the model over time. This can be thought of as a process of hypothesis formulation and testing (as could the model construction process), and it should now be obvious that SD provides very


powerful support for this process (e.g., learners can now interact with models they have created and observe consequences).

ITFORUM DISCUSSION

There were a number of very insightful responses to our paper posted by I T F O R U M participants. There is simply not space to permit reference to and inclusion of all those remarks, and we ask that those who are not referenced not to infer that their comments were not helpful. Suffice it to say that they were all helpful in focusing our understanding on where and to what extent SD might be useful in constructing learning environments. Researchers excited by their own area of interest are perhaps inclined to think that only their approaches and solutions are interesting, with the consequence that exaggerations too easily occur. The overall significance of the I T F O R U M discussion for us was that it reminded us that there are in fact serious limitations with regard to when and where it would be appropriate to implement the ideas expressed earlier in this paper.

There are a number of differences between Internet postings and papers in refereed journals, and one significant difference pertains to standards required for references. Because Internet fora are intended to encourage the exchange of ideas with minimal delays, there are virtually no imposed standards for references. Quite often, one finds that an Internet reference to a World Wide Web page is in error or that the address has changed. Frequently, Internet references to published papers are incomplete as the author is sending a response from his workstation and may not have the complete reference at hand. Fortunately, we did receive several complete references to published work of relevance to this paper (Hannafin & Hooper, 1993; Mellar, Bliss, Boohan, Ogborn, & Tompsett , 1994) - - thanks to Chet Hedden and Martyn Wild for these contributions. In addition, we t h a n k Steve Alessi for pos t i ng the W o r l d Wide Web address (http://www.hps-inc.com) for obtaining information on Stella-II, an SD modeling tool.

The responses contained many substantive ideas in addition to the references to published works and other information available on the World Wide Web. In the remainder of this section, we shall focus on those ideas introduced by Chet Hedden, Steve Alessi, and Rob Foshay, so as to provide a sense for the substance of the I T F O R U M discussion (again our apologies to others who made valuable contributions). In keeping with the less formal nature of Internet discussions, we have chosen to include the first names of the discussants and to refer to them primarily by their first names.


Comments by Chet Hedden

Chet Hedden (University of Washington) asked whether bits of knowledge could be represented as stocks in an SD model. It is perhaps risky to suggest that knowledge is accumulated and can be modeled as a level (amount) in a container (learner's mind) as such a view sounds overly simplistic and static. Suggesting a diversification into a number of containers representing various kinds of knowledge only avoids the issue. In some cases, however, the accumulation of knowledge and its interpretation can be reasonably modeled this way. For example, various containers can be used to accumulate various kinds of knowledge (procedural vs declarative, context-specific vs generic, etc.). Such models could be used to establish a more efficient flow for a particular kind of task in a particular organization. The mix of knowledge of different kinds required to perform a task might be characterized by the weights assigned to the different levels in the calculation of processing efficiency. The learning value of this kind of model would relate to its value in understanding inefficiencies in particular organizational structures.

Clearly, we face a challenge in the operationalization and representation of knowledge. We are inclined to believe that this operationalization must result from a careful investigation of the context at hand. In the modeling of mental models and processes using SD, what is usually done is to replicate a model that represents a real system in a simplified form so as to capture bounded rationality. Elaboration of this notion is beyond the scope of this paper, but we can say that this idea has been used to simulate a model of a complex system and its mental representation in interactions with information acquisition, policy design and selection, and the decision-making cycle.

Chet suggested that we should search for a completely new approach with learning outcomes that are more observable than the well-established, but not demonstrably effective, principles currently in use. Chet then indicated that one of our SD-based ID principles was nothing more than an instance of a corrective feedback strategy (Hannafin & Hooper, 1993). We agree that our example of an SD-based ID principle (using an unexpected simulation result to direct the learner's progress) is an instance of the principle of corrective feedback. We had Gagn6's (1985) seventh instructional event in mind and should be severely reprimanded for not making that reference more explicit (the editor of this journal would certainly have never allowed our remark to pass without proper reference!). We see absolutely no conflict in using SD to provide a more detailed elaboration of generic or macro-level design principles and in saying that such principles are SD-based. This returns us to Chet's first comment, viz., that we should consider an entirely new set of principles. We partly agree and we partly disagree on this issue. In general, we believe in building on (elaborating in more useful detail, as in the previous


example) established principles rather than starting completely anew. The latter approach would require a new foundation (philosophy of society, person, and mind; epistemology; learning theory; design methodology; etc.), an enterprise far beyond what we feel that SD has to offer that might be useful to the instructional community.

However, when it comes to understanding the relationship between structure and behavior, we agree with Chet that quite new principles must be developed and utilized. This understanding is not primarily reflected in an ability to treat a nonlinear feedback model characterized by uncertainties and delays as a mathematical model, although such understanding is often useful. Understanding dynamic systems is partly reflected in an individual's ability to explain the structural origin of a problematic behavior and to effectively suggest structural modifications that robustly correct that behavior. Such understanding is clearly a constructive enterprise that often proceeds slowly and in relatively unpredictable jumps.

Chet wanted to know more about the instructional building blocks of insight that we might try to establish in the minds of our students and that we expect them to utilize when faced with systems and models of a more complex nature - - a very probing query. One such building block to understanding dynamic systems is a generic structure (a number of these have been identified in the SD literature). These can be introduced relatively informally at first. Subsequently, these generic structures are generalized, interpreted, and applied to a number of additional domains (Gagnr's ninth event, transfer of learning, is relevant here).

Other pieces of the SD building block set are the two kinds of diagrams for portraying SD models in educational contexts - - the stock-and-flow diagrams and the causal loop diagrams. Those who recognize the significance of understanding the relationship between structure and behavior in dynamic systems need the same information which is required to simulate the behavior of a system. That is, we need the information embedded in the stock-and- flow diagram. A feedback-loop diagram is information poor since we do not distinguish between the state of the system, represented by the containers, and the change in that state, represented by the flows. On the other hand, we recognize the need for simplified summaries of the structures that dominate the behavior of the model at any point in time, For this purpose, causal loop diagrams are excellent. In any case, whether we work with one or the other form of representation, we need to integrate a description of the behavior in a representation of the underlying structure (applying the notion of graduated complexity in terms of the SD tool kit is a worthwhile enterprise in this regard - - thanks for your help on this, Chet).

Chet also wondered what was counter-intuitive about the particular predator/prey model to which we were referred (the unexpected simulation


result already referenced earlier). A number of simple models exhibit a behavior that most people find difficult to comprehend at first. Here are some examples related to predator/prey models. When not in equilibrium, such systems typically oscillate. First, the prey population grows increasingly as a result of a low number of predators. Then, because of the increased food supply, the predator population starts growing, reducing the rate at which the prey population grows, and, eventually, causing it to diminish. This reduction in food supply (prey) reduces the growth of the predator population and eventually causes that population to diminish so as to set the stage for another increase of the prey population. A typical task for students would be to come up with a policy that dampens the oscillations. Here are three such policies associated with the prey: (a) reduce the fertility of the prey (somehow), (b) hunt prey at a rate proportional to the size of the population, or (c) set a target for the prey population and hunt the surplus prey at a rate proportional to that surplus.

As it turns out, the first two policies are extremely nonrobust (rarely alleviate the oscillations). They bring about the desired dampened oscillatory behavior only when initiated in a very narrow window of opportunity within the prey-cycle. Otherwise, their effects are counter-productive in the sense that the oscillations are amplified. This is an unexpected (counter-intuitive) result arising from the fact that the associated dynamics of the predator population is not well understood. It is the reaction of the predator population that causes this amplification. Although the third policy is very similar to the second, it is completely robust and succeeds in stabilizing the system in ecological equilibrium. Again, the result is counter-intuitive in the sense that this equilibrium does not change even if we vary the target prey population size. The reason is, again, that the predator population counteracts our efforts to change the size of the prey population. All we obtain by hunting more intensively, is to shift the burden of regulating the prey population from the predators to ourselves. This can easily be seen when we give up hunting. If our goal has been in achieving the natural equilibrium, then the predator population is in equilibrium and the system remains stable.

One of Chet's most serious challenges concerned the question of whether values are implicitly or explicitly built into a model. It is quite clear that a model can include human values and associated aspirations, targets, or goals. By choosing different sets of values, we will be able to represent a variety of perspectives or attitudes. Indeed, one appropriate application of SD-based learning environments is in those domains in which values (e.g., in the form of government policies) can be explicitly modeled and the consequent behavior observed, thus linking policy design and decision making to behaviorally observable outcomes - - one of Chet's earlier concerns.


Chet sees the value of a learning environment which supports hypothesis testing and provides the example of a game-based simulation in which animated animals are roaming around in the woods near a town; some are sick, and some are predators eating some prey. In such cases, a static model, such as a spreadsheet might indeed be appropriate, as Chet suggested. An SD representation is extremely helpful when we attempt to draw fruitful conclusions from hypothesis testing beyond a binary conclusion that the hypothesis failed or passed the test. When we want to know why our hypothesis failed and how the hypothesis could be improved, it becomes very useful to have a dynamic model.

Comments by Steve Alessi

Steve Alessi (University of Iowa) has taught SD and reported that SD is not easy to teach, with the implication that this might restrict its use as an underlying instructional methodology. We have experienced similar difficul- ties (one author of this paper is a well known system dynamicist who has spent countless hours trying to shine the SD light into the dimmer reaches of the other author's graying matter, without much success). Moreover, we believe that SD-based learning environments are not always and perhaps not often appropriate. As we have said several times, however, when the system to be understood involves complex behaviors (delays, feedback loops, nonlinearities, etc.), then SD-based methodologies become quite useful.

The next logical question was also raised by Steve: When the domain is appropriately complex, to what extent should students become involved with the SD modeling process? We have argued for an approach that is learner dependent. As learners demonstrate understanding at one level (perhaps making use of a causal loop diagram in a presentation aimed at stating the problem in a predator/prey situation), then they are ready to be introduced to more of the SD. The next step (or a prior step) might be simple exposure to the results of a simulation based on an underlying SD model, followed by questions and perhaps simple hypothesis testing. Steve rightly sees this as a continuum, and we believe that effective SD learning environments will treat the use of the SD-based simulations as a continuum. Unfortunately, existing SD learning environments often operate at only one or two locations on this continuum, typically a very simplistic and not very interactive causal-loop representation or an extremely complex and interactive stock-and-flow model/simulation.

Steve asked if it was wise to suggest such a complex methodology as a basic instructional strategy we should all use in our teaching. Clearly the methodology is complex, but so is much of our world (in spite of what politicians would like us to believe). Complexity characterizes many real


systems, which is why we try to make instructional use of the SD methodology. Moreover, our view is that we are surrounded by dynamic systems, and a number of disciplines (e.g., economics, politics, physics) demonstrate shortcomings when addressing dynamic problems. This clearly illustrates the need for improved public understanding of dynamic phenomena. SD can serve as a methodological element in a number of subjects, in particular in the natural and social sciences. Many dynamic phenomena can be addressed using SD. It is our assessment that few dynamic phenomena are adequately taught and learned in colleges and universities, precisely because teachers are not trained to introduce these phenomena and students are not trained in using adequate tools. In short, we are cautiously optimistic, as is Steve, that there is a future for SD-based learning environments.

Comments by Rob Foshay

Rob Foshay (TRO Learning, Inc.) pointed out that SD-based simulations might be very helpful in a learning environment when they are directed at acquisition of procedural knowledge, but that they would probably be extremely inefficient in supporting the acquisition of declarative knowledge. Rob also reiterated Steve's comment that SD is quite difficult to teach.

Rob also introduced the constructivism debate into the discussion. He argued that constructivist-motivated simulations (of the kind we have described earlier) are not only ill suited to support acquisition of declarative knowledge, but they may fail to facilitate acquisition of more complex procedural knowledge unless the variety of pertinent mental models are better elaborated and more explicitly supported in the learning environment. He further suggests that failure to understand how learners are creating and articulating mental models as they progress through a curriculum was the cause for the failure of the inquiry-based curriculum in the 1960s.

We did not want to take part in the debate on constructivism during the course of the ITFORUM discussion and tried to avoid the topic as we thought that it would serve to detract from the focus of the discussion (one author of this paper failed to follow the simple principle of avoiding unnecessary distractions in the subsequent ITFORUM discussion and regrets this, with apologies to both Rod Sims and Tom Reeves). We are generally in agreement with Rob's concerns on this issue, however.

We do recognize the inefficiencies of "pure" constructivism and our utilization of generic models for teaching and learning constitutes one of our serious attempts to enable our students to structure their learning and get a head start in understanding complex subject matter. In addition, we are inclined to agree that the traditional SD tools provide only a glimpse of a


much richer reality. There is a need to demonstrate what the components of a model actually represent. Using references (electronic links) from the SD model to multimedia presentations of that reality is one way to alleviate that problem. This should not substitute for student interaction with the real world (as opposed to interactions with models of the real world). When time and resources allow, it can be most instructive to investigate an actual system and collect source material that can be used along with the model and with simulation results. To complete this last sequence idea, we also agree with a discussant in a subsequent ITFORUM discussion who said that the design of effective learning environments should take into account h u m a n - h u m a n (specifically, learner-teacher) interactions as well as human-compute r interactions.

CONCLUSION

The primary thesis in this paper has been to suggest that using SD in a systematic way to implement ID principles is likely to lead to the construction of highly engaging and meaningful learning environments. We do not claim that SD is appropriate for all learning situations, but it has been tried in some selected situations within the SD community and in secondary education settings, and the results are generally very promising (Davidsen, 1994, in press; Mandinach & Cline, 1994; Morecroft & Sterman, 1994).

Towne (1995) has made a very compelling argument that effective simulation-based learning environments are constructed with instructional strategies built into the simulation engine so that as simulations are created they have inherent instructional value. The learning environments described in this paper comply with Towne's recommendations and should be regarded as an extension of the principle of tight coupling of simulations with instruction in complex domains well suited for modeling using the techniques of SD.

In closing, we are very grateful to Lloyd Rieber, the ITFORUM moderator, and his assistants at the University of Georgia, for inviting us to initiate this discussion on ITFORUM. As we have already indicated, we are especially grateful to the respondents for having caused us to think more carefully about when and how SD-based learning environments might be effective.

NOTE

~The Grimstad Group consists of P. I. Davidsen (University of Bergen), J. J. Gonzalez (Agder College of Engineering), D. J. Muraida (Armstrong Laboratory), J. M. Spector (Armstrong Laboratory and University of Bergen), and R. D. Tennyson (University of Minnesota).


REFERENCES

Anderson, J. R. (1995). Learning and memory: An integrated approach. New York: Wiley. Davidsen, P. I. (1993). System dynamics as a platform for educational software production. In

B. Z. Barta, J. Eccleston, & R. Hambusch (Eds.), Computer mediated education of information technology professionals and advanced end-users (pp. 27-40). Amsterdam: North Holland.

Davidsen, P. I. (1994). The systems dynamics approach to computer-based management learning environments: Implications and their implementations in Powersim. In J. D. W. Morecroft & J. D. Sterman (Eds.), Modeling for learning organizations (pp. 301-316). Portland, OR: Productivity Press.

Davidsen, P. I. (in press). Educational features of the system dynamics approach to modelling and simulation. Journal of Structural Learning.

Forrester, J. W. (1961). Industrial dynamics. Cambridge, MA: MIT Press. Forrester, J. W. (1985). "The" model versus a modeling "process". System Dynamics Review,

1(1), 133-134. Forrester, J. W. (1992). Policies, decision, and information sources for modeling. European

Journal of Operational Research, 59(1), 42-63. Gagnr, R. M. (1985). The conditions of learning (4th ed.). New York: Holt, Rinehart, and

Winston. Goel, V., & Pirolli, P. (1989). Motivating the notion of generic design within information

processing: The design space problem. AI Magazine, 10(1), 18-36. Gonzalez, J. J., & Davidsen, P. I. (1995). Integrating systems thinking and instructional

science. In R. D. Tennyson (Ed.), Automating instructional design: Computer-based development and delivery tools (pp. 355-376). Berlin: Springer-Verlag.

Gonzalez, J. J., & Vavik, L. (1994). Experiences and prospects derived from the Norwegian R&D project in automation of instructional design. In R. D. Tennyson (Ed.), Automating instructional design, development, and delivery (pp. 79-92). Berlin: Springer-Verlag.

Grimstad Group. (I 995a). Employment of system dynamics in modeling instructional design (ISD4). In R. D. Tennyson (Ed.), Automating instructional design." Computer-based development and delivery tools (pp. 603-609). Berlin: Springer-Verlag. [The Grimstad Group consists of P. I. Davidsen, J. J. Gonzalez, D. J. Muraida, J. M. Spector, & R. D. Tennyson.]

Grimstad Group. (1995b). Using system dynamics to model courseware development: The project dynamics of complex problem solving. In K. M. George, J. H. Carrol, E. Deaton, D. Oppenheim, & J. Hightower (Eds.), Proceedings of the 1995 ACM Symposium on Applied Computing (pp. 32-35). New York: ACM Press.

Grimstad Group. (1995c). Applying system dynamics to courseware development. Computers in Human Behavior, 11, 325-339

Hannafin, M. J., & Hooper, S. R. (1993). Learning principles. In M. Fleming & W. H. Levie (Eds.), Instructional message design: Principles from the behavioral and cognitive sciences (2nd ed., pp. 191-231). Englewood Cliffs, NJ: Educational Technology Publications.

Mandinach, E. B., & Cline, H. F. (1994). Classroom dynamics: implementing a technology- based learning environment. Hillsdale, NJ: Erlbaum.

Mellar, H., Bliss, J., Boohan, R., Ogborn, J., & Tompsett, C. (Eds.). (1994). Learning with artificial worlds: Computer based modelling in the curriculum. London: Farmer Press.

Morecroft, J. D. W., & Sterman, J. D. (Eds.). (1994). Modeling for learning organizations. Portland, OR: Productivity Press.

Nelson, W. A., Magliaro, S., & Sherman, T. M. (1988). The intellectual content of instructional design. Journal of Instructional Development, 11(I), 29-35.

Reigeluth, C. M. (Ed.). (1983). Instructional-design theories and models: An overview of their current status. Hillsdale, NJ: Erlbaum.

Richmond, B. (1993). Systems thinking: Critical thinking for the 1990s and beyond. System Dynamics Review, 9(2), 113-134.


Rowland, G. (1992). What do instructional designers actually do? An initial investigation of expert practice. Performance Improvement Quarterly, 5(2), 65-86.

Senge, P. (1990). The fifth discipline: The art and practice of the learning organization. New York: Doubleday.

Simon, H. A. (1982). Models of bounded rationality. Cambridge, MA: The MIT Press. Spector, J. M, (1994). Integrating instructional science, learning theory and technology, In

R. D. Tennyson (Ed.), Automating instructional design, development, and delivery (pp. 523-546). Berlin: Springer-Verlag.

Spector, J. M., Poison, M. C., & Muraida, D. J. (Eds.). (1993). Automating instructional design: Concepts and issues. Englewood Cliffs, NJ: Educational Technology.

Sterman, J. D. (1988). People express management flight simulator. Cambridge, MA: Sloan School of Management.

Sterman, J. D. (in press). Learning in and about complex systems. System Dynamics Review. Tennyson, R. D. (1994). Knowledge base for automated instructional system development. In

R. D. Tennyson (Ed.), Automating instructional design, development, and delivery (pp. 1 - 11). Berlin: Springer-Verlag.

Towne, D. (1995). Learning and instruction in simulation environments. Englewood Cliffs, NJ: Educational Technology.

APPENDIX A. SAMPLE EXTRACTS OF THE ITFORUM DISCUSSION

The following are verbatim extracts of some of the ITFORUM discussion mentioned in the main course of the paper. They are included here to reflect the nature and substance of the discussion that followed the presentation of the main body of our paper.

Chet Hedden's Comments

A most interesting approach to thinking about ID! I'll keep my reactions brief and relevant to specific portions of the text:

Mike and Paal wrote: "For the purposes of this discussion, "learning" refers to intentional learning and involves persisting, stable, and observable changes in an individual."

Comment: Are these changes equivalent to acquisition of knowledge? Shouldn't learning involve some knowledge acquisition? Don' t numbers 4, 5, and 6 below imply this?

Mike and Paal wrote: "Learning environments can be described in terms of the following attributes ... (4) a knowledge base pertinent to a subject matter area, possibly with links to other knowledge bases (e.g., a collection of learning materials or an instructional database); (5) learning goals, objectives, and desired outcomes (these may be negotiable); and (6) a set of possible relationships and interactions among the actors, roles, knowledge base, and settings which facilitate or contribute to accomplishment of goals and objectives."

Comment: These are nice distinctions, and I shall steal them, if I may!


Mike and Paal wrote: "SD suggests that a complex system can be described in terms of stocks (containers) of things or substances, the quantity of which may change over time."

Comment: Are "bits of knowledge" stocks, too? How do you divide up bits of knowledge into different "containers"?

Mike and Paal wrote: "The change in the levels of the stocks over time constitutes the behavior of a system."

Comment: May we assume, then, that an increase in the "level" (quantity?) of the knowledge stock (bits of it) indicates an increase in learning?

Mike and Paal wrote: "We might start with a rather vague learning goal, such as understanding how the population of a particular species is effected by a predator species in that environment. In other words, our task is to design a learning environment to facilitate learning about, for example, how a prey population fluctuates in relationship to a predator population."

Comment: Far from vague, this learning goal addresses a specific quantitative measure: the relative sizes of two populations. The behavior of the system can be described by the "principle" of inflow and outflow you mention.

Mike and Paal wrote: "To make this exercise more personally meaningful and motivating, we leave it to readers to supply their favorite prey/predator pairs (e.g., rabbits and cougars, rats and cats, little fish and big fish, etc.)."

Comment: More than personally meaningful, is not the choice of species in fact the question of greater interest and value? Environmental policy making should not dismiss the differences between species and their unique patterns of interaction on the basis of personal favorites. The principle to be learned in this simplified scenario remains that of inflow and outflow.

Mike and Paal wrote: "Let's accept for the time being a small set of more or less well-established ID principles ... The problem then arises how best to implement such principles."

Comment: Is this really "the problem"? Shouldn't you be looking for a completely new approach with, hopefully, greater observable learning outcomes than those well-established but not demonstrably effective principles have thus far yielded?

Mike and Paal wrote: "Learners can create and modify SD models, and these models will reflect changes and growth in their knowledge about a subject."

Comment: Yes! But how will you divide up and measure the bits of this knowledge "stock" as called for by the SD model?

Mike and Paal wrote: "Suppose, for example, that in our imaginary learning environment we want to initiate learning about predator/prey dynamics with an epitomizing and motivating example."

Comment: Why do you think this example is motivating?


Mike and Paal wrote: "We may have a ready made SD model concerning some particular predator/prey situation that developed counter-intuitively."

Comment: Why counter-intuitively? (And what do you mean?) Mike and Paal wrote: "This model could be presented to learners,

identifying key components of the model . . ." Comment: How, exactly, would it be presented? Mike and Paal wrote: "Learners might then be asked to predict the

behavior of this model over time." Comment: Yes... Mike and Paal wrote: "Then, we run the simulation based on the

underlying model to show how the model actually behaves over time." Comment: Okay. Mike and Paal wrote: "I f the outcomes of the simulation are significantly

different from expected outcomes (the learner has already been asked to predict the behavior) ... then we will have implemented an important ID principle using SD as a methodology."

Comment: No. Your student has simply eliminated one hypothesis, and you have given him/her the correct solution in one step. This is nothing more than the "corrective feedback strategy" (Hannafin & Hooper, 1993, p. 221).

Mike and Paal wrote: "Students will be asked to construct an initial model of the situation, and the courseware will provide two kinds of support: tools for building the model ... and hints for using the tools for the specific purpose at hand . . . . "

Comment: I do believe that this is a good design strategy. It is similar to the ID model I use with adventure games.

Mike and Paal wrote: "Our learning goal involved the possibility of introducing a new factor into the model - - a policy aimed at controlling one of these populations so as to moderate or eliminate the spread of a disease, for example."

Comment: But you might want to know why you have this pohcy! For example, the disease may be an important "balancing" component of a natural environment - - like forest fires, not all of which should be extinguished. You might decide *not* to control the population or moderate the disease. On what basis, in this example, would you decide? The learning goal you have selected results only in a quantitative analysis of several variables affecting relative population sizes over time. But how would you teach the more complex *implications* of this quantitative analysis that depend on subjects making explicit their own values vis-a-vis species preference and other factors - - whether intuitive or counter-intuitive? Are there "value stocks" too?


Mike and Paal wrote: "This can be thought of as a process of hypothesis formulation and testing (as could the model construction process), and it should now be obvious that SD provides very powerful support for this process (e.g., learners can now interact with models they have created and observe consequences)."

Comment: What kind of simulation are you describing here? Will you have animations of animals roaming around in the woods near the town, some sick and some eating the others? Wouldn't a spreadsheet work just as well for this type of analysis and hypothesis testing? If so, why do you need to explain it in system dynamics terms?

Looking forward to your reply! Chet Hedden

Hannafin, M. J., & Hooper, S. R. (1993). Learning principles. In M. Fleming & W. H. Levie (Eds.), Instructional message design: Principles from the behavioral and cognitive sciences, (2nd ed.). Englewood Cliffs, NJ: Educational Technology Publications.

Rob Foshay's Comments

The increased emphasis on teaching and learning of higher-order thinking skills, coupled with the fresh insights from advances in cognitive theory, constructivist epistemology, and the technologies of computer-based simulation have created many promising new avenues for advancement of the science and technology of instructional design. In this context, it is difficult not to be infected by the enthusiasm of the authors for the potential of System Dynamics (SD) as a tool for instructional designers and for learners. At the same time, the authors' careful attempts to moderate their claims about the potential of SD by giving attention to its limitations as well as its strengths is exemplary - - and unusual - - in a discussion of this sort. As practitioners who must face daily the full complexity of real-world problems of teaching and learning, instructional designers do well to remember the limitations as well as the strengths of their tools. Only by doing so can we resist the long-demonstrated tendency of much of the education and training communities to succumb to the seductions and excesses of the "Next Big Thing" in theory and practice, while recognizing the value of advancements in our tools and underlying theories.

The intent of my comments on ITFORUM was to assist in the task of placing SD in the context of other tools available to the instructional designer and their underlying theories, and thus to help designers appreciate SD's true value to them and their learners. Perhaps it is best to start by asking (1) what

15o Spector and Davidsen

definition we should use of higher order thinking skills, and (2) what analytical tools and prescriptive principles of learning environment design are needed to teach these skills. Then we can see the role that SD can play in this task.

Implicit in my comments was the differentiation of declarative and procedural knowledge, consistent with cognitive learning models such as J. Anderson's ACT model (Anderson, 1995). It is perhaps reasonable within this framework to at least approximately equate the terms "procedural knowledge", "problem solving", and much of "higher order thinking skills". Also implied in this view, it seems to me, is the central importance of the mental model as the key to the synthesis of declarative knowledge into procedural knowledge in the way that Anderson's model describes. A full discussion of these assertions is well beyond the scope of this reply, so it is offered here only to help the reader recognize (and either accept or reject) a relevant portion of the frame of reference from which my comments were made.

If the reader accepts this view, then it seems logical that learning environments intended to foster development of higher order thinking skills should somehow address the need to acquire declarative knowledge and mental models as pre- or co-requisite with the need to synthesize procedural knowledge. In this view, it is possible to criticize the "inquiry" movement of the 1960s for its failure to give appropriate weight to all three kinds of learning. A "pure" constructivist approach to instruction can be criticized for the same limitation. In both cases, a common result may be inefficiency in acquiring the declarative knowledge, and perhaps the mental model components of knowledge, due to excessive reliance on problem solving, reflection and inference as the preferred strategies for teaching and learning.

In this context, SD appears to be most useful as a tool for analysis of mental modeling. It seems to combine some of the strengths of other more familiar methods, such as procedural flow charting and system state tables, by capturing more of a system's complexity. This very strength, however, could also be one source of the difficulty in teaching SD to ID students or other learners in any but its simplest forms. As the authors point out, the SD approach to limiting the level of detail by which the system is represented could be very helpful here.

Another strength of SD may also serve as a limitation for learners. As acknowledged by the authors in the ITFORUM discussion, the abstract nature of system representations using SD may make it difficult for learners to make the connections between their experience (whether in simulations or reality) and the system structure as represented by SD. As the authors point out, a computer-based instructional simulation which provides a simulta- neous view of and dialog on the simulated system and its (SD?) structure,


could be very helpful in supporting teaching and learning of mental models and procedural knowledge.

In conclusion, it also may be appropriate to comment on the process by which this exchange of views was developed. In effect, the balance of seriousness of purpose, permissiveness and spontaneity of the ITFORUM discussion reflected in the paper and its associated comments has had the effect in a few weeks of eliciting variety of questions for further inquiry which might have taken months to emerge by the conventional means of theoretical discussion in published papers and conferences. If the result is acceleration of the growth of the "stock of knowledge" (reference intended) for ID, then we all benefit.

Reference Anderson, J. R. (1995). Learning and memory: An integrated approach.

New York: Wiley.

Steve Alessi's Comments

Since I haven't seen any reactions to Spector & Davidsen's article yet, I'll jump in with some comments, and probably become shark-bait as a result.

I have been enthusiastic about the Systems Dynamics approach ever since reading Nancy Roberts' (and several other authors) book "Introduction to Computer Simulation: A System Dynamics Modeling Approach". I teach SD and Stella in my graduate course on simulation design, and students use Stella to do modeling and then use those models inside other authoring systems to create instructional simulations. Of course, the Roberts book was really about using modeling as a primary instructional strategy for teaching thinking and problem solving (not for designing CAI simulations) and I have always found their arguments for that to be very convincing (much more so than the arguments for using Logo to teach thinking and problem solving). I especially liked the argument by Roberts and her co-authors that our national (and local) policy makers look for simple answers, trying to change complex systems by manipulating a single variable (e.g. more taxes to "fix" the deficit or more prisons to "fix" crime). Invariably their simple fixes change the complex systems in unintended ways, often with the exact opposite outcome they wanted. If only more of those policy makers had studied Systems Dynamics ... well, you get the point.

However, the approach has, sadly, never caught on (as Logo, unfortunately in my opinion, did for a while). There are probably a number of reasons. As Bill Barowy (at BBN) points out in his work, kids have a lot of trouble understanding the concept of modeling. It is very difficult for teachers to teach. It is not in the school curriculum. And, to be honest, I 'm


not sure we have evidence yet that it works. Mike and Pal referenced Mandinach & Cline (1994), Davidsen (in press) and Morecroft & Sterman (1994) claiming they show "very promising" results. I have only read Mandinach & Cline. Their book gives a very good description of designing and implementing a Systems Dynamics based curriculum in schools, but by their own claim (Chapter 6) they had not yet begun research on effectiveness at the time of that book. I hope their continued research will show positive effects, but I would not reference that book as current evidence of any. Do the other two references (Davidsen or Morecroft & Sterman) do a better job of supporting the claim of very promising results? Since Mike and Pal have read these, I would be eager for them to say a little more about that.

But even if positive results are there, the thing Mike and Pal do not say much about is how very difficult it is to teach systems dynamics. All the books and articles I have read (e.g. Roberts et al., Mandinach & Cline, Barowy et al.) say this. Students have trouble distinguishing stocks and flows. Every science teacher experiences this problem when he or she sees the difficulty students have distinguishing quantities versus rates of change. My own (graduate) students handle those fairly well, but have lots of trouble with time delays and with what to include within and outside of the system boundary. Well, I'll dispense with the SD tech talk. Suffice to say it may be a very worthwhile method of thinking and analyzing things but it's also a very complex method. Many teachers have tried it and given up. Is it wise to suggest such a complex methodology as a basic instructional strategy we should all use in our teaching?

Another difficulty I had while reading the article was whether they were suggesting SD as an instructional design strategy or as a teaching strategy. My difficulty was cleared up by the end, but it was at its worst in the middle of page 6 (pages according to my printout of the longer HTTP version) where they say, "we expect that most readers are more interested in improving the quality of instruction and learning than in bank accounts". Here and elsewhere (including in the title) I had an expectation that the authors were going to use SD as an ID strategy. After all, the economics (bank account) example is just as good if we were teaching a business or home accounting lesson, as is the predator/prey example if we are teaching ecology. I was lead to believe that later in the article Mike and Pal would show causal loop diagrams of the instructional design process, or of the factors affecting student learning in an SD-based learning environment. But it soon became clear that they were using SD as an instructional strategy. I don't have a problem with that. I 'm just pointing out that for a while I was led to believe something different was in store.

Something the authors do not deal with much in the article, but which I am most interested in, is the two ends of the "simulation use continuum", both


of which are possible with the SD methodology. On one end of the continuum you can use SD to create CAI simulations which students will learn from by using the simulation (perhaps in a directed mode, perhaps in a more discovery or research mode, but that's another continuum). On the other end of the continuum you can have students learn by using SD to do the modeling themselves. Mike and Pal's emphasis seems to be on the latter end of the continuum. But they did discuss how students might begin with a simple and already developed model, study it, modify it, and then move on to the more creative mode of developing new models. How to design that kind of learning sequence (when to use, when to create) is what I find most interesting.

Lest anyone misinterpret me, I am enthusiastic about the approach that Mike and Pal are suggesting. My suggestions to them (for publication purposes) would be to beef up the support for SD effectiveness (if they can find it), and remove the ambiguity about whether they are suggesting SD as a design strategy or an instructional strategy. My suggestions to others interested in simulation design is to pursue the very interesting question of designing along the "simulation use continuum" and what research needs to be done to give designers advice in that regard.

A few more thoughts on the Simulation Dynamics article: An interesting point which I think probably will begin to recur in many ID

contexts is the distinction between an instructional design strategy and a design strategy. In their response Mike and Pal indicted their agreement that it was probably SD and an instructional strategy more than as an instructional design strategy. But in other (private) comments, Mike made the good point that if SD is appropriate for organizing and representing the content domain, it is in part an ID strategy, or at least the distinction between ID-strategy and I-strategy becomes fuzzy. This is a good point and the one which I think we will see recurring in other discussions. SD is less an instructional strategy (like feedback, or questioning) and more a content creation and organizing tool. Other content creation and organizing tools, like expert systems will probably present the same issue. What are the implications of such distinctions? I suspect the implications are for metacognition (of the students) and the way we design cognitive strategies as a part of learning environments. My suspicion (though I have not thought about it much) is that the purer instructional strategies are not as good prospects for the cognitive strategy portion of a complete learning environment, while those tools that are somewhere between ID-strategy and I-strategy are better candidates.

In their reply to some of Chet's comments there was discussion of whether individuals ought to be the unit of analysis, in contrast to aggregates of individuals. I think it was being discussed with regards to business


simulations. This goes to the heart of the distinction between continuous simulations and discrete simulations. Continuous simulations are solved with differential equations while discrete simulation with statistical and prob- ability functions. But which one is appropriate is not always clear cut. Take the example of population dynamics, human or otherwise. Obviously, populations are discrete (we don't have halves of people in reality). But population dynamics simulation are almost always dealt with as continuous simulations and with calculus rather than statistics for the underlying math models. Why? Because the numbers are generally very large (millions or billions of people) and the time period for discrete change (births) is tiny (second) in comparison with the time frames real population change occurs over (decades). So it makes more sense to treat most population simulation as continuous, that is, the unit being counted is millions of people, which is pretty continuous. Now, in a large business simulation with the overall time frame being years, this will probably be true as well. But if you are doing a small business simulation over a short time frame (months) it might be reasonable to model the simulation as a discrete simulation. It all depends on the number of units, the time increment size, and the overall time frame size.

Last, just a point of useful information. High Performance Systems (the makers of Stella) have just instituted a World Wide Web site at

http://www.hps-inc.com/

Thanks to Mike and Pal for their article. I would encourage others to look into SD more. Though a somewhat difficult topic, I think it's worth the effort.

Sorry, everyone. I misread directions and did not realize we were supposed to "ponder" the Systems Dynamics article for a week before responding.

So in the meantime I'll add my 2 cents about the interactivity issue raised by Rod. Actually, I would like to relate it to the Systems Dynamics paper. The same issue that Tom Reeves discusses (whether we should create interactive multimedia for student consumption, versus students should create their own interactive multimedia in order to "learn by teaching") is central to the Systems Dynamic paper because Mike and Pal distinguish between learning from a simulation and creating ones own simulation. In my response I stressed that this is a continuum and that different points are appropriate for different students, different times, etc. Such a continuum applies to simulations, expert systems, interactive multimedia, videos, books, you name it.

Tom takes the position that eventually one end of the continuum will be proven the "better" one, namely, the end where students do the creating. A very constructivist point of view, of course. While we would probably all


agree that research will determine who is right, I rather suspect that there is no "better" end of the continuum. Our mistake in the past was always favoring the end with students as consumers. I think it would be equally bad to go to the other extreme. A good curricula probably should strive to move students from the consumption end to the creation end, the rate being a function of student, content, and environmental variables (e.g., how hard the content is and how motivated the students are).

There are some smaller but important auxiliary points to make. I think some other responders might have already said some of these in slightly different fashions. One is efficiency. Starting students out on the creation end of the continuum might be both frustrating and inefficient. On the other hand, spending too much time on the consumption end of the continuum might impede higher order learning and transfer. A sequence of moving students from the consumption end to the creation end of the continuum might (and I believe will) be a good compromise between efficiency and outcomes.

Another auxiliary point is accuracy. The fact that you allow a student to create something (a simulation, an expert system, or a multimedia composition) does not guarantee that what they create is at all accurate or true (assuming you believe in some objective truth). Are all student created simulations or compositions to be considered equally good? If they create something very incorrect, and if they learn it very well by virtue of their having created it, won't they have [been] learning false information very well! I think we have to deal with this potential problem. And I believe that sequencing instruction along the "consumption-creation" continuum might be a possible solution.

Steve Alessi, The University of Iowa

Documents

Creating engaging courseware using system dynamics