Modeling the Role of Frontal Lobes in Sequential Task … levine... · 2021. 1. 12. · 1994, Dr. Bapi will be at the School of Computing, University of Plymouth, Devon PL4 8AA, Plymouth,

1994

Pergamon

SPECIAL ISSUE 0893-6080(94)E0041-I

Neural Networks, Vol. 7, Nos. 6/7, pp. 1167-1180, 1994 Copyright © 1994 Elsevier Science Lid Printed in the USA. All rights reserved

0893-6080/94 $6.00 + .00

Modeling the Role of Frontal Lobes in Sequential Task Performance. I. Basic Structure and Primacy Effects

RAJU S. BAPI AND DANIEL S. LEVINE

Department of Mathematics, University of Texas at Arlington

(Received 7 October 1993; revised and accepted 15 February 1994)

Abstract--An avalanche model o f motor sequence encoding is presented. The ultimate aim is to reproduce data showing that monkeys with frontal lobe damage can learn an invariant sequence o f movements i f it is rewarded, but cannot learn to perform an), one o f several variations o f a sequence (f all are rewarded. In this article, we present simulations o f the primary learning o f sequences, and demonstrate parameters that can lead to a primacy effect in recalling items o f this sequence from long-term memory. Two different versions o.f the avalanche network and their simulations are presented, o f which the second version includes a layer o f sequence detectors. Suggestions are made .for including yet another layer to classify these temporal sequences and group them together based on reward. Analogies are drawn between the classifier layer and the frontal lobes, and between the avalanche module and part of the basal ganglia.

Keywords--Frontal lobes, Frontostriatal interactions, Sequences, Externally ordered sequences, Internally ordered sequences, Spatiotemporal patterns, Avalanche model, Pattern classification.

1. INTRODUCTION

The frontal lobes have been implicated in forming strategies for goal-directed behavior (Fuster, 1989; Nauta, 1971; Stuss & Benson, 1986). This general function seems to involve coordination of subsystems that integrate motivational and cognitive information (e.g., Milner, 1964; Pribram, 1961 ) with other subsystems that link past events or actions across time (Brody & Pribram, 1978; Fuster, 1985, 1989; P in to-Hamuy & Linck, 1965 ) and anticipate future events or actions (Gevins et al., 1987; Ingvar, 1985).

The motivational-cognitive linkages involving the frontal lobes have previously been simulated in neural networks by Leven and Levine (1987), Levine and Parks (1992), Levine and Prueitt (1989), and Parks and Levine (1994). The network architectures used in those simulations were based on principles such as adaptive resonance (Carpenter & Grossberg, 1987) and opponent processing (Grossberg, 1972).

In this article, we look at an example of linking events across time. Specifically, we discuss networks that model data of Brody and Pribram (1978) and

Requests for reprints should be sent to Professor Daniel S. Levine, Department of Mathematics, University of Texas at Arlington. 411 S. Nedderman Drive, Arlington, TX 76019-0408. After August 15, 1994, Dr. Bapi will be at the School of Computing, University of Plymouth, Devon PL4 8AA, Plymouth, United Kingdom.

Pinto-Hamuy and Linck (1965) on the performance of movement sequences by frontally damaged monkeys. These data, described in the Experimental Data Sec- tion, show that intact frontal lobes are not necessary for primary learning of invariant sequences, but are necessary for learning sequences whose order is flexible. This is a prime example of the importance of the prefrontal cortex for complex behavioral rule formation, which is a necessary component of planning. Hence, frontally damaged people or animals tend to be dis- tractible and unable to stick to plans (e.g., Grueninger & Pribram, 1969; Wilkins, Shallice, & McCarthy, 1987).

In recent years, there has been a wealth of cognitive and behavioral data on frontal lobe lesions (for sum- maries, see Fuster, 1989; Levin, Eisenberg, & Benton, 1991; Perecman, 1987; Stuss & Benson, 1986), but little on the performance of sequences by lesioned animals, the focus of this article's simulations. There has been some neuropsychological data on sequential learning by frontally lesioned human patients (e.g., Karnath, Wallesch, & Zimmermann, 1991; Petrides & Milner, 1982; Shallice, 1982, 1988 ); these more recent results also support the general notion that the greater the degree of flexibility required in the response, the more pronounced is the involvement of the prefrontal cortex.

Our neural network construction will incorporate these differences among types of sequence learning. This

1167

1168 R. S. Bapi and D. S. Levine

article will introduce an architecture, based on an extension of the avalanche (Grossberg, 1969, 1970, 1978, 1986), for learning and encoding invariant sequences. This architecture will not include representations of flexible sequences, but will be designed with the extension to flexible sequences in mind; that extension will appear in a subsequent article. Hence, the network will not actually include analogs of connections involving the prefrontal cortex until the later companion article.

The function of learning goal-directed sequences involves classification of spatiotemporal patterns. A neural network learning spatiotemporal patterns has to deal with spatial aspects of the pattern, the ordering of the constituent items, time lags between successive items, and the duration of each item. So, a general spatiotemporal pattern processing neural network (STPPNN) stores, recognizes, and recalls patterns dealing with the above aspects successfully. In the next section, we review previous work on STPPNNs.

2. BACKGROUND

Grossberg ( 1969, 1970) developed the outstar avalanche for spatiotemporal pattern storage and performance. A space-time pattern is sliced into a sequence ofn spatial patterns separated by equal time intervals. Each of the spatial patterns is stored in an outstar Oi and the signal from the source node .x-i of O~ arrives at xi+] with a delay. So, given an input signal at xl, the entire pattern is reproduced sequentially from outstars O~, 02, 03, . . . . O,. Grossberg ( 1978, 1986) discussed various issues in serial learning and goal-directed plan formation, the answers to which suggested modifications for the basic outstar avalanche. Our model is inspired by this discussion and extends the architecture suggested therein.

Grossberg ( 1978, pp. 265-269) modified the basic avalanche architecture to make sequence performance less ritualistic and more sensitive to external feedback.

AROUSAL

F (2)

F (3)

F (1) % "ll IDxl÷l,1 x 1+2,1

INPUT FIGURE 1. Avalanche network for sequence generation. (Adapted from Grossberg, 1978, with the permission of Academic Press.)

Frontal Lobes and Sequences 1169

The resulting network is shown in Figure 1. Each one of a sequence of movements has three representations, and these are stored in layers ~71, ~72, and 5 t 3. During performance of one of the movements, its representation is kept active via reverberation between 5tl and ~73. The 5~ 2 representation of that movement inhibits the reverberation in the same column once the movement is performed, and excites the representation of the next movement via a learnable connection. The arousal node controls starting and stopping of the sequence based on external feedback, and also regulates the speed of performance. Our network will build on this basic model, extending it to include a sequence detector layer and a classification mechanism.

Jordan (1986) approached the sequential performance problem by a modified back propagation network. The input nodes, also called context nodes, are partitioned into two sets, namely, plan units and feedback units. Plan units represent external inputs and there are two types of feedback, one coming from these nodes themselves and one from the output. The net effect is to have a decaying memory of past events blended with current plans. The weights between input and hidden layers and between hidden and output layers are adjusted using the delta rule (Rumelhart, Hinton, & Williams, 1986). General shortcomings of networks of this genre are discussed in Grossberg ( 1984, 1987). In particular, Grossberg highlighted how the inadequate definition of functional units of cognitive processing and of the principles subserving the process of category formation can lead to unresolved stability-plasticity di- lemmas in list processing networks. Dehaene, Chan- geux, and Nadal ( 1987 ) proposed a layered neural network using synaptic triads made up of three neurons A-B-C as building blocks. A synaptic triad tracks the transition from C to A. These synaptic triads can be linked together in a hierarchical structure to recognize any complex sequence. By randomly connecting neuronal clusters in the higher layer to other clusters via modifiable connections and using a local Hebbian learning rule, they observed formation of sequence detectors and selection of prerepresentations (sequences that are built into the network structure a priori) among those that resonate with the input sequence. This approach is limited by the immense degree of connectivity required among clusters before learning can select a set of clusters to represent an arbitrary, nontrivial temporal sequence. It fails to recognize those sequences that are not in its prerepresentations.

Wang and Arbib (1990) proposed a dual neuron model for short-term memory (STM) whose activity oscillates with decreasing amplitude. These dual neurons are connected to a sequence detector. Sequential inputs are presented to the dual neurons and the detector node is taught to learn this sequence in a supervised manner. The distribution of the weights between dual neurons and the detector node represents order

information. The weights are modified by a Hebbian learning rule and a normalization rule. A quantity called input potential increases monotonically during learning trials, and once it saturates its value is used as the threshold for the detector node. Thus sequence sen- sitivity is built into the model. This model is then ex- tended to include a recurrent network of two layers to reproduce complex sequences with repetitions and with varying interitem intervals. Although learning and performance use similar mechanisms for recognizing and reproducing sequences, these two processes are handled separately. In a generalized STPPNN, there is a syn- ergistic interaction among storage, recognition, and reproduction / recall processes. Given a new sequence, the network has to recognize that it is being presented with a sequence not present in its database before commit- ting a chain of detectors to store it. It is not clear how the mechanisms used in Wang and Arbib's architecture, are adequate to handle these dynamics.

Various authors (Banquet & Contreras-Vidal, 1993; Bradski, Carpenter, & Grossberg, 1991, 1992; Mannes, 1992; Nigrin, 1990, 1993; Vogh, 1993 ) have attempted to use the insights developed in Grossberg ( 1978, 1986). An essential feature of these attempts is to build an STM layer that preserves the order of item presentations in the amplitudes of node activations. They use a long- term memory ( LTM ) invariance principle (Grossberg, 1978, 1986) that guarantees the preservation of the order of past events as new events are perturbing the system. Bradski et al. ( 1991, 1992) developed a working memory model, STORE (sustained temporal order recurrent), that stores the order of items in decaying amplitudes of STM activations. He proposed a cascade model, ARTSTORE: ART2 (2D aspects ) - - S T O R E - - ART2 (3D objects) to recognize 3-dimensional objects as an application of this STPPNN. Banquet and Con- treras-Vidal (1993) used this STORE model along with adaptive resonance theory (ART) and the spectral timing network (Grossberg & Merrill, 1992 ) to study timing, temporal order, and probability contexts during sequentially delivered inputs. They proposed this model for hippocampo-cortical relations.

Nigrin ( 1990, 1993) used the basic STM mechanism augmented by a masking field architecture (Cohen & Grossberg, 1986) to store and recognize spatiotemporal patterns in real time. He introduced mechanisms to learn the asymmetric inhibitory connections among lists of varying lengths in the masking field architecture. He explicitly transformed space-time patterns into spatial patterns before processing them in the network; in other words, his network does not store the transitions between successive items.

Mannes (1992) and Vogh ( 1993 ) constructed ART- like networks for spatiotemporal pattern processing with explicit reset mechanisms. An essential difference between these approaches is that Vogh's SMART (sequential memory ART) stores transitions between


items explicitly, which can be used for reproduction of sequences; whereas Mannes used a special readout mechanism without storing transitions. Further, Mannes used the ideas proposed in Grossberg (1978, 1986) for pattern storage and reset of category nodes, whereas Vogh developed novel mechanisms that use cell packets with information of past and future item activities affecting sequence storage and categorization. Vogh, like us, represented temporal information in the transitions between items.

Our architecture has more similarities with the latter models (discussed above) developed on Grossberg's network organizing principles. However, it differs from all of these as the architecture is proposed to model frontal lobe data on sequential performance. The requirements for learning to perform a sequence of motor patterns are slightly different from those for learning to encode a sequence of sensory patterns. Specifically, our network includes storage (in connection weights) of transitions between items of a sequence, like that of Grossberg (1978, 1986) and Vogh (1993) but unlike the other previous networks discussed herein. One reason for explicit storage of transitions is to encode the expectation of the next item based on occurrence of the present item. Another reason is to enable learning and performance of inflexible sequences, even if a sequence detector layer is not present. The experimental data that we are modeling on inflexible sequences (Brody & Pribram, 1978; Pin to-Hamuy & Linck, 1965) do not include reorderings of the sequences being learned; hence, the transition weights are sufficient to enable a robust encoding of the learned sequence. However, the same architecture can lead to difficulties in encoding flexible sequences; for example, if ABC and ACB are both learned, the transitions from A to B and C can interfere with one another. In the second part of this article, we introduce an elaboration of our network that overcomes this difficulty, using top-down signals from a sequence detecting layer with the addition of mechanisms for deciding the correct sequence detector to be activated.

Some other design issues are discussed in the next section. Experimental data will then be presented with a view to motivating the architecture and to discussing general issues raised by the data on STPPNN.

3. DESIGN ISSUES

A general architecture for processing spatiotemporal patterns has to deal with spatial and temporal aspects (order information, duration of items, and interitem interval) of a pattern. Ultimately, this must be joined with other neural mechanisms that code the timing of events, which are not present in our model but appear in other networks that can be integrated with ours (e.g., Grossberg & Merrill, 1992; Grossberg & Schmajuk, 1989).

At every instant of time, a space-time pattern can be approximated by a spatial pattern. So, a storing, recognizing, and recalling mechanism for spatial patterns is required. The issues of invariance of scale, translation, and rotation need to be addressed for spatial pattern processing networks. This problem has received wide attention and there are many architectures available now (Carpenter & Grossberg, 1987; Rumelhart et al., 1986, etc.). Although our network does not enjoy the invariance properties of general spatial pattern processing networks mentioned above, it includes mechanisms to classify spatial patterns of activations.

A fundamental characteristic of a temporal pattern is the order of items in it. A mechanism is needed that preserves the order when storing, recognizing, and recalling. The duration of various items in a temporal pattern can be varying across items as well as during different presentations of the same pattern. Mechanisms to store individual durations of items in a sequence as well as average duration of each item across presentations need to be developed. Time interval between items also can be varying across items in the same pattern as well as during different presentations of the same pattern. Of the architectures reviewed in the previous section, only Banquet and Contreras-Vidal (1993), Grossberg ( 1978, 1986), Mannes (1992), and Wang and Arbib (1990) deal with all the above issues in the design of their networks.

Apart from the above issues, a new architecture has to address sequence boundary detection, recognition that a sequence has been coded already before coding it as a new entity, robust coding of embedded lists, and a robust mechanism for reproducing/recalling the sequences (keeping in view the order, duration, and interval aspects). Such an architecture also needs the ca- pacity to learn the minimal set of items (context) required to determine a sequence.

To clarify the specific requirements for our network, we will now present the experimental data that we model on sequential performance in normal monkeys and monkeys with frontal lobe lesions. These data involve the ability of monkeys to learn two general types of motor sequences: invariant sequences, that must be performed in a definite order for reward to be obtained, and flexible sequences, whereby a set of movements must be made but can be performed in any order. The monkey's learning of these motor patterns appears to be largely independent of exact durations of and intervals between movements. Design issues relevant for modelling these data will be outlined.

4. EXPERIMENTAL DATA

Some effects of frontal lesions on performance of sequential tasks were studied in Pribram's laboratory by Pinto-Hamuy and Linck (1965) and Brody and Pri- bram (1978) in juvenile macaque monkeys. Postop-

Frontal Lobes and Sequences I 171

erative retention was assessed in two types of test: one, in which the subjects were to respond in such a way that they had to push all of several cued panels without repetitions but in any order ( internally ordered or flexible sequence test), and another in which they had to respond in an exact order by pushing a series of panels based on given cues (externally ordered or invariant sequence test). The hypothesis was that frontally damaged subjects would have difficulty retaining or learning flexible sequences because that would call for interaction between an internal representation of a sequence and a flexible recall based on previous action performed in order to avoid repetitions. In the case of invariant sequences, on the other hand, the frontals would not have difficulty because there are sensory cues (lighted panels) available for guiding motor actions.

Monkeys were trained on a four-by-four array of stimulus-response panels. In the experiments of Brody and Pribram (1978), there were three separate externally ordered sequences (E-O-Sq) consisting of two or three panels the monkeys had to press in order: "red- green" (large solidly colored circles); "0"-"2"; and "0"- "2"-"6" (white patterns representing numbers against a dark field). The internally ordered sequences (I-O- Sq), which the monkeys could press in any order, consisted of "blue"-"yeliow" (a blue filled plus sign and a yellow filled triangle); "4"-"5"; and "4"-"5"-"7" (white patterns against a dark field). On each trial in the E- O-Sq case, the two stimuli (or three in the case of the three-element sequence) appeared in randomly placed locations on the four-by-four panel array. In order to receive a reward the monkey pressed first the red and then the green panel irrespective of their location. The stimuli disappeared after each press within a trial and reappeared immediately in a new random configura- tion, making spatial strategies irrelevant. The trial ended if the first panel pressed was not red, or if the second press was incorrect, or if the subject was successful. Only successful completions were rewarded. The internally ordered sequence was presented in ex- actly the same manner except that the monkey was permitted to choose different orders from trial to trial as long as any given sequence contained no repetitions. The stimuli used in Pinto-Hamuy and Linck (1965) were similar, except that some sequences consisted of the same stimulus (a red circle) but in different locations, creating a possible confusion between stimulus attribute and spatial location that was eliminated in Brody and Pribram's later work.

The experimental results for one subject in Pinto- Hamuy and Linck (1965) are illustrated in Figure 2, which shows the percentage of correct responses for one example each of the externally and internally ordered sequences both pre- and postoperatively. Al- though Brody and Pribram (1978) obtained similar qualitative results, they observed that the impairment in performance was dependent on the level of sophis-

tication (experience with the test apparatus) achieved. In particular, they observed that the ease with which each monkey could solve the internally ordered sequences depended on the degree to which they limited the number of possible sequences used during a session.

Experiments on human subjects have tended to bear out the differential involvement of the prefrontal cortex on flexible and invariant sequence learning tasks. Pop- pen, Pribram, and Robinson ( 1965 ) explored the effects of frontal lesions on sampling and search in human patients, wherein the subjects were to modify their strategies for winning candy, working through a sequence of 20 programs. In each program, only one of the geometric figures displayed randomly on a four-by- four panel set-up was rewarding. These patients had similar difficulties to the frontally lesioned monkeys in maintaining the strategy required to complete the task to criterion. Shallice (1982) found that frontally lesioned patients are selectively damaged on tasks that involve planning and generating subtasks, such as the Tower of London test that involves stacking differently colored beads on one of three posts in a prescribed arrangement. Karnath et al. ( 1991 ) studied frontal patients and controls on a maze where the subjects saw only a part of the maze at a time. He found that frontal patients can solve mazes effectively when they have learned enough about the environment to find the maze

80

rJ} 7O IJJ o') Z 60 O 13. 09 LLI 50 r r

I-- O 4o ILl £E Or" 30 O O

2O o~

10

Invariant (G-R) Flexible (G-O-4)

RETENTION TEST FOR SEQUENCES

I Pre-operative I Post-operative

FIGURE 2. Results on the effects of frontal lesions on learning of externally and internally ordered sequences by monkeys. (Adapted from Pinto-Hamuy and Linck, 1965, wittt the permission of Academic Press.)


steps routine, but do much worse than normals before then.

Involvement of the frontal lobes in coordination of movement sequences is also supported by evoked potential data on the contingent negative variation or ex- pectancy wave (see Fuster, 1985, for discussion). On the basis of those results and some of the others we have mentioned, Fuster (1989) strongly postulates that the prefrontal cortex is involved in the formation of temporal structures of behavior. Based on neuropsychological and electrophysiological evidence, he suggests that dorsolateral prefrontal cortex is involved in the functions of provisional (short-term) memory and pre- paratory set, and the orbital prefrontal cortex is involved in interference control, especially through its connections to the thalamus. Fuster ( 1989, p. 166 ) further suggests that the prefrontal cortex is essential not for motor acts per se, but for the orderly and purposive

execution of novel and complex behavioral structures. Hence, the actual initiation of motor behavior, which

is modeled by the avalanche network of Figure 1, has to involve a different area of the brain than the frontal cortex. Some evidence exists that the corpus striatum is involved in the initiation of willed movements (e.g., Gray et al., 1991; Rolls & Williams, 1987). Also, striatal damage, but not neocortical damage, interferes with performance of a stereotyped grooming sequence in rats (Berridge & Whishaw, 1992 ). Moreover, the corpus striatum is well known to be strongly influenced by feedback connections with the frontal lobes. Our network does not reproduce the exact structure ofstriatal- frontal connections, which are indirect, via the thalamus and motor cortex, but may be a first approximation to a more anatomically detailed model of these connections.

However, the experimental results on sequences are

AROUSAL

F (2)

F (3)

F (1) \

INPUT

,j FIGURE 3. Modification of the network of Figure 1 actually used in our first stage of simulations.


generally in line with the above observations that frontal lobes are not involved in the representation of externally ordered sequences but are required if a flexible reply is needed, as in the case of internally ordered sequence performance. In the next section, a network architecture to model these data will be described.

5. NETWORK ARCHITECTURE

As the experimental data suggest, our network is in two stages. Stage 1 represents the sequence itself and the transitions between items in the sequence. To simplify the model, we assume that all the items are of equal duration and have identical interitem intervals. Also, we assume that spatial aspects of stimuli are prepro- cessed and the network receives sequential inputs of items. In other words, our network has to essentially deal only with order aspects of a sequence. Stage 2 adds sequence detector nodes, and a third stage (to be discussed in another article) will add a classification mechanism.

The first stage, shown in Figure 3, is almost identical to the network of Figure 1. The movements in a sequence are presented to the ~7 ~ layer successively. The input node xi.t receives a unit activation. The node xi. excites nodes xi,2 (in the ~2 layer) and xi,3 (in the ~3 layer) in the same column. Activity reverberates between xij and xi,3. When x;.2 receives sufficient excitation, it inhibits reverberation in its column and excites xi÷~.~. When a new movement is performed by the network at xi+l.i, the weight from xi.2 to xi+L~ is strength- ened by a Hebbian learning rule representing the transition between movements. Thus when the network performs a sequence of movements, the transitions between successive movements are stored in this chain of weights.

Grossberg ( 1978, 1986) suggested that as items are presented serially, STM has a tendency to store them with a recency gradient in activations; that is, the later the item is presented, the larger the corresponding activity. However, when it is time to store the sequence in LTM, it is desirable to have a prirnao, gradient, that

7

F(2)

F

I . /

7 (1)

1 1 l SEQUENTIAL INPUT OF ITEMS

FIGURE 4. Second stage of the sequence network, including a sequence detector layer.


is, the earlier the item is presented, the larger the corresponding weight, because that would enable the network to recall the sequence in the correct order. He further suggested that this order reversal from recency to primacy can be achieved by competitive-cooperative interactions among the item activities in STM combined with top-down and bot tom-up LTM-STM interactions. Accordingly, our Stage 2 network, shown in Figure 4, includes a layer of sequence detectors (5 t 4), analogous to that in Wang and Arbib (1990), and two- way learnable connections between 5 t 3 and 5 t 4. It also includes on-center offsurround interactions within each of the layers 5 t 3 and 5 t 4. With a suitable choice of parameters and signal function (faster-than-linear), the

i~4 layer will have winner-take-all dynamics. Thus a sequence is encoded only by one detector.

While the item activations are reverberating in the 5t 3 layer, a sequence detector at the ~7 4 layer samples the order information in the spatial pattern of activities. This order information is stored in the distribution of

top-down weights from 5 t4 to ~7 3. The first sequence detector is chosen randomly and is rewarded for the length of sequence presentation.

6. S I M U L A T I O N AND RESULTS

Simulations of the Stage 1 network shown in Figure 3 were conducted. The aim was to study how an avalanche network with learnable connections from the arousal node to the 5 t 3 layer nodes responds to sequential inputs. Equations for this network are given in the Appendix.

In Figure 3, the arousal node is connected to the 5 ~ 3 layer nodes via learnable connections. Previously (Fig- ure 1 ), it was connected to 5 t 2 layer nodes. The change was made in order to study whether order information can be captured in the weight distribution. This has proved to be true, and we added a layer of sequence nodes (5 t4) in the second stage of simulations. Here the arousal node is activated by the sum of all the inputs.

(a)

0.9

0.8

Z- 0.7 ._~

0.6

e- 0.5 0

m 0.4

i.~ 0.3

0.2

0.1

• Trans. wt. ( w t , 2 ) , ~

I I I I 25 50 75 100

"13me Steps

(b) 1 0.9

..~ o.e p -

• ~ 0.7

0.6

i f - 0.5

0.4

~ 0.3 0 '~. 0.2

0.1

0 25 50 75 10o

13me Steps

FIGURE 5. (a) Transition weight distribution between successive items in the three-element sequence for simulation of the network of Figure 3. For convenience, all weights are scaled by the maximum weight, which is 0.8375. (b) Distribution of weights from the arousal node to nodes x~.a. These show a primacy effect. All weights are scaled by the maximum weight, which is 3.9138.


1.

0 .9

O 0.8.

-~ . .

0.4.

~ 0.3

0.2

~U,. 0 I

0 10 20 30 40 50 60 70 80 "rime Steps

9o 10o

FIGURE 6. Evolution of activities of the ~73 layer nodes in the network of Figure 4, indicating a primacy effect in item activations due to learning. (Activations are scaled by their maximum, 2.7664.)

In our simulations, the items of a three-element sequence were presented in succession for five time units each, followed by five time units with a reward input, and then five more time units with no input. Hence, the entire sequence presentation lasts for a total of 25 time units.

The results of simulating the network of Figure 3 are shown in Figure 5. Figure 5 (a)shows how the transition weights from xi,2 to xi+~.= represent the sequence transitions after 100 time steps (four presentations of a three-element sequence). Figure 5 (b) shows the distribution of weights from the arousal node to the ~ 3 layer. There is a noticeable primacy effect in weight distribution after only four presentations of the sequence (i.e., 100 time steps). We have observed the same qualitative results with perturbations in initial

weights, activations, and some of the parameters. In the Appendix, we discuss some mathematical analysis of what parameter choices lead to primacy. This experi- ment confirms the basic intuition behind the avalanche model in Figure 1 and lays the foundation for the second stage of simulation.

The second stage of simulation was based on the network of Figure 4. We decided to add the layer of sequence detectors (~ 4) to aid in future extensions involving sequence classification and reordering. Also, we added learnable connections from xi.2 to all other xj.~, j --/= i, rather than only j = i + 1 as previously. Consequently, we needed to test whether the primacy effect shown in Figure 5 could be preserved with these additions. Simulations were conducted by presenting the items in sequence and rewarding a detector node

1

0.9 [ ] Trans wts.(wl,2) [ ] Trans wts.(wl,3) 0.8 [ ] Trans wts. (w 21 )

~m 0.7 .¢: [ ] Trans wts.(w 23)

0e (w3,)

r- 0.5

"~'° 0.20,1°'3°'4 [-]iI Trans wts'(w32)

0 25 50 75 1 oo

"rime Steps

FIGURE 7. In the network of Figure 4, actual transitions between items in the sequence 11-to-2 and 2-to-3) are differentially enhanced in preference to other weights. (All weights are scaled by their maximum, 1.8610.)


(chosen randomly at first and then fixed for subsequent presentations) throughout the sequence presentation. The time course of presentation of the items of the sequence was the same as in Stage 1. Again, after only about four presentations (100 time steps), there was clear indication that the network was showing the ex- pected primacy effect. Figures 6-8 show the results.

Figure 6 shows the node activations in the fit 3 layer. In the first stage of simulations, STM activity in this layer had often shown a recency distribution. Now, however, the activities show a primacy distribution. This shows the effect of the STM-LTM interactions helping to enhance the activity of the first item in preference to others. The activities of all three fit 3 nodes are shown before learning (in the course of the first presentation of the sequence, that is, time steps 5-25 ); during learning (in the course of the second and third presentations, that is, time steps 30-50 and 55-75); and after learning (in the course of the fourth presentation, that is, time steps 80-100). Our graphs show that in the course of

learning, fit 3 node activities move from a recency to a primacy distribution.

Figure 7 shows how the sequential weights grow over time. The actual transitions from items 1 to 2 and items 2 to 3 are represented by the fact that the weights wl.2 and w2.3 grew somewhat more rapidly than the other transition weights. In the first stage, only the weights w~,2 and w2.3 were present. So the second stage shows that even if all the items are initially connected, order between items can be learned via differential enhance- ment of the appropriate interitem weights. This represents the invariant order between items in an externally ordered sequence.

We have seen in Figure 6 that the order is organizing itself into a primacy distribution of STM activities. This information is further captured in the distribution of bottom-up (fit 3 to fit 4) and top-down (fit 4 to fir 3) weights, as shown in Figure 8a, and b, respectively. The bottom-up weights from fit 3 to the remaining detectors and the top-down weights from the remaining detectors

Q o

.m

CL

E

0 rn

25 5O

"13me Steps 75 100

3 a)

c~ E .o

,e-

o

(b)

0 25 50 75 100

"13me Steps

FIGURE 8. (a) In the network of Figure 4, weights from the ~3 layer nodes to detector 1 capture the order information in the sequence with a primacy distribution. (All weights are scaled by their maximum, 3.3654.) (b) Weights from detector I to ~ = nodes are distributed with a primacy effect to enable the network to recall the first item with the largest activation. (All weights are scaled by their maximum, 4.4523.)


to ~7 3 have not shown any tendency to store the sequence in their distribution, which is a desirable effect. Because these other weights have not shown any preference to store this sequence, we have not shown the graphs for these. Thus, if this sequence is presented or performed again, bot tom-up weights prime the first detector with greater strength, and in turn the first detector will be able more readily to strengthen this sequence a t 5 t3.

7. D I S C U S S I O N

The resonance between bottom-up input and top-down expectation will be useful in extending our model to include a classifier of sequences, partly based on reinforcement, involving a mechanism similar to ART (Carpenter & Grossberg, 1987 ) or ARTMAP ( Carpen- ter, Grossberg, & Reynolds, 1991 ). For reordering the sequence as in the data of P in to -Hamuy and Linck (1965) and Brody and Pribram (1978), we need to introduce a mechanism that oversees this resonance or dissonance between bot tom-up and top-down patterns and makes a decision to reorder the sequences to es- tablish resonance.

The model based on the network of Figure 4 has been tested with perturbations in initial activations, initial weights, and various parameters. The qualitative phenomena shown in Figures 6-8 were robust under small perturbations. For larger perturbations we have observed a parameter bifurcation. As the coupling coef- ficients for LTM and STM interactions (n and g in our equations) are varied, the distribution of 5 t3 node activations and of various bot tom-up and top-down weights moves from recency to dominance of the mid- die of the list, and then to primacy.

Our current network does not include an encoding of flexible sequences. However, our two-stage simulations have established a foundation for adding mechanisms to our network that are required for the effective learning and performance of flexible sequences. A subsequent article will deal with how sequences encoded at the ~7 3 layer can be classified using yet another layer. For example, in modeling learning of an internally ordered sequence, one might add to the network a supervised ARTMAP mechanism (Carpenter et al., 1991 ), so that if several different variations on a sequence (e.g., G-O-4 or O-4-G) are all rewarded, the tendency to perform any one of the variations is in- creased. The network would also need to keep track of which movements have already been performed, in order to engage the appropriate exemplar of such a class of sequences. Also, one might add a masking field mechanism (Cohen & Grossberg, 1986) to regulate the length of encoded sequences.

Such an ARTMAP mechanism could, in principle, learn any subset of sequences that is rewarded but may not be able to generalize to sequences it has not ex-

plicitly learned. An open question, which no experimental results to date seem to have addressed, concerns the ability of monkeys to generalize from a few examples to an entire rule. For example, if a monkey has learned that G-O-4, G-4-O, O-4-G, and 4-O-G are rewarded, can it conclude that O-G-4 and 4-G-O will also be rewarded? Humans have this capability, but it might depend on an explicit encoding of a rule (press each panel once with no repetition) that is not present in the network of the current article and may be beyond the scope of the subsequent article as well. Understand- ing generalization is part of understanding how the prefrontal cortex is involved in learning such rules that, dependent on the data, could be at any one of several levels of complexity. Other examples include learning to alternate between two identical food dispensers (Goldman & Rosvold, 1970; Pribram, Plotkin, An- derson, & Leong, 1977) and learning to go to whichever object is the most novel (Pribram, 1961; see Levine & Prueitt, 1989, for discussion). ART by itself can only decide within one level of complexity, but some type of multilayer combination of ART and ARTMAP modules with vigilance criteria for transferring control between levels might be able to approximate human rule-forming capacities.

REFERENCES

Banquet, J. E, & Contreras-Vidal, J. (1993). Learning temporal contexts and priming-preparation modes for pattern recognition. World Congress on Neural Networks. Portland, Oregon (Vol. 2, pp. 126-130). Hillsdale, NJ: Lawrence Erlbaum Associates.

Berridge, K. C., & Whishaw, 1. Q. (1992). Cortex, striatum, and cerebellum: Control of serial order in a grooming sequence. Ex- perimental Brain Research. 90, 275-290.

Bradski, G., Carpenter, G. A., & Grossberg, S. (1991). Working memory networks for learning multiple groupings of temporally ordered events: Applications to 3-D visual object recognition. In- ternational Joint Conference on Neural Networks. Seattle ( Vol. l, pp. 723-728). Piseataway, NJ: IEEE.

Bradski, G., Carpenter, G. A., & Grossberg, S. (1992). Working memories for storage and recall of arbitrary temporal sequences. In Proceedings of lJCNN-92, Baltimore, Maryland (Vol. 2, pp. 57-62). Piseataway, N J: IEEE.

Brody, B. A., & Pribram, K. H. (1978). The role of frontal and parietal cortex in cognitive processing: Tests of spatial and sequence functions. Brain, 101, 607-633.

Carpenter, G. A., & Grossberg, S. (1987). A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer vision, graphics, and image processing, 37, 54- l 15.

Carpenter, G. A., Grossberg, S., & Reynolds, J. H. ( 1991 ). ARTMAP: Supervised real-time learning and classification of nonstationary data by a self-organizing neural network. Neural Networks, 4, 565- 588.

Cohen, M. A., & Grossberg, S. (1986). Neural dynamics of speech and language coding: Developmental programs, perceptual grouping, and competition for short term memory. Human Neu- robiology, 5, 1-22.

Dehaene, S., Changeux, J.-P., & Nadal, J. P. (1987). Neural networks that learn temporal sequences by selection. Proceedings of the National Academy of Sciences USA. 84, 2727-273 I.

Fuster, J. M. (1985). The prefrontal cortex: Mediator of cross-temporal contingencies. Human Ntntrobiology, 4, 169-179.


Fuster, J. M. ( 1989 ). The prefrontal cortex. New York: Raven. Gevins, A. S., Morgan, N. H., Bressler, S. L., et at. (1987). Human

neuroelectric patterns predict performance accuracy. Science, 235, 580-584.

Goldman, P. S., & Rosvold, H. E. (1970). Localization of function within the dorsolateral prefrontal cortex of the rhesus monkey. Experimental Neurology, 27, 291-304.

Gray, J. A., Feldon, J., Rawlins, J. N. P., Hemsley, D. R., & Smith, A. D. ( 1991 ). The neuropsychology ofschizophrenia. Behavioral and Brain Sciences. 14, 1-84.

Grossberg, S. (1969). Some networks that can learn, remember, and reproduce any number of complicated space time patterns, i. Journal of Mathematics and Mechanics, 19, 53-91.

Grossberg, S. (1970). Some networks that can learn, remember, and reproduce any number of complicated space time patterns, Ii. Studies in Applied Mathematics, 49, 135-166.

Grossberg, S. ( 1972 ). A neural theory of punishment and avoidance. I1: Quantitative theory. Mathematical Biosciences, 15, 253-285.

Grossberg, S. ( 1978 ). A theory of human memory: Self-organization and performance of sensory-motor codes, maps, and plans. In R. Rosen & E Snell (Eds.), Progress in theoretical biology (Vol. 5, pp. 233-374).

Grossberg, S. ( 1984 ). Unitization, automaticity, temporal order, and word recognition. Cognition and Brain Theory, 7, 263-283.

Grossberg, S. (1986). The adaptive self-organization of serial order in behavior: Speech, language, and motor control. In E. C. Schwab & H. C. Nusbaum (Eds.), Pattern recognition by humans and machines. Vol. 1: Speech perception (pp. 187-294). New York: Academic Press.

Grossberg, S. (1987). Competitive learning: From interactive activation to adaptive resonance. Cognitive Science, ! !, 23-63.

Grossberg, S., & Merrill, J. W. L. (1992). A neural network model of adaptively timed reinforcement learning and hippocampal dynamics. Cognitive Brain Research. 1, 3-38.

Grossberg, S., & Pepe, J. ( 1971 ). Spiking threshold and overarousal effects in serial learning. Journal of Statistical Physics. 3, 95-125.

Grossberg, S., & Schmajuk, N. A. (1989). Neural dynamics of adaptive timing and temporal discrimination during associative learning. Neural Networks, 2, 79-102.

Grueninger, W. E., & Pribram, K. H. (1969). Effects of spatial and non-spatial distractors on performance latency of monkeys with frontal lesions. Journal of Comparative and Physiological Psy- chology, 15, 195-240.

Hartman, P. ( 1982 ). Ordinary differential equations. Boston: Birk- h~iuser.

Ingvar, D. (1985). Memory of the future: An essay on the temporal organization of conscious awareness. Human Neurobiolog~, 4, 124- 136.

Jordan, M. I. (1986). Serial order: A parallel distributed processing approach. Technical Report ICS Report 8604, Institute for Cog- nitive Science, University of California, San Diego.

Karnath, H. O., Wallesch, C. W., & Zimmermann, P. ( 1991 ). Mental planning and anticipatory processes with acute and chronic frontal lobe lesions: A comparison of maze performance in routine and non-routine situations. Neuropsychologia, 29, 271-290.

Leven, S. J., & Levine, D. S. (1987). Effects of reinforcement on knowledge retrieval and evaluation. In M. Caudill & C. Butler (Eds.), Proceedings of the First International Conference on Neural Networks (Vol. 2, pp. 269-279). San Diego: IEEE/ICNN.

Levin, H. S., Eisenberg, H. M., & Benton, A. L. (Eds.) ( 1991 ). Frontal Iobefimction and dysfimction. New York: Oxford University Press.

Levine, D. S., & Parks, R. W. (1992). Frontal lesion effects on verbal fluency in a network model. International Joint Conference on Neural Networks, Baltimore, Maryland. June 7-11, 1992 (Vol. 2, pp. 39-44). Piscataway, N J: IEEE.

Levine, D. S., & Prueitt, P. S. (1989). Modeling some effects of frontal lobe damage: Novelty and perseveration. Neural Networks, 2, 103- 116.

Man nes, C. (1992). A neural network model of spatio-temporal pattern recognition, recall, and timing. In International Joint Con- ference on Neural Networks, Baltimore, Maryland, June 7-11, 1992 (Vol. 4, pp. 109-114). Piscataway, NJ: IEEE.

Milner, B. (1964). Some effects of frontal lobectomy in man. In J. M. Warren & K. Akert (Eds.), The frontal granular cortex and behavior ( pp. 313-334 ). New York: McGraw-Hill.

Nauta, W. J. H. ( 1971 ). The problem of the frontal lobe: A reinter- pretation. Journal of Psychiatric Research, 8, 167-187.

Nigrin, A. L. (1990). The real-time classification of temporal sequences with an Adaptive Resonance Circuit. In Proceedings of the IJCNN-90, Washington, DC ( Vol. I, pp. 525-528 ). Hillsdale, NJ: Lawrence Erlbaum Associates.

Nigrin, A. L. ( 1993 ). Neural networks for pattern recognition. Cam- bridge, MA: MIT Press.

Parks, R. W., & Levine, D. S. (1994). Neural network modeling of verbal fluency in Alzheimer's disease. Submitted for publication.

Perecman, E. (Ed.) (1987). The frontal lobes revisited. New York: IRBN.

Petrides, M., & Milner, B. (1982). Deficits on subject-ordered tasks after frontal and temporal-lobe lesions in man. Neuropsychologia, 20, 249-262.

Pinto-Hamuy, T., & Linck, P. (1965). Effects of frontal lesions on performance of sequential tasks by monkeys. Experimental Neu- rology, 12, 96-107.

Poppen, R. L., Pribram, K. H., & Robinson, R. S. (1965). Effects of frontal Iobotomy in man on the performance of a multiple choice task. Experimental Neurology 11, 217-229.

Pribram, K. H. ( 1961 ). A further experimental analysis of the behavioral deficit that follows injury to the primate frontal cortex. Experimental Neurology, 3, 432-466.

Pribram, K. H., Plotkin, H. C., Anderson, R. M., & Leong, D. ( 1977 ). Information sources in the delayed alternation task for normal and "frontal" monkeys. Neuropsychologia, 15, 329-340.

Rolls, E. T., & Williams, G. V. (1987). Sensory and movement related neuronal activity in different regions of the primate striatum. In J. S. Schneider & T. I. Kidsky (Eds,), Basalganglia and behavior: Sensor), aspects and motor fimctioning (pp. 37-59). Toronto: Hans Huber.

Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel distributed processing ( Vol. I, pp. 318-362). Cambridge, MA: MIT Press.

Shallice, T. ( 1982 ). Specific impairments of planning. Philosophical Transactions of the Royal Society of London. Series B, 298, 199- 209.

Shallice, T. ( 1988 ). From neuropsychology to mental structure. Cam- bridge, UK: Cambridge University Press.

Stuss, D. T., & Benson. D. E (1986). The frontal lobes. New York: Raven.

Vogh, J. (1993). Sequential memory with ART: A self-organizing network capable of learning sequences of patterns. World Congress on Neural Networks. Portland. Oregon (Vol. 2, pp. 540-544). Hillsdale, N J: Lawrence Eflbaum Associates.

Wang, D. L., & Arbib, M. A. (1990). Complex temporal sequence learning based on short-term memory. Proceedings of the IEEE, 78, 1536-1543.

Wilkins, A. J., Shallice, T., & McCarthy, R. (1987). Frontal lesions and sustained attention. Neuropsychologia. 25, 359-365.

A P P E N D I X

Following are the equations and parameter values used for simulation of networks of Stages 1 and 2 as shown in Figures 3 and 4, respectively. The Runge-Kutta method was employed to numerically solve the equations. In both stages, there are three nodes in all the layers (hence, i , j = I, 2, 3).

F r o n t a l L o b e s a n d S e q u e n c e s 1 1 7 9

Stage 1.

f c , . , = - a - ' q . , + { ( ~ w ~ . , [ x , . 2 - - O ] + ) + l i + c x , . 3 } ( r - x , . , ) ( A . l a ) 1÷i

j~F 2

X,.~ = -ex~.2 + fxi.~(r - x,.~) (A. lb )

"~'i,3 = -bx, ,3 + {cx,.t + gw^ .dx^ - F] ÷ } (r - x,.3)

- d [ x , . ~ - O]+x~.3;

O if w > 0 where [w] ÷ = (A. lc)

if w < 0

.~-̂ = - a x ^ + k ( ~ I , + R ) ( r - x ^ ) . (A . ld )

In the x~., equation, only w~.2 and w2.3 are present in Stage I. The equations for x,j reflect shunting reverberatory excitation between each x,.~ and the corresponding x,,~; excitation from x,.~ to the corresponding x,.2, which in turn inhibits the corresponding x j . Each x,.~ is also excited by the arousal node x^. The arousal node is excited both by all sequential inputs and by the reward R when it is on. The parameters F and 0 represent thresholds on x^ and x~.2 respectively. Transition weights:

~i,ij = ( - l w ~ j + mx~.~)[x,.2 - O]+; (A. le)

weights from arousal node to fit 3 layer:

*i'A.i = ( -hv^., + tr~xi. 3) [x^ - F] +. (A. I f)

Parameter Values for Stage 1.

a = b = 10, c = 1.4, d = 22, e = 10, f = 1,

g = 1.9, k = 2 .5 ,1= I , m = 3, r = 5 . 5 , 0 = 0 . 4 ,

F = 0.57, R = I (while the reward is on).

Stage 2. Equations for time derivatives of x,.,, x,.2, x^, w~j, w^., are the same as in Stage 1. The equations for x,.3 are changed to include self-excitation, lateral inhibition, and the contribution from sequence detector nodes in the fit 4 layer:

.X',,3 =-bx~,3 + {¢Xt.IWgWA,i[XA--~]+'i-H~ wfDA~,4+hH(xi.3)} .t E F ,4

× ( r - x,.3) - d[x,.2 - O] +x~.3 - h(xi.a - s) ~ H(Xk.3); k÷l

I where H ( y ) 1 + e c*-~'~

.~',., = - a x , . , + {n ~ w ~ U . , ) . 3 + p J ( x , . 4 ) + R 1 jEF 3

X ( r - - x , , 4 ) - - p ( x , , a - - S) ~ J(XkA); k'~J

(sigmoid function).

where j ( y ) = ),2 (square function).

The fit 4 node activities x,.4 are influenced by self-excitation and lateral inhibition combined with contributions from all the fit 3 nodes. Bot- tom-up weights from fit3 to fit4:

~i,eV,., = (-Iw~y. + ~x~.,)x,.3;

top-down weights from fit 4 to fit 3:

Parameter Values for Stage 2.

a = b = 10, e = 1.4, d = 22, e = 10, f = l , g = 1.9, h = 0,

k = 2 . 5 , / = l , m = 3 , n = 0 . 1 2 , p = 0 , r = 5.5, s = 0 . 1 ,

O = 0.4, I" = 0.57, ~ = 0.3, R = 1 (while the reward is on).

Note: the parameter settings h = 0 and p = 0 above mean that in our current simulations there are no on-center off-surround interactions among fit3 and fit4 nodes. These on-center off-surround interactions were included for future modeling in which more than one sequence can be learned by the network. We ran simulations of the current network in which there was competition between items in the fit 3 layer, that is, h > 0, and found that positive h did not markedly enhance or reduce the possibility of primacy effects. This means that the effects modeled here are likely to persist in the more complex network, but that the greater complexity is not necessary for primacy in the present model. As for the other competition parameter, p, because in the present simulations only one sequence is stored at fit4, winner-take-all dynamics is not required for effective sequence storage at the current stage. A positive value of p, however, is likely to be necessary at the next stage whereby multiple sequence detectors are activated.

Some heuristics for how different parameter settings affect primacy versus recency can be obtained from mathematical analysis of a sim- plified version of our Stage I eqn (A.1). From eqn (A. I f ) , we see that the proportion between the arousal-to-x3 weights wA., for different i is asymptotically the same as the proportion between the xi3 values; hence our analysis will concern the relative sizes of the .x-,as.

Following the method used by Grossberg and Pepe ( 1971 ) in their model of serial list learning, we study eqn. (A. 1 ) with all learnable connections removed, what was called the barefield. Our bare field equations are

-x~.t = -a3:i.l + (0.,3 + I, )(r - x~.t)

.f,.2 = -ex, .2 + fx',.l(r - x,.2)

.~,.3 = -bxi .3 + cxi.t(r - x,.3) - d[x,.2 - O]+x,.3.

We assume for simplicity that O = 0. Also, based on our simulations, we express the inputs to the different item nodes as functions (pulse series) that follow each other at equal time lags r, labeling the input to item node I as J( t ) . Then we find that the equations for representations of the i th item can be represented (dropping the "'i" sub- scripts) as

Yq = - a r t + (cx3 + J ( t - ( i - I ) r ) ) ( r - xl)

-x'2 = - o : 2 + f i q ( r - x2)

x3 = -bx3 + c : q ( r - x3) - d~2x3.

Grossberg and Pepe studied similar equations by treating the item number i as a continuous variable. Following their analysis, we note that primacy occurs if the value ofx3 at any given time is highest for i = 1 and lowest for i = 3; that is, the derivative of x3 with respect to i is always negative. Also, while the input function J ( t ) in our simulations is a series of on-off pulses, Grossberg and Pepe assumed for simplicity that it is continuously differentiable in time, and we will do the same here. Grossberg and Pepe assumed a single pulse, but our heuristic analysis remains valid if there is a series of pulses as in our simulations.

The derivatives of the solutions of (A.2) with respect to i are solutions of the linear variational equations of ( A.2 ) (see Hartman, 1982, pp. 95-96) . In general, the variational equations for the time- dependent, parameter-dependent, vector dynamical system

d?f = 7(.~, t, i) dt

with fixed initial conditions is

d-2 ~;,,,~# = ( - l w [ f f + m.xj.3)x,.4. -~- = A( t ) + ~(t) (A.3a)


where A(t) is the Jacobian matrix of .7 with respect to ~ at the solution, and ~ = 07/0i. For our specific eqn (A.2), we can compute the matrix function A(t) and the vector function ~ to be

A(t)

[ - a - c x 3 - J ( t - ( i - l ) ~ ' ) 0 -.'xl) ] =l f ( r - x 2 ) - e - f x l C(rO / (A.3b)

[ c(r - x3) -dx3 - b - cxl - dx2J

- r ( r - x,)J'(t - (i - l)r)] = 0 . (A.3c)

0

Recall that the partial derivatives ofxa, x2, and x3 with respect to i are solutions of eqn (A.3). At the first introduction of the input, J ' ( t ) > 0. Equations (A.3b) and (A.3c) show that the values of Oxt/ 0i, Ox2/Oi, and Ox3/Oi never become positive, so primacy is occurring. But while a pulse is being shut off, J'(t) < 0, hence it is possible for O,~h/0i to become positive and thereby make dx3/Oi positive. This can be prevented by large enough negative values in the diagonal terms

of the matrix for A(t) in eqn (A.3b). Ifthe values ofOxt/Oi, Ox2/Oi, and 0x3/0i can be prevented from going above 0 during the shutting- off periods, they will remain negative during the next period when the pulse is introduced, and so primacy will occur.

For example, large negative terms in the diagonal, and therefore primacy, can be achieved by large enough values of the parameter c representing the strength of reverbatory interactions between each x~.~ and the corresponding x,.3, or the parameter d representing the strength of inhibition from each x~.2 to the corresponding xi.3. The influence of both those parameters on primacy versus recency was verified in our simulations.

N O M E N C L A T U R E

x i j ac t iva t ion o f n o d e i in layer 5 t j

wi j we igh t on the c o n n e c t i o n f r o m n o d e i to n o d e j

Ii i n p u t to n o d e i in layer ~7

R reward i n p u t to a rousa l and de t ec to r nodes

Documents

Modeling the Role of Frontal Lobes in Sequential Task … levine... · 2021. 1. 12. · 1994, Dr. Bapi will be at the School of Computing, University of Plymouth, Devon PL4 8AA, Plymouth,