57
The Martian Program 2009: Theory of the Stage I Infant Model An LCNTR Tech Brief Richard B. Wells and Quinn MacPherson April 20, 2009 Abstract: This tech brief presents an overview and general explanation of the Martian Research Program here at the University of Idaho's MRC Institute. It begins with a strategic overview of the long-term research objectives, followed by an overview of the current near-term research objectives. The Martian system in its current state of development is next explained and discussed. Finally, the specific research aims for the next twelve months are presented. 1

The Martian Program 2009: Theory of the Stage I Infant Modelrwells/techdocs/Martian Program... · The Martian Program 2009: Theory of the Stage I Infant Model An LCNTR Tech Brief

  • Upload
    others

  • View
    25

  • Download
    0

Embed Size (px)

Citation preview

The Martian Program 2009:

Theory of the Stage I Infant Model An LCNTR Tech Brief

Richard B. Wells and Quinn MacPherson April 20, 2009

Abstract: This tech brief presents an overview and general explanation of the Martian Research

Program here at the University of Idaho's MRC Institute. It begins with a strategic overview of

the long-term research objectives, followed by an overview of the current near-term research

objectives. The Martian system in its current state of development is next explained and

discussed. Finally, the specific research aims for the next twelve months are presented.

1

The Martian Program 2009 Wells and MacPherson April 20, 2009

I. Overview of Basic Hypotheses, Premises, and Research Objectives

The Martian is an intelligent agent model, implemented using neural networks, for studying

the development of intelligence in Homo sapiens. This research is based upon several hypotheses.

The background and reasoning underlying this set of basic hypotheses has been presented in two

previous works by this author [WELL1-2]. The most fundamental of these is the Development

Stages Hypothesis: The development of later cognitive and intelligent capacities are built upon

foundations laid during earlier developmental stages. This means that all later intellective and

practical capabilities are fundamentally shaped and influenced by learning episodes taking place

early in life. At the time this is being written, the Martian (which we nickname 'Marty') is an

infant at a developmental level corresponding to that of the first days of life of a human infant.

The immediate research aims of the Program at this time involve the study of the development of

early sensorimotor intelligence. Specifically, the work is aimed at understanding the functional

capacities of brain function required for the infant to progress from innate sensorimotor reflex

behaviors to its first acquisition of the simplest habits acquired during the first month of life.

In H. sapiens these simplest first habits are exhibited by such behaviors as sucking the fingers,

dry sucking, habituation of the looking and listening reflexes, etc. [PIAG1]. We take as our

second working hypothesis that, despite specific differences found in these various behavioral

modes, the fundamental brain functioning for all such sensorimotor modalities follows a common

developmental process, specifically a central process of general equilibration [PIAG2]. This, like

the first hypothesis, is based upon theories of developmental psychology. These theories are of a

general and qualitative character at present. The Martian research therefore constitutes a quanti-

tative examination and test of these psychological theories.

This hypothesis makes it possible to explore these research questions by means of an artificial

agent model. This is because if it is true that the development of the earliest sensorimotor habits

follows a common functional schema, the specific details of particular sensorimotor modalities

become irrelevant to the mathematics of the research. This allows the use of abstract senses and

an abstract body structure with its associated motor capabilities without the risk that the results

obtained will be fundamentally inapplicable to H. sapiens at a later time. If it should be found that

the second hypothesis is incorrect, at the minimum this research will demonstrate this, thus

narrowing the field for future research. If the hypothesis is not refuted by this research, then the

ground is laid for the next stage of research into early sensorimotor intelligence development,

which will then constitute a further test of the qualitative psychological theory.

We adopt as a working premise the following supposition: H. sapiens is born with no innate

2

The Martian Program 2009 Wells and MacPherson April 20, 2009

knowledge of objects and no copy-of-reality mechanism, but is born with a set of innate reflex

structures (endowed by biology) and some set of innate affective preferences for particular types

of physical stimuli (e.g. a baby doesn't like pain, does like gentle rocking, etc.). This premise

rules out the use of constructs from rationalist metaphysics and appears to be consistent with the

bulk of empirical findings reported over a great many years by experiments and observations in

developmental psychology. This premise leads to our third main hypothesis: Cognition and

cognitive learning in the earliest stages of life are affectivity-driven. All cognitive learning

capabilities of the Martian agent therefore take place under the impetus of affective signals and

all cognitive learning achievements must therefore be possible through unsupervised learning

mechanisms (that is, no externally-supplied desired responses or lessons imparted by an external

teacher are permitted). This hypothesis appears to be entirely consistent with empirical findings

from work carried out in treating young children suffering from severe autism [GREE].

The general context of this research can be called psychophysical. Psychophysics is defined as

the area of psychology concerned primarily with the quantitative relationship between physical

stimuli and the psychological experience of them. The Martian research program deals with

neural network systems and the signal processing taking place within them. This is a physical

model of brain function. However, psychological experience is a term that at root involves

concepts and ideas of a supersensible nature (that is, psychological quantities cannot be directly

measured and are known to us only by introspection and by the self-reports of psychology test

subjects). Inherent in all psychophysical work is the problem of interpretation and evaluation of

the operational characteristics of the Martian brain model in psychological terms. This

psychological dimension is called "mind." Put into other words, the nature of the Martian agent is

constrained by a paradigm of mind. Because some such paradigm is unavoidable in any critical

evaluation of findings, the next section of this tech brief provides a sketch of the paradigm

employed in this research program.

II. The Mind of a Martian

There is at the present time no one theory or model for defining what "mind" is or how the

context of "mental" and "mind" phenomena is to be understood or treated. In the words of Arthur

and Emily Reber, mind is "the battered offspring of the union of philosophy and psychology."

Some scientists regard "mind" as nothing other than an epiphenomenon; others see it as an

emergent property of brain; still others shun the use of the term "mind" altogether. Nonetheless,

some presuppositions about mind characteristics are required if models such as the Martian are to

have a psychophysical interpretation for applying the research findings to H. sapiens. In most

3

The Martian Program 2009 Wells and MacPherson April 20, 2009

cases, particularly within the fields of artificial intelligence research or connectionist psychology,

mind models are proposed on a rather limited and ad hoc basis. A number of representative

examples is provided in the collection of papers published jointly in the 1997 book Mind Design

II [HAUG].

The paradigm employed in the Wells Laboratory of LCNTR for the psychological dimension

(mind) is practical. This is to say that it is metaphysically meaningless to talk about mind

concepts other than in practical terms of "what mind does" for the agent. Wells' mind model is the

result of a ten year research project published in electronic form on the LCNTR web site in 2006

[WELL3]. This theory has not yet appeared in textbook form (which would make the theory more

accessible to a greater number of people), but fortunately a full understanding of this theory is not

a prerequisite for beginning researchers in the Martian Program at this stage of the project. A

simpler encapsulation of some of its consequences will adequately serve our purposes in its stead.

The psychological dimension involves two separate but highly interrelated practical

objectives: knowledge representation and the Martian's interaction with its environment. The

practical task in simplified form is illustrated in Figure 1 below. This type of model is commonly

known as a "reasoning loop" model. It requires the agent to represent to itself an "internal world"

and to use this representation to produce behavioral actions through which it interacts with its

environment. Key issues in the knowledge representation problem have been previously reviewed

in the technical literature, e.g. [WOOD], although most of these works are in paradigmatic

contexts prior to [WELL3] and tend to implicitly invoke a number of rationalist presuppositions

or copies-of-reality hypotheses that the Martian presuppositions and hypotheses reject.

A cursory examination of Figure 1 shows this model includes four ideas – perception,

judgment, expectation, and evaluation – that belong strictly to the psychological dimension of the

problem. We define perception as representation with consciousness. The various signals within a

Figure 1: Agent reasoning loop model. The agent's body is implicit in the environment block of this model.

4

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 2: Block diagram model of the actor-critic system. The agent's body is implicit in the environment

block of this model and is the immediate source of both objective and affective stimuli.

Martian agent are representations of information and of knowledge, but what does one mean by

the term "consciousness"? Here our definition must and can only be a practical one: we define

consciousness as the representation that another representation is present within the agent. As we

will later see, the neural network system of the Martian includes a number of signaling functions

that produce arousal signals (activations of various network functions). These arousals serve as

the network representations that "another representation is present" within the Martian brain.

Within the restrictions imposed by our hypotheses, these signals are all affectivity signals; that is,

they correspond to such psychological phenomena as feelings, emotions, or motivations. These

signals drive all cognitive functions of the Martian as well as its innate reflex behaviors. A model

of this class is generally called an actor-critic model. Figure 2 illustrates the actor-critic concept.

We define judgment as the act of subsuming particular representations under more general

ones. In classical logic a predication – e.g. a subject-predicate construct – was the most common

example of a judgment prior to the advent of symbolic logic in the late nineteenth and early

twentieth centuries. The judgment problem in our psychological dimension is twofold: producing

general representations from particular ones and finding particular representations to connect

under already-formed general ones.

Expectation is a form of anticipation and prediction. In everyday language, this function is

often called imagination because the expectation function is concerned with "things that haven't

happened yet" but are "anticipated to happen" by the agent. For the infant Martian, these

expectations will typically take the form of subjective anticipations – i.e. they have to do with the

5

The Martian Program 2009 Wells and MacPherson April 20, 2009

anticipation of affective rather than objective outcomes. This is because, at this stage of sensori-

motor intelligence, the Martian has not yet formed distinct cognitive representations (concepts) of

objects that it regards as belonging to its "world." Such concepts come much later and, when they

do make their first appearance, they will be founded upon the Martian's practical cognitions of its

first acquired habits. That is why this stage of the research is propaedeutic for all later stages.

We define evaluation as the determination of the relationship between perceptions and the

spontaneous actions of the agent. The evaluation function plays a regulatory role in determining

the acts carried out by the other functions depicted in Figure 1. It belongs to the psychological

phenomenon we typically call "reasoning." It, too, is generally non-cognitive in character and so

finds its logical place in the critic part of an actor-critic model.

Figures 1 and 2 are representative of the level of modeling of the psychological dimension one

finds in the bulk of the literature. These models are adequate for the context of the infant Martian

during the first stage of sensorimotor development. They are, however, abstractions and

simplifications of the more detailed model found in [WELL3]. For the sake of completeness in

our discussion of this section, the full Wells model, which is a synthetic union of the ideas

expressed in Figures 1 and 2, is depicted in Figure 3 below.

Figure 3: Detailed functional block diagram of the Wells mind model.

6

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 4: Overall block diagram of the current (2008) infant Martian network system.

III. The Martian System

Figure 4 depicts the overall network system diagram of the infant Martian in its current (2008)

form. Marty has a very simple body capable of moving around in an environment ("the world"). It

has no a priori knowledge of its world. Motor units ("muscles") allow body movements and

sensory nerves in the body allow it to sense the appearances of objects in this world as well as the

state of its own body.

The other blocks in Figure 4 depict Marty's central nervous system. These networks are

proxies for functions found in the central nervous system of H. sapiens but are not intended to be

anatomically equivalent to human brain and spinal cord structures. This requires a word of

explanation. With the exception of olfaction, all the sensory and motor systems found in H.

sapiens demonstrate a rather remarkable homogeneity at the functional level. This is to say that,

in spite of numerous and important differences at the cellular level, all our sensory systems share

a very similar signal processing organization with very common features. Likewise, a very

common architectural theme is demonstrated by our various motor systems regardless of whether

7

The Martian Program 2009 Wells and MacPherson April 20, 2009

these are spinal cord motor systems or the motor systems involved by the cranial nerves. Within

the motor system, different subsystems do have specific differences; but these differences are

largely overshadowed by the functional commonality they share – i.e. they control muscles that

move body parts. Because it is presently impractical to attempt to model in detail the entire

human body, the Martian agent is instead given a functional body that retains the key common

features of real mammalian organisms while at the same time minimizing the computational

burden our simulations must take on. We call these functional proxies "mock" systems to

emphasize their functional role while reminding ourselves that they are not anatomical models.

This heuristic approach to computational neuroscience is called the Grossberg method, after

Stephen Grossberg of Boston University who first developed it in the early 1970s.

Marty's muscles are driven by a proxy spinal cord ventral horn network. Ventral horn outputs

cause their target muscles to contract; in the absence of ventral horn signals the muscles relax.

The ventral horn network is driven by a proxy motor cortex drive network (AN F1 and F3) that

supplies high level movement commands that are decoded in the ventral horn network.

Sensory nerve signals are conveyed from the body to the brain via a proxy spinal cord dorsal

horn network. This network produces five types of output signal tracts: (1) external sense signals

that proxy the effect of the environment on the Martian's organs of external sense (analogous to

vision, audition, touch, etc.); (2) group Ia signals that provide information about the rate of

contraction or expansion of the muscles (muscle velocity); (3) group II signals that provide

information about muscle stretch or contraction; (4) group Ib signals that provide information

about muscle/tendon tension; and (5) group III signals that provide nociceptor information, i.e.

signals that the Martian will interpret as pain. At present the group Ib and group II signals are not

used by the system, although in the 2009 stage of the program these signals are expected to have

a role to play. These signals are the sole and entire source of information about its body and its

world available to the Martian.

The external sense signals are projected from the dorsal horn network to a proxy primary

sensory cortex network composed of two parts. The external perception window (EPW)

represents the sensory field of the Martian and produces mapped sensory signals arranged in a

two-dimensional retina for each specific sensory modality. At present the Martian has only one

external sensory modality but the system is organized so that additional modalities can be added.

The reason for the current restriction to only a single modality comes from the fact that during the

first stage of sensorimotor intelligence the human infant is unable to coordinate and integrate the

sensorimotor schemes for his different senses (e.g., vision is not coordinated with prehension,

etc.). Sensorimotor intelligence develops during this stage independently for each different major

8

The Martian Program 2009 Wells and MacPherson April 20, 2009

sensorimotor modality [PIAG1]. These raw sensory map signals are then projected to a contrast-

enhancing on-center/off-surround (OCOS) network that produces the sensory signals actually

received by the rest of the system. The reason for including the OCOS network is because the

sensory pathways in H. sapiens generally consist of a sequence of such networks and so the

OCOS function provides a proxy for the type of information actually available to the higher

cortical systems [MART], [KAND1].

The OCOS projects to three network subsystems, the mock higher sensory cortex S, a motor

reflex mock brain stem proxy called Resonator 1 (R1), and a mock brain stem/limbic subsystem

proxy comprised of two more resonator networks, R2 and R3. Network subsystem S performs an

early cognition function and makes projections to various motor cortex subsystems. R1 models

innate higher-level reflex responses to sensory stimuli (e.g., a mock sucking reflex) and projects

indirectly to motor cortex functions via an intervening cortical network called the assimilation

network (KBN). R1 and its pathway are justified by the fact that in H. sapiens the various non-

olfactory sensory signal tracts also make side projections to brain stem structures involved with

motor responses (e.g. the superior colliculus, inferior colliculus, etc.). Networks R2 and R3 proxy

affective functions of the mock limbic system. These networks respond with innate preferences,

i.e. affective responses to particular sensory stimulus patterns, and comprise the first stage of the

Martian critic function.

The mock dorsal horn also projects velocity signals (group Ia afferents) and nociceptor signals

(group III afferents) to a reflex hub (RH) network. The RH network is a functional proxy for

involuntary reflex responses to painful stimuli associated with the spinal cord flexor reflex

afferent (FRA) pathway and brain stem motor nuclei. The RH network projects indirectly to the

motor cortex drive network via a mock thalamus function, called dipole network 2 (DN2). This

essentially relays the FRA reflex response on to the motor cortex drive network. The signal

processing functions carried out by this motor control loop are an abstracted and greatly

simplified model of the reflex functions actually found in the mammalian spinal cord and brain

stem structures [WELL4]. It is not presently known whether this proxy system is capable of

exercising very fine control of Martian body movements, nor whether it will eventually prove

necessary to add additional FRA functions running directly between the dorsal horn network and

the ventral horn network. This is, at present, an open research question.

The affectivity network (LN) is a mock limbic system. Its purpose is to produce

representations of the affective state of the Martian and to generate arousal level signals for the

higher sensory cortex (S), dipole network 1 (DN1), and the motor cortex drive network (AN F1

and F3). It receives inputs from Resonators 2 and 3 as well as nociceptor inputs directly from the

9

The Martian Program 2009 Wells and MacPherson April 20, 2009

dorsal horn network. These arousal signals "capture the attention" of network S and energize the

motor drive network's projections to the ventral horn. The LN is part of the critic function of the

Martian.

The innate motor scheme reflex signals from Resonator 1 are relayed to the motor cortex drive

network via a mock cortex Assimilation Network, nicknamed the "know better network" (KBN).

The purpose of the KBN is to allow the Martian's cognitive function (network S) to inhibit the

innate motor reflex scheme pathway so that Marty can effect voluntary control of its movements.

Network S receives inputs from the OCOS network, group Ia (velocity) signals from the

dorsal horn, and arousal signals from the affectivity network and dipole network 1. When aroused

by the LN, network S learns EPW and velocity signal patterns and associates these patterns with

specific motor control sequences in the motor cortex network (AN F2). This constitutes a form of

psychological learning through conditioning [GROS1-2] by which sensory patterns are endowed

with practical meanings implications [PIAG3]. In this context, the meaning of a sensory signal

pattern (OCOS and group Ia signals) is "what can be done with" this objective information. The

conditioned association of objective sensory information with motor responses is realized by

means of dipole network 1 (DN1), which arouses motor association learning in both network S

and in the higher motor cortex network (AN F2). The immediate objective for this phase of the

research is for Marty to develop its first acquired habits built upon its innate sensorimotor reflex

schemes. Network S also projects outputs to dipole network 2 (DN2) in the reflex hub pathway.

Eventually this projection is intended to let the higher cortical functions override the pain reflex

schemes, but at this stage in the research program this connection to DN2 is not used.

Practical motor learning and voluntary movement control is implemented by the combination

of the higher motor cortex (AN F2) and the motor cortex drive network (AN F1 and F3). The

combination of these two network systems is called an avalanche network (AVNET). The

avalanche network concept was first proposed by Grossberg in the 1970s [GROS3]. Oddly

enough, relatively little follow-on work has been done with the AVNET and, to the best of our

knowledge, the Martian is the first system to implement practical avalanche networks as principal

elements of the actor function in an actor-critic system. The avalanche network learns sequences

of, in our case, motor commands, associates these sequences with an executive command signal

(issued by network S), and executes these practical motor schemes upon receipt of the associated

S network command signal.

IV. The Sensorimotor System of the Martian Body

The Martian body system is designed using the concept of parallel sensorimotor modalities. A

10

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 5: Basic mechanical scheme of the Martian body for a sensorimotor modality.

particular sensori-motor modality follows the general scheme illustrated in Figure 5. At present

the Martian has only one such modality but later additional ones can be easily added using

parallel body parts, possibly with mechanical variations on this scheme and with cross-coupling

interactions among different parts of the Martian body.

The world of the Martian is represented by parallel two-dimensional pixel grids called retinas.

Objects are defined in this external world by assigning numerical values to the pixels. We

presently have one retina. The world also includes a second and overlapping retina in which the

pixel values define pleasant, unpleasant, or neutral "skin" stimuli. At present this second plane is

not implemented in the Martian simulator and nociceptor stimulus is limited to muscle and joint

nociceptors implemented in the simulation code. One specific sub-objective for the 2009 research

program is to augment the muscle/joint nociceptors with skin receptors using the method just

described. The current Martian world is a 16 x 13 grid.

The Martian moves in this world by moving a sensory field window (a 5 x 5 grid for the

present sensorimotor modality). Motion up-down ("north-south") and left-right ("west-east") is

implemented using four muscles driven by the output layer of the ventral horn. The muscles are

modeled using a linearized and simplified Hill muscle model [WELL5] and the perception field is

given a mass so that the Martian body is modeled by a simple contractile element-spring-mass-

11

The Martian Program 2009 Wells and MacPherson April 20, 2009

damper system. Without loss of generality and to eliminate unnecessary mathematical

complications, the motor flexor-extensor pairs (east-west, north-south) are each one-dimensional.

This is equivalent to saying the east-west motor pair translates north and south when the north-

south motor pairs move the Martian up and down, and likewise for the north-south motor pair.

The Martian world has its coordinate system referenced to the center of the northwest pixel as

shown in Figure 6 below. World coordinates are non-negative. Martian body dynamics are

expressed in a special per-muscle coordinate system, the MacPherson coordinates (also shown in

Figure 6). North, south, east, and west positions are measured from the center of the outermost

world row or column to the outermost row or column of the Martian's sensory field window. All

these position coordinates are non-negative. Positive velocities and accelerations are in the

direction of stretching the muscle. Positive tensions are in the direction of contracting the muscle.

Each muscle has a zero-spring-force slack length L0 (= 2) that is reached when its corresponding

MacPherson coordinate Pi = L0. The Martian body dynamics are governed by the set of

MacPherson equations

( ) ( )( ) (

( )( )

⎥⎦

⎤⎢⎣

⎡=⎥

⎤⎢⎣

⎥⎦

⎤⎢⎣

⎡−−

=⎥⎦

⎤⎢⎣

⎡⋅

⎥⎦

⎤⎢⎣

⎡−−

=⎥⎦

⎤⎢⎣

⎥⎦

⎤⎢⎣

⎡−−−−

⋅+⎥⎦

⎤⎢⎣

⎡⋅+⎥

⎤⎢⎣

⎡⋅=⎥

⎤⎢⎣

SN

EW

SN

EW

SS

EE

SN

EW

NS

WE

S

E

SN

EW

SN

EW

DU

RL

SN

EW

vvvv

PPPP

FFFF

vvvv

m

TTTT

FF

LPLPLPLP

Kvvvv

Byyyy

QTTTT

&&

&&

&&

&&

00

00

)

. (1)

Figure 6: The MacPherson per-muscle coordinate system.

12

The Martian Program 2009 Wells and MacPherson April 20, 2009

Here Ti is tension in the ith direction, Fi is the force in the ith direction, vi is the velocity in the

ith direction, and Pi is the position in the ith MacPherson coordinate. We will denote the respective

MacPherson matrices as T, F, V, and P. Q is the contractile force constant of the muscle

contractile element, B is the muscle damping factor, K is the muscle linear spring constant, and m

is the body mass. The Martian's parametric values are [Q m K B] = [20.0 1.0 0.1 2.0].

The motion of the sensory field window is continuous, which means that pixels in the field

window are not necessarily aligned with those of the world. The numerical value of each pixel in

the field window (which equal the outputs of the sensory cortex EPW in Figure 4) is the sum of

the values of the world pixels it overlaps weighted by the area of overlap.

V. The Dorsal Horn, Ventral Horn, and OCOS Networks

The dorsal horn network is a functional proxy for the neural networks located in the dorsal

horn of the mammalian spinal cortex. This is not implemented in the Martian model by an

explicit neural network. Instead the dorsal horn outputs consist of the EPW data, as described

above, plus the MacPherson matrices V, T, and P, which provide the group Ia, group Ib, and

group II afferent outputs, respectively. Because muscle velocities returned by this proxy can be

negative numbers (whereas biological networks cannot explicitly signal negative magnitudes), a

word of explanation is in order here. In the mammalian spinal cortex the signaling patterns of the

group Ia intrafusal muscle fiber neurons is a complex result of the interaction of muscle stretch or

contraction and the action of gamma motor neurons in the ventral horn that signal to intrafusal

muscle fibers [WELL6]. Precisely how velocity information is conveyed from the spinal cord to

the brain stem and cerebellum is not presently well understood, but it is known that this function

involves ascending tract interneurons in the spinal cord and projections via the ventral spino-

cerebellar tract to brain stem nuclei and the cerebellum [WELL4, pt. 4]. The dorsal horn proxy

merely fulfills this overall function by providing the information directly.

The dorsal horn also provides nociceptor (pain) information as group III afferent signals. At

present this is limited to signals that indicate over-compression of muscles, i.e. when Pi < L0. The

group III signals are represented by a MacPherson matrix given as h[L0 – P] where L0 is a 2-by-2

constant matrix with values L0 and h is the Heaviside extractor function, h[x] = x if x > 0, h[x] = 0

if x ≤ 0. Strictly speaking, group III signals actually represent over-stretching of the antagonist

muscle rather than pain from over-compression of the agonist muscle, but the calculation method

is simpler and equivalent using the current implementation. At a later date we will augment the

group III information by adding a "skin plane" to the world representation, the entries in which

will denote pleasant or unpleasant body sensations associated with objects in the world. These

13

The Martian Program 2009 Wells and MacPherson April 20, 2009

will be represented using additional sets of dorsal horn signals, the groups IV and V afferents,

which will correspond to sense signals conveyed to the brain via the lateral spinothalamic tract

and the dorsal columns of the spinal cord [SHEP, pp. 284-5].

The ventral horn network is illustrated in Figure 7. It receives eight descending motor

command signals from the F1 field of the motor cortex drive network, corresponding to directions

of movement. These signals are combined by a combining layer W which weights the relative

contributions of the drive signals, i.e. sR = E + 0.5(NE + SE) and etc. for the other motor paths.

The combined signals are projected to the motor neuron layer Y. Motor neurons are grouped into

two agonist-antagonist pairs ("flexors and extensors") that laterally inhibit each other. The motor

neuron networks are defined by a system of simultaneous steady-state equations of the form

][ jii yshy ⋅−= β (2)

where yj is the output activity of the antagonist motor neuron, h is the Heaviside extractor, β = 1.

This network is a greatly simplified approximation of real ventral horn neural circuits

[WELL4]. In real mammals, the ventral horn network contains complex local reflex circuit feed-

back loops and voluntary movements are implemented by descending control signals from the

brain that "co-opt" these reflex pathways – essentially "fooling" the ventral horn networks into

carrying out their reflex responses according to brain rather than muscle spindle signals. The

consequence of this approximation is to place a heavier burden on the Martian's brain networks

for fine control of motion. In particular, posture commands – commands for the body to freeze

and assume a particular posture – become more difficult. We currently require a better model for

Figure 7: Ventral horn network.

14

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 8: OCOS Network.

implementing the innate posture reflexes observed in human infants, and this is one of the

research objects for the Martian program in 2009.

The OCOS network consists of a bank of parallel on-center/off-surround contrast-enhancing

networks, one for each pixel in the EPW. The per-pixel arrangement is illustrated in Figure 8.

OCOS contrast enhancement is common in ascending sensory pathways to the brain and the

OCOS network mimics this function. Each pixel node, i, and its immediate neighbors, j, project

their sensory values pi or pj to its associated OCOS netlet. This netlet produces an on-center/off-

surround output and an off-center/on-surround output according to the equations

. (3)

( )

( ) ( )⎥⎥⎦

⎢⎢⎣

⎡⋅+−⋅=

⎥⎥⎦

⎢⎢⎣

⎡⋅−=

=

=

4

1

4

1

125.015.0

25.0

jji

offi

jji

oni

pphy

pphy

The network employs a virtual expansion of the EPW at its boundaries in order to avoid making

adjustments to (3) for those pixels at the outermost rows and columns of the EPW. The virtual

rows and columns are merely copies of the outermost rows and columns of the EPW. The two

output signals are represented by column vectors Yon and Yoff, respectively. All pixel values are

non-negative and restricted to the range from 0 to 1 in value.

VI. Resonators 1, 2 and 3

Resonators are one of the basic signal processing networks in adaptive resonance theory. The

Martian architecture uses fixed-weight resonators to implement innate reflexes and affective

preferences in the infant Martian. All three resonators in the Martian are identical except for the

number of inputs, outputs, and fixed weights W and Z. Figure 9 shows their network structure.

15

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 9: Block diagram of the ART resonator network.

The resonator consists of an input network, GN2, that automatically normalizes the input

vector and two field networks that interact to produce stable signal vectors Xi and Xj (called the

short term memory or STM of the network). Let Y denote the input vector. The output vectors for

GN2 and the F1 field have the same number of elements, n, as Y. The F2 field has N inputs and

outputs, and typically N < n in an ART resonator. The two fields are coupled by weight matrices

W and Z, which jointly constitute the long-term memory (LTM) of the network. The theory of

operation for this network is provided in [WELL7, chapters 14 and 15].

Figure 10 illustrates the general form of a GN2 network. Each input signal projects to an Instar

node belonging to a class of Instar "neuron" known as a shunting-node Instar of class 1, SNI(1). It

also projects inhibitory inputs to the other nodes of the network. Input signals Vi correspond to

signals Ii + Ji in Figure 10. The Martian resonators do not use a secondary input vector J. The

output signals produced by GN2 are given by

( )( )( ) ⎥

⎢⎢

+

−⋅

−⋅

=∑∑

=

=

nj j

nj jii VA

nVV

nBnhxh

:1

:1

1 (4)

where B = 1 and A = 0.5 are network parameters, h is the Heaviside extractor and Vi ≥ 0.

Figure 10: Anatomy of GN2. The J input is not used in the infant Martian.

16

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 11: Block diagram of the type-1 ART resonator network.

GN2 produces output vectors having Euclidean lengths that are remarkably insensitive to the

magnitude of the input vector. In this sense, it can be said to perform an approximate automatic

gain control function. It also performs some foreground-background contrast enhancement in the

sense that input signals less than the average value of the components of the input vector produce

corresponding output signals equal to zero (background level blanking).

The resonator fields are shown in more detail in Figure 11. Within each field, each shunting-

node Instar projects and receives inhibitory signals to/from the other nodes in that field. Each F1

node projects to every F2 node through a weight vector Wi. The rows of the W matrix are

comprised of these forward-projecting weight vectors. Each F2 node makes a feedback projection

to every F1 node through a weight vector Zj. The columns of the feedback matrix Z are comprised

of these feedback vectors. These matrices are fixed in the Martian resonators and matrix Z is

merely the transpose matrix of matrix W. Each Wi vector is normalized to unit Euclidean length.

F1 and F2 are governed by a set of first-order non-linear differential equations (described in

[WELL7]) and the resonator STM patterns in each field undergo a complex set of changes until

the network as a whole comes to fixed-point equilibrium. This equilibrium is called the resonance

of the network. Each Martian resonator has n = 25 nodes in F1 and N nodes in F2, one node for

each output of the network. Resonators 1, 2 and 3 have N = 8, 5, and 6, respectively. Weight

matrices W are N x n and matrices Z are n x N. All xi and xj signals in fields 1 and 2, respectively,

are non-negative. The lateral inhibition present in each field takes place through a sigmoid

activation function (a saturating Heaviside extractor).

17

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 12: Fixed weight patterns for Wi and Zj in the Martian resonators. Filled squares denote 1; unfilled

squares denote 0.

The resonators respond preferentially to particular input patterns presented by GN2. There are

a total of fifteen such patterns, defined and named in Figure 12. These patterns represent innate

preferences or reflex conditions for the Martian system. Resonator 3 responds to six innate

preferences for off-center/on-surround output vectors Yoff from the OCOS network. In the order

of the F2 node enumeration for Resonator 3, these patterns are SquareS, squareB, Weight_N,

Weight_S, Weight_E, and Weight_W. It is important to note that these responses are to the GN2

output vector rather than the OCOS vector. GN2 eliminates the background level of the Yoff

vector. Resonator 3 is part of the Martian's mock limbic system and the Martian finds these six

patterns "exciting" and will react by seeking to maximize this excitation.

Resonator 2 is also part of the Martian's limbic system. Its outputs also represent innate

preferences but in the case of Resonator 2 these affective perceptions are representative of what

in German is called "loost" (Lust). Like the word gestalt, this word does not translate well into

English and does not mean the same thing as the English word "lust." It denotes a kind of

"motivated wanting" that corresponds to what philosophers and artists call the sense of aesthetics

and which Kant called "taste" (as in "good taste in art"). These preferences likewise correspond to

what American philosopher George Santayana called "the sense of beauty." The theoretical basis

for the presence of this type of affectivity within an intelligent agent is a requirement of the mind

model of Figure 3 and is part of the process of reflective judgment illustrated in that figure.

Resonator 2 receives the Yon vector as its input and responds to five "beautiful" patterns:

18

The Martian Program 2009 Wells and MacPherson April 20, 2009

SquareW, lineVW, lineHW, lineDpW, and lineDnW. Resonator 2, Resonator 3, and the

affectivity network (LN) jointly make up part of the process of aesthetical reflective judgment

within the reflective judgment block of Figure 3. The outcome of this part of the system is arousal

the perception network S and energizing of the motor cortex drive field F1.

Resonator 1 is also part of the Martian's reflective judgment but its role is non-aesthetic. In

Wells' theory [WELL3, chapters 16, 18-19] this function is called teleological reflective

judgment. The immediate function of Resonator 1 is to tie OCOS patterns (specifically, Yon

patterns) to innate motor reflex reactions. It responds to eight innate patterns: the patterns

Weight_E, N, W, and S (which are also used by Resonator 3) as well as additional patterns

Weight_NE, Weight_NW, Weight_SW, and Weight_SE. Its outputs indirectly drive the motor

avalanche field F1 via the assimilation network (KBN). The design of the Resonator 1 subsystem

is not as straight-forward as is the case of the other two resonators. The patterns Resonator 1 is

predisposed to respond to are EPW patterns rather than OCOS output patterns. In other words,

Resonator 1 is presented with OCOS output signals but is designed to resonant with EPW

patterns. The reason is that Resonator 1 produces motor command signals for moving the Martian

toward objects in the environment that correspond to the "exciting" and "beautiful" preferences

mechanized by the other two resonators. The design of Resonator 1 was carried out empirically

by MacPherson and Wells in June of 2008 with the result now implemented in the Martian. The

details of this design are documented in [MacP1].

VII. The Affectivity Network LN

The affectivity network (LN) is the primary arousal network for the Stage I Martian system. It

models the minimum functionality required for arousing reflex reactions and stimulating the

learning of objective patterns by the S network. As such, it is not a complete design for the

general limbic function and the LN model will be undergoing some changes and improvements in

2009 dictated by findings obtained from the 2008 Martian studies. It receives inputs from

Resonator 2 and Resonator 3 and produces arousal signals sent to the S network, dipole network

1, and the F1 field of the motor cortex drive network. Figure 13 illustrates the LN subsystem.

The LN contains four nodes that define the affective state of the Martian. Its equations are

0else;0,

00

2.00

8.00.00.00.15.05.00.10.0

0.00.05.00.00.00.00.05.0

3res3res2

4

3

2

1

4

3

2

1

=≥

⎥⎥⎥⎥

⎢⎢⎢⎢

+Γ+++

⎥⎥⎥⎥

⎢⎢⎢⎢

⎥⎥⎥⎥

⎢⎢⎢⎢

−−−

−−

=

⎥⎥⎥⎥

⎢⎢⎢⎢

ii xxXX

xxxx

xxxx

432 BBB

&

&

&

&

(5)

where Xres2, Xres3, and Γ3 are column vector outputs from Resonator 2, Resonator 3, and the group

19

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 13: Affectivity Network in the 2008 version of the Martian agent.

III (nociceptor) afferents, respectively. The input distribution matrices B2, 3, 4 are currently defined

as

⎥⎥⎥⎥

⎢⎢⎢⎢

=

⎥⎥⎥⎥

⎢⎢⎢⎢

=

⎥⎥⎥⎥

⎢⎢⎢⎢

=

0000000011110000

;

00000061616161610

000000000001

;

00000000000000000001

432 BBB .

Except for the threshold restriction that all xi in the network be non-negative, the LN is a linear

and time-invariant system with four repeated eigenvalues λ = –0.5.

A consequence of this design is that the Martian will only attempt to learn an enclosure pattern

(corresponding to the squareS Yoff and SquareW Yon patterns of Figure 12). This was adequate for

our initial 2008 experiments with the Martian system but is in fact contrary to the intent that the

Martian is to "value" all the "beautiful" patterns defined for Resonator 3. It also places some

restrictions on the abilities of the system that play a role in the system's present inability to learn

primary circular reaction schemes, which is the key capability exhibited in Stage II of sensori-

motor development [PIAG1], [WELL8]. Therefore, in order to allow us to study a more complex

set of Stage I dynamics and to prepare for research work on the problem of the development of

primary circular reactions, the 2009 Martian program will extend the current model of the LN

subsystem. The working concept is illustrated in Figure 14.

The new principal element for this enhanced LN subsystem is a network called a type-2

Grossberg classifier [WELL7, chapter 14]. Figure 15 illustrates this network in block diagram

form. A type-2 Grossberg classifier (CL(2)) is a pattern-recognizing network consisting of two

fields. Its v1 field is essentially a Grossberg normalizer that projects through a weight matrix to

the inputs of the v2 field. The v2 field is essentially a competitive network capable of winner-take-

all operation from lateral inhibition among class-3 of shunting-node Instars [WELL7, chapter 14].

20

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 14: Concept of the enhanced affectivity network (LN) for the 2009 research work.

Figure 15: Block diagram representation of the type-2 Grossberg classifier network.

This enhancement will necessitate changes to the LN dynamical equation. Projections from

CL(2) to the LN nodes x1 and x2 will be through independent weight vectors. We conjecture that

this enhancement will provide a more accurate implementation of the reflective judgment

function of Figure 3. It also provides a structural form for the application of feedback from the S

network to the affectivity function by which objective perceptions stored in the S network

subsystem can affect affectivity. (This is the actor-to-critic feedback path shown in Figure 2). A

feedback link of this sort is necessary if the Martian is to be able to "anticipate a future

satisfaction" expected from the application of a sensorimotor scheme. Such an ability is a

function for acquiring learned somatic markers [DAMA1], i.e. learned "likes and dislikes."

VIII. The Reflex Hub

The reflex hub (RH) network provides a reflex reaction to painful (nociceptor) stimuli. When

the accumulated effect of group III signaling surpasses a pain threshold, the RH network is

activated and sends a motor command to the motor cortex drive (F1 avalanche) network reversing

21

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 16: The tract from reflex hub to avalanche F1 layer.

the direction of the Martian's current motion. This is accomplished by stimulating dipole network

2 (DN2) such that motor signals for the current direction of travel are inhibited and motor

commands for moving in the opposite direction are stimulated. The general schema is illustrated

in Figure 16.

The reflex hub contains 26 nodes. Two of these nodes (x0 and xw) are affectivity-function

nodes. The remaining 24 nodes implement the motor reflex action. They are arranged in eight

slabs of three layers each as illustrated in Figure 17. The output layers of these eight slabs

laterally inhibit each other, forming a competitive layer detecting the most-active slab. The input

layer of each slab (x1,j) receives velocity (group Ia) and motor avalanche F1 signals conveying the

Figure 17: Reflex hub network.

22

The Martian Program 2009 Wells and MacPherson April 20, 2009

direction-of-travel information. Under normal conditions the output of node x0 is 0 and second

layer nodes x2,j integrate the outputs of the first layer. Feedforward from the second to the third

layer is inhibited by node x0. When x0 rises above a pain threshold, Γn, the first layer nodes are

inhibited and the feedforward path from the second to the third layer is enabled. This sets up a

positive feedback loop between the second and third layers, which results in a saturating output

from the x3,j third layer. Lateral inhibition between the slabs typically results in a winner-take-all

competition in which only the most-active slab produces a non-zero output signal. The network

dynamical equations are

( )

( )

( ) ( )

.81

,

2

23,2023,3,333,3

,332,112,222,2

.1101,111,1

00

00000

L

&

&

&

&

&

=

Γ⋅Γ−⋅+⋅−−=

++−=

⋅+⋅+Γ−⋅−−=+−=

−⋅+−=

∑≠

j

xfxhxxx

xxxx

Vxxhxxxxx

xhxx

jnjm

mxjj

jjjj

jvjffnnjj

wwww

n

ααα

ααα

αααααα

ναα

(6)

Here,

( )⎩⎨⎧

>≤

=0,0,0

xxx

xh

is the Heaviside extractor activation function and

( )⎪⎩

⎪⎨

Γ≥ΓΓ<<

≤=Γ

xxx

xxf

,0,

0,0,

is the sigmoid activation function. The parameters of the system are: α0 = 2; αn0 = 0.1; αw = 5⋅10-4;

α0w =0.003; α11 = α22 = α33 = 1; αn1 = 3; α12 = 1; α23 = 4; α32 = 4; αx = 0.25; αf1 = 0; αv = 1; Γn =

0.75; Γ23 = 0.5. The pain input, ν, is given by the sum of the group III nociceptor signals. Signals

xf1.j are the output signals from the F1 field of the motor cortex drive avalanche network with the

enumeration index j corresponding to motor commands E, NE, N, NW, W, SW, S, and SE,

respectively. Velocity inputs Vj correspond to directions E, N, W, and S for odd-numbered j and

zero for even-numbered j. (This is because there are only four group Ia signals). Because αf1 = 0,

only half of the RH nodes are actually active in the Martian. An optimum feedback gain from the

motor command outputs of the F1 avalanche layer to the RH has not yet been determined and so

at present this feedback loop is effectively disconnected. This will change in 2009.

Once x0 rises above the pain threshold, it inhibits the direct input to the RH hub and movement

is entirely controlled by the self-saturating short term memory (STM) of the second and third

23

The Martian Program 2009 Wells and MacPherson April 20, 2009

layers. In effect, this is a "flight response" from a painful stimulation. We nickname the x0 signal

the "ouch" signal. It corresponds to consciousness of pain. The xw signal is nicknamed the "wa-

wa" signal; metaphorically, it represents a vocalization reflex to pain ("crying") and when it rises

above a threshold level of 10, xw inhibits all motor drive to the ventral horn, allowing the

Martian's muscles to pull it to a point of equilibrium near the center of its environment.

The x3 signals project to the drive inputs of dipole network 2. In the absence of object feature

signal input from the S network, this activates the output layer of the dipole network (Figure 16).

This layer projects an inhibitory signal to the corresponding node in the avalanche F1 layer and an

excitatory signal to the antagonist F1 signal (e.g., inhibit the N signal and energize the S signal).

This reverses the Martian's primary direction of movement and constitutes the behavioral effect

of the pain reflex.

The function represented by the RH network is not well-localized in mammalian physical

anatomy. It represents in part the flexor reflex afferent (FRA) pathway in the spinal column

[WELL4], in part reflex motor center functions in brain stem, and in part motor control function

in cerebellum. Thus, the RH network is an abstract representation of this overall reflex function in

mammalian anatomy.

IX. Posture Reflex

Infants in late Stage I and in Stage II of sensorimotor development exhibit a variety of posture

reflexes that accompany the execution of primary circular reactions [WELL8]. This is particularly

observable in the case of the sucking reflex [PIAG1, observation 7, 16, 18]. Observations suggest

that this reflex is driven by recognition (objective perception of total body/sensory state) and is

therefore a learned assimilation of the state of the subject with the posture reflex. It appears to be

the case that the infant associates a perceptible body state with an affective reaction

corresponding to satisfaction with this state. The 2008 version of the Martian model does not

include any functional capacity to realize such a posture reaction, which is a more advanced

capability and was outside the research aims for the year 2008. However, for the research aims of

2009 it appears to be necessary – based on analysis of current Martian behavior and capabilities –

to add the capacity for such a posture reflex. For the Martian, the analog of a posture reflex is to

position the body at rest in a location that maximizes and maintains the Martian's sense of

satisfaction that is to accompany an affective feeling of "beauty" (section VI).

This posture function is similar in most respects to the well-known regulator problem in

control theory. In the regulator problem, the control law of the system brings the system to rest in

a state where an "error signal" between a set point reference and the position (or other state) of

24

The Martian Program 2009 Wells and MacPherson April 20, 2009

the system is minimized. However, how this is done in mammals, including H. sapiens, is

presently very far from well understood. Classical control system configurations are not clearly

evident in the neurological organization of the mammalian nervous system and the multiple-

input/multiple-output and nonlinear nature of the posture control problem produces a quite

difficult problem for control system theory.

It appears to be well established that voluntary movement, including the posture reflex, works

fundamentally by the mechanism of having descending signals from brain to the ventral horn co-

opt and "take over" the spinal cord's innate reflex pathway [WELL4, pt. I and IV]. This is known

as the generalized reflex afferent (GRA) hypothesis. Wells has privately conjectured that spinal

cord organization in the ventral horn is such as to imply that the control system employed in

mammals may be a type of nonlinear control scheme known in the engineering world as a

"variable structure switching control" (VSSC) system. It may be possible to implement a VSSC

system for posture control in the Martian through a modification of the affectivity network (LN)

and augmenting the reflex hub network to add a posture reflex mechanism driven by a control

signal derived from LN node x1. At the date of this writing, this idea is still under development

and will be described in the last section of this paper and discussed in a later report [WELL9].

X. The Assimilation Network

Figure 18 depicts the assimilation network (KBN). It consists of a parallel bank of eight

simple networks and its purpose is to allow the Martian's higher cognitive center (the S network)

to take over voluntary control of movements. To do so, it must be able to block the reflex motor

signals coming out of Resonator 1 and this is what the assimilation network does.

Each of the assimilator banks consists of two nodes, x1.n and x2.n. x2.n receives an excitatory

input from one of the eight output signals from Resonator 1. It also receives an inhibitory input

from its x1.n node and projects an excitatory signal (positive feedback) to this node. The x1.n node

receives input signals sj from all the output (field F2) nodes of the S network. The connection

weights zj,n for these inputs are adaptive and adapt according to an Outstar adaptation rule. The

weights have initial values of zero. When all sj inputs are zero, or when the corresponding zj,n is

zero, node x1.n merely relays the Resonator 1 motor command signal. When an sj input is non-zero

it excites its x2.n node by an amount determined by its zj,n weight. If, in addition, the xR.n input

from Resonator 1 is non-zero, the weight undergoes adaptation. The dynamical equations are

( ) ( )( )

.0, .2.112.2.222.2

.221.,1.111.1

.1...

≥−⋅+−=

+⋅Γ−+−=

⋅Γ−−⋅Γ−⋅=

∑ =

nnnRRnn

nnjsjmjnn

nsnjjRnRnj

xxxxx

xzsHxx

xzshxhz

ααα

αα

λ

&

&

&

(7)

25

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 18: The assimilation network.

Here H is the Heaviside step function and h is the Heaviside extractor function. The projection

from xR.n to the synaptic knobs zj.n represents a metabotropic modulation of the weight value. An

Outstar adaptation rule of this sort is called a facilitated Outstar adaptation rule or f-OAR. Γs and

ΓR are thresholds. λ is an adaptation rate constant. The current parameter values for the KBN are:

α11 = 1; α22 = 1; α12 = 5; αR2 = 1; α21 = 0.005; Γs = 0; ΓR = 0.2; and λ = 1.5.

The S network signals sj are binary valued, i.e., sj ∈ 0, 1. With Γs = 0, the steady-state

adapted value for weights zj.n = 1 and for the range of signal values and parameters given to the

Martian, activation of an S network signal applied to a fully adapted weight is sufficient to

entirely inhibit output from its x2.n node. Note that both x1.n and x2.n are assumed to be conditioned

by a Heaviside extractor activation function so that these values are always non-negative.

XI. The Dipole Networks

Dipole networks 1 and 2 (DN1, DN2) are comprised of banks of Grossberg non-recurrent

dipole layers [GROSS2]. Figure 19 illustrates the Grossberg dipole layer. Each layer receives one

or more drive inputs, D, and a bias input B. It also receives m signals, sj, from the S network.

These signals are weighted by adaptive weights wj.i (i = 3 or 4, j = 1 to m) as they go into hidden

layer Instars I3 and I4. These weights are plastic weights, i.e. adaptive changes to the weight

values are long-term changes. Input nodes I1 and I2 project to I3 and I4 through elastic weights z1

and z2, i.e. the changes undergone by these weights are non-permanent and return to basal levels

26

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 19: Basic dipole layer used in dipole networks DN1 and DN2.

when input activity ceases. In the absence of drive (D) or sj inputs, the outputs of the dipole layer

are zero. The elastic weights are what permit drive signal inputs to produce a non-zero output at I5

and a rebound output at I6 upon removal of the drive input [GROSS2]. The dynamical equations

of the dipole layer network are

(8)

( )( )

( )( )

( )

( )

( ) ( )( ) ( )

( ) ( ).

4,3,DB

B

OO

..

...

11

22

1111

2222

13.1133

14.2244

4355

3466

55

66

γργθγ

ηγαα

δγβδγβ

ζε

ζε

κεκε

λλ

Γ−⋅+⋅−=

∈⋅Γ−⋅+⋅−=++⋅−=

+⋅−=⋅Γ−⋅−−⋅=⋅Γ−⋅−−⋅=

⋅+⋅Γ−⋅+⋅−=

⋅+⋅Γ−⋅+⋅−=

−⋅+⋅−=−⋅+⋅−=

Ω−⋅=Ω−⋅=

=

=

jijij

isjijijij

m

jjj

m

jjj

sh

ixshwwxxxx

zxhzzzxhzz

swzxhxx

swzxhxx

xxhxxxxhxx

xhxh

&

&

&

&

&

&

&

&

&

&

The last pair of equations in (8) is called the Wells-Hill adaptation rule. Dipole layer parameters

are the same for DN1 and DN2. The bias input for DN1 is currently fixed at B = 6; for DN2 the

bias is B = 1. Current parameters are: λ = 32; Ω = 0; ε = 4; κ = 1; ζ = 4/3; Γ = 1/2; β = 1; γ = 3; δ

= 2/3; α = 3; η = 4; Γs = 1/2; θ = 1; Γγ = 1/2; ρ = 1. Note that the γ parameter is not the same as γj.i

27

The Martian Program 2009 Wells and MacPherson April 20, 2009

in (8). The weight values wj.i are constrained such that 0 ≤ wj.i ≤ 10.

Dipole network DN1 contains only a single dipole layer. It receives its drive input D1 from

signals originating in the affectivity network. D1 is binary valued and is equal to 1 if and only if

the state of the affectivity network meets a set of conditions; otherwise it is zero. The conditions

for setting D1 = 1 are: LN node x3 > Γpain = 0.8 or x1 > ΓLA = 0.45. Here Γpain is a pain arousal

threshold and ΓLA is an attention arousal threshold for non-painful affective stimulation. D1 is also

sent to the S network, where it serves as an arousal signal causing that network to pay attention to

the sensory map signals it is receiving from the OCOS network. If the S network recognizes the

OCOS pattern, it will respond by activating one of its sj outputs, which initiates adaptation of the

corresponding wj.i weights in DN1. For this reason, D1 is nicknamed the is looking signal.

Application of the drive signal will activate output O5 of DN1, but the elastic decay of z1 will

eventually cause inactivation of O5 and activation of O6 unless a signal sj is received and its wj.i

adapts to a large enough value such that sj can maintain O5. Calculations indicate that only about

9 simulation time steps can elapse before O5 inactivates in the absence of sj. However, we have

not explicitly observed these signals to verify this and we do still require a robustness analysis of

the system dynamics here because this dynamic is one of the crucial determiners of the speed of

motor learning in the motor avalanche network. This is part of our 2009 project work.

It is to be noted also that while drive signal D1 arouses attention to the OCOS signal by the S

network, D1 itself does not arouse pattern learning in the S network. A second arousal signal, Λ,

is required for this. Λ is similar to D1 except that its threshold conditions are LN x3 > Γpain2 = 1.0

or x1 > ΓLA2 = 0.60. Why this difference was implemented in the 2008 Martian was not

documented and so the reason for this different and higher learning-induction threshold is no

longer understood. An analysis of the learning dynamics in light of this is required because of a

concern that this threshold difference between attention and learning induction may lead to lack

of robust or reliable motor learning in the Martian system. This, too, is part of the 2009 work.

Dipole network 2 (DN2) contains eight banks of dipole layers. The parameters and equations

for each layer are the same as for DN1 with the exception that its bias input is B = 1. In the case

of DN2, the drive signals are supplied directly by the eight outputs of the reflex hub network. At

present the object feature signal path gains wj.i are set to zero and cannot change. This is because

the S network connection to DN2 is being reserved for possible application at a later stage of

Martian sensorimotor development. Application of the drive signal D2 from the reflex hub will

activate its corresponding output O5 and trigger the pain withdrawal reflex.

Output O6 is not currently being used in the Martian. This output produces a rebound signal

that activates briefly when the signal driving O5 is removed and could be used as a change-of-

28

The Martian Program 2009 Wells and MacPherson April 20, 2009

condition indicator for future functional capacities of the Martian.

XII. The Motor Avalanche Network I: The AN F2 Network

The avalanche network was discovered and developed by Grossberg in 1969 [GROSS4-7] as a

method for learning and reproducing arbitrary complicated space-time patterns. In its basic form

an avalanche network minimally consists of two network fields, AN F1 and AN F2. The F2 field is

the fundamental timing network and the F1 field learns the spatial patterns at different moments in

time. It can learn as many different patterns as there are nodes in the F2 networks that drive it.

In the Martian the motor cortex network is a bank of three AN F2 avalanche fields. Each

consists of a chain of basic vertices, ( ) ( )221 vv n− , each of which is composed of three nodes. The

superscript in this notation denotes the F2 field. Figure 20 illustrates the F2 avalanche chain. The

internal makeup of each vertex is illustrated in Figure 21.

Figure 20: The F2 avalanche chain.

Figure 21: Details of an avalanche F2 vertex.

29

The Martian Program 2009 Wells and MacPherson April 20, 2009

The mathematical design of the F2 chain for the Martian is documented in [WELL10]. The

network is inert until it is adequately aroused by an arousal signal, A, that is applied to each

vertex in the F2 chain. This arousal signal must exceed a minimum value, Amin, and cannot exceed

a maximum value, Amax. The propagation speed of the avalanche is determined by the magnitude

of A. This arousal signal is generated by the O5 node of dipole network 1. The F2 network

internally generates a related signal, A , depicted in Figure 20 as the "Not-A" signal. The purpose

of this signal is to clear the initial conditions within the vertices of the chain and reset it to

properly execute an avalanche sequence. A is an analog signal. When stimulated by an input

pulse, I1, applied to the first vertex in the chain, the chain will produce a series of output signals,

one at a time, from each vertex in the chain. The timing and duration of these output signals are

determined by the dynamics of the network of Figure 21. The active vertex projects to each

vertex in the motor cortex drive network (AN F1 and F3 in Figure 4). At the end of its timed

interval, the F2 vertex stimulates the next vertex in the chain and resets itself. This cascade of

successive vertex activity is called the avalanche.

The dynamics of each vertex are identical and are governed by the fourth-order nonlinear

differential equation

( )( ) ( ) ( )

( )( )( )[ ] ( )

( )[ ] ( )

( )( ) ( ) ( )

( )[ ] ( )( ) ( )

( )( ) ( ) ( )

( )[ ] ( )( )[ ] ( )

( )

( ) ( ) ( )( )2

22323023

233323

2223213

21131

23

222212

21121

22

221,1331

23313

2111

21 ,

i

iAiiii

iiii

iciciiiii

xt

Axxgxgtx

tAxxgtx

tIxgwuxgxtx

⋅Γ⋅−Γ−Γ⋅=Γ

⋅−−Γ−−Γ−=

+−Γ−=

+Γ−⋅+Γ−+−= −−

δβ

βααα

αα

αα

&

&

&

&

. (9)

The mathematical details of (9) are explained in [WELL10]. With only four equations making up

(9), the number of available parameters results in an underconstrained system. The design of an

F2 avalanche is therefore something of an art inasmuch as many parameters can be chosen to

optimize or at least establish broader network system performance factors. A design heuristic for

this is provided in [WELL10]. The Martian parameters are: α11 = 0.5; α22 = 0.125; α33 = 0.75; α13

= 1.0; α21 = 2.0; α31 = 1.5; α32 = 2.0; Γ31 = 0.066667; Γc = 20.0; Γ12 = 1.0; Γ13 = 0.66667; Γ23 =

20.8; βA = 10. MacPherson's 2008 implementation of the Martian leaves threshold parameter Γ23

set at a constant value, i.e. the fourth equation in (9) effectively has β and δ set at zero and the

function performed by the time-varying Γ23 is taken over by the "Not-A" signal instead.

Activation function g3 is a sigmoid function with parameter u3 = 1. Activation function gc is

the unit Heaviside step function activation. This is the activation function placed between node 2

and the first node of the next vertex in the avalanche chain. Use of this activation function for

propagating the avalanche was decided upon after experimenting with the properties of the

30

The Martian Program 2009 Wells and MacPherson April 20, 2009

avalanche action realized by this system. The connection weight between vertices in the chain is

set to w = 0.50. The Heaviside step function activation provides for reduced timing jitter in the

avalanche in regard to when successive nodes in the chain are fired. Activation function g1 is the

Heaviside extractor activation function. Activation function g2 is also the Heaviside extractor

function in the MacPherson implementation of the Martian.

The dynamics of the "Not-A" signal are determined by

10 +⋅−⋅−= AAtdAd βα (10)

with α0 = 0.1 and β = 4.0. (10) functionally replaces the fourth equation in (9). All signal

variables in system (9)-(10) are operated upon by a Heaviside extractor activation function that

limits all signal values to non-negative quantities. This is not shown explicitly in (9)-(10) because

the mathematical notation required to make this explicit makes the equations needlessly difficult

to understand. It is to be understood instead that the Heaviside extractor is always involved in the

calculation of every signal variable in the network.

XIII. The Motor Avalanche Network II: The AN F3 Network

In Grossberg's minimal avalanche network the F2 field projects directly into the F1 field. This

has certain advantages in some applications but does permit the existence of gaps in learning

sequence when one F2 vertex deactivates before the next becomes fully active and begins driving

the motor cortex drive field. The Martian system eliminates this gap by interposing an additional

avalanche field, the AN F3 field, between AN F2 and AN F1. Figure 22 illustrates the AN F3 field.

Figure 22: The AN F3 field of the motor cortex drive network. H is the unit Heaviside step function.

31

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 23: A Y1 node in avalanche field F3. H is the Heaviside unit step function.

Figure 24: A Y2 node in avalanche field F3. h is the Heaviside extractor activation function.

AN F3 is composed of two layers, the Y1 layer and the Y2 layer. The nodes in these layers are

illustrated in Figures 23 and 24. Both are in essence nothing other than forms of McCulloch-Pitts

neuron models, the earliest formal neuron model and the forerunner of the logic gates used in the

digital computer [NEUM]. In difference equation form, the Y1 node is defined by the equation

( ) ( ) ( )( )( ) ( ) ( ) ( ) ( )[ ]C1112

21 −−+Γ−=∆+ + tytyxHHtty llll (11)

where H is the Heaviside unit step function and C = H(θ – A) is a "not aroused" signal denoting

that the arousal signal A from dipole network 1 is not activated. Threshold θ = 0.6 is the

minimum arousal level for AN F2 with the parameters employed in the Martian implementation

[WELL10]. Threshold Γ = 17 is the threshold required for the AN F2 output signal to be active.

It must here be noted that the 2008 version of the Martian design does not follow the design

described here. Two differences in particular must be noted. First, the 2008 version of Y1 left out

the reset term C. This was a simple error in programming and it can produce erroneous results

(frozen y1 outputs) under certain conditions that can arise in the course of Martian operation. One

of the minor tasks for 2009 is to fix this error. The second difference is that the 2008 version of

the Martian does not use the arousal output from DN1 directly. Rather, it passes this signal

through a form of Heaviside function such that A = 1.5 when DN1 issues an arousal signal and A

32

The Martian Program 2009 Wells and MacPherson April 20, 2009

= 0 otherwise. This value is within the required operating range 0.6 < A < 1.6 set up by the AN F2

parameter set but limits the speed of the avalanche cascade to a single fixed rate. This is probably

acceptable for the 2009 Martian Program objectives, but it does have the consequence of not

allowing variable execution speeds in the Martian's voluntary motor responses. However, this is

currently appropriate until a more flexible affectivity network is developed.

Functionally, the operation of Y1 is quite simple. A Y1 node becomes aroused when its

corresponding AN F2 node issues an output signal. It remains aroused until the next Y1 node in

the avalanche chain becomes active. This prevents gaps in the motor learning sequence. The

system dynamics are such that the F2 output signal and the next Y1 node are never jointly active

with the set of parameters used in AN F2. Signal C ensures that AN F3 becomes inactive when

arousal of AN F2 is removed.

Y2 directly drives the output avalanche field AN F1. The Y2 output nodes are governed by the

equation

( ) ( )( )Γ−+⋅⋅= 2112312 Ayhy ll αα (12)

where the parameters are: α31 = 0.1; α12 = 4.0; Γ = 2. Here h is the Heaviside extractor activation

function. Signal A2 is called the motivation arousal signal. Its purpose is to allow for a future

enhancement of the Martian that will permit the strength of its output motor drive to be tied to the

degree of affective arousal of the Martian, i.e. its "level of motivation." This concept has not yet

been concretely defined for the Martian model since the proper interpretation of "motivation"

must involve further refinements of the reflective judgment function in Martian affectivity.

The MacPherson implementation of the 2008 version uses a pragmatic proxy for the A2 signal.

It is derived from the assimilation network using whichever KBN x1 node is currently the most

active. The expression currently being used is

( )[ ]uxgA ,max 132 Γ−⋅= α

where g3 is the sigmoid activation function, α = 155, Γ = 0.005, and u = 0.005. This limits the

range of A2 to 0 ≤ A2 ≤ 0.775. As noted, the reasoning here is pragmatic rather than properly

neurological or psychological. By making A2 depend on the strength of the most active KBN x1

node, a proxy for affective arousal of the Martian by the cognitive subsystem (the S network) is

introduced. However, this is neither a theoretical model nor a long-term solution for the

"motivation" question in affectivity. It is a temporary device and will later be replaced in the

Martian model by a proper theoretical model.

XIV. The Motor Avalanche Network III: The AN F1 Network

The AN F1 network is the motor learning field of the motor cortex drive system. It is derived

33

The Martian Program 2009 Wells and MacPherson April 20, 2009

from Grossberg's original work but includes a number of enhancements to Grossberg's basic

model. Figure 25 illustrates this network.

AN F1 has three distinct sources of signal input and produces the eight motor command

signals sent to the ventral horn network. Its input sources are: (1) the innate motor scheme stimuli

from the assimilation network; (2) the FRA reflex stimuli from the reflex hub network; and (3)

descending voluntary motor commands from the Y2 layer of AN F3. The first two sources are

combined to produce unconditioned stimuli signals uj feeding directly into the F1 output nodes.

Using the notation defined in Figure 25, the unconditioned stimuli inputs are given by

( ) ( )( )jajjj IIGxGhxu .5.5DN2.2KBN3 5.0 −⋅−⋅⋅+= (13)

where j runs from 1 to 8, GKBN = 0.8, GDN2 = 0.025, x3 is the third node in the affectivity network,

x2.j is the corresponding output signal from the assimilation network for F1 node j, I5.j is the

corresponding O5 output from dipole network 2, and I5.ja is the antagonist O5 output from DN2.

Details of the F1 node are illustrated in Figure 25. Each of the J = 8 F1 nodes receives a net

input signal uj as defined by (13) plus N weighted input projections from the AN F3 field, one for

each Y2 node in F3. The projection weight from the nth Y2 node to the jth F1 node is zj.n. Each F1

node also receives as a control input a non-specific arousal signal H[Σ(un) – Γ]. The sum is taken

Figure 25: The AN F1 avalanche network subsystem.

34

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 26: Details of the AN F1 node

over all J uj inputs to the F1 node, H is the Heaviside unit step function, and Γ = 0.1 is a threshold.

When this non-specific arousal is non-zero, it permits adaptation of the projection weights zj.n.

The adaptation rule, described below, is a modification of the Wells-Hill rule and is called the

Wells-Hill-MacPherson rule. These rules are variants of the Grossberg-Carpenter Outstar rule.

A remark about the neurological relevance of the non-specific arousal term is in order. At the

network system level of modeling employed in the Martian, individual nodes do not represent

single biological neurons. Rather, every node at this level models some population of cells, the

collective interactions of which are hypothesized to result in the function described by the node

equations. Weight adaptation is the functional representation of what on a biological scale would

be mechanized by plastic changes in the strengths of synaptic connections at the cell level. All

such changes are ultimately biochemical (metabotropic) in nature. The non-specific arousal term

is merely the functional equivalent of saying that some (presumably) metabotropic signaling

pathways exist within the putative neural population being modeled and that long-term changes in

synaptic strength are controlled and modulated by the presence of this non-specific arousal.

The dynamical equations describing the jth node are

. (14) ( )

nnn

J

jjjnnjnnj

j

N

nnnjjj

y

uHxyzz

uyzxx

.2

1.2..

1.2.

⋅+⋅−=⎥⎥⎦

⎢⎢⎣

⎡Γ−⋅⋅⋅+⋅−=

+⋅⋅+⋅−=

=

=

ζγεγ

δγ

βα

&

&

&

35

The Martian Program 2009 Wells and MacPherson April 20, 2009

The parameters of (14) are: α = 2.5; β = 3; δ = 50; ε = 9; and ζ = 300. Variable γn controls

memory retention in the Wells-Hill rule. We further presume that the dynamics of the γn variable

are much faster than the rest of the system such that γn reaches steady state long before the other

adaptation dynamics fully get underway. This is equivalent to merely using the steady-state value

nn y .2⋅=εζγ (15)

in equations (14) rather than numerically solving the γn equation. Note that when the F3 input is

zero, γn is also equal to zero. Likewise, when the non-specific arousal signal is zero, no weight

adaptation occurs. Further note that because γn is determined by F3, the γn for each Y2-F1 pathway

is identical and independent of F1 node j. Signals y2.n are binary-valued as given by (12).

Motor learning is carried out in the F1 field and is stored in the connection weights zj.n.

Because the motor pattern being learned is the input vector U of inputs uj, weight adaptation must

be inhibited when the Martian is executing a voluntary movement and the assimilation network is

inhibiting the reflex response input from Resonator 1. This is the function implemented by the

Wells-Hill-MacPherson rule. In general, only one y2.n node is active (non-zero) at a time and in

the presence of the non-specific arousal the solution of (14) approaches the steady-state limits

n

j

n

jnj

n

j

n

jj

yu

yu

z

yu

yu

x

.2.2.

.2.2

150750450150750

300

−=

⋅⋅⋅−⋅

⋅⋅→

−=

⋅⋅⋅−⋅

⋅→

βδεζαδε

βδεζαζ

(16)

using the parameter values given above. Using the earlier parameters for the F3 field, the

activated value of y2.n = 0.1(2 + A2). By comparison, when the F3 field is inactive the F1 output

approaches the steady-state value xj = uj/α = uj/2.5. When A2 is maximal (= 0.775), (16) yields the

result xj → uj/2.375. Thus, during motor learning the motion commands to the ventral horn

network increase by at most a multiplicative factor of approximately 1.053 (5%). The

significance of this is that motor learning does not change the reflex movement F1 is learning by

very much.

On the other hand, during execution of a voluntary movement, and assuming the assimilation

network has inhibited the reflex pathway so that uj = 0, the F1 output signal becomes

jn

nn

njj u

yyy

zx ′⋅

′⋅⋅⋅−⋅⋅⋅

⋅=⋅⋅

→.2

.2.2

.

βδεζαδε

αβ

αβ

where y′2.n is the F3 activation when motor learning occurred previously and u′j is the reflex signal

learned at that time. Assuming that once again arousal A2 is maximal and using the steady state

values of (16), this results in xj → uj/4.755. Thus, the execution of a voluntary movement results

36

The Martian Program 2009 Wells and MacPherson April 20, 2009

in a motor command signal of only slightly more than one-half of that same movement's reflex

command strength.

This is a fundamental property inherent in the basic Grossberg avalanche network structure

and is not due to the addition of the F3 field to this structure. It is a characteristic of the basic

Grossberg-Carpenter Outstar learning rule schema and one that continues to be reflected in the

Wells-Hill-MacPherson rule (which is merely a variant of the Grossberg-Carpenter rule). In the

case of the Martian, it reflects the fact that in the 2008 Martian there is no feedback from the

perception network S to the affectivity network, which is equivalent to saying that the Martian's

"volition" in executing a voluntary movement is not reflected in any affective change in the Critic

structure implemented by the affectivity subsystem. This is inconsistent with the psychological

facts because human infants clearly exhibit an aroused affective condition when responding with

the voluntary execution of an acquired sensorimotor scheme to the perception of an actual

"exciting" situation that has been assimilated into a sensorimotor scheme.

In terms of the general actor-critic model, this reflects the lack of a feedback connection from

the actor function to the critic function, shown in Figure 2, insofar as this feedback connection

directly involves the higher sensory cortex network S. Because the under-arousal of motor

commands demonstrated above is inherent in the basic avalanche network/assimilation network

structure of the 2008 Martian, this finding points out the unsurprising result that progression from

sensorimotor stage I to sensorimotor stage II (first acquired habits) will require a more

functionally complete model of the affectivity network (LN) capable of accepting affective

stimulation from the cortical networks as well as the direct sensory pathways. Development of a

theory for the specific form of this enhancement is a program research objective for 2009.

Accomplishment of this aim requires a more thorough empirical investigation of the current

Martian sensorimotor learning behaviors in terms of the actual dynamics in (16), which is

likewise a specific aim for the 2009 research program.

XV. The Higher Sensory Cortex S Network

Objective learning in the Martian model takes place in the S network. Objective learning is the

learning of particular objects conveyed to the Martian's brain by means of its external senses. The

word "object" used here as a technical term means "that which the representations of the

Martian's higher sensory cortex represent." It is a very broad term and one loaded with a great

many metaphysical implications [WELL3, 11] that we will not go into in this paper. For our

purposes here, it is sufficient to distinguish two species of objects, which we will call things and

events. A thing is here defined to mean a specific individual entity in the Martian's world that the

37

The Martian Program 2009 Wells and MacPherson April 20, 2009

Martian eventually comes to identify as being endowed with certain aspects of existence such as

permanence (persistence in time) and reality. An event is that which is represented by the Martian

as a temporal sequence of representations the Martian comes to integrate into a single totality that

can probably be best described as a "happening."

There are a great many theoretical issues that come into play as soon as we begin to consider

the problem of objective representation because this problem has a great many levels to it. The

human infant in the first few stages of sensorimotor intelligence has no understanding of things or

events in the sense that adults understand these terms. Human beings do not come equipped with

any "copy of reality" mechanism (as Aristotle believed) nor with any rationalist "innate ideas" (as

Descartes believed). Both metaphysical premises have testable consequences and, when put to the

test, empirical psychology finds decisively against both these metaphysical prejudices [PIAG4-6].

But although the infant is born with no objective knowledge and requires a long period of time

(infancy) to develop its first objective knowledge of concrete things and events, it is nonetheless a

fact that the infant eventually does develop this kind of knowledge and the question is: How?

Here the previous theories put forth as attempted explanations can be broadly divided into two

camps: performance model theories and self-organization theories. In terms of popularity, the

performance model theory approach has undeniably been the more popular prejudice. However,

all such theories to date have suffered from severe and fundamental limitations, not the least of

which is that they all – either explicitly or implicitly – eventually find themselves forced to posit

an homunculus in order to make the model work. This is an absurd foundation and suffers from

the fatal disadvantage of simply being a false premise. Self-organization theories, on the other

hand, have demonstrable advantages and are, in our opinion, unquestionably the better-founded

theoretical starting point. This has, we believe, been convincingly demonstrated in the work of

Grossberg and his colleagues, e.g. [GROS8].

This, however, puts another issue in play: In a hierarchical series of levels of representations,

what is the representational starting point in the neocortex? Is it, metaphorically, a "picture" or

"feature fragment" that comes to be united with other such fragments? If so, are these pictures in

some sense "icons" or "partial recognitions" of things and/or events? There are some merits in

this view, e.g. [GROS8] and [DAMA2-3] as well as some demerits [JAME], [PIAG2], [KANT].

The opposing theory holds that the first objective representations of which the subject (infant)

becomes conscious are not "fragments" but, rather, sensorimotor representations of events in their

entirety with no consciousness "breakdown" of the "pieces" or "pictures" that make up the

representation of the event. This premise is entirely in accord with the Piagetian model [PIAG2]

and, furthermore, is a key theorem in the Wells model of human mind [WELL3].

38

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 27: Grossberg's macrocircuit model of recognition and recall processes. STM = short term memory.

Among neural network theorists the "assembly from icon fragments" theory has undoubtedly

been the most favored approach. Perhaps the most important work along this line has been done

by Grossberg and his colleagues. Figure 27 illustrates the Grossberg model of this process

[GROS8]. It consists of a hierarchy of increasing higher-level representations (the S(j) field

hierarchy illustrated in the figure). It begins at the lowest objective level, S(2), with feature

fragments ("iconic sensory features") and proceeds up the chain to "items" and their temporal

"order" at S(3). From there it goes on to progressively more refined and flexible object

representations. The model also calls for sensory field representations to stimulate corresponding

motor representations that travel back down the chain and eventually end up producing motor

responses. This model is in large part, although not entirely, consistent with Damasio's qualitative

brain architecture proposition [DAMA2-3], and its connection with motor representations gives it

a partial agreement with Piaget's theory [PIAG2]. To date, Grossberg's model has presented what

are probably the best practical demonstrations of theoretical progress of systems not founded on

the flawed premises of the "performance model" school of thought.

On the other hand, there are some objections that can be raised about the Grossberg macro-

circuit model. The most fundamental of these is that there is no explicit role for affectivity in its

learning process. The Grossberg model is, to use the terminology of computer science, a "data

path" model and its self-organizing principles are fundamentally driven by entirely objective

items of information. It is not an actor-critic architecture.

39

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 28: The Sherman-Guillery model of thalamocortical organization.

The second objection that can be raised is the one-way communication between its sensory

structures and the motor networks. Motor feedback into the sensory system occurs only at the

lowest level of the system and not between higher levels of sensory and motor representation.

This is a considerably less serious objection than the first. The basis for raising this objection

comes from its possible inconsistency with the sensorimotor interactions model of Piaget

[PIAG2]. However, the Piagetian model is very qualitative and admits to an interpretation under

which the Grossberg macrocircuit model is not inconsistent with it.

On the other hand, the Grossberg model is consistent with anatomical findings regarding the

organization of the thalamocortical system of the brain [SHER]. Figure 28 illustrates the

Sherman-Guillery model of thalamocortical organization. Their finding is that sensory cortex

information projected from one part of the neocortex to another via the thalamus carries motor

command information with it such that neocortical function at the destination point of the

projection involves motor command as well as object perception information. The Grossberg

model seems to be wholly consistent with this aspect of the S-G model. The congruence between

these two models is remarkable when one considers that Grossberg et al. formulated the

macrocircuit model several years before the Sherman-Guillery anatomical findings were made. In

this sense, Grossberg's theory made a prediction and Sherman-Guillery confirmed it. This must be

regarded as a considerable and very noteworthy triumph for Grossberg and his theory.

A third objection, which we regard as one that is just as serious as the first, is the "iconic"

nature of the representations in field S(2) of Figure 27. Grossberg tells us the S(2) representations in

40

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 29: A possible actor-critic version of Grossberg's macrocircuit with affectivity as the critic.

the macrocircuit are similar to "still pictures" that make up the "motion picture" of unfolding

events [GROS8]. He provides us with a qualitative description of how, as part of the larger

system of Figure 27, this suffices to eventually produce time-ordered "item and sequence"

representations in the higher levels of the sensory network. As part of this description, he points

out that the overall complex process he describes is consistent with and an implementation of

Piagetian circular reactions. The totality of S(1), S(2), M(2), and M(1) along with the kinaesthetic

feedback shown in Figure 27 are necessary in order for all this to be possible. Also necessary for

this possibility is the presence of "prewired positional gradients" represented in the initial

conditions of the adaptive weights of the system in order to bias the initial actions of the system

to evolve in the proper direction needed to achieve the end result. The mathematical system he

describes is such that "no serial buffer [of information] is necessary."

We allow that the mathematical analysis of this proposition is correct and that the macrocircuit

model is mathematically possible. The question, though, is whether it is fully consistent with the

totality of the requirements that must be met by the overall process of judgment and

understanding characteristic of the phenomenon of mind. In particular, we question the

correctness of what seems to be the presupposition that the initial representations in S(2) come

about as "still pictures" of a presumably independent character. What makes the system "learn"

these particular "pictures" and what is the mechanism for induction of this learning? The need for

41

The Martian Program 2009 Wells and MacPherson April 20, 2009

"prewired positional gradients" carries in it the premise of innate objective knowledge, and this is

the Critical epistemological issue with the macrocircuit model as Grossberg presents it. On the

other hand, an actor-critic form of the macrocircuit using affectivity as the critic function could

resolve this issue. Figure 29 illustrates a conceptual form such a system could take.

XVI. The S Network in the 2008 Version of the Martian

The Martian research work carried out in 2008 explored the implications of the "still picture"

hypothesis when the induction of "still picture" learning must be driven by affective signaling and

without the presence of innate objective structures within the neural system model. In this section

the details of this work are presented and the findings are discussed. The following section

discusses the research aims for 2009 that resulted from these findings.

Figure 30 illustrates the S network subsystem used in the 2008 version of the Martian. It

consists of three network systems: the S1 network; the hold network; and the S2 network. The S1

and S2 networks are Carpenter-Grossberg ART2 networks [CARP]. S1 and S2 each consist of a

single ART2 network. The ART2 network is described in more detail below. The hold network a

bank of networks, each similar to an avalanche AN F2 node except for the addition of a fourth

node. It is also described in detail below. Upon arousal by the affectivity network, S1 "learns" and

classifies input signal vectors it receives from the OCOS network. It has a very limited storage

capacity and is currently capable of storing only three pattern classifications. This has been

sufficient for purposes of our initial experiments to study Martian behavior but is inadequate for

more general purposes and would need to be expanded. Its outputs xj, j = 1 to 3, are real variables

Figure 30: S network subsystem in the 2008 version Martian.

42

The Martian Program 2009 Wells and MacPherson April 20, 2009

but its projections to the dipole networks and the hold network are binary valued with values 0 or

1. These projections are given by H(xj – Γ) where H is the Heaviside unit step function and

threshold Γ = 30.

S1 receives arousal inputs from affectivity network nodes LN x1 and x3 via an attentional

subsystem. The current implementation has two levels of arousal determined by separate

thresholds. The lower level of arousal enables data input from the OCOS network into S1 and

activates the network but inhibits adaptation ("learning") this pattern until the higher level of

arousal surpasses its threshold. When LN arousal is below the lower threshold, inputs to S1 are

inhibited and the network is quiescent. S2 receives various arousal signals generated within the

attentional subsystem. Figure 31 illustrates in flowchart form the logic of arousal signal

generation within the attentional subsystem. A1 and E1 are arousal signals for the S1 network; the

former enables adaptation by that network while the latter enables (releases from inhibition) the

OCOS data pathway into S1. E2, is looking, ooreset, and S2.hasFired are arousal signals for the S2

network. Signal A2 is an arousal signal for the attentional subsystem itself and is generated from

feedback input it receives from the hold network. λ1 and λ2 denote supra-threshold inputs from

the affectivity network.

The logical operations depicted in Figure 31 can all be implemented using a simple network of

McCulloch-Pitts elements. These are equivalent to the "logic gates" used in computer design, as

was shown by von Neumann [NEUM]. Figure 32 illustrates the attentional subsystem network.

Figure 31: Logic flowchart for the arousal signals generated in the attentional subsystem of the S network.

43

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 32: Network diagram of the attentional subsystem for the S network. Flipflops have a unit delay.

Signal E2 enables the data path input to S2. This pathway consists of the group Ia velocity

signal vector from the dorsal horn network. This data pathway corresponds to the kinaesthetic

feedback pathway depicted in the Grossberg macrocircuit of Figure 29. The is looking signal is an

arousal signal that indicates the S1 network is active and "paying attention" to its input signal.

This arousal signal is inhibited by the S2.hasFired signal. When it is not inhibited, it enables

(releases from inhibition) the data pathway input to S2. This logical combination of the two

signals produces E2, as shown in Figure 32. Signal ooreset ("onset/offset reset") is an orienting

signal used within the S2 ART2 network itself. Its function will be described when the ART2

network is discussed. E2 and ooreset are the only two signals actually projected into S2 by the

attentional subsystem.

The use of the name "attentional subsystem" for this network deserves comment. Following

Kant [KANT2], Wells defines "consciousness" as the representation that another representation is

present in the system and this representation is attended to by the system. In the context of neural

networks, representations are signals. Wells' definition departs from that used by other theorists

and, in particular, it departs from Grossberg's definition ("consciousness is short term memory in

an ART network in a stable resonant state"). Grossberg's definition is speculative ontology and

lacks an objectively sufficient ground for metaphysical objective validity [WELL3]; the

Kant/Wells definition is a practical and objectively valid definition.

Given this definition of consciousness, "attention" (in the context of neural networks) is the

representation (signaling) specifying which representations are those to be attended to (i.e., which

signals within the system are the "conscious representations"). This is precisely the function of

the attentional subsystem, hence the name given to this network.

44

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 33: General structure of an ART2 network. Inhibition of the input pathway I is not shown here.

S1 and S2 each consist of a single ART2 network; in the future we may expand S2 to a parallel

bank of ART2 networks, one for each Field 2 (F2) output of S1. Except for the number of inputs,

the ART2 vigilance parameter, and the implementation of the onset/offset reset function, the

ART2 networks in S1 and S2 are identical. Figure 33 provides the general network diagram for the

ART2 network. The networks used in the Martian were developed by Carpenter and Grossberg

[CARP]. Their theory of operation and the parameters used in the Martian are described in detail

in [WELL7, chapter 17]. Because this theory is somewhat lengthy, it will not be repeated here

and the reader is referred to [WELL7, chapter 17] for the details.

The ART2 network in S2 does not employ the onset/offset inhibition function block shown in

Figure 33. Instead, this function is provided for it by the attentional subsystem and projected into

it as the ooreset signal. One other difference between S1 and S2 is the level of the ART2 vigilance

parameter, ρ. The S1 network uses a vigilance of ρ = 0.90 while S2 uses ρ = 0.98. Both S1 and S2

use an adaptation rate constant η = 0.02. All other parameters are as given in [WELL7, chapter

17, section 2].

Figure 34 illustrates the hold network. It consists of a parallel bank of networks, each as

depicted in Figure 34, with one bank for each ART2 Field 2 (F2) output from S1. It receives an

arousal signal, A, from the O5 output of dipole network 1. The arousal signal is given by

(17) [ Γ−⋅= 5OH5.1A ]where H is the Heaviside unit step function and Γ = 0.5.

The hold network receives its trigger input stimulus from its assigned F2 output from S1. This

output is clipped by a Heaviside unit step function with a threshold of Γ = 30. This binary-valued

45

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 34: The hold network.

signal then passes through a simple McCulloch-Pitts network that implements a monostable

multivibrator function (that is, it produces a single, unit duration pulse of amplitude 1.5 each time

the output of the Heaviside activation function makes a transition from 0 to 1). The purpose of the

hold network is to delay triggering the S2 network, and therefore the avalanche network, until the

arousal from dipole network 1 has reached a sufficient level to signal a motor learning response

(driven by the affectivity network) or an acquired, habituated cognitive motor response (driven by

the S1 network).

The hold network is essentially identical to an AN F2 field node except for the addition of one

more node. The purpose of this fourth node is to provide a well-timed arousal stimulus to S2. This

stimulus is transmitted via the attentional subsystem network as shown in Figure 32. The

dynamical equations for the hold network are

( )( )( ) (( )2322424444

4342341311313333

1211212222

33133131111 ,

Γ−⋅+−=Γ−⋅−Γ−⋅+−=

+Γ−⋅+−=)

+Γ−⋅+−=

xgxxxgxgxx

AxgxxIuxgxx

ααααα

αααα

&

&

&

&

. (18)

Here g1 is the Heaviside extractor activation function, g2 is the Heaviside unit step function, and

g3 is the sigmoid activation function with saturation u3 = 1. The network parameters are: α11 =

0.5; α13 = 1; α22 = 0.125; α21 = 2; α33 = 0.75; α31 = 1.5; α34 = 2; α44 = 1; α42 = 1; Γ31 = 1/15; Γ12 =

1; Γ13 = 2/3; Γ43 = 0.1; and Γ23 = 20.8.

It has so far appeared to be the case that the hold network does not require a reset mechanism

46

The Martian Program 2009 Wells and MacPherson April 20, 2009

similar to the one employed in the avalanche network. However, it is mathematically possible for

a failure mechanism to arise. If S1 and DN1 should stimulate the hold network into operation and

then something should cause the arousal to be removed before x2 reaches the threshold for

activating x4, the memory loop x1-x3 would remain energized and the next DN1 arousal input

would cause a false arousal signal to be sent to S2 via the attentional subsystem. Like the AN F2

network, this mode can be counteracted by the simple expedient of a "Not-A" signal generated

from the arousal input and applied as an inhibitory input to node x3.

XVII. The Performance Characteristics of the Martian and Objectives for 2009

Specifications for the general characteristics of the Martian system are as follows [WELL1]:

1. The Martian will find high-contrast edges in its sensory field (EPW) to be "interesting" and will prefer to settle into position and dwell on images of contrast boundaries.

2. The Martian will find images with no contrast among pixels to be "boring" and will not pay attention to them.

3. The Martian will find pixel patterns in the EPW that form enclosures to be "interesting."

4. The Martian will have an aversion response if it tries to move outside its environment.

5. The Martian will have a "drive" to move in such a way as to try to make its sensory field match any previously learned feature category (that is, anything learned in S1).

6. The Martian will experience an increasing degree of satisfaction the more successful it is in finding a match between the sensory field and one of its learned categories. Contrariwise, it will experience an increasing degree of dissatisfaction from lack of success in accomplishing this.

7. The Martian will experience a "sense of complete satisfaction" when its sensory window is positioned such that the interest it affectively finds in its sensory data is maximal; this complete satisfaction will implicate that further motor commands will be neutralized except as necessary to maintain this degree of satisfaction.

8. The Martian will experience an attractive drive when a learned pattern is recognized by its sensory cortex system and will respond by attempting to maximize this attraction.

9. The Martian will experience an aversive drive when motor movements lead to a reduction in satisfaction in its recognition of the sensory field.

The current version of the Martian accomplishes some of these performance objectives but is not

presently capable of accomplishing all of them. The research goals for 2009 are aimed at making

the Martian capable of achieving more, preferably all, of these stated objectives.

The following summary encapsulates the main performance/behavioral characteristics of the

Martian in its current version [MacP2]:

1. In most cases observed, the Martian will approach an enclosure object (a 3 × 3 square with a hole in the center pixel) and settle into a movement limit cycle in which it over-shoots this object but then returns to it. This movement is entirely reflex driven and the dynamics appear to be a function of the initial position of the Martian relative to the

47

The Martian Program 2009 Wells and MacPherson April 20, 2009

object. In most cases, one or more learning episodes occur during this reflex action.

2. The S network has demonstrated the ability to both recognize and discriminate among sensory image patterns. However, the Martian does not succeed in learning a circular reaction, i.e. it does not succeed in learning how to perform this limit cycle movement.

3. Cases have been observed where the Martian's reflex limit cycle overruns the enclosure object at a velocity that is too fast to permit a learning episode to occur. Before it has time to become sufficiently aroused, it is past the object. The object "disappears" from the Martian's perception window and it "loses interest" in it.

4. Under some conditions, the Martian is capable of such violent reflex movement that it passes beyond the edge of its environment. When this happens the simulator crashes. One could say metaphorically that the Martian "dies" when this happens. Thus, objective (4) is not met in the manner intended, although it is true that the Martian exhibits an aversion to "dying."

5. The Martian's reflex responses have demonstrated the ability to settle on high-contrast boundaries. However, the upper brain system (S network and motor cortex) sometimes make this difficult by interfering with the reflex movement. This behavior tends to implicate incomplete assimilation of the object perception at the assimilation network.

6. When settling on a contrast boundary, the Martian's ART system sometimes exhibits mismatch resets in its ART search and "loses recognition" of the object. This is because the Martian's position is continuous and when it is positioned only halfway over a contrast line (half "on" the high contrast row or column in the environment and half "off" this row or column) the OCOS network produces surround canceling of the image. In other words, on-center/off-surround processing tends to make the image "fade in and out" as the Martian moves over the boundary. The current "still picture" object learning implemented in the S network is not capable of associating these changes in the image pattern, nor is the present storage capacity of S1 large enough to permit it to learn the multiplicity of appearances it is presented with during the settling operation. This behavior characteristic of the Martian is indicative of a fundamental flaw in the "still picture" model discussed in section XV and appears to provide empirical support for the opposing hypothesis that first objective learnings are of an entirety of closed sensorimotor cycles [WELL3, chapter 21].

7. In one test, during which the Martian learned to recognize the enclosure pattern object, this object was then moved a short distance to a different location in the environment. The Martian followed the object and remembered the general motion in its avalanche chain. This demonstrates that the current system is capable of learning patches or pieces of recognition-driven movement schemes if the conditions are not too challenging for it.

8. In cases where the Martian recognizes an image pattern but has very little movement (near-rest conditions), the S2 subsystem within the S network will not learn a velocity profile and will not initiate an avalanche cascade to learn the reflex motor commands that are doing the "station keeping" to put the Martian into position on or near its target object. This is a fundamental characteristic of the ART network itself, which requires that its inputs rise above some minimum threshold of activation. This demonstrates that velocity-only kinaesthetic feedback (group Ia tract) is insufficient to achieve the objectives of the system. It is conjectured that providing S2 with group Ib and/or group II afferent signals might adequately address this shortcoming. Carrying out an experiment to test this conjecture is one of the 2009 research objectives.

9. In one test, the Martian's "object of attraction" was deliberately placed in a position the

48

The Martian Program 2009 Wells and MacPherson April 20, 2009

Martian could not reach without its nociceptor (group III) pathway becoming activated. After approaching the object, the Martian reacted to this with a pain reflex response. This demonstrated that the reflex hub pathway is operating as intended. The Martian also recognized the situation in the S network. When subsequently the Martian's motor reflex system drove it to again approach the object, the Martian recognized the situation and initiated an avalanche-driven motor response and moved away from the object. This constitutes a learned "flight response" and demonstrated that the current Martian is capable of developing at least some acquired and objectively-triggered movement schemes of a one-time-learning-event character.

Illustrations of various signals from the Martian's different subsystems are provided in

[MacP2]. All in all, we regard the 2008 Martian program as successful in accomplishing a great

many things, particularly when one considers that it was during this program when the first

Martian system (the current version) was built. Studies of the Martian's behaviors have so far

provided data that point directly to needed additions in the basic structure of the Martian brain

system. These needs constitute the principal research aims for the 2009 program.

Findings (5) and (6) above appear to establish that there is a role and a need for direct

feedback from the Martian's sensory cortex to the affectivity subsystem. The current system has

no functional structure capable of achieving objective (6) above. The actor-critic macrocircuit

model of Figure 29 does not include objective feedback from S(2) to the affectivity subsystem and

yet there is not a sufficient establishment of sensory learning in the S network (which corresponds

to S(2) in the Grossberg macrocircuit) to justify "higher learning" at the S(3) level on fundamental

psychological grounds, regardless of whether or not the mere mathematical possibility of such

learning exists as Grossberg et al. claim is the case on analytical grounds. Empirical psychology

clearly demonstrates the role of "frustration" is key in triggering accommodation (induction of

adaptation) in infant learning [PIAG1]. The current version of the Martian lacks the feedback

between the S network and the affectivity subsystem to produce a frustration response. Our 2008

findings appear to implicate that there must be a feedback link to the critic (affectivity subsystem)

even at the S(2) level of the macrocircuit model of Figure 29. The development of additional

structures in the sensory cortex (S network) and the affectivity network (LN) to provide for

"success/failure" perception (cognitive dissonance and cognitive-driven satisfaction) is the first

principal objective for the 2009 program.

This objective is also related to the shortcoming revealed in finding (5). Because there is no

feedback between the S network and the LN, the Martian's objectively-driven movement attempts

can clash with its reflex actions during the assimilation process. In effect, there is no

success/failure "gain control" in the motor pathway nor does the present system provide a means

for such gain control to arise as an emergent behavior of the system. The motivation arousal

signal in the AN F3 field is currently inadequate to provide proper control of the Martian's

49

The Martian Program 2009 Wells and MacPherson April 20, 2009

voluntary movements, nor does the system contain a gain control mechanism for motor command

signals projecting down to the ventral horn. During stages I and II of early sensorimotor

intelligence, such control can clearly not call upon objective perception alone, and this implicates

the requirement of a more complete system for cortico-limbic feedback.

In this context, it is worth mentioning that within the psychology community there has been a

long-standing controversy over whether affect drives cognition fundamentally [ZAJO] or

cognition drives affectivity [LAZA]. Our findings to date appear to favor a more moderate

hypothesis, namely that it is not a case of one or the other (cognition or affectivity) dominating

the behavior of the system. Rather, our position is that it is not a competition at all; it is a dance.

The current LN (Figure 13) is the simplest system possible capable of acting as a functional

proxy for the affectivity theory of [WELL3]. The 2008 findings motivating the first principal

research objective for 2009 are also the findings that implicate extending the LN proxy to at least

the level depicted in Figure 14. However, even this figure lacks explicit feedback from the S

network, and it is highly probable that such feedback will be required to accomplish general

objective (6).

Finding (2) appears to establish that the "still picture" hypothesis is not sufficient to make

possible the development of first acquired habits and lead to the ability to spontaneously develop

primary circular reactions in Stage II of sensorimotor intelligence in an actor-critic model system.

We do not find this very surprising; Wells' theory [WELL3, chapter 21] in fact predicts exactly

this outcome. This does not mean the current S network should be altogether replaced. Finding

(9) clearly establishes that the function it provides does have a place within the overall system,

and its existence appears at this time to be necessary to support the later development of the next

level in the affectivity-driven macrocircuit model. However, it does appear to mean that the

current sensory cortex model is incomplete and that an additional structure capable of learning a

temporal sequence in its entirety is required. Providing this capability is the second principal

research objective for 2009.

Figure 35 provides a concept illustration of how we will approach this task by expanding the

sensory cortex model. Analysis of Martian behaviors related to findings (1), (2), (6), and (8) show

that by the time the Martian recognizes its "objects of desire" the reflex-driven movement

commands needed to voluntarily return to and settle on this object have already passed. The

avalanche sequence it learns subsequently is not the correct sequence for voluntary movement to

acquire the target object. Simply put, the Martian cannot presently anticipate that a particular

motor plan will lead to affective success. (At this stage we cannot invoke objective success in any

50

The Martian Program 2009 Wells and MacPherson April 20, 2009

Figure 35: Concept illustration of a sensory sequence learning/recognition network for the sensory cortex.

objectively valid way as a means to success because the infant at this stage does not possess the

acquired concept structure needed to judge objectively). It lacks the structure needed for the

expectation function in the reasoning loop structure of Figure 1.

If the Wells theory is correct, the capability necessary for the acquisition of first objective

representations requires the addition of a short-term memory capacity that an affective arousal

will cause to be learned in longer term memory. The proposed method for accomplishing this is

what Figure 35 depicts. We will add a new avalanche network subsystem. Learning (weight

adaptation in the F1 field) in this network will be temporary. This is to say the weights connecting

the F3 field and the F1 field will be allowed to decay over time. Input to the F1 field will come

from the OCOS network, i.e. this avalanche network will learn an objective spatio-temporal

sequence rather than a motor command sequence. The avalanche network will be constructed as a

ring, i.e. the last node in the F2 field will be connected back to the first node so that the avalanche

runs all the time when it is in an unaroused "forward mode" of operation.

Receipt of an affective arousal will cause the avalanche to reverse direction (hence the name

bidirectional avalanche network in Figure 35). During this mode, the learned pattern sequence

will be transmitted to an ART2 network having the memory capacity to store the entire sequence.

51

The Martian Program 2009 Wells and MacPherson April 20, 2009

Subsequently, when the system is again in unaroused forward mode operation, this ART2

network will monitor the OCOS signals. If it recognizes any pattern that was previously part of

the learned sequence, it will provide a feedback signal to the expanded LN subsystem (signaling a

condition for an affective arousal) and will trigger a motor avalanche in anticipation that the

motor sequence it learns will lead to successful acquisition of the target. If the OCOS sequence

being observed subsequently "breaks" with the previous learned sequence, this will constitute a

cognitive dissonance leading to abortion of the motor avalanche and permitting its learned

sequence to decay (failure to induce long-term storage of the aborted sequence).

This constitutes an entirely new kind of neural network subsystem, one that to our best

knowledge has never before been attempted. It is therefore anticipated that this research will

produce many detailed findings and will necessitate a number of significant modifications to the

existing motor cortex network structure. It is, in other words, a major new undertaking. It appears

likely that the structure of Figure 35 will require a bank of such subsystems, rather than merely

one, and a control means for switching between banks so that a second bank is monitoring the

actual perceptual sequence while the first is replaying its acquired STM into the ART2 network.

Figure 35 is a concept illustration, not a block diagram for the complete system we will need to

develop.

Implicit in the post-recognitory operation of the network, in which the ART2 network

recognizes a piece of a learned sequence and initiates a motor avalanche, is the need to be able to

initiate a learned motor avalanche at different places in the motor avalanche chain. Working out a

robust way to do this is implied in research objective 2.

Principal research objective 3 addresses one obvious shortcoming in the current Martian,

mentioned briefly in section V. This is its inability to servo itself in on target when it first gets an

affective signal triggering S1 to learn an objective appearance that satisfies its built-in innate

predisposition for the object. In effect, the current Martian "knows how to apply the gas" to its

movements but is not capable of "applying the brakes." This is why it always overshoots its target

and is a fundamental reason for the failure of the "still pictures" model. The very same problem

will exist in the sequence-learning approach of principal objective 2. It is a shortcoming in the

Martian structure that is contradictory to observable behaviors in human infants and so represents

a fundamental shortcoming in the architecture of the current Martian system. We call this "the

posture problem." The Martian does not know how to invoke a posture reflex. We had thought it

probable that a posture reflex would be a natural consequence of the reflex subsystem built into

the current version. However, our experiments to date have demonstrated this is not the case.

Research objective 3 is to provide the Martian with an explicit resting-posture reflex function.

52

The Martian Program 2009 Wells and MacPherson April 20, 2009

Toward this end, the proposed method is to add additional affectivity and reflex structures to

take advantage of the Martian's reflex hub network. The reflex hub is by design a network

capable of "applying the brakes" to movement, although it is probably more accurate to say the

reflex hub "throws it into reverse and guns it" rather than that it "applies the brakes." However,

this functional character of the reflex hub just happens to dovetail rather naturally with a

conjecture made by Wells in 2003 regarding the nature of the reflex control system in mammals.

Wells speculates that, rather than a classical control system, the basic sensorimotor control

scheme in mammals may be a type of control system known in the engineering world as a

"variable structure switching control system" (VSSC). Such systems are not uncommon in, for

example, control of large industrial electric motors. Wells believes certain features found in the

spinal sensorimotor system of mammals [WELL4] are suggestive of the presence of a VSSC

system in nature. The potential advantage of such a system arises from the possibility that it

might constitute a simpler means to control highly nonlinear systems.

The VSSC concept for principal objective 3 is illustrated in Figure 36. x is an arousal signal

originating in the affectivity network (LN or some enhancement of the existing LN). This signal

is sent to a hysteresis network that encodes the present and past state of affective arousal and

produces a reflex hub control signal y. In Figure 36, y = 1 invokes a reflex reaction. It is likely

that this signal must also act as a gain control enabling AN F1 output signals to participate in the

reflex response. (The AN F1 to RH gain in the present Martian is set to zero). Hysteresis network

actions are defined by two thresholds, Γ1 and Γ0, that determine the desired direction of travel.

Figure 36: VSSC Concept for Martian posture control.

53

The Martian Program 2009 Wells and MacPherson April 20, 2009

Functionally, the hysteresis network is an asynchronous finite state machine. It is defined by

the state table shown in Figure 36. Because this is a relatively simple logical operation, the

hysteresis network can be implemented using McCulloch-Pitts elements.

It is theoretically possible that a system such as depicted in Figure 36 could produce growing

oscillations in the Martian's movements rather than settling down into a well-controlled limit

cycle operation that keeps the Martian "on target." A mathematical description of the dynamics of

this system is likely to be unobtainable in closed form due to the inherent nonlinearities involved

in this VSSC system. It is therefore likely that a considerable amount of empirical

experimentation will be required to determine gain settings and other parameters. It is not

unlikely that the hysteresis control function might also need to inhibit the normal reflex pathway

to avalanche network AN F1 since these normal command signals will be fighting against the

reflex hub command signals during change-in-direction operation. The quality with which the

Martian posture is achieved will depend upon the spacing between the two thresholds shown in

the figure, with wider spacing probably leading to greater "palsy" in the posture response. The

posture dynamics are also likely to depend on the speed with which the LN responds to its input

data. A system such as depicted in Figure 36 has never been attempted before in the existing

neural network literature, and so we are again breaking new ground here.

There is still a significant amount of quantitative analysis that needs to be carried out on the

current version of the Martian. Are the lengths of the present motor avalanche chains sufficient?

Should the speed of the motor avalanche be made to depend on the velocity of the Martian's

movements? If so, how can this be accomplished in an affectivity-driven manner? Are the current

learning rates in the S and AN F1 subsystems properly matched to the reflex dynamics or does the

learning (adaptation of the weights) significantly lag behind reflex-driven events?

How can the presently existing difference between reflex-driven motor command intensities

and objectivity-driven (voluntary) motor command intensities be eliminated so that the

objectively-driven motor scheme execution is a truer reflection of motion control that took place

when the sequence was learned initially? The current mismatch is a fundamental consequence of

the adaptation scheme employed in AN F1. There is an inherent tradeoff in the current system

between not having motor learning unbalance the reflex motion and having motor learning

produce an effective drive signal to the ventral horn when the assimilation network has inhibited

the reflex pathway. The system's current parameters were established to favor operation of the

Martian's innate reflex pathway and there are very good reasons for this bias in the parameters.

Therefore, overcoming the existing problem is likely to involve the need for either new structures

within the avalanche network subsystem, a change in the AN F1 learning rule, or both.

54

The Martian Program 2009 Wells and MacPherson April 20, 2009

Answers to all of these questions will almost certainly require better instrumentation

capabilities be added to the existing Martian simulator, which is a GUI-based simulator. Signals

that are currently unobservable during simulation must be made observable, and this must also be

done in light of memory and computational limitations in the computer itself. These

instrumentation issues are therefore tightly coupled with the analysis questions posed above, and

collectively these two activities combine to define our fourth and last principal objective for the

2009 program.

References

[CARP]: Carpenter, Gail A. and Grossberg, Stephen (1987), "ART2: self-organization of stable category recognition codes for analog input patterns," Applied Optics, vol. 26, no. 23, pp. 4919-4930.

[DAMA1]: Damasio, Antonio R. (1994), Descartes' Error, NY: Avon Books.

[DAMA2]: Damasio, Antonio R. (1989), "The brain binds entities and events by multiregional activation from convergence zones," Neural Computation, vol. 1, pp. 123-132.

[DAMA3]: Damasio, Antonio R. (1989), "Time-locked multiregional retroactivation: A systems-level proposal for the neural substrate of recall and recognition," Cognition, 33 (1989) 25-62.

[GREE]: Greenspan, Stanley I. (1997), The Growth of the Mind, Reading, MA: Addison-Wesley.

[GROS1]: Grossberg, Stephen (1972), "A neural theory of punishment and avoidance I: Qualitative theory," Mathematical Biosciences 15, 39-67.

[GROS2]: Grossberg, Stephen (1972), "A neural theory of punishment and avoidance II: Quantitative theory," Mathematical Biosciences 15, 253-285.

[GROS3]: Grossberg, Stephen (1978), "A theory of human memory: Self-organization and performance of sensory-motor codes, maps, and plans," Progress in Theoretical Biology, vol. 5, pp. 233-374), NY: Academic Press.

[GROS4]: Grossberg, Stephen (1969), "Embedding fields: A theory of learning with physiological implications," Journal of Mathematical Psychology, 6, 209-239.

[GROS5]: Grossberg, Stephen (1969), "On learning, information, lateral inhibition, and transmitters," Mathematical Biosciences, 4, 255-310.

[GROS6]: Grossberg, Stephen (1969), "Some networks that can learn, remember, and reproduce any number of complicated space-time patterns, I," Journal of Mathematics and Mechanics, vol. 19, no. 1, pp. 53-91.

[GROS7]: Grossberg, Stephen (1970), "Some networks that can learn, remember, and reproduce any number of complicated space-time patterns, II," Studies in Applied Mathematics, vol. 49, no. 2, pp. 135-166.

[GROS8]: Grossberg, Stephen (1986), "The adaptive self-organization of serial order in behavior, speech, language, and motor control," in Pattern Recognition by Humans and Machines, Vol. I: Speech Perception, E.C. Schwab and H.C. Nusbaum (Eds.), NY: Academic Press.

[HAUG]: Haugeland, John (1997), Mind Design II, Cambridge, MA: MIT Press.

[JAME]: James, William (1890), The Principles of Psychology, NY: Dover Publications, Dover Edition, 1950.

55

The Martian Program 2009 Wells and MacPherson April 20, 2009

[KAND1]: Kandel, E.R. and Jessel, T.M. (1991), "Modality coding in the somatic sensory system," in Neural Science, 3rd ed. (Eric R. Kandel, James H. Schwartz, and Thomas M. Jessel, eds.), Norwalk, CN: Appleton & Lange, Chap. 26, pp. 367-384.

[KANT]: Kant, Immanuel (1787), Kritik der reinen Vernunft, Berlin: Druck und Verlag von Georg Reimer, 1911 edition.

[KANT2]: Kant, Immanuel (1800), Logik, in Kant's gesammelte Schriften, Band IX, Preukilchen Akademie der Wissenschaften (eds.), Berlin: Walter de Gruyter, 1923.

[LAZA]: Lazarus, Richard S. (1991), "Cognition and motivation in emotion," American Psychologist, vol. 46, no. 4, pp. 352-367.

[MacP1]: MacPherson, Q. (June, 2008), "The potentially useful effects of contrast enhancement for ART resonators," LCNTR Technical Report, The MRC Institute, University of Idaho, Moscow, ID. For a copy contact Richard B. Wells, Director, LCNTR.

[MacP2]: MacPherson, Q. (July, 2008), "Martian Program Project Report: July 31, 2008," LCNTR Technical Report, The MRC Institute, University of Idaho, Moscow, ID. For a copy contact Richard B. Wells, Director, LCNTR.

[MART]: Martin, John H. (1991), "Coding and processing of sensory information," in Neural Science, 3rd ed. (Eric R. Kandel, James H. Schwartz, and Thomas M. Jessel, eds.), Norwalk, CN: Appleton & Lange, Chap. 23, pp. 329-340.

[NEUM]: von Neumann, John (1956), "Probabilistic logics and the synthesis of reliable organisms from unreliable components," John von Neumann – Collected Works, A.H. Taub (ed.), vol. 5, pp. 329-378, NY: Macmillan, 1961-63.

[PIAG1]: Piaget, Jean (1952), The Origins of Intelligence in Children, Madison, CN: International Universities Press.

[PIAG2]: Piaget, Jean (1977), The Development of Thought, NY: Viking Press.

[PIAG3]: Piaget, Jean and Garcia, Rolando (1991), Toward a Logic of Meanings, Hillsdale, NJ: Lawrence Erlbaum Associates.

[PIAG4]: Piaget, Jean (1970), Genetic Epistemology, NY: W.W. Norton.

[PIAG5]: Piaget, Jean (1971), Psychology and Epistemology, NY: Viking Press.

[PIAG6]: Piaget, Jean (1972), The Principles of Genetic Epistemology, NY: Basic Books.

[SHEP]: Shepherd, Gordon M. (1994), Neurobiology, 3rd ed., NY: Oxford University Press.

[SHER]: Sherman, S. Murray and Guillery, R.W. (2006), Exploring the Thalamus and Its Role in Cortical Function, Cambridge, MA: The MIT Press.

[WELL1]: Wells, Richard B. (2007), "Affective control of learning processes in network systems architectures: A research project," LCNTR Tech Brief, Moscow, ID: The University of Idaho, http://www.mrc.uidaho.edu/~rwells/techdocs.

[WELL2]: Wells, Richard B. (2007), "Meanings-based networks: A new learning paradigm for ART network system models," LCNTR Tech Brief, Moscow, ID: The University of Idaho, http://www.mrc.uidaho.edu/~rwells/techdocs.

[WELL3]: Wells, Richard B. (2006), The Critical Philosophy and the Phenomenon of Mind, provide the web link

[WELL4]: Wells, Richard B. (2003), "The spinal sensorimotor system," parts I-IV, LCNTR Tech Brief, Moscow, ID: The University of Idaho, http://www.mrc.uidaho.edu/~rwells/techdocs.

56

The Martian Program 2009 Wells and MacPherson April 20, 2009

[WELL5]: Wells, Richard B. (2003), "Kinetics and muscle modeling of a single degree of freedom joint, part I: Mechanics," LCNTR Tech Brief, Moscow, ID: The University of Idaho, http://www.mrc.uidaho.edu/~rwells/techdocs.

[WELL6]: Wells, Richard B. (2003), "Kinetics and muscle modeling of a single degree of freedom joint, part II: Spindles and sensory neurons," LCNTR Tech Brief, Moscow, ID: The University of Idaho, http://www.mrc.uidaho.edu/~rwells/techdocs.

[WELL7]: Wells, Richard B. (2009), Introduction to Biological Signal Processing and Computational Neuroscience (draft manuscript), available from the author.

[WELL8]: Wells, Richard B. (April, 2009), "An update on the modeling of early development in sensorimotor intelligence: 2009 status," LCNTR Technical Report, The MRC Institute, University of Idaho, Moscow, ID. For a copy contact Richard B. Wells, Director, LCNTR.

[WELL9]: Wells, Richard B. (2009), "Use of the affectivity and reflex hub networks to implement posture control in the Martian," LCNTR Technical Report, to appear.

[WELL10]: Wells, Richard B. (2008), "Avalanche networks for spatio-temporal learning, Part I: The F2 field node," LCNTR Technical Report, May 22, 2008. Contact the author for a copy.

[WELL11]: Wells, Richard B., Introduction to Mental Physics, to appear. Contact the author for details.

[WOOD]: Wood, W.A. (1986), "Important issues in knowledge representation," Proceedings of the IEEE, 74 (10), pp. 1322-1334.

[ZAJO]: Zajonc, R.B. (1980), "Feeling and thinking: Preferences need no inferences," American Psychologist, vol. 35, no. 2, pp. 151-175.

57