Neural Networks 22 (2009) 126–133

Contents lists available at ScienceDirect

Neural Networks

journal homepage: www.elsevier.com/locate/neunet

2009 Special Issue

Cortical basis of communication: Local computation, coordination, attentionFrederic AlexandreLORIA-INRIA BP 239, F-54506 Vandoeuvre, France

a r t i c l e i n f o

Keywords:Computational neurosciencesCommunicationVisual attentionCoordinationModel of the cortex

a b s t r a c t

Human communication emerges from cortical processing, known to be implemented on a regularrepetitive neuronal substratum. The supposed genericity of cortical processing has elicited a series ofmodeling works in computational neuroscience that underline the information flows driven by thecortical circuitry. In the minimalist framework underlying the current theories for the embodiment ofcognition, such a generic cortical processing is exploited for the coordination of poles of representation,as is reported in this paper for the case of visual attention. Interestingly, this case emphasizes howabstractinternal referents are built to conform to memory requirements. This paper proposes that these referentsare the basis for communication in humans, which is firstly a coordination and an attentional procedurewith regard to their congeners.

© 2009 Elsevier Ltd. All rights reserved.

1. Introduction

The cerebral cortex is known to be composed of millions ofsimilar elementary circuits, repeated throughout its surface, thecortical columns (Ballard, 1986; Burnod, 1989;Mountcastle, 1978).A fascinating challenge with the study and modeling of the cortexis to understand how the wide spectrum of functions generated bythis ultimate structure in the evolution process can emerge fromthis repetitive paving. The corresponding approach of modelingin Computational Neuroscience is to consider the cortical columnas the adequate level of computation (Alexandre, Guyot, Haton, &Burnod, 1991) and to define its functioning and learning rules insuch a way that whatever the cortical area or processing pathwaywe consider, only the nature of the inputs and the size andconnectivity scheme of the considered cortical regions will specifyits resulting activity.Three kinds of data from Neuroscience are used to guide

modelers in this domain. At themicroscopic level, it is fundamentalto be informed about the inner structure of the cortical column,seen as a circuit of a hundred of neurons, including its inputs,outputs and dynamics. At the mesoscopic level, the organizationof a network of automata, proposed as models of corticalcolumns, must be consistent with current knowledge aboutthe main information flows in the brain. Columns must begathered in cortical areas; cortical areas must be interconnectedwith a precise connectivity scheme; the cortical processing isalso frequently dependent on extra cortical information flows(including perceptive input and motor output) that have to bedefined at this level. At the macroscopic level, behavioral studies

E-mail address: frederic.alexandre@loria.fr.URL: http://www.loria.fr/∼falex.

0893-6080/$ – see front matter© 2009 Elsevier Ltd. All rights reserved.doi:10.1016/j.neunet.2009.01.006

are also necessary to figure out what kind of information isprocessed and how it is manipulated, stored and exploited tointeract with the outer world.Other non-bio-inspired theories in machine intelligence are

perhaps more intuitionist. They rely on a logic, sequential andstructural approach and have a tendency to build symbolic andabstract representations of information (models of the world),whereas the distributed, local, numerical and asynchronousapproach in Computational Neuroscience is rather slanted towardthe emergence of properties and toward the intensive use ofinteractions with the real world to build the minimal necessaryinternal representations. Experiments from psychophysics andother developmental issues are therefore necessary not to followour misleading intuition.In this paper,my goal is to explain that current studiesmodeling

cortical processing with distributed simple local computationscan be directly used to emulate complex behaviors related tocommunication. I will report several studies that have beencarried out to define the local functioning of a cortical automaton(Section 2). Elementary behavioral functions using such localcomputations will then be presented in Section 3. I will refer tothem as coordination functions since they allow for the linkage ofrepresentational poles in the cortex. Section 4 will report recentdevelopments of a more ambitious study about mechanisms ofselective attention. One of the goals of that study is to showthat elaborated functions like attention, central in cognitionand often presented as related to consciousness (Taylor, 1997),can emerge from the distributed asynchronous computations oflocal automata and can avoid many representational hypothesesdifficult to implement and justify. Similarly, the claim of thispaper is that communication capabilities can also emerge fromthe same distributed asynchronous computations. In Section 5, I

F. Alexandre / Neural Networks 22 (2009) 126–133 127

Table 1Fundamental principles of information processing are applied to various information flows in the laminar organization of the cortex.

Principle of information processing Cortical layer Information flow

Sensory filtering 4 Feedforward and thalamic inputSequence processing 2/3 Lateral intra-areaMulti-modal integration 2/3 Cortico-corticalOutput 5/6 Feedback and extra cortical output

will argue that an artificial system, endowed with a distributedsubstratum of computation inspired by the local mechanisms ofcortical circuitry presented here, could be able to exhibit somefundamental communication capabilities as observed in evolvedliving beings. Section 6 will conclude on the amount of work thatremains to be done to obtain more elaborated communicationchannels like those observed in spoken language.

2. Principles of local distributed computation

The local circuitry in the cortex has been studied for a longtime. Many studies have laid emphasis on the cortical column,seen as the elementary computing block of the cortex (Ballard,1986; Burnod, 1989; Grossberg & Swaminathan, 2004; Korner,Gawaltig, Korner, Richter, & Rodemann, 1999; Mountcastle, 1978).The cortical column is perpendicular to the surface of the cortexand passes through the six layers of the cortex. A very basic andfunctional description of these layers can be proposed as follows:Layer 4 receives extra cortical (thalamic) and feedforward corticalinput; layers 1, 2 and 3 are cortico-cortical connections; and layers5 and 6 send feedback in the cortex and projections outside thecortex. Apart from the granular layer 4, all these projections areperformed by pyramidal excitatory neurons and other excitatoryand inhibitory interneurons achieve the internal logic of theautomaton, coupling and uncoupling these layers. The interplaybetween these information flows is the basis of neural fieldapproaches developed by many authors Amari (1977), Rougier(2006), Taylor (1999) and Wilson and Cowan (1972). Othershave proposed a cortical automaton including several inputs andoutputs and internal learning and functioning rules (Alexandreet al., 1991; Grossberg & Swaminathan, 2004; Korner et al., 1999).From an evolutionary perspective, the cortex can be seen as

a new evolved loop, added to more primitive perception–actionloops (Ballard, 1986). In this view, the cortex offers a substratumcharacterized by modal maps where specific information canbe displayed, by associative areas where subtle intermodaldependencies can be expressed and by an original relaxationdynamics (Doya, 1999). These powerful properties are obtainedfrom the layered organization of the cortex. They can be describedthrough four principles of information processing, as described inthe following subsections and summarized in Table 1.

2.1. Thalamic input and feedforward

Thalamic and cortical feedforward integration is certainlythe most classical principle of cortical functioning. Corticalcolumns in sensory areas are known to be selective to specificstimuli and to perform filtering on the incoming perceptiveinformation, from thalamus to primary sensory areas or fromprimary to secondary sensory areas. Kohonen has popularizedthis operation with the self-organizing map principle (Kohonen,1989). It combines adaptive feedforward links conveying thalamicinputs and learning relevant filters with static lateral inhibitionimplementing a competition between filters. More precise anddevelopmental details have been introduced by Miikkulainen,Bednar, Choe, and Sirosh (1996) and Taylor has proposed adynamic and distributed version of such a processing, wherebubbles of activity are generated and maintained from an initialstimulation (Taylor, 1999).

The learning rule proposed by Kohonen highlights the mainprinciples of plasticity related to the thalamic feedforwardinformation flow. The cortex plays a role of reduction ofdimensionality from the huge set of possible thalamic inputs toa reduced set of prototypes selected to represent the statisticaldistribution of the inputs. Prototypical learning (quantization)selects the best representatives. Lateral inhibition ensures theirsparseness. The bidimensional nature of the cortex allows for theemergence of a topography where close units are selective to closestimuli (Kohonen, 1989).

2.2. Lateral propagation of information within an area

Apart from stereotyped bell-shaped (e.g. with DOG: differenceof Gaussians) pattern of connection allowing for competitionwithin population of close columns (and evoked in the previoussubsection), lateral connectivity within cortical areas plays animportant role in temporal information processing in the cortex(Bringuier, Chavane, Glaeser, & Fregnac, 1999; Jancke, Chavane,Naaman, &Grinvald, 2004). Events and stimuli in theworld have animportant temporal dimension and representing only static stimuliis not sufficient. Spatio-temporal self-organization mechanismshave been proposed, which exploit lateral excitatory connectivityin cortical areas (Wiemer, 2003). The principle can be illustratedwith the image of drops of water falling on a liquid surface:consider a sequence of events, each event stimulating a corticalunit. For event n, the stimulation of a cortical unit will generate alateral wave of excitation within the map. This wave will collidewith the waves emitted by the cortical units excited by eventsn − 1 and n + 1. Cortical units situated at the collision loci ofthe waves will be able to learn the corresponding co-excitationand consequently the sequence of activation within the map.A Bayesian mechanism has been proposed (Koechlin, Anton, &Burnod, 1999) for the learning of such sequences. It also enablesanticipation, with the pre-activation of the next event in thesequence. Its principle relies on a Hebbian learning between thethalamic or feedforward activation of the unit representative of anevent and the cortical activation of the unit representative of thepreceding event in the sequence.Thismechanismof lateral propagation of excitation is described

here as encoding sequences of events but it could have a moregeneral use. This processing principle of the cortex is not yetfully mastered but seems to be a fundamental property to bebetter understood. For example, the cerebellum shares somestructural properties with the cortex (tilled with similar micro-circuits) but does not have such a lateral connectivity (Doya,1999). Its appearance in the cortex could have endowed it withdynamic relaxation properties, allowing for propagation of waveson its surface for the search of solutions to multiple constraintexpressions (Burnod, 1989).

2.3. Cortico-cortical connections

Connections between cortical areas have been studied for twogeneral cases of activity: multi-modal integration and feedbackprojections. In both cases, the problem is to interconnect twocortical maps representing information of different kinds, eitherto create a mixed representation of both (multi-modality) or to

128 F. Alexandre / Neural Networks 22 (2009) 126–133

enable one map to be controlled or monitored by the activity ofthe other (feedback).A large amount of work has studied multi-modality as well

as feedback, most of which are somewhat consistent as far asfunctioning and learning rules are concerned. The general idea is toadvocate for the implementation of a gating effect between maps.Differently from afferent connections having an integrative effectonto cortical units, such cortico-cortical connections will ratherhave a multiplicative influence onto existing activity in the map:a strong signal will enhance the activity whereas a weak signalwill inhibit it. Such a logic has been frequently implemented insigma–pi units (Alexandre & Guyot, 1995; Fix, Vitay, & Rougier,2006) for which Hebbian learning rules can be easily derived.

2.4. Sub-cortical output and feedback

Competition and association between cortical columns yield apattern of activation in cortical maps that has to be transformedinto actions or decisions in the outer world or into a feedbackinfluence onto lower maps. This is the role of deeper layers 5 and 6of the cortex, particularly in themotor and premotor cortex. Layers5 and 6 also establish feedback links between hierarchical sensoryareas.

2.5. In summary

We have presented here some principles of information proce-ssing generally considered as responsible for cortical organizationand involving the laminar architecture of the cortex. This func-tional view defines a basic automaton of computation, the corticalcolumn, considered as the crossroad of several information flowsin the cortex. Local specialization and, coupling and uncoupling inthe automaton enhance functional circuits. We now illustrate howelementary behavioral functions exploit these mechanisms.

3. Coordination

Many models of the cerebral cortex have been illustrated byapplications devoted to the discrimination of perceptive stimuli(e.g. visual recognition) and to the orientation of actuators inspace (e.g. target reaching). These two kinds of tasks illustratea fundamental property of the cortex, structured into twopathways (Goodale & Humphrey, 1998): the parietal axis (wherepathway) which associates perceptive and body poles of thecortex; and the temporal axis (what pathway) which associatesthe perceptive and the limbic poles of the cortex, the latter beingknown to be the locus of multi-modal episodic memory (Squire,1992) (pathway toward the hippocampus).

3.1. Identification and location

Identification and location might appear as rather differenttasks. Particularly, imagining that they emerge from the samelocal logic of computation and that only the nature of the dataand the structure of the network specify the task can appearas a rather difficult exercise. Effectively, even if many modelsare consistent with the general principles mentioned above,their detailed implementation generally makes identification andlocation models incompatible. This is not acceptable in theframework of the unified theory of cortical processing that thispaper tries to promote. I first explain here why the tasks are notso different as they could appear at first sight and then proposea framework of interpretation that could correct some excesseswhich in my opinion are often made in classical models.AsmentionedbyNorman (2002) andbyGoodale andHumphrey

(1998), it is important to underline that identification and locationtasks have been designed by researchers from different origins andwith different goals. Identification is a classical task in statistics andartificial intelligence (e.g. classification, recognition tasks) and the

Table 2Two apparently different tasks (identification and location) rely on similarprinciples.

Task Identification Location

Associative cortex Temporal ParietalLinking sensory pole with Limbic pole Body poleExtracts hints related to Emotional value Position in space

general goal of such a task is to activate only one output unit (theclass number) for each possible perceptive input. Location ratherrefers to geometrical and behavioral aspects and the goal here isto compute, within a population of neurons, transformations fromone reference frame to another. Moreover in this latter case, theunderlying task is more ecological and compatible with Gibsoniantheories (Gibson, 1979), known to be difficultly compatible withsymbolic artificial intelligence promoted by the former domain.Are these theories and principled approaches only a distorting

mirror which hides the similarity of the tasks? The first argumentfor a positive answer comes from a description of the tasks interms of information flows. First of all, it must be recalled thatparietal and temporal axes can be seen as providing an associationbetween different poles of representation in the cortex. In a moredynamical view, it can be said that they coordinate poles ofrepresentation. Basically, the parietal axis performs a coordinationbetween perceptive (visual, auditory) and body representation,whereas the temporal axis performs a coordination betweenperceptive stimuli and the inner representations that they triggerin the limbic system (attractive vs repellent, love vs hate, etc.).In both cases, learning the coordination corresponds to extractinghints that are discriminant in the association between the polesinto consideration. Later, this learning can be used to activate apopulation of neurons to orient the body toward a target or toactivate few neurons representative of a class to be recognized, butin both cases, the same kind of distributed network establishingrelations between poles of representation is exploited (cf. Table 2).This unified view of coordination between poles of representationpromotes the principle of purposive perception (Goodale &Humphrey, 1998): the axes of processing depend on the purposeof the poles to be associated and consequently determine the hintsto be extracted in the intermediate representations.Ballard, Hayhoe, Pook, and Rao (1997) have illustrated such

a dual view in a task of identification or location of stimuli.From high-dimensional feature vectors extracted from the imageof objects, they first explain how a neuronal distribution ofactivity can be memorized in internal representations. Then,they underline the similarity of identification and location tasks:Identification corresponds to building a neuronal activation fromthe current foveal view, to compare it with a large number ofmemorized internal representations and to select the one bestmatching the current view. Location corresponds to activating onlyone internal representation (the object we wish to locate in thescene), to find the place in the visual fieldwith the closest neuronalactivation and to return its location. In both cases, the resources arethe same but they are transformed by the way they are used.

3.2. The situatedness of cognition

The most synthetic way to reconcile identification and locationis to remember that the phenomenonwe are interested in here, thehuman cognition, emerges from the interactions of the body withthe world. As evoked above, the body can orient toward objectsto better perceive and analyze them. The analysis is performed onthe basis of physical measurements and more or less elaboratedemotional reactions of the body generated by the stimuli. Thiswill elicit other orientation activities of the body (approach andgrasping or avoidance). This elementary behavior is made possibleby the capacity to coordinate the perceptive representation of

F. Alexandre / Neural Networks 22 (2009) 126–133 129

the outer world (including the perception of the body itself)with the answer that it generates either in the physical bodyreference frame (parietal axis) or in the more emotional referenceframe linked to the limbic system (temporal axis). In both cases,this capacity of coordination is acquired from experience andinteraction with the world. In this perspective, the cortex is seenas a multi-modal device, learning the self-organization of thevarious frames of representation sent to it by the outer (perception,proprioception) and the inner (emotion, rhythms) world, and alsothe self-organization of the associative substratum linking thepoles of representation. Menard and his colleagues have shownthat a self-organization process can also act on such multi-modalinformation. This process displays information in such away that itbuilds in associative areas an intermediate representation betweenthe poles (Ménard & Frezza-Buet, 2003).In the elementary tasks mentioned here (identification and lo-

cation), only two pathways are considered but the same explana-tion applies to other more complicated behaviors. Particularly, thefrontal lobe of the cortex is known to stand for the motor pole ofrepresentation (Fuster, 1997). Thanks to its links with the basalganglia and the posterior part of the cortex, the prefrontal cortexassociates the consequences ofmotor actionwith sensory informa-tion from the inner and outerworld, self-organized in the posteriorpart of the cortex as mentioned above, and combines this infor-mation with goal oriented rewards computed in the basal gangliato decide the temporal organization of behavior which best satis-fies the constraints. This aspect is out of the scope of this paperand will be only evoked in the concluding part. Nevertheless, it isworth mentioning that the present approach also claims that unitsin the prefrontal cortex obey the same rules of local computationand global organization in poles of representation, as in the poste-rior cortex.Moreover, the theory of the situatedness of cognition gives

other principles about information representation which are ofhigh interest to understand which relations and coordinationsare learned in the cortex. Contrary to intellectualist symbolicapproaches and their abstract, complex and hierarchical represen-tations, the situatedness of cognition proposes minimizing repre-sentational contents (O’Regan & Noe, 2001) and privileges simplestrategies, more directly coupling perception and action (Brooks,1991) and more efficient to react quickly in the changing environ-ment.A key aspect of this theory of intelligence is the Gibsonian no-

tion of affordance (Gibson, 1979): perception is not a passive pro-cess and, depending on the current task, objects are discriminatedas possible tools that could be used to act in the environment.Whereas a scene full of details can be memorized in very differ-ent and costly ways, a task-dependent description is a very eco-nomical way that implies minimal storage requirements. Hence,remembering becomes a constructive process.Ballard et al. (1997) have done a decisive step to operationalize

this concept. From the idea that the orientation of the bodyis important for cognition, they propose that eye movements,performed at an average rate of one third of a second, are theright level of description of the computational interface betweenneuronal and behavioral processes: the embodiment level. Theypropose considering eye fixation as a deictic pointer (with thesamemeaning and use as in computer science) that binds variablesextracted from the environment. In this framework, themeaning ofthe pointer corresponds to task-relevant components of the objectbeing fixated. It must be underlined how, as in computer science,the pointer mechanism is clever and economical: the pointer canbe manipulated internally for complex operations. For the sameoperation to be performed on different objects, it is not necessaryto store all the details in memory but just to change the referentof the pointer. Also, with such a deictic strategy, the organismcan keep track of relevant targets in the environment by only

storing themovement of the eye necessary to foveate it. We do notmemorize details of the object but we knowwhich eye movementto perform to get them: Theworld itself is considered as an externalmemory.All these principles can possibly be applied to coordination

tasks (identification and location) but are not necessary formodelers working at the level of only one processing pathway:it is still possible for them to be influenced by a moresymbolic approach and to build a fully detailed representationof information; the system is generally effective and only itsbiological plausibility can be discussed. The same cannot be saidabout more complex tasks, where several processing pathwaysand poles of representation are implicated. My opinion is thatsuch systems are difficult to build and master with a too richrepresentation of information just because this approach is notsustainable, as is demonstrated at this level of complexity. Inthe next section, I present some experiments, based on theembodiment of cognition, which successfully implement complextasks with a minimal representation.

4. Attention

Wenow considermore complex tasks, involving several framesof representation and, in the framework of the above theory,several pointers to index the world. In this section, amongthe variety of body orienting movements, real and virtual eyemovements only are considered: fixation and attention, seen asmechanical and neural deictic devices.Concerning fixation, Ballard et al. (1997) have deeply studied

an experimental procedure, in which a subject has at his disposal aset of colored blocks to copy amodel onto a workspace. Behavioralstudies reveal the strategy generally chosen by subjects, faithfulto the minimal internal memory principle and described by aset of eye movements. One typical iteration of the block copyingtask can be described as follows: the eye goes to the model toacquire the color of the next block to be copied (identification),goes to the set of blocks to find one with the same color (location),the hand picks up the foveated block while the eye goes backto the model to acquire the position of the considered block;the hand with the block moves toward the workspace and dropsthe block at the place where, meanwhile, the eye has indicatedthe similar place as in the model. It can be remarked that thismultiple fixation strategy drawsminimally on internal memory bybinding objects to variables, at the cost of a longer execution time.This behavioral study is directly linked to several computationalmodels, related to computer vision and reinforcement learning but,as far as I know, not to connectionist approaches. I now presenta connectionist model that we are currently building, based onsuch a unified theory of cortical processing. This model aimsat implementing covert and overt visual attention, as a way tointernally and externallymanage eyemovements and their relateddeictic properties.

4.1. Local functioning rules

The model is composed of a set of two-dimensional discreteneural fields, which are direct extensions of the model proposedby Amari (1977), in 2D and with a discrete implementation oftopological maps. Each of the maps corresponds to a corticalarea, composed of units corresponding to cortical automata, aspresented above. As described below and sketched in Fig. 1, a setof maps is implemented to represent two information flows, theWhere and the What pathways, receiving external informationfrom a visual input (retina) and able to internally focus on a targetand to produce cameramovements to center a desired target in themiddle of the retina (fovea).

130 F. Alexandre / Neural Networks 22 (2009) 126–133

Fig. 1. General structure of a model of visual overt and covert attention: feature and spatial processing are respectively performed in models of temporal and parietalcortical axes. Features are extracted in the Feature maps and combined in the Saliency map. The Focus map selects only one pattern because it is salient or because it isdesired thanks to the monitoring of the Feature processing axis, encoding Target and Perceived features. The Working memory maps memorize the patterns that have beenexplored (inhibition of return) and the Memory anticipation map anticipates eye movements and update the Working memory. In the covert mode, the Switch mechanisminhibits the Focus map to put the focus on the next selected target. In the overt mode, a camera movement is performed if the target corresponds to the perceived feature. Inaccordance with a unified theory of cortical processing, all these apparently different processing are in fact produced by maps of cortically-inspired units driven by identicalfunctioning rules. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

As a consequence, at the level of the cortical automaton, severalkinds of input and output have to be specified. The feedforwardflow between areas and the lateral flowwithin areas are integratedwith weighted summations. Cortico-cortical connections betweenareas are implemented by a multiplicative rule (weighted sumof the product of activity of afferent neurons). The weightingfunctions are chosen as a Gaussian for the lateral flows and asa difference of Gaussians for the feedforward flows. They areconstant for the cortico-cortical connections. At themoment, theseweights are fixed from biological data (Miikkulainen et al., 1996);current work aims at defining learning rules to adapt them.

4.2. Emerging properties

From these local uniform functioning rules, several interestingproperties emerge from a structured network, depending only onthe connectivity.Saliency: The feedforward weights can be chosen in such a waythat, in a topological map receiving an external visual input, eachunit responds best to a stimulus in a given receptive field andwith a given orientation or color. This allows implementing Featuremaps (cf. Fig. 1) acting as topological filters on these features.These elementary feature maps converge onto the Saliency mapwhich displays a multi-modal representation. In all these maps,the Gaussians are narrow and several stimuli can be represented.Gaussians are larger in the Focus map that receives informationfrom the Saliency map. As a consequence, a stronger competitiontakes place in the Focus map and only one blob of activity cansurvive. This competition can be biased by feedback informationfrom one map in the What axis (Feature processing axis in Fig. 1)which can specify one feature of the desired target (e.g. a redpattern or orientation on the left). Conversely, the Focus map canbias the What axis in order to display the identity of the patternthat wins the competition in the Focus map (e.g. red patternoriented on the left).

Inhibition of return: If the goal of the task is to successively focusonto each pattern with specified features, a mechanism is desiredto prevent the system from going several times to the same place.This mechanism is known as the inhibition of return (Posner &Cohen, 1984) and is implemented with a couple of maps calledWorking memory in Fig. 1 and described in Vitay, Rougier, andAlexandre (2005). One map in the Working memory receivestopological (one to one) excitations from the Focus and the Saliencymaps and sends reciprocal excitatory topological connections toanother similar map that helps in maintaining excitation: bothexternal inputs are necessary to create excitation but only one(from the Saliencymap) is sufficient tomaintain it. This cumulativemechanism creates a dynamical memory of the locations that havebeen explored. When the focus of attention has to change (’switch’order in the covert mode), this memory will inhibit the Focus mapand force it to look for another location.Anticipation: A camera movement completely modifies the visualreference frame and all that has been built in the Workingmemory could be lost. To avoid this perturbating phenomenon,a Memory anticipation map is built (cf. Fig. 1). It is computedas a convolution product of the Working memory and the Focusmap. Just before a movement is triggered, the Focus map givesthe information of the next location that will be attended. Thanksto the multiplicative effect of the Memory anticipation map, theactivity of the Working memory is translated by the movementvector represented in the Focusmap.When the cameramovementis performed, this prediction of the future state of the Workingmemory meets the feedforward activity sent by the Saliency mapand the representation remains valid and stable.

4.3. Integrated behavior

The properties evoked above are only a functional interpre-tation of the collective behavior of few interconnected maps ofcortically-inspired automata. This level of interpretation can also

F. Alexandre / Neural Networks 22 (2009) 126–133 131

Fig. 2. Screenshot of the model. This is an implementation with maps and units of the functional circuit presented in Fig. 1. Left part: The perceived world is in the redsquare which can be moved (overt attention) or can be internally explored (switch: covert attention). Observed (bottom row) and desired (middle row) hints of color (Blue;Green) and orientation (Pi/4; 3Pi/4). Upper row: Move or Switch order. Right part: implementation in maps of cortically-inspired units of anticipation, Working memoryand inhibition of return for the appropriate analysis of the scene. The perceptual map stands for the Saliency map; the Premotor map stands for the Focus map. TheWorkingmemory is composed of wm and thal-wm maps. The four upper left maps related to inhibition correspond to the Switch mechanism, not described here. Refer to the textand to Fix et al. (2007a, 2007b, 2006) for details. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

be used to relate the whole system to such a complex behavior asvisual attention.Covert attention does not imply eye movements and allows

for parallel feature search and sequential conjunction search(involving several features)with inhibition of return. Treisman andGelade (1980) explain this property in their Feature IntegrationTheory where a global feedforward processing is only possibleinside one feature map and an iterative sequential processincluding feedback operates in the conjunctive cases.An initial model (Vitay et al., 2005) including the Focus and

Working memory maps is able to perform covert attention anddisplays similar properties (Fix, Rougier, &Alexandre, 2007a): if thedesired target is encoded only in a feature map, it can directly biasthe competition in the Focus map; else a selection correspondingto only one feature dimension is made, must be compared to thedesired target and eliminated (thanks to the inhibition of return)or selected (hence a search time proportional to the number oftargets).We have recently extended the system (adding the Memory

anticipation map) to allow for eye movements (overt attention).This is consistent with the premotor theory of attention (Rizzolati,Riggio, Dascola, & Ulmitac, 1987) that considers that covertand overt attentions share several neuronal structures. As eyemovements perturbate the frames of representation, there is aneed to anticipate the consequences of eye movements to updatethe Working memory (Fix, Rougier, & Alexandre, 2007b), asillustrated in Fig. 2. Here the goal is to center the retina on a bluetarget. In the figure, the red square corresponds to the retina andcontains a blue target at the bottom. This target is selected by thefocus map; in the Anticipationmap, it is anticipated that the targetwill be in the middle of the retina and the Working memory willbe translated accordingly, as the movement is performed.

In this system, the visual search can be also much morecomplex; it is for example possible to sequentially focus onall patterns with the same characteristics, the search for onepattern being itself a conjunctive search. In this case, forexample, the goal would be to focus on all blue bars witha Pi/4 orientation. First, the covert attention will successivelymake more salient several Blue or Pi/4 patterns until one withboth characteristics in found (else, the Switch neuron eliminatesa bad candidate with inhibition of return and asks for thenext candidate). When a matching candidate is found, overtattention can apply and ask for a real camera movement totranslate this pattern in the middle of the retina. Just before themovement, the Memory anticipation mechanism will anticipatethe modification in the frame of representation and modify theWorking memory accordingly. Here too, it must be underlinedthat this functional interpretation emerges from distributedasynchronous computations of cortically-inspired maps of units.Thismechanism can also be interpreted in the framework of the

situatedness of cognition. In the ‘‘feature axis’’ corresponding tothe temporal axis, a kind of Saliencymap uses a filtering process tochoose targets from intrinsic characteristics of perceived objectsor because they are ‘‘desired’’ (the goal of the task). Then, on thisbasis, in the ‘‘spatial axis’’ standing for the parietal axis, only thelocations of stimuli, that can be considered as pointers towardobjects in the real world, are manipulated. Particularly, a gatingmechanism using sigma–pi units combines the activity of a focusmap indicating the next target to focus on with the activity ofthe Working memory to anticipate what should be the state ofthe Working memory after the eye movement (cf. Fig. 1 and referto Fix et al. (2006) for details). After this internal process of pointermanipulation, when the eye movement is performed, it is possible

132 F. Alexandre / Neural Networks 22 (2009) 126–133

to relate this information about location to the real objects in theworld and to their characteristics.Fig. 2 illustrates themodel running in a simplified environment,

where the behavior of the system can be finely observed. Currentwork complexifies the input filters to apply the system to naturalimages. Concerning the size of the current model, it should beunderlined that it consists of eighteen maps, fifteen thousandunits and over six million connections, which illustrates the factthat fully distributed asynchronous computations can bemastereduntil that level, provided that a solid theoretical frameworkensures the consistency of the whole.

5. Communication

In the preceding sections, I have presented the principles ofa theory of intelligence based on the situatedness of cognition,together with several behavioral and biological related facts. Ihave shown the use of such a framework for tasks related tovisual scene exploration with or without eye movements. Themain goal of this paper is to argue that this framework is alsofundamental to endowing artificial beings with efficient and bio-inspired communication skills.It was indicated above that the orientations of any parts of the

body can be used as deictic pointers toward the real world and thatthis principle is used by human beings to keep track of relevant‘‘objects’’ (with a broad meaning) in the world. Firstly, since allhumans use that strategy, consciously or unconsciously observingthe orientation of the body parts of other humans can be usefulfor being informed about their intentionality: This is the first wayto communicate. Secondly, the spoken language can be integratedto that framework if we go beyond the visual modality put to thefore for the moment. From the multi-modal integration principlesevoked at the beginning of this paper, it can be understood that an‘‘object’’ that is evoked, observed ormanipulated can be associatedwith an articulated sound. Similarly to pointers to the outer world,this sound can also be considered as a pointer to the object it usedto be associated with.Concerning non-verbal communication, referents can be

achieved by pointing with the hand, the head, the eyes, etc. andthe listener is able to bind the real object to those deictic pointers(e.g.: do it where I am fixating). Concerning spoken communica-tion, deictic pointers are useful to interpret indexicals (e.g.: this ismine). In both cases, we find again that principle to represent in-formation from hints toward the outer world as it is needed ratherthan to build complete and complex internal representations of theworld.All these phenomenological descriptions particularly match

with what we know about the role of mirror neurons incommunication (Rizzolatti, 2005). Besides, they are ideally locatedat the crossroads of multi-modal sensory information and actionrepresentation in the cortex. They are also characterized by ageneric spectrum of responses, independent from the agent andthe sensory modality and, as part of the cortex, share the samestructuration in cortical columnswith the other parts of the cortex.

6. Perspectives

The preceding section is itself a prospective part because alot of work remains to be done to operationalize such concepts.Nevertheless, it seems to me that the way forward is rather clearto proceed in the domain. The main prerequisite is to be able tooperate several (artificial and/or real) agents creating, observingand manipulating such deictic pointers in a real world.It must be also underlined that I mainly developed here

the first level of the theory, based on deictic strategies at athird of a second or so, concerning orientations of parts of the

body or uttered words. A lot remains to be done to considerbehaviors with longer constants of time, requiring manipulationsand combinations of elementary pointers. Similarly to computerscience, it is possible to imagine creating pointers of pointersand preserving the genericity of the computations. Two cerebralmechanisms must be evoked in that perspective. Firstly, theperirhinal cortex and the hippocampic system are reputed tostore high level abstract representations (Squire, 1992). Secondly,the prefrontal–striatal–thalamic–cortical loops own a variety oftime constants that allow for the organization of behavior intime respecting several kinds of constraints (Fuster, 1997). Atthe moment, I have described separated elements of behavior.Such mechanisms are necessary to integrate them in behaviorswith longer constant of time while preserving their reactivity andflexibility. First, this is necessary to develop longer term dynamicalmemory (to have the illusion of a stable world in the visual case;to remember the general meaning of the sentence while utteringa word). Second, this is necessary to give a stronger meaningfulstructure to the behavior (intelligent exploration of the world inthe visual case; syntactic rules in language). My claim is that suchpowerful processes could emerge from these mechanisms andthat these mechanisms could be implemented with the same localcomputational logic as the one described in this paper.Finally, concerning both aspects, they should be developed in

parallel to solid knowledge concerning the underlying cerebralcircuitry. Here also, there exists a large amount of evidence, evokedabove, relating most aspects to posterior and prefrontal cortex,hippocampus and basal ganglia. Modeling experiments could alsobring up new predictions to be confronted with biological andbehavioral reality.

References

Alexandre, F., & Guyot, F. (1995). Neurobiological inspiration for the architectureand functioning of cooperating neural networks. In Proceedings internationalworkshop on artificial neural networks.

Alexandre, F., Guyot, F., Haton, J.-P., & Burnod, Y. (1991). The cortical column: A newprocessing unit for multilayered networks. Neural Networks, 4, 15–25.

Amari, S. (1977). Dynamic of pattern formation in lateral-inhibition type neuralfields. Biological Cybernetics, 27, 77–78.

Ballard, D. H. (1986). Cortical connections and parallel processing: Structure andfunction. Behavioral and Brain Sciences, 9, 67–120.

Ballard, D. H., Hayhoe, M. M., Pook, P. K., & Rao, R. P. N. (1997). Deictic codes for theembodiment of cognition. Behavioral and Brain Sciences, 20, 723–767.

Bringuier, V., Chavane, F., Glaeser, L., & Fregnac, Y. (1999). Horizontal propagation ofvisual activity in the synaptic integration field of area 17 neurons. Science, 283,695–699.

Brooks, R. (1991). Intelligence without representation. Artificial Intelligence, 47,139–159.

Burnod, Y. (1989). An adaptive neural network: The cerebral cortex. Masson.Doya, K. (1999).What are the computations of the cerebellum, the basal ganglia andthe cerebral cortex? Neural Networks, 12, 961–974.

Fix, J., Rougier, N., & Alexandre, F. (2007a). From physiological principles tocomputational models of the cortex. Journal of Physiology, 101(1–3), 32–39.

Fix, J., Rougier, N. P., & Alexandre, F. (2007b). A top-down attentional systemscanning multiple targets with saccades. In: From computational cognitiveneuroscience to computer vision.

Fix, J., Vitay, J., & Rougier, N. P. (2006). A computational model of spatial memoryanticipation during visual search. In Proceedings of the third workshop onanticipatory behavior in adaptive learning system.

Fuster, J. M. D. (1997). The prefrontal cortex : Anatomy, physiology, and neuropsychol-ogy of the frontal lobe. Lippincott Williams and Wilkins Publishers.

Gibson, J. J. (1979). The ecological approach to visual perception. Boston: HoughtonMifflin.

Goodale, M. A., & Humphrey, G. K. (1998). The objects of action and perception.Cognition, 67, 181–207.

Grossberg, S., & Swaminathan, G. (2004). A laminar cortical model for 3d perceptionof slanted and curved surfaces and of 2d images: Development, attention andbistability. Vision Research, 44, 1147–1187.

Jancke, D., Chavane, F., Naaman, S., & Grinvald, A. (2004). Imaging cortical correlatesof illusion in early visual cortex. Nature, 428, 423–426.

Koechlin, E., Anton, J.-L., & Burnod, Y. (1999). Bayesian inference in populations ofcortical neurons: A model of motion integration and segmentation in area mt.Biological Cybernetics, 80, 25–44.

Kohonen, T. (1989). Springer series in information sciences: Vol. 8. Self-organizationand associative memory. Springer-Verlag.

F. Alexandre / Neural Networks 22 (2009) 126–133 133

Korner, E., Gawaltig, M.-O., Korner, U., Richter, A., & Rodemann, T. (1999). A modelof computation in neocortical architecture. Neural Networks, 12, 989–1005.

Ménard, O., & Frezza-Buet, H. (2003). Multi-map self-organization for sensorimotorlearning: A cortical approach. In IEEE international joint conference on neuralnetworks.

Miikkulainen, R., Bednar, J. A., Choe, T., & Sirosh, J. (1996). Self-organization,plasticity, and low-level visual phenomena in a laterally connected map modelof the primary visual cortex. Psychology of Learning and Motivation.

Mountcastle, V. (1978). An organizing principle for cerebral function: The unitmodel and the distributed system. In Gerald M. Edelman, & Vernon B.Mountcastle (Eds.), The mindful brain. Cambridge, MA: MIT Press.

Norman, J. (2002). Two visual systems and two theories of perception. Behavioraland Brain Sciences, 25, 73–144.

O’Regan, J. K., & Noe, A. (2001). A sensorimotor account of vision and visualconsciousness. Behavioral and Brain Sciences, 24, 939–973.

Posner, M. I, & Cohen, Y. (1984). In H. Bouma, & D. Bouwhuis (Eds.), Attention andperformance: Vol. X . Components of visual orienting (pp. 531–556). LawrenceErlbaum.

Rizzolati, G., Riggio, L., Dascola, I., & Ulmitac, C. (1987). Reorienting attention acrossthe horizontal and vertical meridians. Neuropsychologia, 25, 31–40.

Rizzolatti, G. (2005). Themirror neuron system and its function in humans. Anatomyand Embryology, 210(5–6), 419–421.

Rougier, N. P. (2006). Dynamic neural field with local inhibition. BiologicalCybernetics, 94(3), 169–179.

Squire, L. R. (1992). Memory and the hippocampus: A synthesis from findings withrats, monkeys, and humans. Psychological Review, 99, 195–231.

Taylor, J. G. (1997). Neural networks for consciousness. Neural Networks, 10(7),1207–1225.

Taylor, J. G. (1999). Neural bubble dynamics in two dimensions. BiologicalCybernetics, 80, 5167–5174.

Treisman, A., &Gelade, G. (1980). A feature-integration theory of attention.CognitivePsychology, 12, 97–136.

Vitay, J., Rougier, N., & Alexandre, F. (2005). A distributed model of spatial visualattention. In S. Wermter, G. Palm, & M. Elshaw (Eds.), Lecture notes in computerscience: Vol. 3575. Biomimetic neural learning for intelligent robots: Intelligentsystems, cognitive robotics, and neuroscience (pp. 54–72). Springer.

Wiemer, J. C. (2003). The time-organized map algorithm: Extending the self-organizing map to spatiotemporal signals. Neural Computation, 15, 1143–1171.

Wilson, H. R., & Cowan, J. D. (1972). Excitatory and inhibitory interactions inlocalized populations of model neurons. Biophysical Journal, 12, 1–24.