Biological solutions to the selection problem Peter Redgrave Neuroscience Research Group Department of Psychology, University of Sheffield

Biological solutions to the selection problem

Peter RedgraveNeuroscience Research Group

Department of Psychology,

University of Sheffield

Abbreviations:CPu, caudate putamenGP, globus pallidusMRF, medullary reticular formationSC, superior colliculusSNr, substantia nigra pars reticulataSTN, subthalamic nucleus

CPu

GP

SNr

Vm

SC

MRF

STNEN

The Team• Kevin Gurney

• Tony Prescott

• Mark Humphries

• Fernando Montes Gonzales

• Khepera Robot

Staking out the ground

• Level versus content of consciousness– “A useful distinction can be made between factors

influencing the overall level of consciousness and those determining its content”.

• Critical role for attention– “The studies of perception….demonstrate that stimuli can

be highly processed and yet not enter awareness. Attention might be the critical mechanism by which preprocessed stimuli are selected for awareness”.

Frith, C., R. Perry, et al. (1999). "The neural correlates of conscious experience: an experimental framework." Trends Cogn Sci 3: 105-114.

Outline of talk

• Selection - a fundamental computational problem

• Theoretical solutions

• Basal ganglia as a biological solution

• Tests of the selection hypothesis– Simulation

– Robotics

• Is the robot conscious ?

A General Architecture of the Brain

• Critical concept:

– Independent functional units - modules

– Each with:

• critical sensory input

• specific objectives, actions or movements as output

Necessary implication : Selection Problem

Behavioural output(Feeding)

Fluid balance(Drinking)

Predisposing Conditions

MotorResources

Energy balance(Feeding)

Threat(Escape)

• Multiple command systems

• Spatially distributed

• Sensory specificities

• Output specificities (often exclusive)

• Processing in parallel

• All act through final common motor path

Theoretical Solutions

• Recurrent reciprocal inhibition– Selection an emergent

property– Positive feedback– Winner-take-all

• Centralised selection– Localised switching

– Dissociates selection from perception and motor control

MotorPlant

MotorPlant

InputSaliencies

InputSaliencies

Problems of Scale

• Recurrent reciprocal inhibition– Each additional

competitor increases connections by n(n-1)

• Centralised selection– Each additional

competitor adds 2 further connections 3 competitors

6 connections3+1 competitors6+2 connections

8 competitors16 connections


3+1 competitors6+6 connections


Basal Ganglia: a biological solution to the selection problem

Rat

Human

Evolutionary conservatism

“The basal ganglia in modern mammals, birds and reptiles (i.e. modern amniotes) are very similar in connections and neurotransmitters, suggesting that the evolution of the basal ganglia in amniotes has been very conservative.”

Medina, L and Reiner, A.

Neurotransmitter organization and connectivity of the basal ganglia in vertebrates: Implications for the evolution of basal ganglia. Brain Behaviour and Evolution (1995) 46, 235-258

Functions attributed to the Basal Ganglia• Sensory

– Sensory perception (Brown et al 1997)– Analgesia (Chudler and Dong 1995)

• Cognitive– Attention (Jackson and Houghton 1995)– Temporal processing (Gibbon et al 1997) – Working memory (Levy et al 1997)– Habit learning (Gaffan 1996)

• Motor– Planning, selection and execution of motor strategies (Robbins and Brown 1990)– Initiating movement (Denny-brown 1962)– Scaling speed/size of movement (reviewed in Mink 1996)– Building action repertoires (Graybiel 1995)– Automatic execution of movement sequences (van den Bercken and Cools 1982)

• ?– Suppression of epileptic seizures (Depaulis 1994)

Clinical associations• Parkinson’s disease

– Disease of the motor system…….– But with increasingly appreciated cognitive overtones – set switching

• Schizophrenia– Cognitive, sensory and motor symptoms– Inability to suppress competing systems ?

• Addictions– Cravings dominate

• Other basal ganglia-related dysfunctions– Attention deficit disorder– Obsessive compulsive disorder– Tourette’s syndrome– Altzheimer’s disease


CPu

GP

SNr

Vm

SC

MRF

STNEN

Basal Ganglia: External Connectivity• External command systems

– Cortical – Limbic– Midbrain

• Command inputs– Sensory– Cognitive– Affective

• Command outputs– Converge on brainstem and

spinal motor generators

• Links with basal ganglia– Closed loop connections

Redgrave et al (1999) Neuroscience, 89, 1009-1023

Basal Ganglia Architecture : Loops I

Alexander, G. E., M. R. DeLong, et al. (1986). "Parallel organization of functionally segregated circuits linking basal ganglia and cortex." Ann. Rev. Neurosci. 9: 357-381.

Loops: a specific example

Middleton, F. A. and P. L. Strick (1996). "The temporal lobe is a target of output from the basal ganglia." Proc Natl Acad Sci USA 93(16): 8683-8687.

Phasic/excitatory

Phasic/inhibitory

Tonic/inhibitory

Phasic/Disinhibitory(PositiveFeedback)

Basal Ganglia: Repeating microcircuitry

• External command systems– Cortical – Limbic– Midbrain

• Command functions– Sensory– Cognitive– Affective


CPu

GP

SNr

Vm

SC

MRF

STNEN

Selection by inhibition and disinhibition

MotorResources



Threat(Escape)

Basal Ganglia

ExcitationInhibition

Behavioural output(Feeding)

Fluid balance(Drinking)


MotorResources


Threat(Escape)

The Selection Problem

Potential resolution

Serial Selection in the Basal Ganglia

Striatum

Inputs (Cortex/Thalamus)

Output Nuclei

Up-state/down-state filtering

1) Up-down states of medium spiny neurones

Local inhibitory circuits

2) Local inhibition in striatum

Local recurrent circuits4) Recurrent inhibition in output nuclei

Subthalamus

3) Diffuse/focused projection onto output nuclei

Focused inhibition

Diffuse excitation

Model: Analysis

MotorResources



Threat(Escape)

Basal Ganglia

ExcitationInhibition

Model neurons - leaky integrators with piecewise linear output

Analytic equilibrium solution(Kevin Gurney)

Gurney, K., T. J. Prescott, et al. (2001). "A computational model of action selection in the basal ganglia. I. A new functional anatomy." Biol Cybern 84: 401-410.

Model: Common currency

Koechlin, E. and Y. Burnod (1996). "Dual population coding in the neocortex: A model of interaction between representation and attention in the visual cortex." J. Cog. Neurosci. 8: 353-370.

Grandmother cell– out of fashion

A

B1

B2

NeuralActivity

Space

A B1

B2Stimuli

Population Coding– spatial distribution of activity = feature– area under curve = salience

Simulation results

Dynamic switching between channels on basis of changes in input salience

Computer Model: Conclusions

• Relatively easy to: – Take the basic architectural features of the basal

ganglia – Simulate them in a computer model– And have the model select between channels on the

basis of relative input salience (strength)

• Provides an existence proof of the hypothesised selective function

Next strategy

• Embody the model – Generates realistic (environmentally driven) sequences of

input– Forces interpretation of outputs in terms of actions

• Aim: To test if model can generate action sequences in a behaving robot– Research sought to model behavioural switching in a

foraging rat

Action Selection: Rat foraging

• Motivations– Hungry : 24hrs food

deprived– Frightened: placed in

open arena

• Behaviour– Initially keeps close to

walls and corners– Collects food– Returns to corner to

eat

Robot Action Selection: The foraging Khepera

• Motivations– Hunger– Fear

• 5 behavioural sub-systems– Wall seek– Wall follow– Can seek– Can pick-up– Can deposit

• 8 Infra-red sensors detect– Walls– Corners– Cans

• Gripper sensors detect– Presence/absence of can

The Physical Interface: Side-step problems of motor control

BasalGanglia

Sensor Data

Wheels/Gripper

PerceptualSub-systems

ActionSub-systems

At each time step: Basal ganglia/thalamus• Computes sub-systems saliences• Resolves competition• Disinhibits winning sub-system

Thalamus

Extrinsic Variables1) Perceptual sub-systems (Wall, corner, can, gripper)

3) Action sub-systems/current state

(Wall seek, corner seek, can

seek, can pick-up, can deposit)

MotivationalSub-systems

Intrinsic Variables2) Motivational sub-systems (Fear, Hunger)

Robot Action Selection: Model Dynamics

SelectionsCan Seek

Can Pick-up

Wall Seek

Wall follow

Can Deposit

Sensors

Motivations

FearHunger

Model dynamics

Saliences

Robot Ethogram

• Competitor with highest salience controls motor plant– Clean switching

– Little distortion

– Infrequent dithering

• Platform to test the effects of simulated: – Lesions

– Stimulation

– Pharmacological treatments

cylinder-seek

-1.5

-0.5

0.5

1.5

2.5

3.5

wall-seek

-1.5

-0.5

0.5

1.5

2.5

3.5

corner-seek

-1.5

-0.5

0.5

1.5

2.5

3.5

Salience BG output

Sig

nal

leve

l

time

cylinder-seek

0

0.2

0.4

0.6

0.8

1

Behaviour selection

corner-seek

0

0.2

0.4

0.6

0.8

1

wall-seek

0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

-1.5

-0.5

0.5

1.5

2.5

3.5

cylinder-pickup cylinder-pickup

Conclusions

• Selection hypothesis confirmed in simulation and control of robot action selection

• Represents a generic task performed by the basal ganglia– High level behavioural objectives

– Actions

– Movements

• Consistent with early development and evolutionary conservation

• Explains basal ganglia ‘involvement’ in so many tasks

Wider implications

• Decision making– Head vs heart

• Cortical vs sub-cortical competitors

– Free will ? • Is the central selector a dumb switch ?

• Personality – A statistical profile of the winners and loosers

Implications for content of consciousness

• Selective attention/conscious awareness– Equivalent to current ‘winning’ channel and associated circuits ?

– No single location for consciousness

– Dynamically switches from region to region

• Dissociations in neuropsychology– Blindsight vs peripheral neglect

– Achomatopsia – loss of experience of colour

– Procedural memory in amnesics

– Prosopagnosia (failure to recognise familiar faces)

– Impulsiveness of pre-frontal patients

Conscious awareness in artificial systems

• Does the robot have any awareness ?

• Has limited representations of ‘hunger’ and ‘fear’

• Has representations of critical aspects of external world

• Uses biologically inspired architecture to select appropriate actions

Critical role for working memory

• Cannabis– Subjective awareness of “self”

• Receptors in human brain– High densities in basal ganglia

• Problem of selection– Disruption of working memory

loop ?

Glass, M., M. Dragunow, et al. (1997). "Cannabinoid receptors in the human brain: A detailed anatomical and quantitative autoradiographic study in the fetal, neonatal and adult human brain." Neuroscience 77: 299-318.

Critical role for episodic memory

• Diazepam – Unconscious consciousness

• Receptor distribution– High densities in cortex and

cerebellum

• Not related to basal ganglia– Selection fine

Young and Kuhar (1979) Autoradiographic localisation of benzodiazepine receptors in the brains of humans and animals Nature, 280, 393-395

Documents

Biological solutions to the selection problem Peter Redgrave Neuroscience Research Group Department of Psychology, University of Sheffield