How Do Brain Areas Work How Do Brain Areas Work Together When We Think, Together When We Think, Perceive, and Remember?Perceive, and Remember?

J. McClellandJ. McClellandStanford UniversityStanford University

Examples of situations in which Examples of situations in which there must be coordination there must be coordination

between brain areasbetween brain areas• Cross-modal integration for perception, Cross-modal integration for perception,

action and spatial awarenessaction and spatial awareness• Speech perception (McGurk Effect…)Speech perception (McGurk Effect…)• Coordination of what and where in Coordination of what and where in

perceptionperception• Coordination of meaning and syntax in Coordination of meaning and syntax in

languagelanguage• Attention and working memoryAttention and working memory• Comprehension of language in a natural Comprehension of language in a natural

settingsetting• Memory for events and experiencesMemory for events and experiences

Feed-Forward or Feed-Forward or Cooperative Cooperative

Computation?Computation?

Occ-OFC Phaselocking

Fusiform-OFCLocking whenRecog succeeds

Fusiform-OFCLocking whenRecog fails

Coherent Engagement of Two Coherent Engagement of Two Brain Areas in Selective Attention Brain Areas in Selective Attention

(Gotts, Desimone, et al)(Gotts, Desimone, et al)

Hupe, James, Payne, Lomber, Hupe, James, Payne, Lomber, Girard & Bullier (Nature, 1998, Girard & Bullier (Nature, 1998,

394, 784-787)394, 784-787)• Investigated effects of Investigated effects of

cooling V5 (MT) on neuronal cooling V5 (MT) on neuronal responses in V1, V2, and V3 responses in V1, V2, and V3 to a bar on a background to a bar on a background grid of lower contrast.grid of lower contrast.

• Cooling typically produces a Cooling typically produces a reversible reduction in firing reversible reduction in firing rate to the cell’s optimal rate to the cell’s optimal stimulus.stimulus.

• Similar effects occur in Similar effects occur in many other tasks and many other tasks and contexts.contexts.

*

Could it be…Could it be…

• That conscious experience depends on That conscious experience depends on coherent activity distributed across many coherent activity distributed across many brain areas?brain areas?

• That our ability to think deeply and That our ability to think deeply and insightfully depends on coherent engagement insightfully depends on coherent engagement of multiple brain areas that each contribute of multiple brain areas that each contribute their own crucial element to successful their own crucial element to successful thinking?thinking?

• That individual differences in the ability to That individual differences in the ability to achieve such coherent engagement underlie achieve such coherent engagement underlie successful performance in intellectually successful performance in intellectually challenging and cognitively demanding tasks?challenging and cognitively demanding tasks?

The time has comeThe time has come

• Multi-regional cooperative computation is Multi-regional cooperative computation is the rule in nearly all aspects of human the rule in nearly all aspects of human cognitioncognition

• The reason we don’t generally think this way The reason we don’t generally think this way is that we have not until now had to tools to is that we have not until now had to tools to study cooperative engagement of brain areasstudy cooperative engagement of brain areas

• We now have the tools – over the next 5 We now have the tools – over the next 5 years we can use them convergently to years we can use them convergently to understand how parts of the brain work understand how parts of the brain work together.together.

Experimental Experimental ApproachesApproaches

• Investigate timing and synchronization of neural activity Investigate timing and synchronization of neural activity across brain areas using simultaneously recorded spikes, across brain areas using simultaneously recorded spikes, local field potentials, scalp EEGs, and/or MEG.local field potentials, scalp EEGs, and/or MEG.

• Investigate effects of congruity between inputs in different Investigate effects of congruity between inputs in different modalities on neural processes using above methods.modalities on neural processes using above methods.

• Couple the above with fMRI and DTI tractography to pin-Couple the above with fMRI and DTI tractography to pin-point cooperating brain areas.point cooperating brain areas.

• Use animal models as well as cognitive neuroscience Use animal models as well as cognitive neuroscience methods in humans. Small-scale and large-scale systems.methods in humans. Small-scale and large-scale systems.

• Explore the roles of experience and development of long-Explore the roles of experience and development of long-range connectivity in effective cross-regional engagement.range connectivity in effective cross-regional engagement.

• Explore how intrinsic properties of neurons, local circuit Explore how intrinsic properties of neurons, local circuit structure, neuro-modulatory influences support mutual structure, neuro-modulatory influences support mutual engagement within and between brain areas.engagement within and between brain areas.

• ……

Convergent Computational Convergent Computational ApproachesApproaches

• Pursue the computational theory of cross-regional Pursue the computational theory of cross-regional cooperative computation. Why does such cooperative cooperative computation. Why does such cooperative engagement occur?engagement occur?

• Explore biologically realistic models of the mechanistic Explore biologically realistic models of the mechanistic basis for cooperative engagement across brain areas.basis for cooperative engagement across brain areas.

• Explore learning methods that would support mutual Explore learning methods that would support mutual engagement of neural populations in diverse brain engagement of neural populations in diverse brain areas.areas.

• Exploit advanced data analysis methods and new Exploit advanced data analysis methods and new statistical concepts to infer directional (and bi-statistical concepts to infer directional (and bi-directional) influence among brain areas, and to directional) influence among brain areas, and to incorporate known structural and biophysical incorporate known structural and biophysical constraints.constraints.

Applicable Topic AreasApplicable Topic Areas

• PerceptionPerception• Cross-modal Cross-modal

integration of integration of sensory sensory informationinformation

• Motor ControlMotor Control• AttentionAttention• Working MemoryWorking Memory• Spatial CognitionSpatial Cognition

• Language Language perception, perception, comprehension and comprehension and productionproduction

• ReadingReading• Mathematical Mathematical

CognitionCognition• Semantic CognitionSemantic Cognition• Causal ReasoningCausal Reasoning• Episodic MemoryEpisodic Memory