Chapter 9 Dynamic Modelling of Brain Functions and Cognitive … · 12/16/2002 Nik Kasabov - Evolving Connectionist Systems Chapter 9 Dynamic Modelling of Brain Functions and Cognitive

12/16/2002Nik Kasabov - Evolving Connectionist Systems

Chapter 9Dynamic Modelling of Brain

Functions and Cognitive Processes

Prof. Nik [email protected]://www.kedri.info

Nik Kasabov - Evolving Connectionist Systems

• Evolving Structure of the Brain and Evolving Cognition

• Dynamic modelling of brain states based on EEG signals

• Dynamic modelling of cognitive processes based on brain imaging

• Modelling perception – the auditory system• Dynamic modelling of visual and multi-modal

systems • Computational models of the entire brain

Overview


Evolving Structure of the Brain and Evolving Cognition

• The human brain can be viewed as a dynamic, evolving information processing system – probably the most complex one

• As an embryo, the brain grows and develops mainly based on genetic information

• The cognitive processes of learning in the brain evolve throughout a life time, e.g. spoken language learning

• Most appropriate sources of data for brain modelling tasks come from instrumental measurements of the brain activities. » Electroencephalography (EEG), » Magnetoencephalography (MEG)» functional Magnetic Resonance Imaging (fMRI).


A brain’s pathways for auditory, visual and sensory motor information processing

Different areas of the human brain transfer different signals, shown as lines. (Fig 9.2)


Evolving Brain Structures

• The synergism of • (1) Developmental (the

development of a single brain from an embryo), AND

• (2) Evolutionary (evolving multiple individuals through many generations; genes) processes


Evolving visual patterns


Evolving musical patterns...

• Music of different composers has different features, e.g.: Bethoven, Mozart, Bach, Vivaldi, Heavy Metal,….

• How is an area of the brain evolved on a particular music so that similar musical patterns can be generated by an experienced listener or a professional musician?

• Can we create (evolve) systems that learn Bach and then generate Bach-like music?


Dynamic modelling of brain states based on EEG signals

• The moving electrical charges associated with a neuronal action potential emit a minute, time varying electro-magnetic field

• Historically, expert analysis via visual inspection of EEG has tended to focus on the activity in specific wavebands, such as delta, theta, alpha, and beta

• Noise contaminants in EEG are called “artefacts”. • The physical movement of the test subject can contaminate an EEG

with the noise (“artefact”) generated by the action potentials of the skeletal muscles.

• It is possible to use advanced DSP technique, such as Independent Component Analysis (ICA) to filter this and other types of noise from the brain activation signal.

• Since the electrodes are placed at fixed locations on the scalp, the subject’s movement will not violate the spatial integrity of the data.


Dynamic Modeling in Cognitive Engineering

• Analysis of the dynamics of brain states when dealing with cognitive tasks

• Feature extraction –independent component analysis

• ECOS for dynamic modeling

• Rule extraction• Scientific

visualization of the dynamic processes

• Future work: real-time systems


Dynamic modelling of brain states based on EEG signals

EEG signals recorded in eight channels from a person in a normal stateand in an eileptic one (manifested after time unit 10) (Fig 9.4)


Case Study for Dynamic Modelling of EEG data using EFuNN

• Combining adaptive on-line ICA, as a pre-processing technique, with ECOS, for on-line classification of brain states based on EEG signals

• 4 classes of brain states, 37 single trials, each of them including the following stimuli: » Auditory stimulus data;» Visual stimulus data;» Mixed Auditory-Visual stimulus data; » No stimulus presented data.

• Process:1. For each subject, stimuli were presented and raw EEG data was

recorded.2. An evolving fuzzy neural network EFuNN was trained and tested for

each stimulus. 3. ICA technique was applied on the raw EEG data and one EFuNN

was trained for each stimulus; results were evaluated and “noise”(insignificant) independent components were removed.


Evolving speech structures in the brain visualised through fMRI techniques


Evolving brains...

• There is a pre-define area of the brain that is allocated for language and visual information processing but the way they evolve may change this definition

• This is a process of evolving, unfolding, revealing, changing

Picture from Voneida & Vardaris,”The Animated Brain”


Evolving brain structures...

• Examples: Evolving musical patterns

• Left hemisphere involved in the formally trained musicians and logical processes

• Right hemisphere is activated through the listening processes

Picture from T.Voneida and R.Vardaris)


Modelling Perception - Auditory

• The human brain comprises many functional areas, each of them being a potential subject to modelling and analysis.

• Issue of developing a computer model of the hearing apparatus which transforms speech signals into brain signals.

• Model is adaptive, so new features can be included in the model and can be further tuned. Features can then be further used as inputs to phoneme or word recognition models based on ECOS.

• A Schematic diagram of a model of the auditory system of the brain (fig. 9.8)

SignalModel ofthe cochlea

AcousticModel Word

ModelLanguageModel


Modelling Perception - Auditory

• Precise modelling of hearing functions and cochlea – extremely difficult, but not impossible to achieve task. » useful for both helping people with disabilities and

for the creation of better speech recognition systems.

• The ear is the front-end auditory apparatus in mammalians. » Task is to transform the environmental sounds into

specific features and transmit them to the brain for further processing.

» The ear consists of three divisions: the outer ear, the middle ear, and the inner ear


Diagrams of ear and human basilar membrane

Outer ear, Inner ear, Middle ear (Fig 9.9)

Human basilar membrane shows approximate positions of maximaldisplacement to tones of different Frequencies.


Dynamic Modelling of Integrated Auditory and Visual Systems• Issue of integrating auditory and visual

information in one information processing model. » may lead to better information processing and

adaptation in the future intelligent systems.

• Model of multi-modal information processing in the brain.

• Deacon’s model for multi-modal information processing (Fig. 9.11)

Frontal Lobe Parietal Lobe Occipital Lobe

Broca’sarea

Primary MotorCortex

SupplementaryMotorCortex

PrefrontalCortex

Wernicke’sarea

Angulargyrus

PrimaryAuditoryCortex

CorticalVisual Area

Auditoryinput(speech)

Visualinput(reading)

Outputto oral-vocal


Architecture of a multi-modal system• Consists of two subsystems – auditory, and visual. • Biologically plausible• Auditory subsystem consists of five modules:

» Pre-processing» Elementary sound recognition» Dynamic sound recognition» Word detection (for spoken words)» Language structure detection

• Visual subsystem also consists of five modules:» Pre-processing» Elementary feature recognition» Dynamic feature recognition» Object Recognition (e.g. shapes and parts)» Object/Configuration Recognition (e.g. faces)


Computational Models of the entire brain

• The brain - most complex information processing machine. It processes data, information and knowledge at different levels.

• Four levels of information processing in the brain:1. Genetic level (gene information is processed in a cell – in a

neuron). 2. Single neuronal level (information is processed in a single

neuron). 3. Neural network level (information is processed in ensembles of

neurons that form a functionally defined area)4. Entire brain level (information is processed in the whole brain

as many interacting modules)• Modelling the entire brain far from achieved

» will take many years to achieve this goal» each step is a useful step towards understanding the brain

and towards the creation of intelligent machines.


Summary• Issues of modelling dynamic processes in the

human brain• Processes are very complex and their modelling

requires dynamic, adaptive techniques.• Is it possible to create a truly adequate model of

the human brain? • How could precise modelling of the human

hearing apparatus help to achieve progress in the area of speech recognition systems?


Further Readings• Principles of the brain development (Quartz and Sejnowski, 1997; Purves and

Lichtman, 1985; Eriksson, et al 1998; van Owen,1994; Wong,1995; Amit,1989; Arbib, 1972, 1987, 1998, 1995 and 2002; Churchland and Sejnowski, 1992; Taylor, GJ 1998; Deacon, 1988, 1998; Freeman, 2001; Grossberg, 1982; Joseph,1998; Wolpert et al,1998).

• Similarity of brain functions and neural networks (Rolls and Treves, 1998). • Computational models based on brain-imaging (Taylor J.G., 2000). • Hearing and the auditory apparatus (Harmann,1998; Glassberg and Moore,

1990; Allen, 1995).• Modelling perception – the auditory system (Wang and Jabri, 1998; Abdulla,

2001; Abdulla and Kasabov, 2002; Kuhl, 1994; Liberman,1967).• Modelling visual pattern recognition (Fukushima, 1987; Fukushima et al, 1983) • EEG signals modelling (Freeman, 1987; Freeman and Skarda, 1985; Leichter,

et al, 2001).• MRI (magnetic resonance images) processing (Hall et al, 1992)• Multi-modal functional brain models (Neisser, 1987; Deacon, 1988, 1998)• Computational brain models (Matsumoto et al, 1996; Matsumoto,2000).• Dynamic interactive models of vision and control functions (Arbib, 1998).• Learning in the hippocampus brain (McClelland, McNaughton and O’Reilly R,

1995; Durand et al, 1996; Eriksson et al, 1998; Grossberg and Merrill, 1996).• Dynamic models of the human mind (Port and van Gelder, 1995).

Documents

Chapter 9 Dynamic Modelling of Brain Functions and Cognitive … · 12/16/2002 Nik Kasabov - Evolving Connectionist Systems Chapter 9 Dynamic Modelling of Brain Functions and Cognitive