2 1 Learning

7/31/2019 2 1 Learning

1/34

Learning Process

CS/CMPE 537 Neural Networks

7/31/2019 2 1 Learning

2/34

CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 2

Learning

Learning?

Learning is a process by which the free parameters of a

neural network are adapted through a continuing process

of stimulation by the environment in which the network

is embedded

The type of learning is determined by the manner in

which the parameter changes take place

Types of learning

Error-correction, memory-based, Hebbian, competitive,

Boltzmann

Supervised, reinforced, unsupervised

7/31/2019 2 1 Learning

3/34


Learning Process

Adapting the synaptic weight

wkj(n + 1) = wkj(n) + wkj(n)

7/31/2019 2 1 Learning

4/34


Learning Algorithms

Learning algorithm: a prescribed set of well-defined rules for

the solution of a learning problem In the context of synaptic weight updating, the learning algorithm

prescribes rules for w

Learning rules Error-correction

Memory based

Boltzmann

Hebbian

Competitive

Learning paradigms Supervised

Reinforced

Self-organizing (unsupervised)

7/31/2019 2 1 Learning

5/34


Error-Correction Learning (1)

ek(n) = dk(n) yk(n)

The goal of error-correction learning is to minimize a

cost function based on the error function Least-mean-square error as cost function

J = E[0.5kek2(n)]

E = expectation operatorMinimizing J with respect to the network parameters is the

method of gradient descent

7/31/2019 2 1 Learning

6/34


Error-Correction Learning (2)

How do we find the expectation of the process?

We avoid its computation, and use an instantaneousvalue of the sum of squared errors as the error function(as an approximation)

(n) = 0.5kek2(n)

Error correction learning rule (or delta rule)

wkj(n) = ek(n)xj(n)

= learning rate

A plot of error function and weights is called an errorsurface. The minimization process tries to find theminimum point on the surface through an iterative

procedure.

7/31/2019 2 1 Learning

7/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 7

Memory-based Learning (1)

All (or most) of the past experiences are stored

explicitly in memory of correctly classified input-outputexamples: {(xi, di)}i = 1, N

Given a test vectorxtest , the algorithm retrieves the

classification of the xi closest to xtest in the trainingexamples (and memory)

Ingredients Definition of what is closest or local neighborhood

Learning rule applied to the training examples in the local

neigborhood

7/31/2019 2 1 Learning


Memory-based Learning (2)

Nearest neighbor rule

K-nearest neighbor rule

Radial-basis function rule (network)

7/31/2019 2 1 Learning


Hebbian Learning (1)

Hebb, a neuropsychologist, proposed a model of neural

activation in 1949. Its idealization is used as a learningrule in neural network learning.

Hebbs postulate (1949) If the axon of cell A is near enough to excite cell B and

repeatedly or perseistently takes part in firing it, some growth

process or metabolic change occurs in one or both cells such

that As efficiency as one of the cells firing B is increased.

7/31/2019 2 1 Learning


Hebbian Learning (2)

Hebbian learning (model of Hebbian synapse)

1. If two neurons on either side of a synapse are activatedsimultaneously, then the strength of that synapse is

selectively increased

2. If two neurons on either side of synapse are activated

asynchronously, then that synapse is selectively weakened oreliminated

Properties of Hebbian synapse Time-dependent mechanism

Local mechanism

Interactive mechanism

Correlational mechanism

7/31/2019 2 1 Learning


Mathematical Models of Hebbian Learning (1)

General form of Hebbian rule

wkj(n) = F[yk(n), xj(n)]

F is a function of pre-synaptic and post-synaptic

activities.

A specific Hebbian rule (activity product rule)wkj(n) = yk(n)xj(n)

= learning rate

Is there a problem with the above rule?No bounds on increase (or decrease) of wkj

7/31/2019 2 1 Learning



Generalized activity product rule

wkj(n) = yk(n)xj(n) yk(n)wkj(n)

Or

wkj(n) = yk(n)[cxk(n) - wkj(n)]

where c = / and = positive constant

7/31/2019 2 1 Learning



7/31/2019 2 1 Learning



Activity covariance rule

wkj(n) = cov[yk(n), xj(n)]

= E[(yk(n) y)(xj(n) x)]

where = proportionality constant and x and y are

respective means

After simplification

wkj(n) = {E[yk(n)xj(n)] xy}

7/31/2019 2 1 Learning


Competitive Learning (1)

The output neurons of a neural network (or a group of

output neurons) compete among themselves for beingthe one to be active (fired) At any given time, only one neuron in the group is active

This behavior naturally leads to identifying features in input

data (feature detection)

Neurobiological basis Competitive behavior was observed and studied in the 1970s

Early self-organizing and topographic map neuralnetworks were also proposed in the 1970s (e.g.

cognitron by Fukushima)

7/31/2019 2 1 Learning



Elements of competitive learning

A set of neurons A limit on the strength of each neuron

A mechanism that permits the neurons to compete for the right

to respond to a given input, such that only one neuron is active

at a time

7/31/2019 2 1 Learning



7/31/2019 2 1 Learning



Standard competitive learning rule

wji = (xi wji) if neuron j wins the competition

0 otherwise

Each neuron is allotted a fixed amount of synaptic

weight which is distributed among its input nodesi wji = 1 for all j

7/31/2019 2 1 Learning



7/31/2019 2 1 Learning


Boltzmann Learning

Stochastic learning algorithm based on information-

theoretic and thermodynamic principles The state of the network is captured by an energy

function, E

E = -1/2 kj wkjsiskwhere sj = state of neuron j [0, 1] (i.e. binary state)

Learning process

At each step, choose a neuron at random (say k) and flip itsstate sk(to - sk) by the following probability

w(sk-> -sk) = (1 + exp(-Ek/T)]-1

The state evolves until thermal equilibrium is achieved

C

7/31/2019 2 1 Learning


Credit-Assignment Problem

How to assign credit and blame for a neural networks

output to its internal (free) parameters ? This is basically the credit-assignment problem

The learning system (rule) must distribute credit or blame in

such a way that the network evolves to the correct outcomes

Temporal credit-assignment problem Determining which actions, among a sequence of actions, are

responsible for certain outcomes of the network

Structural credit-assignment problem Determining which internal components behavior should be

modified and by how much

S i d L i (1)

7/31/2019 2 1 Learning


Supervised Learning (1)

S i d L i (2)

7/31/2019 2 1 Learning

23/34


Supervised Learning (2)

Conceptually, supervised learning involves a teacher

who has knowledge of the environment and guides thetraining of the network

In practice, knowledge of the environment is in the form

of input-output examplesWhen viewed as a intelligent agent, this knowledge is current

knowledge obtained from sensors

How is supervised learning applied?

Error-correction learning Examples of supervised learning algorithms

LMS algorithm

Back-propagation algorithm

R i f t L i (1)

7/31/2019 2 1 Learning

24/34


Reinforcement Learning (1)

Reinforcement learing is supervised learning in which

limited information of the desired outputs is known Complete knowledge of the environment is not available; only

basic benefit or reward information

In other words, a critic rather than a teacher guides the

learning process Reinforcement learning has roots in experimental

studies of animal learning Training a dog by positive (good dog, something to eat) and

negative (bad dog, nothing to eat) reinforcement

R i f t L i (2)

7/31/2019 2 1 Learning

25/34


Reinforcement Learning (2)

Reinforcement learning is the online learning of an

input-output mapping through a process of trail anderror designed to maximize a scalar performance index

called reinforcement signal

Types of reinforcement learningNon-associative: selecting one action instead of associating

actions with stimuli. The only input received from the

environment is reinforcement information. Examples include

genetic algorithms and simulated annealing.

Associative: associating action and stimuli. In other words,

developing a action-stimuli mapping from reinforcement

information received from the environment. This type is more

closely related to neural network learning.

S i d V R i f t L i

7/31/2019 2 1 Learning

26/34


Supervised Vs Reinforcement Learning

Supervised learning Reinforcement learning

Teacher detailed informationavailable

Critic only reward informationavailable

Instructive feedback system Evaluative feedback system

Instantaneous and localinformation

Delayed and general information

Directed information howsystem should adapt Undirected info system has toprobe with trial and error

Faster training Slower training

U i d L i (1)

7/31/2019 2 1 Learning

27/34


Unsupervised Learning (1)

There is no teacher or critic in unsupervised learningNo specific example of the function/model to be learned

A task-independent measure is used to guide the internal

representation of knowledge The free parameters of the network are optimized with respect

to this measure

U i d L i (2)

7/31/2019 2 1 Learning

28/34


Unsupervised Learning (2)

Also known as self-organizing when used in the context

of neural networks The neural network develops an internal representation of the

inputs without any specific information

Once it is trained it can identify features in the input, based on

the task-independent (or general) criterion

S i d V U i d L i

7/31/2019 2 1 Learning

29/34


Supervised Vs Unsupervised Learning

Supervised learning Unsupervised learning

Teacher detailed informationavailable

No specific information available

Instructive feedback system Task-independent feedback system

Poor scalability Better scalability

L i T k

7/31/2019 2 1 Learning

30/34


Learning Tasks

Pattern association

Pattern recognition Function approximation

Control

Filtering Beamforming

Ad t ti d L i (1)

7/31/2019 2 1 Learning

31/34


Adaptation and Learning (1)

Learning, as we know it in biological systems, is a

spatiotemporal process Space and time dimensions are equally significant

Is supervised error-correcting learning spatiotemporal? Yes and no (trick question )

Stationary environment Learning one time procedure in which environment

knowledge is built-in (memory) and later recalled for use Non-stationary environment

Adaptation continually update the free parameters to reflect

the changing environment


7/31/2019 2 1 Learning

32/34




7/31/2019 2 1 Learning

33/34



e(n) = x(n) - x(n)

where e = error; x = actual input; x = model output

Adaptation needed when e not equal to zero

This means that the knowledge encoded in the neural networkhas become outdated requiring modification to reflect the new

environment

How to perform adaptation?

As an adaptive control system As an adaptive filter (adaptive error-correcting supervised

learning)

Statistical Nature of Learning

7/31/2019 2 1 Learning

34/34

Statistical Nature of Learning

Learning can be viewed as a stochastic process

Stochastic process? when there is some element ofrandomness (e.g. neural network encoding is not unique

for the same environment that is temporal) Also, in general, neural network represent just one form of

representation. Other representation forms are also possible.

Regression model

d = g(x) +

where g(x) = actual model; = statistical estimate of error

Documents

2 1 Learning