7/31/2019 2 1 Learning
1/34
Learning Process
CS/CMPE 537 Neural Networks
7/31/2019 2 1 Learning
2/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 2
Learning
Learning?
Learning is a process by which the free parameters of a
neural network are adapted through a continuing process
of stimulation by the environment in which the network
is embedded
The type of learning is determined by the manner in
which the parameter changes take place
Types of learning
Error-correction, memory-based, Hebbian, competitive,
Boltzmann
Supervised, reinforced, unsupervised
7/31/2019 2 1 Learning
3/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 3
Learning Process
Adapting the synaptic weight
wkj(n + 1) = wkj(n) + wkj(n)
7/31/2019 2 1 Learning
4/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 4
Learning Algorithms
Learning algorithm: a prescribed set of well-defined rules for
the solution of a learning problem In the context of synaptic weight updating, the learning algorithm
prescribes rules for w
Learning rules Error-correction
Memory based
Boltzmann
Hebbian
Competitive
Learning paradigms Supervised
Reinforced
Self-organizing (unsupervised)
7/31/2019 2 1 Learning
5/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 5
Error-Correction Learning (1)
ek(n) = dk(n) yk(n)
The goal of error-correction learning is to minimize a
cost function based on the error function Least-mean-square error as cost function
J = E[0.5kek2(n)]
E = expectation operatorMinimizing J with respect to the network parameters is the
method of gradient descent
7/31/2019 2 1 Learning
6/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 6
Error-Correction Learning (2)
How do we find the expectation of the process?
We avoid its computation, and use an instantaneousvalue of the sum of squared errors as the error function(as an approximation)
(n) = 0.5kek2(n)
Error correction learning rule (or delta rule)
wkj(n) = ek(n)xj(n)
= learning rate
A plot of error function and weights is called an errorsurface. The minimization process tries to find theminimum point on the surface through an iterative
procedure.
7/31/2019 2 1 Learning
7/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 7
Memory-based Learning (1)
All (or most) of the past experiences are stored
explicitly in memory of correctly classified input-outputexamples: {(xi, di)}i = 1, N
Given a test vectorxtest , the algorithm retrieves the
classification of the xi closest to xtest in the trainingexamples (and memory)
Ingredients Definition of what is closest or local neighborhood
Learning rule applied to the training examples in the local
neigborhood
7/31/2019 2 1 Learning
8/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 8
Memory-based Learning (2)
Nearest neighbor rule
K-nearest neighbor rule
Radial-basis function rule (network)
7/31/2019 2 1 Learning
9/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 9
Hebbian Learning (1)
Hebb, a neuropsychologist, proposed a model of neural
activation in 1949. Its idealization is used as a learningrule in neural network learning.
Hebbs postulate (1949) If the axon of cell A is near enough to excite cell B and
repeatedly or perseistently takes part in firing it, some growth
process or metabolic change occurs in one or both cells such
that As efficiency as one of the cells firing B is increased.
7/31/2019 2 1 Learning
10/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 10
Hebbian Learning (2)
Hebbian learning (model of Hebbian synapse)
1. If two neurons on either side of a synapse are activatedsimultaneously, then the strength of that synapse is
selectively increased
2. If two neurons on either side of synapse are activated
asynchronously, then that synapse is selectively weakened oreliminated
Properties of Hebbian synapse Time-dependent mechanism
Local mechanism
Interactive mechanism
Correlational mechanism
7/31/2019 2 1 Learning
11/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 11
Mathematical Models of Hebbian Learning (1)
General form of Hebbian rule
wkj(n) = F[yk(n), xj(n)]
F is a function of pre-synaptic and post-synaptic
activities.
A specific Hebbian rule (activity product rule)wkj(n) = yk(n)xj(n)
= learning rate
Is there a problem with the above rule?No bounds on increase (or decrease) of wkj
7/31/2019 2 1 Learning
12/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 12
Mathematical Models of Hebbian Learning (2)
Generalized activity product rule
wkj(n) = yk(n)xj(n) yk(n)wkj(n)
Or
wkj(n) = yk(n)[cxk(n) - wkj(n)]
where c = / and = positive constant
7/31/2019 2 1 Learning
13/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 13
Mathematical Models of Hebbian Learning (3)
7/31/2019 2 1 Learning
14/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 14
Mathematical Models of Hebbian Learning (4)
Activity covariance rule
wkj(n) = cov[yk(n), xj(n)]
= E[(yk(n) y)(xj(n) x)]
where = proportionality constant and x and y are
respective means
After simplification
wkj(n) = {E[yk(n)xj(n)] xy}
7/31/2019 2 1 Learning
15/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 15
Competitive Learning (1)
The output neurons of a neural network (or a group of
output neurons) compete among themselves for beingthe one to be active (fired) At any given time, only one neuron in the group is active
This behavior naturally leads to identifying features in input
data (feature detection)
Neurobiological basis Competitive behavior was observed and studied in the 1970s
Early self-organizing and topographic map neuralnetworks were also proposed in the 1970s (e.g.
cognitron by Fukushima)
7/31/2019 2 1 Learning
16/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 16
Competitive Learning (2)
Elements of competitive learning
A set of neurons A limit on the strength of each neuron
A mechanism that permits the neurons to compete for the right
to respond to a given input, such that only one neuron is active
at a time
7/31/2019 2 1 Learning
17/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 17
Competitive Learning (3)
7/31/2019 2 1 Learning
18/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 18
Competitive Learning (4)
Standard competitive learning rule
wji = (xi wji) if neuron j wins the competition
0 otherwise
Each neuron is allotted a fixed amount of synaptic
weight which is distributed among its input nodesi wji = 1 for all j
7/31/2019 2 1 Learning
19/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 19
Competitive Learning (5)
7/31/2019 2 1 Learning
20/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 20
Boltzmann Learning
Stochastic learning algorithm based on information-
theoretic and thermodynamic principles The state of the network is captured by an energy
function, E
E = -1/2 kj wkjsiskwhere sj = state of neuron j [0, 1] (i.e. binary state)
Learning process
At each step, choose a neuron at random (say k) and flip itsstate sk(to - sk) by the following probability
w(sk-> -sk) = (1 + exp(-Ek/T)]-1
The state evolves until thermal equilibrium is achieved
C
7/31/2019 2 1 Learning
21/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 21
Credit-Assignment Problem
How to assign credit and blame for a neural networks
output to its internal (free) parameters ? This is basically the credit-assignment problem
The learning system (rule) must distribute credit or blame in
such a way that the network evolves to the correct outcomes
Temporal credit-assignment problem Determining which actions, among a sequence of actions, are
responsible for certain outcomes of the network
Structural credit-assignment problem Determining which internal components behavior should be
modified and by how much
S i d L i (1)
7/31/2019 2 1 Learning
22/34CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 22
Supervised Learning (1)
S i d L i (2)
7/31/2019 2 1 Learning
23/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 23
Supervised Learning (2)
Conceptually, supervised learning involves a teacher
who has knowledge of the environment and guides thetraining of the network
In practice, knowledge of the environment is in the form
of input-output examplesWhen viewed as a intelligent agent, this knowledge is current
knowledge obtained from sensors
How is supervised learning applied?
Error-correction learning Examples of supervised learning algorithms
LMS algorithm
Back-propagation algorithm
R i f t L i (1)
7/31/2019 2 1 Learning
24/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 24
Reinforcement Learning (1)
Reinforcement learing is supervised learning in which
limited information of the desired outputs is known Complete knowledge of the environment is not available; only
basic benefit or reward information
In other words, a critic rather than a teacher guides the
learning process Reinforcement learning has roots in experimental
studies of animal learning Training a dog by positive (good dog, something to eat) and
negative (bad dog, nothing to eat) reinforcement
R i f t L i (2)
7/31/2019 2 1 Learning
25/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 25
Reinforcement Learning (2)
Reinforcement learning is the online learning of an
input-output mapping through a process of trail anderror designed to maximize a scalar performance index
called reinforcement signal
Types of reinforcement learningNon-associative: selecting one action instead of associating
actions with stimuli. The only input received from the
environment is reinforcement information. Examples include
genetic algorithms and simulated annealing.
Associative: associating action and stimuli. In other words,
developing a action-stimuli mapping from reinforcement
information received from the environment. This type is more
closely related to neural network learning.
S i d V R i f t L i
7/31/2019 2 1 Learning
26/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 26
Supervised Vs Reinforcement Learning
Supervised learning Reinforcement learning
Teacher detailed informationavailable
Critic only reward informationavailable
Instructive feedback system Evaluative feedback system
Instantaneous and localinformation
Delayed and general information
Directed information howsystem should adapt Undirected info system has toprobe with trial and error
Faster training Slower training
U i d L i (1)
7/31/2019 2 1 Learning
27/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 27
Unsupervised Learning (1)
There is no teacher or critic in unsupervised learningNo specific example of the function/model to be learned
A task-independent measure is used to guide the internal
representation of knowledge The free parameters of the network are optimized with respect
to this measure
U i d L i (2)
7/31/2019 2 1 Learning
28/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 28
Unsupervised Learning (2)
Also known as self-organizing when used in the context
of neural networks The neural network develops an internal representation of the
inputs without any specific information
Once it is trained it can identify features in the input, based on
the task-independent (or general) criterion
S i d V U i d L i
7/31/2019 2 1 Learning
29/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 29
Supervised Vs Unsupervised Learning
Supervised learning Unsupervised learning
Teacher detailed informationavailable
No specific information available
Instructive feedback system Task-independent feedback system
Poor scalability Better scalability
L i T k
7/31/2019 2 1 Learning
30/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 30
Learning Tasks
Pattern association
Pattern recognition Function approximation
Control
Filtering Beamforming
Ad t ti d L i (1)
7/31/2019 2 1 Learning
31/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 31
Adaptation and Learning (1)
Learning, as we know it in biological systems, is a
spatiotemporal process Space and time dimensions are equally significant
Is supervised error-correcting learning spatiotemporal? Yes and no (trick question )
Stationary environment Learning one time procedure in which environment
knowledge is built-in (memory) and later recalled for use Non-stationary environment
Adaptation continually update the free parameters to reflect
the changing environment
Adaptation and Learning (2)
7/31/2019 2 1 Learning
32/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 32
Adaptation and Learning (2)
Adaptation and Learning (3)
7/31/2019 2 1 Learning
33/34
CS/CMPE 537 - Neural Networks (Sp 2006-2007) - Asim Karim @ LUMS 33
Adaptation and Learning (3)
e(n) = x(n) - x(n)
where e = error; x = actual input; x = model output
Adaptation needed when e not equal to zero
This means that the knowledge encoded in the neural networkhas become outdated requiring modification to reflect the new
environment
How to perform adaptation?
As an adaptive control system As an adaptive filter (adaptive error-correcting supervised
learning)
Statistical Nature of Learning
7/31/2019 2 1 Learning
34/34
Statistical Nature of Learning
Learning can be viewed as a stochastic process
Stochastic process? when there is some element ofrandomness (e.g. neural network encoding is not unique
for the same environment that is temporal) Also, in general, neural network represent just one form of
representation. Other representation forms are also possible.
Regression model
d = g(x) +
where g(x) = actual model; = statistical estimate of error