Upload
moses-greene
View
222
Download
7
Embed Size (px)
Citation preview
2
Hopfield network (HN) model• A Hopfield network is a
form of recurrent artificial neural network invented by John Hopfield in 1982
• The network is entirely interconnected – All neurons are both
input and output• with binary threshold
units1, if x > 0
F(x) = -1, otherwise
These are single layered recurrent networks
3
Hopfield network modelApplications
• Recalling or Reconstructing corrupted patterns
• Large-scale computational intelligence systems
• Handwriting Recognition Software• Hopfield is model used to solve
optimization problems such as traveler salesman, task scheduling, etc.
• Practical applications of HNs are limited because number of training patterns can be at most about 14% the number of nodes in the network.
• If the network is overloaded -- trained with more than the maximum acceptable number of attractors -- then it won't converge to clearly defined attractors.
These are single layered recurrent networks
4
Hopfield network model• Hopfield nets serve as content-
addressable memory systems with binary threshold nodes (associative memory)– The information is distributed and stored in the
network connection.• If one element disappear, the information is not lost• We access to information using memory content (CAM
: Content addressable memory)
• The network build progressively its attractors– In the learning phase neurones adjust there
synapses based on there activities and predefined rules
– Hebb rule– Widrow-Hoff rule (delta rule)
6
Hopfield network model
Learning (Hebb rule): Neurone are symetric:
|P|: number of training patterns (input combinations)
is ith bit in the pattern P
0
1
ii
Pp
pj
pijiij
W
ssP
WW
pis
7
• Lets train this network for following patterns• Pattern 1:- ie Oa(1)=-1,Ob(1)=-1,Oc(1)=1
• Pattern 2:- ie Oa(2)=1,Ob(2)=-1,Oc(3)=-1
• Pattern 3:- ie Oa(3)=-1,Ob(3)=1,Oc(3)=1
w1,1 = 0w1,2 = 1/3( OA(1) × OB(1) + OA(2) × OB(2) + OA(3) × OB(3)) =1/3( (-1) × (-1) + 1 × (-1) + (-1) × 1) = 1/3w1,3 = 1/3( OA(1) × OC(1) + OA(2) × OC(2) + OA(3) × OC(3)) = 1/3((-1) × 1 + 1 × (-1) + (-1) × 1) = 1/3( -3)w2,2 = 0w2,1 = 1/3( OB(1) × OA(1) + OB(2) × OA(2) + OB(3) × OA(3)) = 1/3((-1) × (-1) + (-1) × 1 + 1 × (-1) = 1/3( -1)w2,3 = 1/3( OB(1) × OC(1) + OB(2) × OC(2) + OB(3) × OC(3)) = 1/3((-1) × 1 + (-1) × (-1) + 1 × 1) = 1/3(1)w3,3 = 0w3,1 = 1/3( OC(1) × OA(1) + OC(2) × OA(2) + OC(3) × OA(3)) = 1/3( 1 × (-1) + (-1) × 1 + 1 × (-1)) = 1/3( -3)w3,2 = 1/3( OC(1) × OB(1) + OC(2) × OB(2) + OC(3) × OB(3)) = 1/3( 1 × (-1) + (-1) × (-1) + 1 × 1) = 1/3( 1)
Hopfield network model
8
Hopfield network model
Utilization :– A pattern is entered in the network by
setting all nodes (neurones) to a specific value, or by setting only part of the nodes.
– The network is then subject to a number of iterations using asynchronous or synchronous updating.
– This is stopped after a while. The network neurons are then read out to see which pattern is in the network.
9
Hopfield network model
There are two ways of updating them:• Asynchronous: one picks one neuron,
calculates the weighted input sum and updates immediately. This can be done in a fixed order, or neurons can be picked at random, which is called asynchronous random updating.
• Synchronous: the weighted input sums of all neurons are calculated without updating the neurons. Then all neurons are set to their new value, according to the value of their weighted input sum.
10
Hopfield network model
Neurone state modification:
Nj
ti
ti
tjij
Nj
ti
tjij
Nj
ti
tjij
sssW
ssW
ssW
11
1
1
then 0. if
1 then 0. if
1 then 0. if
11
Hopfield network model
Network energy : a stable states have a low energy
This value is called the "energy" because the definition ensures that when units are randomly chosen to update, the energy E will either lower in value or stay the same.
Ni
iiNji
ijij sThresholdssWE *..2
1
,
12
Hopfield network modelShortcomings of HNs
• Training patterns can be at most about 14% the number of nodes in the network.
• If more patterns are used then the stored patterns become unstable; spurious stable states appear (i.e., stable states
which do not correspond with stored patterns).• Can sometimes misinterpret the corrupted
pattern.
13
Learning HNs through example• Moving onto little more complex problem described in Haykin’s
Neural Network Book• They book used N=120 neuron and trained network with 120
pixel images where each pixel was represented by one neuron.• Following 8 patterns were used to train neural network.
Hopfield network model