Upload
rockeysuseelan
View
233
Download
0
Embed Size (px)
Citation preview
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 1/31
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 2/31
` Definition
` Neural Model
` Activation Function
`
Algorithm` Stage one conversion
` Stage two conversion
` Binary Neural implementation
`
Hardware Optimization` References
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 3/31
` Neural network is a sorted triple (N, V, w) with two sets N, V
and a function w
N is the set of neurons
V a sorted set
` {(i, j)|i, j E N} whose elements are called connections between
neuron i and neuron j.
` w : V R defines the weights
w((i, j)), the weight of the connection between neuron i and neuron j.
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 4/31
` Learning capa bility
` Predict the outcome of past trends.
`
Ro bust and Fault tolerant` Process information in parallel at high speed and in a
distri buted manner.
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 5/31
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 6/31
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 7/31
` Identity functionx F(x)=x for all x
` Binary Step function
x F(x)= 1 if x>=
= 0 if x<
` Sigmoid function
x F(x)=
` Bipolar sigmoid function
F(x)=
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 8/31
` Steps
Digitization
Conversion of digitized model into logic gate structure
Hardware optimization by elimination of redundant logic
gates
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 9/31
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 10/31
` Digitization of one Neuron Mathematical Model
` Real values between -1 and +1 can be represented by
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 11/31
` Functionality of the neuron should not be affected
while transforming analogue neuron into an
appropriate digital model.
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 12/31
` Conversion is achieved by transforming the analog input into
digital inputs
` Each analogue neuron input is transformed to its equivalent
group of n b binary inputs.
` Each input defined by initial weight wij into n b su binputs,
whose weights wijp ( p=0,1,« n b-1) is calculated as
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 13/31
` Argument corresponding to the neuron after the first conversion
stage is calculated as
= constant
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 14/31
` After 1st conversion neurons can have both positive and
negative weights.
`
Stage two aims to replace these neurons with equivalent oneshaving only positive weights.
` ie
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 15/31
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 16/31
` R elati s i bet ee t e i t bits s lie t sta e 2 e r s
a sta e e r s is i e b
` T ese t alter ati es ca be c resse i t
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 17/31
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 18/31
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 19/31
` Ar e ts f t e acti ati f cti bef re a after sta e 2
c ersi s l be e al
` ie
` T eref re
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 20/31
` Su bstituting the value of wijp
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 21/31
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 22/31
` Therefore threshold level of the stage two neurons is
` The neuron parameter after stage two can be calculated as a
function of initial analog neuron parameter
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 23/31
` Iterative implementation procedure uses 3 input parameters:
Index defining the current terminal group(F)
The current threshold level (T)
The logic gate type(LGT)
x ANY_GATE
x
AND_GATE
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 24/31
` At first step F=1,T=t(2) , LGT=ANY_GATE
1. If LGT=AND_GATE then go to 7., else 2.
2. Calculate the num ber of X input weights and determine num ber Y
of the cumulative weights which are >T.
If X>1 and Y=0 then go to3.
If X>1 and Y>0 then go to 4
If X=1 and Y=0 then go to 5
If X=0 and Y=1 then go to 7.
If X=0 and Y=0 then go to 6.
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 25/31
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 26/31
` The hardware implementation netlist o btained is repeatedly
analyzed and redundant logic gates with the same input
signals and are of the same type are removed.
` This optimization ends when no more gates can be removed.
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 27/31
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 28/31
[1] A. Dinu and M. N. Cirstea, ³A digital neural network FPGA direct
hardware implementation algorithm,´ in Proc. ISIE, Vigo, Spain, pp. 2307±
2312.
[2]Andrei Dinu, Marcian N. Cirstea ³Direct Nueral Network Hardware
Implementation Algorithm´ IEEE Transactions on Industrial Electronics
Vol 57,No 5, May 2010
[3] Martin T. Howard B Demuth and Mark Beale: Neural Network Design,
Vikas Thomas Learning
[4] Simon Haykin: Neural Networks- A Comprehensive Foundation, Pearson
Education
[5] Maurico A.Leon, James Keller ³Toward Implementation of Artificial
Neural Networks That "Really Work³´, Department of Computer
engineering and Computer Science,University of Missouri Colum bia.
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 29/31
` The novel algorithm treats each neuron is treated as a Boolean
functions with properties that can be ex ploited to achieve
compact implementation.
` Most efficient for low num ber of inputs on each input.
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 30/31
[6] Y. Singh, A. S. Chauhan, "Neural Networks in Data Mining," J ournal of
Theoretical and Applied Information Technology, vol 5. No 6, pp.37²42,
J une 2009
[7] C. M. Bishop, "Pattern Recognition and Machine Learning," New York:
Springer, 2006. ² 703 p.
[8] V. Ganapathy, K. L. Liew, "Handwritten Character Recognition Using
Multiscale Neural Network Training Technique," World Academy of Science,
Engineering and Technology, No 39, pp. 32²37, 2008.
[9] G. P. Zhang, "Neural Networks for Classification: A Survey," IEEE Trans.
on Syst., Man and Cybern, vol. 30. No 4, pp. 451²462, Nov. 2000.
[10] K. R. Farell, R. J. Mommone, K. T. Assaleh, "Speaker Recognition Using
Neural Networks and Conventional Classifiers," IEEE Transactions on
Speech and Audio Processing, vo2. 3. No 1, pp.194²205, 1994.
8/6/2019 Neural Network Algorithm 2
http://slidepdf.com/reader/full/neural-network-algorithm-2 31/31