Embed Size (px)
DESCRIPTION
Neural Networks. MLP for System Modeling. f (.). f (.). f (.). Back Propagation Learning Algorithm. Forward propagation. Set the weights Calculate output. Backward propagation. Calculate error Calculate gradient vector Update the weights. Neural Networks. MLP for System Modeling. - PowerPoint PPT Presentation
Citation preview
Dr.-Ing. Erwin SitompulPresident University
Lecture 5
Introduction to Neural Networksand Fuzzy Logic
President University Erwin Sitompul NNFL 5/1
http://zitompul.wordpress.com
President University Erwin Sitompul NNFL 5/2
Back Propagation Learning AlgorithmMLP for System ModelingNeural Networks
Backwardpropagation
f(.)
f(.)
f(.)
•Set the weights•Calculate output
1
1
( )( ) ( ) ( )
pl l lk k jl
ikj
Ei f net i y i
w
w
1 21
1
( )( ) ( ) ( ) ( )
pl l l l lk k kj j il
iji
Ei f net i w f net i y i
w
w
Forwardpropagation
1( ), ( )l lj ky i y i
•Calculate error•Calculate gradient
vector
•Update the weights
1
1
( ) ( )
( ) ( )
l lk k
ml l lk kj j
j
y i f net i
net i w x i
1 1
1 1 2
1
( ) ( )
( ) ( )
l lj j
nl l lj ji i
i
y i f net i
net i w y i
1
( ) ( ),
l lkj ji
E E
w w
w w
( )lk i
President University Erwin Sitompul NNFL 5/3
Feedforward Network
InputNeuronLayer
NeuronLayer
Output
f(.)
f(.)
f(.)
MLP for System ModelingNeural Networks
President University Erwin Sitompul NNFL 5/4
Feedforward Network
01 2 1 0 2 0y
02 3 1 3 1 0y
21 17 3 9 9 0d
01 ( )y i
02 ( )y i
21 ( )y i
MLP for System ModelingNeural Networks
President University Erwin Sitompul NNFL 5/5
Recurrent NetworksExternal Recurrence
Internal Recurrence
Input NeuronLayer
NeuronLayer
Output
Time Delay
Element
Time Delay
Element
Input NeuronLayer
NeuronLayer
Output
Time Delay
Element
MLP for System ModelingNeural Networks
President University Erwin Sitompul NNFL 5/6
InputDynamicSystem
Output
( )u k ( )y k
Dynamic System
( ) ( , )y k m g
a b( 1), , ( ), ( 1), , ( )y k y k n u k u k n g
System parameter
Input-output data vector
MLP for System ModelingNeural Networks
President University Erwin Sitompul NNFL 5/7
InputDynamic
Model
Output
( )u k ˆ( )y k
Dynamic Model
ˆ( ) ( , , )y k w b g
a b( 1), , ( ), ( 1), , ( )y k y k n u k u k n g
weightsbias
input-output data vector
MLP for System ModelingNeural Networks
President University Erwin Sitompul NNFL 5/8
Neural Network Dynamic Model
Feedforward
ˆ( )y k : model output,estimate of system output
( )y k : system output. . .
. . .
. . .
ˆ( )y k
. . .
( 1)u k
b( )u k n
( 1)y k
a( )y k n
MLP for System ModelingNeural Networks
President University Erwin Sitompul NNFL 5/9
Neural Network Dynamic Model
Recurrent
. . .
. . .
. . .
ˆ( )y k
. . .
( )u k
1z
anz
1z
bnz
MLP for System ModelingNeural Networks
President University Erwin Sitompul NNFL 5/10
Tapped Delay Line (TDL)
( )u k
( 1)u k ( 2)u k
( 3)u k ( )u k n
1z 1z 1z 1z .....
( )u k
( 1)u k ( )u k n
T D L
.....
MLP for System ModelingNeural Networks
Unit 1 Unit 2 Unit 3 Unit n
President University Erwin Sitompul NNFL 5/11
Implementation
InputDynamicSystem
Output
( )u k ( )y k
ˆ( )y k. . .
. . .
T D L T D L
feedforward
external recurrence
MLP for System ModelingNeural Networks
President University Erwin Sitompul NNFL 5/12
ExampleSingle Tank System
2
20.4 m0.012 m
Aa
outq
inq
hLearning Data Generation
A : cross-sectional area of the tanka : cross-sectional area of the pipe
Area of operation
Save data to workspace
MLP for System ModelingNeural Networks
in a
1 ah q v
A A
in
12
ah q gh
A A
President University Erwin Sitompul NNFL 5/13
Example
( 1)u k ( )y k
( 1)y k
Data size : 201 from 200 seconds of
simulation
0 20 40 60 80 100 120 140 160 180 2000
0.02
0.04
0.06
0.08
0.1
0.12
0 20 40 60 80 100 120 140 160 180 2000
0.02
0.04
0.06
0.08
0.1
0.12
Feedforward Network External Recurrent Network
MLP for System ModelingNeural Networks
President University Erwin Sitompul NNFL 5/14
Homework 4
( 1)u k
y k( 2)u k
( 1)y k
( 2)y k
0 20 40 60 80 100 120 140 160 180 200-0.025
-0.02
-0.015
-0.01
-0.005
0
0.005
0.01
0.015
0.02
Delta of 2–2–1 network
4–4–1 Network
MLP for System ModelingNeural Networks
A neural network with 2 inputs and 2 hidden neurons seems not to be good enough to model the Single Tank System. Now, design a neural network with 4 inputs and 4 hidden neurons to model the system. Use bias in all neurons and take all a = 1.
Be sure to obtain decreasing errors.
Submit the hardcopy and softcopy of the m-file.
