Upload
lythu
View
220
Download
0
Embed Size (px)
Citation preview
19. 07. 2002. NATO ASI LTP Leuven, Belgium 1
Neural Networks in Measurement Systems
Gábor Horváth
Department of Measurement and Information Systems
19. 07. 2002. NATO ASI LTP Leuven, Belgium 2
OutlineOutlineMeasurement systemsSensors, transducers, signal processingModelingThe role of neural networksThe importance of high-speed, low cost implementationCMAC: an alternative to MLPModeling capabilityGeneralization capability, analytical results CMAC with improved generalization capabilityGeneral cases, SVM versus CMAC
19. 07. 2002. NATO ASI LTP Leuven, Belgium 3
MeasurementMeasurementCan be defined in relation to modelingMeasurement is an autonomous phase of modeling; it is embedded in modeling
19. 07. 2002. NATO ASI LTP Leuven, Belgium 4
MMeasurementeasurementEmpirical process
to obtain experimental data (observations), primary information collection, orto obtain additional information to the a priori one.
to use the experimental data for obtaining (determining) the free parameters (features) of a model. to validate the model
19. 07. 2002. NATO ASI LTP Leuven, Belgium 5
MeasurementMeasurementThe goal of modelingThe goal of modelingThe goal of modeling
Collecting a priori knowledgeCollecting a priori knowledge
A priori modelA priori model
Experiment designExperiment design
Observations, determiningfeatures, parameters
Observations, determiningfeatures, parameters
Model validationModel validation
Final modelFinal model
CorrectionCorrection
Measurement
Modeling
19. 07. 2002. NATO ASI LTP Leuven, Belgium 6
MMeasurementeasurementTo obtain experimental data (observations),
sensing (primary information collection)To process information
signal conversionsignal processing
19. 07. 2002. NATO ASI LTP Leuven, Belgium 7
MMeasurementeasurement
Object formeasurement
Signal transmission
channel
Measuring device
input
output noise
noise
observationmeasurementresult
prior information
19. 07. 2002. NATO ASI LTP Leuven, Belgium 8
MeasurementMeasurementSpecial tasks with sensors’ signals
Sensor linearizationnonlinear sensor characteristics
Sensor fusionincomplete or unreliable sensors
Virtual sensingthe quantity to be measured cannot be sensed directly, other related quantity can be measured
Remote sensingindirect measurement (e.g. reflected light: motion of a mechanical system)
19. 07. 2002. NATO ASI LTP Leuven, Belgium 9
MeasurementMeasurementSensor fusion (fusion is a process of combining information from different sensors, when no physical law indicating the correct way to combine this information)
the system cannot rely on a single sensor to provide sufficient informationit must rely on fusion of multiple sensor measurements to obtainmore complete and accurate information
Sensors’ signalsuncertain
limited resolutionrandom measurement noiseinaccurate conversion
incomplete
19. 07. 2002. NATO ASI LTP Leuven, Belgium 10
MeasurementMeasurementSensor fusion
complementary fusione.g. several visual sensors pointed in different directions
competitive fusione.g. measuring of a distance: laser range sensor, acoustic
(ultrasonic) sensor pointed at the same object
cooperative fusione.g. fusion of physical measurements
2D images →3D representation
19. 07. 2002. NATO ASI LTP Leuven, Belgium 14
ExampleExampleSensor fusion
common internal representation
19. 07. 2002. NATO ASI LTP Leuven, Belgium 15
ModelingModelingWhy do we need models?What models can be built?How to build models?
19. 07. 2002. NATO ASI LTP Leuven, Belgium 16
ModelingModelingWhat is a model?
Some (formal) description of a system, a separable part of the world. Represents essential aspects of a system
Key concepts: separation, selection, parsimonyMain features:
• All models are imperfect. Only some aspects are taken into consideration, while many other aspects are neglected.
• Easier to work with models than with the real systems
19. 07. 2002. NATO ASI LTP Leuven, Belgium 17
ModelingModelingSeparation:
the boundaries of the system have to be defined. system is separated from all other parts of the world
Selection:Only certain aspects are taken into consideration e.g.
information relation, interactions energy interactions
Parsimony:It is desirable to use as simple model as possible
Occam’s razor The most likely hypothesis is the simplest one that is
consistent with all observationsThe simpler of two theories, two models is to be preferred.
19. 07. 2002. NATO ASI LTP Leuven, Belgium 18
ModelingModelingWhy do we need models?
To simulate a systemto predict the behaviour of the system (prediction, forecasting),to determine faults and the cause of malfunctions, fault diagnosis, error detectionto control the system to obtain prescribed behaviour, operationsto increase observability: to estimate such parameters which are not directly observable (indirect measurement) system optimization
19. 07. 2002. NATO ASI LTP Leuven, Belgium 19
ModelingModelingWhat models can be built?
Approachesfunctional models
• parts and its connections based on the functional role in the system
physical models• based on physical laws, analogies (e.g. electrical analog
circuit model of a mechanical system)
mathematical models• mathematical expressions (algebraic, differential
equations, logic functions, etc.)
19. 07. 2002. NATO ASI LTP Leuven, Belgium 20
ModelingModelingWhat models can be built?
A priori informationphysical models, “first principle” models models based on observations (experiments)
Aspectsstructural modelsinput-output (behavioral) models
19. 07. 2002. NATO ASI LTP Leuven, Belgium 21
Model classesModel classesBased on the system characteristics
Static – dynamic Deterministic – stochastic Continuous-time – discrete-time Lumped parameter – distributed parameter Linear – non-linear Time invariant – time variant …
19. 07. 2002. NATO ASI LTP Leuven, Belgium 22
Model classesModel classesBased on the modeling approach
parametricknown model structurelimited number of unknown parameters
nonparametricno definite model structuredescribed in many points (frequency characteristics, impulse response)
semi-parametric
19. 07. 2002. NATO ASI LTP Leuven, Belgium 23
Model classesModel classesBased on the a priori information (physical insight)
Black-box
White-box
Structure ParametersStructure Parameters
Structure ParametersStructure Parameters
Structure ParametersStructure Parameters
Structure ParametersStructure ParametersGrey-box
Structure ParametersStructure Parameters
Known Missing (Unknown)
Case of classical measurement
19. 07. 2002. NATO ASI LTP Leuven, Belgium 24
ModelingModelingMain steps
collect information
model set selectionexperiment design and data collectiondetermine model parameters (parameter estimation, decision)
model validation
19. 07. 2002. NATO ASI LTP Leuven, Belgium 25
ModelingModelingCollect information
physical insight (a priori information)understanding the physical behaviour
only observations or experiments can be designed application
what operating conditions• one operating point• a large range of different conditions
what purpose• scientific
– basic research, • engineering
– to study the behavior of a system, – to detect faults, – to design control systems,– etc.
19. 07. 2002. NATO ASI LTP Leuven, Belgium 26
ModelingModelingModel set selection
static – dynamiclinear – non-linear non-linear
linear - in - the - parametersnon-linear - in - the - parameters
parametric – non-parametricwhite-box – black-box
19. 07. 2002. NATO ASI LTP Leuven, Belgium 27
ModelingModelingModel structure selection
known model structure (available a priori information)no physical insights, general model structure
general rule: always use as simple model as possible
• linear• feed-forward
•••
19. 07. 2002. NATO ASI LTP Leuven, Belgium 28
Experiment design and data collectionExperiment design and data collectionExcitation
input signal selectiondesign of excitation
time domain or frequency domain identification (random signal, multi-sine excitation, impulse response, frequency characteristics)persistent excitation
Measurement of input-output data no possibility to design excitation signal
noisy data, missing data, distorted data non-representing data
19. 07. 2002. NATO ASI LTP Leuven, Belgium 29
Modeling (some examples)Modeling (some examples)Resistor modelingModel of a ductModel of a steel converter (model of a complex industrial process)Model of a signal (time series modeling)
19. 07. 2002. NATO ASI LTP Leuven, Belgium 30
Modeling (example)Modeling (example)Resistor modeling
the goal of modeling: to get a description of a physical system (electrical component)parametric model
linear modelconstant parameter
variant model
frequency dependent
RRIU =
I
U
I R(I)IIRU )(=
U
DC
cR AC
12)(
)()()()()()(
+===
CRfjRfZ
fIfUfZfIfZfU
π
19. 07. 2002. NATO ASI LTP Leuven, Belgium 31
Modeling (example)Modeling (example)Resistor modeling
nonparametric model
Z
fI
Unonlinear
I
Ulinear
AC
frequency dependent
DC
19. 07. 2002. NATO ASI LTP Leuven, Belgium 32
Modeling (example)Modeling (example)Resistor modeling
parameter estimation based on noisy measurements
I
Ulinear
OutputSystem
+ +nunI
Input
I U
System
nu
OutputInput
+nI
I
+
U
+
System noise
Measurement noise
+
Input noise
19. 07. 2002. NATO ASI LTP Leuven, Belgium 33
Modeling (example)Modeling (example)Model of a duct
the goal of modeling: to design a controller for noise compensation. active noise control problem
19. 07. 2002. NATO ASI LTP Leuven, Belgium 35
Modeling (example)Modeling (example)Model of a duct
physical modeling: general knowledge about acoustical effects; propagation of sound, etc.
no physical insight. Input: sound pressure, output: sound pressure what signals: stochastic or deterministic: periodic, non-periodic, combined, etc. what frequency range time invariant or notfixed solution, adaptive solution. Model structure is fixed, model parameters are estimated and adjusted: adaptive solution
19. 07. 2002. NATO ASI LTP Leuven, Belgium 36
Modeling (example)Modeling (example)Model of a duct
nonparametric (linear) model of the duct (H1)FIR filter with 10-100 coefficients
0 200 400 600 800 1000-45
-40
-35
-30
-25
-20
-15
-10
-5
0
5
frequency (Hz)
mag
nitu
de (d
B)
19. 07. 2002. NATO ASI LTP Leuven, Belgium 37
Modeling (example)Modeling (example)Nonparametric models: impulse responses
19. 07. 2002. NATO ASI LTP Leuven, Belgium 38
Modeling (example)Modeling (example)Model of a steel converter (LD converter)
19. 07. 2002. NATO ASI LTP Leuven, Belgium 39
Modeling (example)Modeling (example)Model of a steel converter (LD converter)
the goal of modeling: to control steel-making process to get predetermined quality steelphysical insight:
complex physical-chemical process with many inputsheat balance, mass balancemany unmeasurable (input) parameters
no physical insight: there are input-output measurement data
19. 07. 2002. NATO ASI LTP Leuven, Belgium 40
The modeling taskThe modeling task
System
NeuralModel
Σε
components(parameters)
temperature
predictedtemperature
+
-
oxygen
measuredtemperature
components(parameters)
Σε
-
+
Copy ofModel
predictedoxygen
InverseModel
model outputtemperature
19. 07. 2002. NATO ASI LTP Leuven, Belgium 41
Neural Networks in MeasurementsNeural Networks in MeasurementsTasks to be solved in measurement systems
Non-linear transformation (linearization)Signal fusion (combining information from manysensors)Non-linear static or dynamic input-output mappingData compression, data separation
19. 07. 2002. NATO ASI LTP Leuven, Belgium 42
Neural Networks in MeasurementsNeural Networks in MeasurementsMain features of neural networks
Neural networks are universal modeling devices(universal approximators) Implement (static or dynamic) linear or non-linear mappinglinearization, static/dynamic (nonlinear) system modeling, sensor fusion
Their parameters are determined by learning fromexamples (measurement observations)Adaptive system (real-time adaptation) Data transformers (data compression, data separation)
19. 07. 2002. NATO ASI LTP Leuven, Belgium 43
Neural Networks in MeasurementNeural Networks in MeasurementNetwork function:
non-linear static mapping:• MLP, • RBF, • CMAC,• SVMnon-linear dynamic mapping• extended networks: local or global memory,
local or global feedbacklinear or non-linear data compression• MLP • PCA• ICA
19. 07. 2002. NATO ASI LTP Leuven, Belgium 44
Neural Networks for Embedded SystemsNeural Networks for Embedded SystemsEmbedded system:
small size, low cost, low power consumption
Real-time operation: high-speed learning and recall
Hardware implementation:analog
critical: accuracy, learning (weight updating)
digitalcritical: multipliers, activation function
19. 07. 2002. NATO ASI LTP Leuven, Belgium 45
Neural networks for Embedded SystemsNeural networks for Embedded SystemsEfficient implementation of the critical elements Special architecture without
multipliersnonlinear activation function
19. 07. 2002. NATO ASI LTP Leuven, Belgium 46
CMAC Neural NetworkCMAC Neural NetworkCerebellar Model Articulation Controller Albus 1975
Σ
discrete input space(quantized input)
aa
i+i+3
u a a y
y
wwwww
ii+i+i+
21
3
C=4
wa
association vector weight vector
u1
2
i+
i+4
5u
u
2
3
aa
aa
j+
j+
123
j j
j+
w
wwww
j+j+j+
1
32
ai
a
a
a
i+1
i+i+
45
CMAC: no multipliers, no activation functions, only encoder, memory, adder
∑=
=1)(:
)(u
ujaj
jwy
19. 07. 2002. NATO ASI LTP Leuven, Belgium 47
CMAC Neural NetworkCMAC Neural Network
a =
Σ
aa
==
Discrete input space
u
aaai+
1
===
111
00
a =a =M
M
0
00
z
y
y
M
-1-2
12
i+
i+
2
C= 4
aassociation compressed weight
ai+3=1z
i+
i+
i+z
w
w
w
w
i+
i+
i+
z
z
2
3 3
2
1 1
ii i
wz
u a a z
vector association vectorvector
19. 07. 2002. NATO ASI LTP Leuven, Belgium 48
CMAC as a basis function networkCMAC as a basis function network
Basis functions
19. 07. 2002. NATO ASI LTP Leuven, Belgium 49
CMAC as a basis function networkCMAC as a basis function networkC overlays of basis functionsBasis functions are arranged in overlays: one-dimensional case
C=8 dtrain=4
C=4
19. 07. 2002. NATO ASI LTP Leuven, Belgium 50
CMAC as a basis function networkCMAC as a basis function networkBasis functions are arranged in overlays: two-dimensional case
Basis functions
)1(1
01
−+= ∏
=−
N -1
iiN CR
CM
C overlays
19. 07. 2002. NATO ASI LTP Leuven, Belgium 51
CMAC modeling capabilityCMAC modeling capability
One-dimensional case: can learn any training data set exactly
Multi-dimensional case: can learn any non-linear mapping from the additive function set (consistency equations)
19. 07. 2002. NATO ASI LTP Leuven, Belgium 52
CMAC generalization capabilityCMAC generalization capability
Important parameters:
C generalization parameterdtrain distance between adjacent training data
Interesting behaviorC=l*dtrain : linear interpolation between the
training points
C≠l*dtrain : significant generalization error non-smooth output
19. 07. 2002. NATO ASI LTP Leuven, Belgium 55
C/2
CMAC generalization errorCMAC generalization error
19. 07. 2002. NATO ASI LTP Leuven, Belgium 56
Analytical model of Analytical model of the generalization errorthe generalization error
When solving a given problem, try to avoid solving a more general problem as an intermediate step(V. Vapnik: The Nature of Statistical Learning Theory)
19. 07. 2002. NATO ASI LTP Leuven, Belgium 57
Analytical model of Analytical model of the generalization errorthe generalization error
Awy =y(u)=a(u)Tw.
→←
→←→←
=
TL
T
T
)(
)2(
)1(
a
aa
AM
=
011111111...000000000000
0...001111111010000000000...000000010111111000000...00000000000111101111
MMM
A
( ) 1† −= TT AAAAdyAw †=∗
19. 07. 2002. NATO ASI LTP Leuven, Belgium 58
Analytical solutionAnalytical solution
dTT yAATATwy 1)( −∗ == d
T yBTAy = ( ) 1−= TAAB
dT
d yAAByv 1)( −==
dTjjv yb=
=
1 1 1 1 1 10... 0 0 0 0 0 0
0...0 0 0 0 0 1 1 1 1 1 1 1 1 000...0 0 0 0 0 0 1 1 1 1 1 1 1 1 00...0 0 0 0 0 0 0 1 1 1 1 1 1 1 1
L
MMM
T
( ) vGyBTA idiT
iy ==
( ) dyTAPh †−= P: interpolation matrix
19. 07. 2002. NATO ASI LTP Leuven, Belgium 59
Analytical solutionAnalytical solution
−
+−−−
−−−
−+
−+
−−−−
+−−−
=
cdc
dccdcc
dcc
cdcd
cd
cdccdc
dccdcc
T
000
11200310
11420
0210011100122
00211001
00011000
MMMMM
MMMMM
LL
MMMMM
LL
MMMMM
TA
19. 07. 2002. NATO ASI LTP Leuven, Belgium 60
FilterFilter--bank model of the CMACbank model of the CMAC
yd
B
G0
Gd-1
y
G0=AAT=B-1
( ) 1−= TAAB
Gi interpolation filters
19. 07. 2002. NATO ASI LTP Leuven, Belgium 61
Analytical resultAnalytical resultThe key problem: to find the inverse of AAT (symmetrical, Toeplitz, banded)
***************
**********
**********
**********
MO
L
−−−
−−−−−−−−−
−−−
=
CdCdCzdC
zdCdCdCCdCdCzdCdCdCCdC
zdCdCdCC
T
2...0...00
0...0...220...0...20...00...2
MMM
AA
Conjecture: the inner part of the inverse of a matrix is the inverse of the inner part of a matrix
=
dCz
19. 07. 2002. NATO ASI LTP Leuven, Belgium 62
Analytical resultAnalytical resultExtension to get a cyclic matrix
−−−
−−−−−−−−−
−−−
=
CdCdCzdC
zdCdCdCCdCdCzdCdCdCCdC
zdCdCdCC
T
2...0...00
0...0...220...0...20...00...2
MMM
AA
−−−−
−−−−−−−−−−−−−−−−
=
CdCzdCdCdC
dCzdCdCdCCdCdCdCzdCdCdCCdC
dCdCzdCdCdCC
Tcyc
...0...2
3...0...222...0...2
2...00...2
MMM
AA
19. 07. 2002. NATO ASI LTP Leuven, Belgium 63
Analytical resultAnalytical result
Using the spectral representation of
TcycAA
nlk
lk en
)1)(1(2
,1
−−
=π
FF FH
19. 07. 2002. NATO ASI LTP Leuven, Belgium 64
Analytical resultAnalytical resultUsing the spectral representation the general form of the inverse can be written as:
HTcyc FΛFAA =
( ) Hk
Tcyc ff FFAA λ=)(
H
k
Tcyc FFAA
=−
λ1)( 1
( ) ( ) ( )12cos24cos222cos21 −
−++
−+
−+=
nzkzdC
nkdC
nkdCC
k
πππλ
K
19. 07. 2002. NATO ASI LTP Leuven, Belgium 65
Analytical resultAnalytical result
( ) 1−= TAAB
=
dCz
∑−
= −−+++−
−−
=1
0, 2cos)()1(2cos)(
1)2cos()(2cos1 n
kji
dnkπCdzd
nzkπzdC
nkπ
nijkπ
nB
According to numerical results the conjecture is truewith high accuracy
19. 07. 2002. NATO ASI LTP Leuven, Belgium 66
Analytical resultAnalytical resultMore special case of z=1
−
−−−−
−
=
CdC
dCCdCdCCdC
dCC
T
0...000
0...0000...0000...0
MMM
AA
19. 07. 2002. NATO ASI LTP Leuven, Belgium 67
Analytical resultAnalytical result
[ ]
−−−
−−−
=−
xx
xx
x
x
10001100
0010011
L
L
MM
L
L
L
KI
The inverse can be determined using [ ] 1−−= KIP x
− 001
19. 07. 2002. NATO ASI LTP Leuven, Belgium 68
Analytical resultAnalytical result
dCCx−
−=
( ) ( )[ ] PKIAAdC
dCCT−
=−−= −− 111
2−<−
−=dC
Cx
P
Θchx 2−=
ifif
19. 07. 2002. NATO ASI LTP Leuven, Belgium 69
Analytical resultAnalytical resultIf n is large enough we can use
P
( )l
llii b
rCr
−−=+ 2
11 )(,B 22 4bCr −= dCb −=
19. 07. 2002. NATO ASI LTP Leuven, Belgium 70
Generalization error (max)Generalization error (max)
1 2 3 4 5 6 7 80
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
C/dtrain
Abs. error
19. 07. 2002. NATO ASI LTP Leuven, Belgium 71
CMAC gCMAC geometrical interpretationeometrical interpretationWeight activation
19. 07. 2002. NATO ASI LTP Leuven, Belgium 72
CMAC gCMAC geometrical interpretationeometrical interpretationWeight activations
19. 07. 2002. NATO ASI LTP Leuven, Belgium 73
Error reductionError reductionHow to reduce this error?
The real reason of the erroneous behavior: non-uniform distribution of the weight values
Regularization
( ) ( ) ( ) ( ) ( )
−++=+ kw
Ckykkkwkw i
dii
)(1 λεµ a
( ) ( ) ( )( ) ( ) ( )2
2
221
−+−= kw
Cky
kykykC id
dλ
19. 07. 2002. NATO ASI LTP Leuven, Belgium 76
More general casesMore general casesInput positions: periodical by two-samples
Basis functions
Training samples
19. 07. 2002. NATO ASI LTP Leuven, Belgium 77
More general casesMore general casesInput positions: periodical by two-samples; superposition
19. 07. 2002. NATO ASI LTP Leuven, Belgium 78
More general casesMore general casesInput positions: periodical by two-samples; superposition
19. 07. 2002. NATO ASI LTP Leuven, Belgium 79
More general casesMore general casesRandom input positions
Red: periodical by two-samples, blue: random
19. 07. 2002. NATO ASI LTP Leuven, Belgium 80
More general casesMore general casesMultidimensional case
without regularization with regularization
19. 07. 2002. NATO ASI LTP Leuven, Belgium 81
HigherHigher--order CMACorder CMACFirst order Bspline basis function
C=8
02
46
8
02
46
80
0.2
0.4
0.6
0.8
1
continuous input space discrete input space
19. 07. 2002. NATO ASI LTP Leuven, Belgium 82
Relation to SVM with Relation to SVM with splinespline Kernel Kernel functionfunction
The response of an SVM with linear Bspline kernelC=ldtrain C≠ ldtrain
19. 07. 2002. NATO ASI LTP Leuven, Belgium 83
Relation to SVM with Relation to SVM with splinespline Kernel Kernel functionfunction
The response of an SVM with linear Bspline kernelC=ldtrain C≠ ldtrain
19. 07. 2002. NATO ASI LTP Leuven, Belgium 84
Relation to SVM with Relation to SVM with splinespline Kernel Kernel functionfunction
The response of an SVM with linear Bspline kernelC=ldtrain C≠ ldtrain
19. 07. 2002. NATO ASI LTP Leuven, Belgium 85
Relation to SVM with Relation to SVM with splinespline Kernel Kernel functionfunction
The effect of the training points in their neighborhood can be described by a linearly decreasing proximity function with 2C support. Binary CMAC corresponds to an SVM with linear Bspline kernel function
BCMAC: primal space SVM with linear Bspline kernel: dual space
19. 07. 2002. NATO ASI LTP Leuven, Belgium 86
Relation to SVM with Relation to SVM with splinespline Kernel Kernel functionfunction
SVM (with Bspline
kernel)
CMAC(binary or higher-order)
No. of basis function
Support vectors (sparse
approximation)
Training rule Quadratic programming
LMS
Learning speed,on-line adaptation
Low?
Highpossible
Generalization General results Special results
Embedded system ? Easy-to-implement
)1(1
11
−+= ∏
=−
N
iiN CR
CM
19. 07. 2002. NATO ASI LTP Leuven, Belgium 87
ConclusionsConclusions• Measurement as an autonomous step of modeling• Sensors + signal processing• Different tasks require nonlinear mapping• Mapping is learned from observations• The importance of embedded system• Network architecture suitable for embedded systems: CMAC • Properties of CMAC from the point of view of:
implementationmodeling capabilitygeneralization capability
• Detailed mathematical analysis of the performance • Modified learning algorithm to improve generalization
capability.
19. 07. 2002. NATO ASI LTP Leuven, Belgium 88
ReferencesReferencesB. Pataki – G. Horváth – Gy. Strausz – Zs. Talata "Inverse Neural Modeling of a Linz-Donawitz Steel Converter" e & i
Elektrotechnik und Informationstechnik, Vol. 117. No. 1. 2000. pp. 13-17.J. Sjöberg, Q. Zhang, L. Ljung, A. Benveniste, B. Delyon, P.-Y. Glorennec, H. Hjalmarsson, and A. Juditsky: "Non-linear
Black-box Modeling in System Identification: a Unified Overview", Automatica, 31:1691-1724, 1995. G. Horváth: “Neural Networks in System Identification” Chapter 3. In: V. Piuri (Ed.) Neural Networks in Measurement Systems
NATO ASI NIMIA, Crema, Italy 2001. Oct. IOS Press, in print.J. Van Dam: “Environment Modelling of Mobile robots. Neural Learning for Sensor Fusion” Ph.D thesis Universiteit van
Amsterdam, 1998.S. Thrun: “Learning Maps for Indoor Robot Navigation” Artificial Intelligence 1998. J. S. Albus: "A New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)", Transaction of
the ASME, Sep. 1975. pp. 220-227. M. Brown – C. J. Harris – P. Parks: "The Interpolation Capability of the Binary CMAC", Neural Networks, Vol. 6. No. 3. 1993.
pp. 429-440. G. Horváth – F. Deák: "Hardware Implementation of Neural Networks Using FPGA Elements", Proc. of The International
Conference on Signal Processing Application and Technology, Santa Clara Vol. II. pp. 60-65. 1993. T. Szabó – G. Horváth: "CMAC and its Extensions for Efficient System Modelling", International Journal of Applied
Mathematics and Computer Science, 1999. Vol. 9. No. 3. pp. 571-598. T. Szabó, G. Horváth: "Improving the generalization capability of the binary CMAC", Proc. of the International Joint
Conference on Neural Networks, IJCNN'2000. Como Italy. Vol. III. pp.85-90. G. Horváth, T. Szabó: “CMAC Neural Network with Improved Generalization Capability for System Modeling” Proc. Of the
IEEE Conference on Instrumentation and Measurement, Anchorage, AK. 2002. Vol. 2. pp. 1603-1608.S. Gunn: “Support Vector Machines for Classification and Regression” ISIS Technical Report and MATLAB toolbox, University
of Southampton, 1998.V. Vapnik: “The Nature of Statistical Learning Theory”, Springer, 1995.