Hopfield neural networks for on-line parameter estimation

Neural Networks 22 (2009) 450–462

Contents lists available at ScienceDirect

Neural Networks

journal homepage: www.elsevier.com/locate/neunet

Hopfield neural networks for on-line parameter estimationHugo Alonso a,∗, Teresa Mendonça a,b, Paula Rocha a,ca Unidade de Investigação Matemática e Aplicações, Universidade de Aveiro, Campus Universitário de Santiago, 3810-193 Aveiro, Portugalb Faculdade de Ciências da Universidade do Porto, Departamento de Matemática Aplicada, Rua do Campo Alegre, 687, 4169-007 Porto, Portugalc Faculdade de Engenharia da Universidade do Porto, Departamento de Engenharia Electrotécnica e de Computadores, Rua Dr. Roberto Frias, s/n, 4200-465 Porto, Portugal

a r t i c l e i n f o

Article history:Received 6 October 2006Accepted 27 January 2009

Keywords:On-line parameter estimationHopfield neural networksLyapunov stability theory

a b s t r a c t

This paper addresses the problem of using Hopfield Neural Networks (HNNs) for on-line parameterestimation. As presented here, a HNN is a nonautonomous nonlinear dynamical system able to producea time-evolving estimate of the actual parameterization. The stability analysis of the HNN is carried outunder more general assumptions than those previously considered in the literature, yielding a weakersufficient condition under which the estimation error asymptotically converges to zero. Furthermore,a robustness analysis is made, showing that, under the presence of perturbations, the estimation errorconverges to a bounded neighbourhood of zero, whose size decreases with the size of the perturbations.The results obtained are illustrated by means of two case studies, where the HNN is compared with twoother methods.

© 2009 Elsevier Ltd. All rights reserved.

1. Introduction

In many cases it is necessary, or useful, to have a model ofa system available on-line while the system is in operation. Theneed for such an on-linemodel construction, reduced to parameterestimationwhen the structure is given beforehand, typically ariseswhen a model is required in order to take some decision aboutthe system: which input should be applied next?; what is thebest prediction of the output?; has a failure occurred and, if so,of what type? (Ljung, 1999). The parameter estimates should thenbe based on observations up to the current time, and thereforeon-line parameter estimation methods must recursively processthe measured data as they become available. Traditional methodsinclude recursive least-squares and the Kalman filter for parameterestimation (Ljung, 1999). Newer alternative methods includeHopfield Neural Networks (HNNs) for parameter estimation (see,for example, (Atencia, Joya, & Sandoval, 2004; Hu & Balakrishnan,2005)). Such networks constitute a particular type of recurrentneural networks. In (Di Claudio, Parisi, & Orlandi, 2000; Parisi, DiClaudio, Orlandi, & Rao, 1997), a different type of recurrent neuralnetwork and a learning algorithmare proposed for the equalizationof digital communication channels. Channel equalization is viewedas a classification task. Thus, acting as a classifier, the networkreceives a pattern from a channel and predicts a class for thatpattern; more specifically, a class from a finite set of classes.Here, our goal is to estimate as accurately as possible the

∗ Corresponding author.E-mail address: [email protected] (H. Alonso).

0893-6080/$ – see front matter© 2009 Elsevier Ltd. All rights reserved.doi:10.1016/j.neunet.2009.01.015

parameterization for a model of a system from a given infiniteparameter universe. In this context, a classification approachwould have serious accuracy limitations, because only a finitenumber of estimates corresponding to classes could be predicted.Therefore, we take a different approach to the problem of on-lineparameter estimation, proposing the use of HNNs.At least two formulations of the HNN have been considered

in the context of the problem of on-line parameter estimation,namely the original Hopfield formulation (Hopfield, 1984) and thesimplified Abe formulation (Abe, 1989). In Hopfield’s formulation,the dynamics of neuron i is governed by the ordinary differentialequation

dpidt(t) = −

1Ci

(1Ripi(t)+

∑j

Wijfj(pj(t))+ Ii

),

where pi is the total input to neuron i, fj is a continuous,nonlinear, bounded and strictly increasing function, and Ci, Ri,Wij, Ii are parameters corresponding respectively to a capacitance,a resistance, the synaptic efficacy or weight associated with theconnection from neuron j to neuron i, and the external inputor bias of neuron i. In Abe’s formulation, it is assumed thatCi = 1 and Ri = ∞. In the context of the problem of on-lineparameter estimation, these two formulations lead to differentways of defining Wij and Ii as functions of the measured data.Hopfield’s formulation is considered in (Hu & Balakrishnan, 2005),and Wij and Ii are defined as integral functions; however, anintegral function is not always explicitly known, and if so, itsevaluation has to be numerically approximated. In turn, Abe’s

http://www.elsevier.com/locate/neunet

http://www.elsevier.com/locate/neunet

mailto:[email protected]

http://dx.doi.org/10.1016/j.neunet.2009.01.015

H. Alonso et al. / Neural Networks 22 (2009) 450–462 451

formulation is considered in (Atencia et al., 2004), and Wij andIi are defined as non-integral, explicitly known functions. Giventhat the parameter estimation is to be carried out on-line, it isclear that one should prefer a problem solving approach based onthis latter formulation. Regarding the main assumptions about thestructure of the model whose parameters are to be estimated, itis assumed in (Atencia et al., 2004; Hu & Balakrishnan, 2005) thatthe model is dynamic and linear in the parameters; moreover, itis assumed in (Hu & Balakrishnan, 2005) that the model is alsolinear in the inputs. In any case, the estimate of a parameteris produced by a neuron in the HNN. Hence, the convergenceproperties of the estimated parameterization are deduced from astability analysis of the HNN. Mind that this stability analysis ismore complex than usual, since Wij and Ii vary in time, i.e., theHNN is nonautonomous. In (Hu & Balakrishnan, 2005), it is shownthat the estimated parameterization asymptotically convergesto the actual parameterization. This is also shown in (Atenciaet al., 2004), but only when the actual parameterization is time-invariant; when the actual parameterization is time-variant, it isshown that the estimated parameterization remains close to theactual parameterization once close to it. In both works, the resultswere achieved assuming no perturbations, like modeling errors,uncertainties and disturbances.In this paper, in the line of previous arguments, we approach

the problem of on-line parameter estimation by considering Abe’sformulation of a HNN. Our work is therefore related to (Atenciaet al., 2004). The only assumption we make on the model struc-ture is that it be linear in the parameters; hence, our results areapplicable not only to dynamical systems, but also to systemswhich are static or include static relations. Regarding the stabilityanalysis of the HNN, the main hypothesis used in (Atencia et al.,2004) to prove that the estimated parameterization asymptot-ically converges to the actual parameterization amounts toassuming that the time-variant weight matrix (Wij) is always non-singular throughout the estimation process; however, this is arestrictive assumption, namely in practice. Here, we present aweaker sufficient condition on (Wij) for the estimated parameter-ization to asymptotically converge to the actual parameterization.Furthermore, we carry out a robustness analysis, proving that, inthe presence of perturbations andunder thisweaker sufficient con-dition on (Wij), the estimated parameterization converges in finitetime to a bounded neighbourhood of the actual parameterization,whose size decreases with the size of the perturbations.The remainder of the paper is organized as follows. In Section 2,

the problem of applying HNNs to on-line parameter estimationis formulated, and our contributions clarified. In Section 3, wedescribe how to apply a HNN to on-line parameter estimation,and then present and prove the results concerning the stabilityand robustness of the HNN estimator. Section 4 is devoted to theillustration of our results through two case studies: the first isrelated to a two-cart system, and the second to a robotic armwith a flexible link. We also analyze the convergence time of theHNN estimator, and compare the performance of our method withthe performance of two traditional methods for on-line parameterestimation, namely recursive least-squares and the Kalman filterfor parameter estimation. Finally, we end the paper with theconclusions, and present the ongoing and future work.

2. Problem formulation

The systems to be considered are assumed to be autonomousand linear in the parameters, i.e., they can always be cast to thereduced form

y(t) = A(t)θ , (1)

for some y : [t0,+∞[→ Rm×1, A : [t0,+∞[→ Rm×n, andwhere θ ∈ Rn×1 is the vector of the unknown parameters to be

estimated on-line. Furthermore, we shall work under the followingassumptions:

A1. y,A ∈ C1;A2. y,A are bounded;A3. y(t),A(t) are known at each time t;A4. θ ∈ ] − c, c[n for some known c > 0.

In order to solve the on-line estimation problem, an algorithmA should generate at time t an estimate θ (t) of θ based on theavailable information: c and (y(τ ),A(τ )) : τ ∈ [t−∆t, t], where∆t ≤ t − t0. Some desirable properties forA are:

D1. ∀t ≥ t0 θ (t) ∈ ] − c, c[n;D2. limt→+∞ θ (t) = θ .

Proeperty D1 defines the feasible region for the estimationproblem, while Property D2 is the ultimate goal of an on-lineestimation algorithm. The distinction between different on-lineestimation algorithms lies in the definition of the trajectory θ (·).Nevertheless, all of them embed a measure of closeness betweenθ (t) and θ , which in the present context is taken as the squarednorm of the error

e(t, θ (t)) = y(t)− A(t)θ(t).

Here, we define an on-line estimation algorithm represented by anonautonomous Hopfield Neural Network (HNN) able to generatea trajectory θ (·) for which Properties D1 and D2 are satisfied, thusasymptotically solving the optimization problem

minθ∈]−c,c[n

sup

t∈[t0,+∞[

12e(t, θ )Te(t, θ )

. (2)

It is shown in (Atencia et al., 2004) that there exists a nonau-tonomous HNN able to generate a trajectory θ (·) that asymp-totically converges to θ in the particular case where ∀t ≥ t0ker(A(t)) = 0 holds and one knows an a priori estimate θ (t0) =θ0 of θ such that θ ∈ θ0+] − 1, 1[n. But if∀t ≥ t0 ker(A(t)) = 0,then the solution to the estimation problem can be obtained at anytime t ≥ t0, since ∀t ≥ t0 θ = −B(t)−1b(t), where

B(t) = A(t)TA(t), (3)

b(t) = −A(t)Ty(t). (4)

In this case, the advantage of using a nonautonomous HNN tosolve the estimation problem has to do with the fact that thereis no need to invert B(t), which can be ill-conditioned. Clearly, anecessary condition for ∀t ≥ t0 ker(A(t)) = 0 to hold is thatm ≥ n, i.e., the system must not be overparameterized. However,a successful application of a nonautonomous HNN to an estimationproblem where m < n is given in (Atencia et al., 2004), whichmotivated us to find more general conditions. Here, we start byintroducing a modified nonautonomous HNN structure that hasthe advantage of requiring no prior knowledge on the choice ofθ (t0), the initial estimate of θ , and which accommodates the valueof c , a parameter that canbe viewed as ameasure of the uncertaintyin the location of θ . Then, we show that θ is a globally uniformlyasymptotically stable equilibrium point of this HNN at t = t01 ifthe following holds: for all nondegenerate interval I ⊂ [t0,+∞[,⋂t∈I ker(A(t)) = 0. It is clear that this is aweaker condition than

the one presented in (Atencia et al., 2004); moreover, it does notimply an order relation betweenm and n.We also prove that, in the

1 Let f : [t0,+∞[×D→ Rn be piecewise continuous in t and locally Lipschitz inx on [t0,+∞[×D, where D ⊂ Rn . The point x? ∈ D is an equilibrium point of thesystem dx

dt = f (t, x) at t = t?≥ t0 if ∀t ≥ t? f (t, x?) = 0 (Khalil, 2002).

452 H. Alonso et al. / Neural Networks 22 (2009) 450–462

presence of perturbations, θ is not, in general, an equilibrium pointof the proposed HNN. In this case, it is shown that every trajectorygenerated by the HNN is globally uniformly ultimately bounded(i.e., converges in finite time to a bounded neighbourhood of theactual parameterization) if the latter condition on ker(A) holds.Finally, note that, in both cases, we assume no upper bound for theuncertainty parameter c , which can be made arbitrarily large.

3. Hopfield Neural Networks for on-line parameter estimation

In what follows, we first describe our extension of Abe’sformulation of an autonomous Hopfield Neural Network (HNN)to a nonautonomous one. Then, we present the methodology forapplying this HNN to the problem of on-line parameter estimation.Finally, we state and prove the results concerning the stability androbustness of the proposed HNN estimator.

3.1. Hopfield Neural Network

Consider a network with N neurons indexed in 〈N〉 =1, . . . ,N. Let pi denote the total input to neuron i, defined as

dpidt(t) = −

(N∑j=1

Wij(t)sj(t)+ Ii(t)

),

where sj represents the state (or output) of neuron j,Wij theweightassociatedwith the connection from neuron j to neuron i, and Ii thebias (or external input) of neuron i. The total input — state relationfor neuron i is given by

si(t) = α tanh(pi(t)β

),

where α, β > 0, and therefore

∀i ∈ 〈N〉 , t ≥ t0 si(t) ∈] − α, α[. (5)

Using matrix notation, this HNN is thus given bydpdt(t) = − (W(t)s(t)+ I(t))

s(t) = α tanh(p(t)β

),

(6)

where p(t), s(t), I(t) ∈ RN×1 andW(t) ∈ RN×N , yielding the statespace representation

dsdt(t) = −

1αβ

Dα(s(t)) (W(t)s(t)+ I(t)) , (7)

where

Dα(s(t)) = diag((α2 − si(t)2

)i∈〈N〉

)(8)

is positive definite and invertible by (5). Henceforth, we shall referto (7) as a HNN. This is a nonautonomous nonlinear dynamicalsystem whose architecture is fully determined by the number Nof neurons, andwhose dynamics is fully characterized by α, β ,W, Iand s(t0). The next subsection explains how to define α,W and I sothat the HNN can be applied to solving our estimation problem.The choice of β is empirical, but does not affect the qualitativeproperties of the network.

3.2. Applying the Hopfield Neural Network to on-line parameterestimation

Since (Hopfield & Tank, 1985), HNNs have been applied tosolving optimization problems, namely those of the form

minv∈R

E(v) =

12vTQv+ vTq

, (9)

where R ⊂ RN is bounded,Q ∈ RN×N , q ∈ RN×1 are time-invariantand Q is symmetric. This application is accomplished by means ofdefining an autonomous HNN, where ∀t ≥ t0 W(t) = Q, I(t) = q,so that a solution to the problem is an attractive equilibrium pointof the network, being the attraction conditions derived from astability analysis. In this way, the continuous time-evolution ofthe neuron states can be interpreted as that of an approximatesolution to the problem. In our case, let us start by considering theoptimization problem (2), where12e(t, θ )Te(t, θ ) =

12y(t)Ty(t)+ E(t, θ )

for

E(t, θ ) =12θTB(t)θ + θTb(t).

It can be seen that ∀θ ∈ ] − c, c[n, t ≥ t0 E(t, θ ) ≥ − 12y(t)Ty(t)

= E(t, θ), since ∀θ ∈ ] − c, c[n, t ≥ t0 12e(t, θ )

Te(t, θ ) ≥ 0and ∀t ≥ t0 1

2e(t, θ)Te(t, θ) = 0. Therefore, ∀θ ∈ ] − c, c[n

supt∈[t0,+∞[E(t, θ )

≥ supt∈[t0,+∞[ E(t, θ), from where it

follows that θ = θ is a solution to

minθ∈]−c,c[n

sup

t∈[t0,+∞[

E(t, θ )

. (10)

Henceforth, this is the optimization problemwe shall look at. Notethat this can be regarded as a time-variant version of (9).Motivatedby the methodology adopted for the time-invariant case, we startby considering a HNN whose state at time t , s(t), is taken as theestimate θ (t) of the sought parameterization θ , i.e., θ (t) = s(t).Implicitly, we are considering a network with as many neuronsas the parameters to estimate, i.e., N = n. Taking α = c , weguarantee by (5) that the trajectory generated by the HNN is inthe feasible region of the estimation problem. The next step is toensure that θ is an equilibrium point of the network at t = t0.Minding the state space representation (7), this is the case if andonly if ∀t ≥ t0 I(t) = −W(t)θ . In order to achieve this condition,and given that

∀t ≥ t0 b(t) = −B(t)θby (4), (1) and (3), we takeW = B and I = b. Hence, the HNN to beused as an on-line estimation algorithm has its dynamics definedby

dθdt(t) =

1cβ

Dc(θ(t))A(t)T(y(t)− A(t)θ(t)). (11)

Remark 1. Sensor networks are proposed in (Barbarossa & Scutari,2007; Schizas, Ribeiro, & Giannakis, 2008) for parameter estima-tion and they can be used for the off-line estimation of θ in (1). Inthis context, all sensors in a network estimate the parameter vec-tor θ , with different sensors using, in general, different data col-lected from the system. Furthermore, a network is said to reacha consensus when all sensors estimate the same value for θ .Noting that parameter estimation in a sensor is a result of the con-vergence of a dynamic process, one may look at a sensor networkas a network of Hopfield-like networks; in fact, the sensor dynam-ics in (Barbarossa & Scutari, 2007) is similar (but not equal) to (11).This idea of combining several networks can also be implementedusing as basic elements our HNNs, and might improve the reliabil-ity of the estimation procedure. In this paper, we concentrate onthe analysis of a single HNN, leaving the investigation of the per-formance of a network of HNNs for future research.


For theoretical purposes, we consider, in what follows, theequivalent formulation to (11) given by

dθdt(t) =

1cβ

Dc(θ(t))B(t)(θ − θ (t)), (12)

cf. (1) and (3). It remains to show that θ is an attractive equilibriumpoint. Note that the so-called energy function E in the optimizationproblem (10) explicitly depends on time, since B and b are time-variant. This implies that dEdt evaluated along the HNN trajectory,

dEdt(t, θ (t)) = −

12θ (t)T

dBdt(t)(2θ − θ (t))

−1cβ(B(t)(θ − θ (t)))TDc(θ(t))B(t)(θ − θ (t)),

is not guaranteed to be negative definite, contrary towhat happenswhen E does not explicitly depend on time. Therefore, E is not aLyapunov function and the usual studies on HNN stability, whichare based on E being Lyapunov, are not valid. We next analyzethe convergence of the HNN estimator, starting by presentinga suitable Lyapunov function which is used to infer about thestability of the equilibrium point θ .

3.3. Convergence of the Hopfield estimator

This subsection is divided in two parts. In the first part, wepresent a stability analysis whose goal is to introduce a sufficientcondition, weaker than the one in (Atencia et al., 2004) (cf. endof Section 2), under which the HNN with dynamics (12) actsas an on-line estimation algorithm able to estimate the actualparameterization θ . Here, the ability to estimate θ means that θis a globally uniformly asymptotically stable equilibrium point.In (Atencia et al., 2004), it is shown that θ is globally attractive.However, an equilibrium point can be attractive without beingstable (Vidyasagar, 2002). Stabilitymeans that the trajectory of theHNN remains close to θ if the initial estimate θ (t0) is sufficientlyclose, being therefore an important feature in practice, and thusworth showing. Then, in the second part, we present a robustnessanalysis whose objective is to prove that, in the presence ofperturbations and under the aforementioned sufficient condition,the trajectories generated by the proposed HNN estimator areglobally uniformly ultimately bounded.

3.3.1. Stability analysisThe stability behavior of θ as an equilibrium point of the HNN

can be determined by examining the stability behavior of theorigin as an equilibrium point of a system obtained from the HNNthrough an appropriate change of variables. Let us then considerthe estimation error defined by

∆(t) = θ − θ (t).

The dynamics governing this error can be obtained from (12), andis given by

d∆dt(t) = f (t,∆(t)), (13)

where

f (t,∆(t)) = −1cβ

Dc(θ −∆(t))B(t)∆(t)

is continuously differentiable with respect to t , since so is B by (3)and assumption A1, and with respect to∆, since f is polynomial in∆ given the way Dc is defined in (8). It is clear that ∆? = 0 is theequilibrium point of the estimation error dynamics (13) at t = t0which corresponds to the equilibrium point θ ? = θ of the HNN(12) at t = t0.

Aswe alreadymentioned, one of themain results of this paper isbased on a Lyapunov stability analysis, where the function definedin the next lemma plays an important role.

Lemma 1. The function V : θ + ] − c, c[n → R defined by

V (∆(t)) = −12c

n∑i=1

ln

((1+

∆(t)ic − θi

)c−θi (1−

∆(t)ic + θi

)c+θi)(14)

is a Lyapunov function for the estimation error dynamics (13).

Proof. Start by noting that

V (0) = 0. (15)

The gradient of V is given by

∂V∂∆

(∆(t)) = Dc(θ −∆(t))−1∆(t),

and the Hessian matrix by

H(V )(∆(t)) = diag

((c2 + (∆(t)2i − θ

2i )

(c2 − (∆(t)i − θi)2)2

)i∈〈n〉

).

At the only critical point of V , 0, the Hessian matrix is positivedefinite, due to assumption A4. Therefore, as θ+] − c, c[n is open,0 is the global minimizer of V , and thus, since V (0) = 0,

∀∆(t) 6= 0 V (∆(t)) > 0. (16)

From (15) and (16), it follows that V is positive definite, and hencea Lyapunov function candidate (Vidyasagar, 2002). Since c, β > 0and B is clearly positive semidefinite,

dVdt(t,∆(t)) = −

1cβ∆(t)TB(t)∆(t) (17)

is negative semidefinite, and therefore V is in fact a Lyapunovfunction (Vidyasagar, 2002).

The next lemma states that in every concentric, closed sub-hypercube of θ + ] − c, c[n containing the origin, the Lyapunovfunction V is lower and upper bounded by suitable comparisonfunctions.

Lemma 2. Consider the set

Sr = ∆ ∈ θ + ] − c, c[n : ‖∆− θ‖∞ ≤ r (18)

for some r ∈]‖θ‖∞, c[. Then, there exist classK functions2 γ1 and γ2such that

γ1(‖∆‖∞) ≤ V (∆) ≤ γ2(‖∆‖∞)

for all∆ ∈ Sr .

Proof. First of all, note that 0 ∈ Sr . Now, let r? =

max∆∈∂Sr ‖∆‖∞. Define ψ : [0, r?] → R by

ψ(ξ) = min∆∈LξV (∆),

where Lξ = ∆ ∈ Sr : ‖∆‖∞ ≥ ξ. It can be shown that ψis positive definite and increasing, but not necessarily strictlyincreasing. Let ∆ ∈ Sr be arbitrarily chosen. Since ‖∆‖∞ ≤ r?by the definition of r?, it follows that ψ(‖∆‖∞) is well defined.Moreover, ψ(‖∆‖∞) = min∆∈L

‖∆‖∞V (∆) ≤ V (∆) ∀∆ ∈ L

‖∆‖∞,

and in particularψ(‖∆‖∞) ≤ V (∆). Therefore ∀∆ ∈ Sr ψ(‖∆‖∞)≤ V (∆), since ∆ was arbitrarily chosen in Sr . Recall that ψ is not

2 A continuous function γ : [0, a] → [0,+∞[ is said to belong to classK if it isstrictly increasing and γ (0) = 0 (Khalil, 2002).


necessarily strictly increasing, and thus not necessarily of classK;however, there exists a classK function γ1 : [0, r?] → R such that∀ξ ∈ [0, r?] γ1(ξ) ≤ ψ(ξ). Then,

∀∆ ∈ Sr γ1(‖∆‖∞) ≤ V (∆).

Finally, define φ : [0, r?] → R by

φ(ξ) = max∆∈UξV (∆),

where Uξ = ∆ ∈ Sr : ‖∆‖∞ ≤ ξ. The function φ can beshown to be positive definite and increasing, but not necessarilystrictly increasing. By similar arguments as before, it can be seenthat ∀∆ ∈ Sr V (∆) ≤ φ(‖∆‖∞). Let γ2 : [0, r?] → R be a classKfunction such that ∀ξ ∈ [0, r?] γ2(ξ) ≥ φ(ξ). Then,

∀∆ ∈ Sr V (∆) ≤ γ2(‖∆‖∞).

Hence, we conclude that

γ1(‖∆‖∞) ≤ V (∆) ≤ γ2(‖∆‖∞)

for all∆ ∈ Sr .

The following lemma guarantees that the trajectory θ (·)generated by the HNN is unique for each initial estimate θ (t0) ofthe actual parameterization θ . This is important for the proof ofour first main result.

Lemma 3. The initial-value problem defined by the estimation errordynamics (13) and the initial condition∆(t0) = ∆0,

d∆dt(t) = f (t,∆(t)), ∆(t0) = ∆0, (19)

has a unique solution over [t0,+∞[.

Proof. The global existence and uniqueness of the solution followsfrom Theorem 3.3 in (Khalil, 2002), since f is continuouslydifferentiable, and hence Lipschitz, and every solution lies entirelyin a compact subset of θ + ] − c, c[n, as we prove next.Consider the set

Ω = ∆ ∈ θ + ] − c, c[n : V (∆) ≤ V (∆0).

All solutions starting from ∆0 lie entirely inΩ , since the negativesemidefiniteness of dVdt implies that ∀t ≥ t0 V (∆(t)) ≤V (∆(t0)) = V (∆0). Now, Ω is bounded by definition, and thuswe only need to show that Ω is closed so that it be a compactset. Let us then consider a sequence (∆(k))k∈N with elements inΩ and such that limk→+∞∆(k) = ∆L. We want to show that∆L ∈ Ω . Start by noting that ∆L ∈ θ + [−c, c]n. Suppose that∆L 6∈ θ + ] − c, c[n, i.e., that ∆L ∈ ∂

(θ + ] − c, c[n

). Then,

limk→+∞ V (∆(k)) = +∞ by (14), and therefore (V (∆(k)))k∈N is notupper bounded, which contradicts the fact that ∀k ∈ N V (∆(k)) ≤V (∆0), i.e., that (∆(k))k∈N ⊂ Ω . Hence, ∆L ∈ θ + ] − c, c[n. Now,since V is continuous, it follows that limk→+∞∆(k) = ∆L ⇒

limk→+∞ V (∆(k)) = V (∆L). In this way, the sequence (V (∆(k)))k∈Nwith elements in the closed set [0, V (∆0)] is convergent, and thusthe limit V (∆L) is in [0, V (∆0)]. Given that∆L ∈ θ+] − c, c[n andV (∆L) ≤ V (∆0), we conclude that∆L ∈ Ω .

Corollary 1. If ∆(t0) 6= 0, then the solution∆(t) to the initial-valueproblem (19) is such that ∀t ≥ t0 ∆(t) 6= 0.

Proof. Assume that ∆(t0) 6= 0. Furthermore, by reductio adabsurdum, suppose that ∃τ > t0 : ∆(τ ) = 0. Lemma 3 guaranteesthat ∆(t) is unique over [t0, τ ]. Then, so is the solution η(t) =∆(τ + t0 − t), t ∈ [t0, τ ], to the initial-value problem

dηdt(t) = −f (τ + t0 − t, η(t)), η(t0) = 0.

But since η? = 0 is an equilibrium point at t = t0, it followsthat ∀t ∈ [t0, τ ] η(t) = 0. Therefore, since η is unique, ∀t ∈[t0, τ ] ∆(τ + t0 − t) = 0, and in particular ∆(t0) = 0, whichcontradicts the initial assumption.

We are now ready to state and prove the first main result of thispaper.

Theorem 1. The equilibrium point ∆? = 0 of the estimation errordynamics (13) is globally uniformly asymptotically stable if

for all nondegenerate interval

I ⊂ [t0,+∞[,⋂t∈I

ker(A(t)) = 0. (20)

Proof. We start by stressing that a necessary condition for∆? = 0to be attractive is that it be isolated. Under (20), ∆? = 0 is in factthe only equilibrium point of the estimation error dynamics (13).To see this, let∆? be an equilibrium point at t = t? ≥ t0. Since Dcis invertible, it follows that ∀t ≥ t? ∆? ∈ ker(B(t)), and given that∀t ≥ t0 ker(B(t)) = ker(A(t)) by the definition of B, we concludethat∆? = 0 if (20) holds.Now, consider the initial-value problem (19), which has a

unique solution over [t0,+∞[ by Lemma 3. Let us assume that∆(t0) = ∆0 6= 0; otherwise ∀t ≥ t0 ∆(t) = 0, since ∆? = 0is an equilibrium point at t = t0. For some r ∈]‖θ‖∞, c[, considerthe set Sr as defined in (18). Take

Ωρ = ∆ ∈ Sr : V (∆) ≤ ρ

for some ρ ∈]0,min∆∈∂Sr V (∆)[. If ∆0 ∈ Ωρ , then ∀t ≥t0 ∆(t) ∈ Ωρ , since the negative semidefiniteness of dVdt impliesthat ∀t ≥ t0 V (∆(t)) ≤ V (∆0). In the following, we prove that,under (20), we have

∃δ > 0 : ∀t ≥ t0 V (∆(t + δ)) < V (∆(t)). (21)

Given that dVdt is negative semidefinite, this is equivalent to

showing that

∃δ > 0 : ∀t ≥ t0 ∃τ ∈ [t, t + δ] :dVdt(τ ,∆(τ )) < 0.

By reductio ad absurdum, suppose that

∀δ > 0 ∃t ≥ t0 : ∀τ ∈ [t, t + δ]dVdt(τ ,∆(τ )) = 0.

Let δ > 0 be arbitrarily chosen. Then, ∃t ≥ t0 : ∀τ ∈ [t, t + δ]

dVdt(τ ,∆(τ )) = 0

⇔ ∆(τ ) ∈ ker(B(τ ))

⇒d∆dt(τ ) = 0

by (17), by the symmetry and positive semidefiniteness of B, andby (13). Therefore, ∆ is constant, say equal to ∆c , in [t, t + δ].Moreover, since ∀τ ∈ [t, t + δ] ∆c ∈ ker(B(τ )), it follows that∆c = 0 by the property of B under which ∀t ≥ t0 ker(B(t)) =ker(A(t)), and by (20). Aswe assumed that∆0 6= 0, this contradictsCorollary 1, and thus (21) holds. Now, it is easy to see that, from(21), we have

∃δ > 0 : ∀t ≥ t0 V (∆(t + δ)) ≤ (1− λ)V (∆(t))

for some λ ∈]0, 1[. Similarly to the proof of Theorem 8.5 in (Khalil,2002), it can be shown that there exists a class KL function3 σ

3 A continuous function σ : [0, a] × [0,+∞[→ [0,+∞[ is said to belong toclassKL if, for each fixed τ , the mapping σ(ξ, τ ) belongs to classK with respectto ξ and, for each fixed ξ , the mapping σ(ξ, τ ) is decreasing with respect to τ andσ(ξ, τ )→ 0 as τ →+∞ (Khalil, 2002).


such that

∀∆0 ∈ Ωρ, t ≥ t0 V (∆(t)) ≤ σ(V (∆0), t − t0). (22)

Hence, if∆0 ∈ Ωρ , then

∀t ≥ t0 ‖∆(t)‖∞ ≤ γ−11 (V (∆(t)))

≤ γ−11 (σ (V (∆0), t − t0))

≤ γ−11 (σ (γ2(‖∆0‖∞), t − t0))

, µ(‖∆0‖∞ , t − t0) (23)

by Lemma 2 and (22), and where the function µ is a class KLfunction by Lemma 4.2 in (Khalil, 2002). Making use of the fact thatthe norms ‖·‖2 and ‖·‖∞ are equivalent in Rn, from (23) it followsthat ∆? = 0 is uniformly asymptotically stable by Lemma 4.5in (Khalil, 2002). Finally, note that limr→c− ∂Sr = ∂(θ+] − c, c[n),and therefore limr→c− min∆∈∂Sr V (∆) = +∞ by (14). In thisway, ρ < min∆∈∂Sr V (∆) can bemade arbitrarily large so that thesetΩρ includes any initial state∆0 ∈ θ + ] − c, c[n. Thus,∆? = 0is globally uniformly asymptotically stable.

Remark 2. Note that, as alreadymentioned in Section 2, condition(20) in Theorem 1 is weaker than the one presented in (Atenciaet al., 2004), which requires that ∀t ≥ t0 ker(A(t)) = 0.Both conditions guarantee that θ is the only time-invariantparameterization for the system.

Remark 3. According to the theorem just proved, the origin is aglobally uniformly asymptotically stable equilibrium point of theestimation error dynamics (13) if condition (20) on ker(A) holds.In fact, assuming that the network parameters c and β are fixed,global uniform asymptotic stability of the origin implies that, foreach ε > 0, there is T = T (ε) ≥ 0 such that ∀t ≥ t0 +T ‖∆(t)‖∞ < ε (Khalil, 2002). This means that the convergencetime of the HNN estimator, with a given precision ε, depends on ε,but not on the initial estimate θ (t0). On the other hand, differentchoices of c and β have an effect on the convergence time, as willbe shown in our practical experiments in Section 4.

3.3.2. Robustness analysisIn the following, we shall study the robustness of the

HNN estimator under perturbations. Start by considering thenetwork (11) to be used in practice as an on-line estimationalgorithm. For many reasons, like modeling errors, uncertaintiesand disturbances, it is possible that one does not know the exactvalues of y(t) and A(t), but rather some related values y(t) =y(t) + υy(t) and A(t) = A(t) + ΥA(t), where υy : [t0,+∞[→Rm×1 and ΥA : [t0,+∞[→ Rm×n. In what follows, υy and ΥA areassumed to be bounded and piecewise continuous. In this context,it can be seen after some straightforward calculations that theestimation error∆(t) = θ − θ (t) has its dynamics given by

d∆dt(t) = f (t,∆(t))+ g(t,∆(t)), (24)

where f is defined as in (13), i.e.,

f (t,∆(t)) = −1cβ

Dc(θ −∆(t))B(t)∆(t),

and

g(t,∆(t)) = −1cβ

Dc(θ −∆(t))((A(t)TΥA(t)+ ΥA(t)TA(t))∆(t)

+ A(t)T(υy(t)+ ΥA(t)θ)).

Therefore, we can think of (24) as a perturbed version of (13),where g(t,∆(t)) is the perturbation term. Note that the origin

(∆ = 0) is not, in general, an equilibrium point of the perturbeddynamics (24). To see this, notice that f (t, 0)+g(t, 0) = g(t, 0) =0⇔ υy(t)+ΥA(t)θ ∈ ker(A(t)T), and it is unlikely that there existsa t? ≥ t0 such that the latter condition holds for all t ≥ t?. Hence,we shall only study boundedness and ultimate boundedness 4 ofthe solutions of (24).Let us begin by noting that the condition ∀t ≥ t0 ∆(t) ∈

θ + ] − c, c[n still holds for the perturbed dynamics (24). This is aresult of how we defined the HNN estimator, since the condition∀t ≥ t0 θ (t) ∈ ] − c, c[n was achieved without assumingany knowledge about y(t) and A(t), just by taking θ (t) = s(t)and α = c in (6) (see Section 3.2). Therefore, letting rmax =max∆∈∂(θ+]−c,c[n)‖∆‖∞, it is easy to see that

∀t ≥ t0 ‖∆(t)‖∞ < rmax,

i.e., that the solutions of (24) are globally uniformly bounded.Mindthat this is true regardless of the size of the perturbation, i.e., of thevalue of ‖g‖∞. However, the knowledge of an upper bound for thesize ‖g‖∞ of the perturbationmay lead to a tighter bound than rmaxfor the asymptotic values of the error size ‖∆(t)‖∞.Let us start by noting that V in (14) is a Lyapunov function

for the unperturbed dynamics (13), but that dVdt is in generalnot definite along the solutions of the perturbed dynamics (24).However, if condition (20) on ker(A) holds, then the next lemmaensures the existence of a different Lyapunov function V for (13),such that dVdt is negative definite along the solutions of (24) at leastwhen these are outside a certain hypercube centered at the origin.Furthermore, the size of this hypercube is a class K function ofan upper bound of ‖g‖∞. Hence, the smaller the perturbation, thesmaller the size of this hypercube.

Lemma 4. Assume that condition (20) on ker(A) holds. Then, there isa continuously differentiable Lyapunov function V : [t0,+∞[×(θ +] − c, c[n)→ R for the unperturbed dynamics (13) that satisfies theinequalities

γ1(‖∆(t)‖∞) ≤ V (t,∆(t)) ≤ γ2(‖∆(t)‖∞) (25)

∂ V∂t(t,∆(t))+

∂ V∂∆T

(t,∆(t))f (t,∆(t)) ≤ −γ3(‖∆(t)‖∞) (26)∥∥∥∥∥ ∂ V∂∆ (t,∆(t))∥∥∥∥∥∞

≤ γ4(‖∆(t)‖∞), (27)

where γi, i = 1, . . . , 4, are classK functions. Furthermore, along thesolutions of the perturbed dynamics (24), dVdt is such that

∀ ‖∆(t)‖∞ ≥ γ (δg)dVdt(t,∆(t)) ≤ −(1− λ)γ3(‖∆(t)‖∞) ≤ 0,

(28)

where γ is a class K function, δg is an upper bound of ‖g‖∞ andλ < 1 is some positive constant.

Proof. Let us begin by noting that ∂ f∂∆is bounded, given that

it is polynomial in ∆, ∆ belongs to the bounded set θ +] − c, c[n, and because B = ATA is bounded since so is A byassumption A2. Now, if condition (20) on ker(A) holds, then theorigin is a globally uniformly asymptotically stable equilibrium

4 Let f : [t0,+∞[×D → Rn be piecewise continuous in t and locally Lipschitzin x on [t0,+∞[×D, where D ⊂ Rn . The solutions of dxdt = f (t, x) are uniformlyultimately bounded with ultimate bound b if there exist positive constants b and c ,independent of t0 , and for every a ∈]0, c[, there is T = T (a, b) ≥ 0, independent oft0 , such that ‖x(t0)‖ ≤ a⇒ ∀t ≥ t0 + T ‖x(t)‖ ≤ b (Khalil, 2002).


point of the unperturbed dynamics (13) by Theorem 1. Using theboundedness of ∂ f

∂∆and this property of the origin, the existence

of V satisfying (25)–(27) follows from the converse Lyapunovtheorem corresponding to Theorem 4.16 and Lemma 4.5 in (Khalil,2002).Next, notice that ‖g‖∞ has an upper bound, say equal to δg ,

given that∆ belongs to the bounded set θ+] − c, c[n,A is boundedby assumption A2 and both υy and ΥA were also assumed to bebounded. In order to show (28), we use the boundedness of ‖g‖∞,inequalities (26)–(27) and the fact that γ4 is an increasing function.Hence, it can be seen that, along the solutions of the perturbeddynamics (24), dVdt is such that

dVdt(t,∆(t)) =

∂ V∂t(t,∆(t))+

∂ V∂∆T

(t,∆(t))d∆dt(t)

=∂ V∂t(t,∆(t))+

∂ V∂∆T

(t,∆(t))(f (t,∆(t))+ g(t,∆(t)))

≤ −γ3(‖∆(t)‖∞)+ δg γ4(‖∆(t)‖∞)≤ −(1−λ)γ3(‖∆(t)‖∞)− λγ3(‖∆(t)‖∞)+δg γ4(rc), λ ∈]0, 1[≤ −(1− λ)γ3(‖∆(t)‖∞)

if ‖∆(t)‖∞ ≥ γ (δg) , γ−13 (λ−1γ4(rc)δg), i.e., at least when thesolutions of (24) are outside the hypercube centered at the originand of radius γ (δg). The fact that γ is a class K function followsfrom Lemma 4.2 in (Khalil, 2002).

The next theorem,which is the secondmain result of this paper,presents some conditions under which the solutions of (24) areglobally uniformly ultimately bounded, with an ultimate boundthat is tighter than the bound rmax and whose size decreases withthe size of the perturbation.

Theorem 2. Consider the perturbed dynamics (24) and assume thatg is not the null function. Furthermore, suppose that condition (20) onker(A) holds. Then, a solution of (24) satisfies

∀t ∈ [t0, t0 + T [ ‖∆(t)‖∞ ≤ µ(‖∆0‖∞ , t − t0) (29)

and

∀t ≥ t0 + T ‖∆(t)‖∞ ≤ b(δg) (30)

for some classKL function µ and some finite T ≥ 0, being b a classK function of the upper bound δg of ‖g‖∞, defined as

b(δg) = γ−11 (γ2(γ (δg))),

with γ , γ1 and γ2 as in Lemma 4. Moreover, the ultimate bound b(δg)is less than rmax if and only if γ (δg) < γ−12 (γ1(rmax)).

Proof. Let us start by introducing the time-variant set

Ωt,ρ = ∆ ∈ θ + ] − c, c[n : V (t,∆) ≤ ρ.

Note thatΩt,ρ contains Sg = ] − γ (δg), γ (δg)[n⋂

(θ + ] − c, c[n)for all t ≥ t0 if ρ ≥ γ2(γ (δg)), since for an arbitrary point ∆ ∈ Sgone has

‖∆‖∞ < γ (δg) ⇔ γ2(‖∆‖∞) < γ2(γ (δg))

⇒ ∀t ≥ t0 V (t,∆) < γ2(γ (δg)),

where we used the fact that γ2 is strictly increasing together with(25). Furthermore, if ρ ≥ γ2(γ (δg)), then a solution of (24)belonging to Ωt?,ρ at some t? ≥ t0 belongs to Ωt,ρ for all t ≥ t?.To see this, assume that ρ ≥ γ2(γ (δg)). Begin by recalling that asolution of (24) always belongs to θ+] − c, c[n, and notice that, ateach t ≥ t?, one has either‖∆(t)‖∞ < γ (δg)or‖∆(t)‖∞ ≥ γ (δg).If ‖∆(t)‖∞ < γ (δg), then ∆(t) ∈ Sg , and therefore ∆(t) ∈ Ωt,ρ ,

given that Ωt,ρ ⊃ Sg . Now, assume that ‖∆(t)‖∞ ≥ γ (δg). Startby noting that we can write

V (t,∆(t)) =∫ t

t?

dVdτ(τ ,∆(τ ))dτ + V (t?,∆(t?))

because V is continuously differentiable. Using (28), it can be seenthat

V (t,∆(t)) ≤ V (t?,∆(t?)).

Hence, if∆(t?) ∈ Ωt?,ρ , i.e., if V (t?,∆(t?)) ≤ ρ, then∆(t) ∈ Ωt,ρ ,given that V (t,∆(t)) ≤ V (t?,∆(t?)) ≤ ρ.Following the previous analysis, consider the set Ωt,ρ1 with

ρ1 = γ2(γ (δg)). Next, we show that there exists a finite T ≥ 0such that

∀t ≥ t0 + T ∆(t) ∈ Ωt,ρ1 . (31)

If ∆(t0) = ∆0 ∈ Ωt0,ρ1 , then (31) holds with T = 0. Now, assumethat∆0 6∈ Ωt0,ρ1 and consider the setΩt,ρ2 with ρ2 = γ2(‖∆0‖∞).It is clear that ∆0 ∈ Ωt0,ρ2 , since V (t0,∆0) ≤ γ2(‖∆0‖∞) from(25). Furthermore, ρ2 = γ2(‖∆0‖∞) > γ2(γ (δg)) = ρ1, giventhat γ2 is strictly increasing and ‖∆0‖∞ > γ (δg), because byassumption ∆0 6∈ Ωt0,ρ1 andΩt0,ρ1 ⊃ Sg . Therefore, for all t ≥ t0,∆(t) ∈ Ωt,ρ2 and Ωt,ρ2 ⊃ Ωt,ρ1 . Noting that Ωt0,ρ2 − Ωt0,ρ1 isnonempty and V is continuous, there must be T > 0 such that∀t ∈ [t0, t0 + T [ ∆(t) ∈ Ωt,ρ2 − Ωt,ρ1 , where Ωt,ρ2 − Ωt,ρ1 =∆ ∈ θ + ] − c, c[n : ρ1 < V (t,∆) ≤ ρ2. Hence, it sufficesto prove that T cannot be infinite to conclude (31). Since ∀t ∈[t0, t0 + T [ ‖∆(t)‖∞ > γ (δg), from (28) and (25) one has

dVdt(t,∆(t)) ≤ −(1− λ)γ3(‖∆(t)‖∞)

≤ −(1− λ)γ3(γ−11 (V (t,∆(t))))

, −γ4(V (t,∆(t))), (32)

where γ4 is a class K function by Lemma 4.2 in (Khalil, 2002).Assume, without loss of generality5 that γ4 is locally Lipschitz.Moreover, consider the initial-value problem

dzdt(t) = −γ4(z(t)), z(t0) = V (t0,∆0). (33)

Applying (the comparison) Lemma 3.4 in (Khalil, 2002) to (32) and(33), it follows that ∀t ∈ [t0, t0 + T [ V (t,∆(t)) ≤ z(t). Butaccording to Lemma 4.4 in (Khalil, 2002), the unique solution of(33) is of the form z(t) = σ(V (t0,∆0), t − t0)with t ≥ t0, being σa classKL function. Therefore,

∀t ∈ [t0, t0 + T [ ρ1 < V (t,∆(t)) ≤ σ(V (t0,∆0), t − t0), (34)

where the left inequality results from the fact that ∀t ∈ [t0, t0 +T [ ∆(t) ∈ Ωt,ρ2 −Ωt,ρ1 . Hence, if T = ∞, then

ρ1 = limt→∞

ρ1 ≤ limt→∞

V (t,∆(t)) ≤ limt→∞

σ(V (t0,∆0), t − t0) = 0,

given that σ is a classKL function. However, ρ1 = γ2(γ (δg)) > 0,since δg > 0 by the assumption that g is not the null function andγ2 γ is positive definite because so are γ2 and γ . Thus, T must befinite.In the following, (29) and (30) are proved. Using (25) and

(34), being γ−11 increasing and σ class KL function, it can be

5 If γ4 is not locally Lipschitz, we can choose a locally Lipschitz classK functionγ5 such that γ4(r) ≥ γ5(r) over the domain of interest. Then, dVdt (t,∆(t))dt ≤−γ5(V (t,∆(t))), and we can continue the proof with γ5 instead of γ4 .


seen that, for all t ∈ [t0, t0 + T [,

‖∆(t)‖∞ ≤ γ−11 (V (t,∆(t)))

≤ γ−11 (σ (V (t0,∆0), t − t0))

≤ γ−11 (σ (γ2(‖∆0‖∞), t − t0)), µ(‖∆0‖∞ , t − t0),

where µ is a class KL function by Lemma 4.2 in (Khalil, 2002).This proves (29). In order to prove (30), start by noting that, for allt ≥ t0 + T ,

V (t,∆(t)) ≤ ρ1 ⇒ γ1(‖∆(t)‖∞) ≤ ρ1 ⇔ ‖∆(t)‖∞≤ γ−11 (ρ1) = γ

−11 (γ2(γ (δg))),

where we used (31) and (25). Now, just take b(δg) =

γ−11 (γ2(γ (δg))) and note that b is a class K function by Lemma4.2 in (Khalil, 2002).Finally, we end the proof of the theorem by noting that b(δg) <

rmax ⇔ γ (δg) < γ−12 (γ1(rmax)) given that γ1 and γ−12 are strictlyincreasing.

Remark 4. The robustness of the HNN estimator under perturba-tions can be investigated using the theorem just proved. It shouldbe stressed that this result considers a condition on ker(A), not onker(A), where A is the perturbed version of A. In fact, the conditionis the same of Theorem 1, which is used to investigate the stabilityof the HNN estimator under no perturbations. This guarantees thata prior experiment design based on the unperturbed matrix Awilllead to suitable estimation results when the HNN is applied underthe presence of perturbations, i.e., using the perturbed matrix A.

4. Experimental results

In this section, two case studies of on-line parameter estimationare considered. The first refers to a two-cart system, whose robustadaptive control is a benchmark problem. The second is relatedto a robotic arm with a flexible link. This system was consideredin (Atencia et al., 2004), where another Hopfield Neural Network(HNN) exhibiting a good performance was proposed, althoughwithout a suitable theoretical support (cf. end of Section 2).Both case studies illustrate the practical application of the theorypresented here. Furthermore, they include an analysis of theconvergence time of theHNNestimator and a comparison betweenthe performance of our method and the performance of two othermethods, namely recursive least-squares (RLS) and the Kalmanfilter for parameter estimation (Ljung, 1999). The implementationof these two methods is the same in both case studies. Hence, webriefly describe it in what follows.The RLSmethod and the Kalman filter were implemented using

QR factorization (Meyer, 2000) in order to guarantee numericalstability. Both methods require the initialization of a n× nmatrixexpressing the confidence in the initial guess θ (t0) of the nparameter vector θ . Here, we take this matrix equal to

( c2

)2 In,where c is the uncertainty parameter of the HNN and In denotesthe identity matrix of order n. Noting that a suitable choice forc and θ (t0) ensures that θ belongs to θ (t0) + ] − c, c[n, themotivation for this matrix initialization arises from the fact that itcorresponds to assuming that θ belongs to θ (t0) + ] − c, c[n with95% probability if θ is seen as a Gaussian random vector withmeanθ (t0) and covariance matrix

( c2

)2 In. The RLS method also requiresthe adjustment of a so-called forgetting factorλ. According to Ljung(1999), typical choices for λ are in the range between 0.98 and0.995. Here, we take λ = 0.99, because this leads, in general, tothe best results. In turn, the Kalman filter assumes the knowledgeof a m × m matrix corresponding to the covariance matrix of

the error perturbing the m equations in (1). We guarantee thebest performance for the Kalman filter by assuming the exactknowledge of this matrix. However, note that, in general, this isnot possible when the system data is real and not simulated asin here. As a final comment, note that the RLS method and theKalman filter are implemented as discretemethods, while theHNNestimator is continuous. In this context, we ensure that allmethodsuse the same system data by applying the two discrete methods tothe data considered in the numerical integration of the differentialequations (11) that define the dynamics of the HNN estimator.

4.1. Two carts system

Consider two carts joined by a spring and a damper asrepresented in Fig. 1. The application of physical laws yields themodelm1x1(t) = k(−x1(t)+ x2(t))+ b(−x1(t)+ x2(t))+ u(t)m2x2(t) = k(x1(t)− x2(t))+ b(x1(t)− x2(t)),

(35)

where xi, xi, xi denote respectively the displacement, velocity andacceleration of cart i, u the force applied to cart 1, and mi, k, bthe parameters corresponding respectively to the mass of cart i,the spring constant, and the damper constant. Assume that themassesmi are known, and that the constants k, b are the unknownparameters to estimate. Hence, (35) can be cast to the reduced form(1) as(m1x1(t)− u(t)m2x2(t)

)︸︷︷︸

y(t)

=

(−x1(t)+ x2(t) −x1(t)+ x2(t)x1(t)− x2(t) x1(t)− x2(t)

)︸︷︷︸

A(t)

(kb

)︸︷︷︸θ

.

A successful parameter estimation is guaranteed if condition(20) on ker(A) holds, i.e., for all nondegenerate interval I ⊂[0,+∞[,

⋂t∈I ker(A(t)) = 0. Start by noting that ∀t ≥

0 ker(A(t)) % 0, given that ∀t ≥ 0 det(A(t)) = 0; in fact,∀t ≥ 0 ker(A(t)) = θ ∈ R2 : (x1(t) − x2(t))θ1 + (x1(t) −x2(t))θ2 = 0. Therefore, it is easy to see that condition (20) holdsif for all nondegenerate interval I ⊂ [0,+∞[ and all r1, r2 ∈ R,there exists t ∈ I such that

x1(t)− x2(t) 6= r1er2t . (36)

In the following, we shall study x1 − x2, and show that the lattercondition holds for suitable initial values on the system statevariables xi, xi and suitable forces u.Let us start by seeing that any solution x1, x2 of (35) is also a

solution of

x1(t)− x2(t)+ ζ1(x1(t)− x2(t))+ ζ0(x1(t)− x2(t)) =1m1u(t),

where ζ1 = bm1+m2m1m2and ζ0 = km1+m2m1m2

. The application of the(unilateral) Laplace transform L to both sides of the precedingequality yields after some manipulation

X1(s)− X2(s) = F(s) (U(s)+m1((x1(0)− x2(0))(s+ ζ1)+ x1(0)− x2(0))) ,

Fig. 1. Two-cart system.


where Xi = L[xi], U = L[u], and

F(s) =1

m1(s2 + ζ1s+ ζ0).

Given the form of the transfer function F , it can be seen thatthe condition concerning (36) holds for general initial values onthe system state variables xi, xi and general forces u, like thoserepresented by Bohl functions.6 In this way, condition (20) holds,and a HNN as defined in (11) is able to carry out a successfulparameter estimation as long as assumptions A2 and A4 hold.First, note that, since ζi > 0, all poles of the transfer functionF have a negative real part; hence, the system is BIBO-stable,and assumption A2 holds for all bounded forces u. Second, notethat assumption A4 holds if the value chosen for the uncertaintyparameter c , which can be made arbitrarily large in the definitionof the HNN, is such that θ ∈ ] − c, c[2. Therefore, a bounded uand a large value for c should be considered so that a successfulparameter estimation can be carried out. Finally, mind that thevalue of the other time-invariant parameter of the HNN, β , doesnot affect the qualitative properties of the network. In whatfollows, we illustrate the performance of a HNN for a particularparameterization of the system.Let us assume, for instance, thatm1 = m2 = 2 [kg], k = 1

[ Nm

]and b = 0.1

[Nsm

]. Consider the HNN as defined in (11) with c = 5

and β = 0.01. Fig. 2 depicts the time-evolution of the estimatedparameterization produced by this HNN for two different initialconditions xi(0), xi(0) (columns) and two different forces u (rows):the first column refers to xi(0) = 0 [m], xi(0) = 0

[ms

], the second

to xi(0) = 0 [m], x1(0) = 1, x2(0) = 2[ms

], and the first row refers

to u(t) = e−t [N], the second to u(t) = sin(π t) [N]. In each of the

6 A Bohl function is a function whose Laplace transform is rational and strictlyproper.

four cases, the initial estimates of k and bwere randomly generatedaccording to the continuous uniform distribution U(]0, c[). Asexpected, the estimation process is well succeeded in all cases.In the remainder of this case study, we take for m1, m2, k and

b the values previously assumed. Furthermore, the force and theinitial conditions considered are respectively u(t) = e−t [N] andxi(0) = 0 [m], xi(0) = 0

[ms

], while the initial estimates of k and

b are respectively 1.5 and 0.6. Similar results and conclusions areobtained for other forces, initial conditions and initial estimates.Next, we analyze the way how the adjustable parameters c and

β of the HNN estimator influence the convergence time. The timeto convergence was defined as the time needed by the HNN to takethe estimation error norm ‖∆‖∞ to a value less than 5×10−2, sincethis allows one to get θ by rounding θ to one decimal place. Fig. 3shows the convergence time of the HNN estimator as a function ofparticular values of c and β , where c > ‖θ‖∞ = 1 and β > 0. Itcan be seen that, for each value of β , the lower the value of c , thelower the convergence time. Furthermore, for each value of c , thehigher the value of β , the lower the convergence time. This latterfinding is particularly important. To see this, start by noting thatwhen there is uncertainty about the location of θ , and hence aboutthe value of ‖θ‖∞, it is necessary to choose a high value for c toguarantee that assumption A4 holds, i.e., that c > ‖θ‖∞, whichmay cause an increase of the convergence time. However, this canbe compensated by choosing a high value for β , in order to stillachieve a suitable convergence time.In the following, we compare the performance of the HNN

estimatorwith the performance of the RLSmethod and the Kalmanfilter. The values of the adjustable parameters of the HNN are nowc = 2 and β = 0.2. The other two methods were implementedas described in the beginning of this section. The comparisonis done under mild and strong perturbations. These result fromerrors affecting the measurements of x1, x1 and x2, where the

Fig. 2. Time-evolution of the estimated parameterization produced by the HNN for two different sets of initial conditions IC and two different forces u, where IC1 = xi(0) =0 [m], xi(0) = 0 [ms ], IC2 = xi(0) = 0 [m], x1(0) = 1, x2(0) = 2 [

ms ], and u1(t) = e

−t[N], u2(t) = sin(π t) [N]. The solid and dashed lines represent respectively the

estimated values for k and b; the square and the circle represent respectively the actual values of k and b.


Fig. 3. Convergence time of the HNN estimator as a function of the adjustableparameters c and β .

measurement noise is assumed to be white7 and Gaussian withzero mean and a variance adjusted in order to achieve a desiredsignal-to-noise ratio (SNR). In the case of mild perturbations, x1and x2 have a SNR of 15 dB and x1 a SNR of 10 dB, whereas in thecase of strong perturbations these values are respectively 10 dBand 5 dB. The mean and the standard-deviation of the estimationerror norm ‖∆‖∞ were computed over 30 realizations of themeasurement noise. The time-evolution of these statistics of‖∆‖∞for the three methods is depicted in Figs. 4 and 5, respectivelyfor the cases of mild and strong perturbations. It can be seen thatthe HNN drives the mean estimation error norm to lower values.Furthermore, when the HNN is used, the estimation process issmoother, much more robust to perturbations and to an increasein the level of perturbations. In conclusion, our method exhibitsthe best performance and, contrary to the competing Kalman filter,does not assume the knowledge of the measurement noise model.

7 White noise is defined as a sequence of independent and identically distributedrandom variables (Ljung, 1999).

Fig. 4. Time-evolution of the estimation error norm ‖∆‖∞ in the case of mildperturbations,where the solid lines refer to theHNN, the dashed lines to the Kalmanfilter and the dotted lines to the RLS method. For all estimation methods, the innerline represents the mean value of ‖∆‖∞ and the outer lines correspond to thebounds determined by the standard-deviation.

Fig. 5. Time-evolution of the estimation error norm ‖∆‖∞ in the case of strongperturbations,where the solid lines refer to theHNN, the dashed lines to the Kalmanfilter and the dotted lines to the RLS method. For all estimation methods, the innerline represents the mean value of ‖∆‖∞ and the outer lines correspond to thebounds determined by the standard-deviation.

Finally, note that an important reason for using an on-linemethod is that we would like the estimation procedure to trackpossible variations in time of the system properties. Here, weconsider the case where the parameterization of the systemundergoes an abrupt change. When a variation like this occurs, theRLS method and the Kalman filter require its detection in orderto cope with it in a suitable manner (Ljung, 1999). With respectto the RLS method, the forgetting factor should be decreased toa small value for one sample to allow for a ‘‘cut off’’ of pastmeasurements. As for the Kalman filter, one should specify a n× ncovariance matrix describing the change in the n parameter vectorθ . Regarding the HNN estimator, practical experiments suggestthat there is no need to detect the variation and to accordinglyadjust the method with the aim of achieving a suitable tracking.This is illustrated in Figs. 6 and 7, where the HNN previouslyconsidered, with c = 2 and β = 0.2, proves to be able to track anabrupt parameter change at t = 3 [π s]. The new values of k andb are respectively 1.5

[ Nm

]and 0.05

[Nsm

]. Immediately after the

detection of the variation, the forgetting factor of the RLS methodis reduced by 50% for one sample and a covariance matrix equalto( c2

)2 In is specified in the case of the Kalman filter. It can beseen that, undermild and strong perturbations, the HNN drives themean tracking error to lower values. Furthermore, it is the mostrobust method.

4.2. Robotic arm with a flexible link

The dynamics of a robotic arm with a flexible link can bemodeled by looking at the single flexible link as a couple of rigidlinks joined by a spring. A schematic representation of the modelbased on this assumption is given in Fig. 8. Let xi, xi, xi denoterespectively the angular displacement, velocity and acceleration oflink i, u the torque applied by a motor to the arm, and li, k, m theparameters corresponding respectively to the length of link i, thespring constant, and the mass of a material object supported bythe arm at the end of link 2. Assuming that the constants α = ml22,γ = ml1l2, k are the unknownparameters to estimate, themodel ofthe arm is defined in the reduced form (1) by (Atencia et al., 2004)(u(t)0

)︸︷︷︸

y(t)

= A(t)

(αγk

)︸︷︷︸θ

,


Fig. 6. Time-evolution of the estimation error norm ‖∆‖∞ in the case of mildperturbations for an abrupt parameter change at t = 3 [π s], where the solid linesrefer to the HNN, the dashed lines to the Kalman filter and the dotted lines to theRLS method. For all estimation methods, the inner line represents the mean valueof ‖∆‖∞ and the outer lines correspond to the bounds determined by the standard-deviation.

Fig. 7. Time-evolution of the estimation error norm ‖∆‖∞ in the case of strongperturbations for an abrupt parameter change at t = 3 [π s], where the solid linesrefer to the HNN, the dashed lines to the Kalman filter and the dotted lines to theRLS method. For all estimation methods, the inner line represents the mean valueof ‖∆‖∞ and the outer lines correspond to the bounds determined by the standard-deviation.

where

A =

x1 + x2 cos(x2)(x1 +

12x2

)− sin(x2)x2

(x1 +

12x2

)0

x1 + x212(cos(x2)x1 + sin(x2)x21) −x2

.As in the previous case study, a successful parameter estimationis guaranteed if condition (20) on ker(A) holds, i.e., for allnondegenerate interval I ⊂ [0,+∞[,

⋂t∈I ker(A(t)) = 0. It

is clear that ∀t ≥ 0 ker(A(t)) % 0; in fact, given t ≥ 0,

θ ∈ ker(A(t))⇔A(t)11θ1 + A(t)12θ2 = 0(A(t)22 − A(t)12)θ2 + A(t)23θ3 = 0.

In particular, we have ker(A(t)) = span(v(t)) for v(t) =(v(t)1, v(t)2, v(t)3) = (−

A(t)12A(t)11

, 1,− A(t)22−A(t)12A(t)23

)T if A(t)11,A(t)23 6= 0. Hence, condition (20) holds if for all nondegenerateinterval I ⊂ [0,+∞[, the set I? = t ∈ I : A(t)11 = 0 ∨ A(t)23 =

Fig. 8. Schematic representation of a simplified model of a robotic arm with aflexible link.

0 is countable (which guarantees that dim(⋂

t∈I ker(A(t)))≤ 1,

i.e.,⋂t∈I ker(A(t)) is at most a line), and for all r1, r2 ∈ R, there

exist t1, t2 ∈ I \ I? such that

v(t1)1 6= r1 ∨ v(t2)3 6= r2,

(which guarantees that ker(A(t)), t ∈ I \ I? is not a fixedline). Minding this observation, we shall justify in what followsthe application of a HNN to the estimation of a particularparameterization θ under a set of initial conditions xi(0), xi(0)and a torque u, namely α = 0.75, γ = 1.5 [kgm2], k = 2

[ Nm

]under x1(0) = 0, x2(0) = π

8 [rad], xi(0) = 0 [ rads ] andu(t) = 1

2 (sin(2t) + sin(2t −2π3 ) + sin(5t)) [Nm] as in (Atencia

et al., 2004).Let I ⊂ [0,+∞[ be a nondegenerate interval. Fig. 9 shows that

the set I? = t ∈ I : A(t)11 = 0 ∨ A(t)23 = 0 is countablefor I ⊂ [0, 4π ]. Now, let r1 ∈ R. Fig. 10 shows that there exists

Fig. 9. Time-evolution of A(t)11 (solid line) and A(t)23 (dashed line).

Fig. 10. Time-evolution of v(t)1 .


Fig. 11. Time-evolution of the estimated parameterization produced by the HNN.The solid, dashed and dashed–dotted lines represent respectively the estimatedvalues for α, γ and k; the square, the circle and the triangle represent respectivelythe actual values of α, γ and k.

t1 ∈ I \ I? ⊂ [0, 4π ] such that v(t1)1 6= r1. Both conclusions canalso be drawn for I ⊂]4π,+∞[, and hence condition (20) holds. Inthis way, a HNN as defined in (11) is able to carry out a successfulparameter estimation as long as assumptions A2 andA4 hold. Sinceassumption A2 holds under the considered setup, we only need totake c > 2 so that assumption A4 holds. Fig. 11 depicts the time-evolution of the estimated parameterization produced by the HNNwith c = 3 and β = 0.01. The initial estimates of α, γ , k arerespectively 1, 2 [kgm2], 2.4

[ Nm

]as in (Atencia et al., 2004).

Next, we analyze the way how the adjustable parameters c andβ of the HNN estimator influence the convergence time. The timeto convergence was defined as the time needed by the HNN to takethe estimation error norm ‖∆‖∞ to a value less than 5×10−3, sincethis allows one to get θ by rounding θ to two decimal places. Fig. 12shows the convergence time of the HNN estimator as a function ofparticular values of c and β , where c > ‖θ‖∞ = 2 and β > 0. Itcan be seen that, for high (low) values of β , the higher (lower) thevalue of c , the lower the convergence time. Furthermore, for eachfixed value of c , the convergence time is a concave function of βand attains a minimum value at intermediate values of β . Hence,for a given value of c , the best convergence time can be achievedby adequately choosing β .In the following, we compare the performance of the HNN

estimatorwith the performance of the RLSmethod and the Kalmanfilter. The values of the adjustable parameters of the HNN are

Fig. 12. Convergence time of the HNN estimator as a function of the adjustableparameters c and β .

Fig. 13. Time-evolution of the estimation error norm ‖∆‖∞ in the case of mildperturbations,where the solid lines refer to theHNN, the dashed lines to the Kalmanfilter and the dotted lines to the RLS method. For all estimation methods, the innerline represents the mean value of ‖∆‖∞ and the outer lines correspond to thebounds determined by the standard-deviation.

Fig. 14. Time-evolution of the estimation error norm ‖∆‖∞ in the case of strongperturbations,where the solid lines refer to theHNN, the dashed lines to the Kalmanfilter and the dotted lines to the RLS method. For all estimation methods, the innerline represents the mean value of ‖∆‖∞ and the outer lines correspond to thebounds determined by the standard-deviation.

now c = 5 and β = 0.01. The other two methods wereimplemented as described in the beginning of this section. As in theprevious case study, the comparison is done undermild and strongperturbations. However, now these result from the simulation ofunmodeled dynamics, where all the components of y and A areaffected bywhite andGaussian noisewith zeromean. In the case ofmild perturbations, the noise variance is (0.05)2 for y and (0.001)2for A, whereas in the case of strong perturbations these valuesare respectively (0.1)2 and (0.002)2. The mean and the standard-deviation of the estimation error norm ‖∆‖∞ were computed over30 realizations of the noise. The time-evolution of these statisticsof ‖∆‖∞ for the three methods is depicted in Figs. 13 and 14,respectively for the cases of mild and strong perturbations. It canbe seen that the HNN drives the mean estimation error norm tolower values. Furthermore, when the HNN is used, the estimationprocess is much more robust to perturbations and to an increasein the level of perturbations. In conclusion, our method exhibitsthe best performance and, contrary to the competing Kalman filter,does not assume the knowledge of the noise model.


Fig. 15. Time-evolution of the estimation error norm ‖∆‖∞ in the case of mildperturbations for an abrupt parameter change at t = 4 [π s], where the solid linesrefer to the HNN, the dashed lines to the Kalman filter and the dotted lines to theRLS method. For all estimation methods, the inner line represents the mean valueof ‖∆‖∞ and the outer lines correspond to the bounds determined by the standard-deviation.

Fig. 16. Time-evolution of the estimation error norm ‖∆‖∞ in the case of strongperturbations for an abrupt parameter change at t = 4 [π s], where the solid linesrefer to the HNN, the dashed lines to the Kalman filter and the dotted lines to theRLS method. For all estimation methods, the inner line represents the mean valueof ‖∆‖∞ and the outer lines correspond to the bounds determined by the standard-deviation.

Finally, just like in the first case study, we consider a scenariowhere the parameterization of the system undergoes an abruptchange. Once again, note that, when a variation like this occurs,the RLSmethod and the Kalman filter require its detection in orderto cope with it in a suitable manner. Regarding the HNN estimator,practical experiments suggest, as before, that there is no need todetect the variation and to accordingly adjust the method with theaim of achieving a suitable tracking. This is illustrated in Figs. 15and 16, where the HNNpreviously considered, with c = 5 and β =0.01, proves to be able to track an abrupt parameter change at t =4 [π s]. The new values of α and γ are respectively 1.25 [kgm2]and 2.5 [kgm2]; the value of k ismaintained at 2

[ Nm

]. Immediately

after the detection of the variation, the forgetting factor of the RLSmethod is reduced by 50% for one sample and a covariance matrixequal to

( c2

)2 In is specified in the case of the Kalman filter. It canbe seen that, under mild and strong perturbations, the HNN drives

themean tracking error to lower values. Furthermore, it is themostrobust method.

5. Conclusions

In this paper, we considered the problem of using HopfieldNeural Networks (HNNs) for on-line parameter estimation.The only assumption we made on the model structure isthat it be linear in the parameters; this is in contrast withprevious literature, where additional assumptions are made.Linearity in the parameters is a common assumption in systemidentification: take, for instance, the ARX structure, widely usedto model system behavior. Hence, our results are of generalapplicability. We presented a weaker sufficient condition underwhich the estimation error converges to zero when there areno perturbations and to a bounded neighbourhood of zerootherwise. In order to illustrate the practical usefulness andapplication of our theoretical results, we considered two casestudies of on-line parameter estimation: the first related to atwo-cart system and the second to a robotic arm with a flexiblelink. Furthermore, it was shown in these case studies that theHNN estimator outperforms the recursive least-squares methodand the Kalman filter for parameter estimation. Ongoing workincludes an extension of our theoretical results to the case ofa time-variant parameterization. Furthermore, motivated by theexistence of hardware implementations of the original HNN (see,for example, (Bhama & Hassoun, 1990)), we plan to have in thefuture our own hardware implementation of the new HNN hereproposed.

Acknowledgments

The first author would like to thank the Fundação para a Ciênciae a Tecnologia (FCT) for the financial support during the courseof this project. This work was supported in part by FCT throughthe Unidade de Investigação Matemática e Aplicações (UIMA),Universidade de Aveiro, Portugal.

References

Abe, Shigeo (1989). Theories on the Hopfield neural networks. In Proceedings of theIEEE international joint conference on neural networks: Vol. 1 (pp. 557–564).

Atencia, Miguel, Joya, Gonzalo, & Sandoval, Francisco (2004). Parametric identifi-cation of robotic systems with stable time-varying Hopfield networks. NeuralComputing and Applications, 13, 270–280.

Barbarossa, Sergio, & Scutari, Gesualdo (2007). Decentralized maximum-likelihoodestimation for sensor networks composed of nonlinearly coupled dynamicalsystems. IEEE Transactions on Signal Processing , 55(7), 3456–3470.

Bhama, Satyendra, & Hassoun, M. H. (1990). Continuous Hopfield computationalnetwork: Hardware implementation. International Journal of Electronics, 69(5),603–612.

Di Claudio, E. D., Parisi, R., & Orlandi, G. (2000). Discriminative learning strategyfor efficient neural decision feedback equalizers. In Proceedings of the IEEEinternational symposium on circuits and systems: Vol. 4 (pp. 521–524).

Hopfield, J. J. (1984). Neurons with graded response have collective computationalproperties like those of two-state neurons. Proceedings of the National Academyof Sciences of the United States of America, 81, 3088–3092.

Hopfield, J. J., & Tank, D.W. (1985). Neural computation of decisions in optimizationproblems. Biological Cybernetics, 52, 141–152.

Hu, Zhenning, & Balakrishnan, S. N. (2005). Parameter estimation in nonlinearsystems using Hopfield neural networks. Journal of Aircraft , 42(1), 41–53.

Khalil, Hassan K. (2002). Nonlinear systems (3rd edition). New Jersey: Prentice Hall.Ljung, Lennart (1999). System identification — theory for the user (2nd edition). NewJersey: Prentice Hall.

Meyer, C. D. (2000).Matrix analysis and applied linear algebra. SIAM.Parisi, Raffaele, Di Claudio, Elio D., Orlandi, Gianni, & Rao, Bhaskar (1997). Fastadaptive digital equalization by recurrent neural networks. IEEE Transactionson Signal Processing , 45(11), 2731–2739.

Schizas, Ioannis D., Ribeiro, Alejandro, &Giannakis, Georgios B. (2008). Consensus inAd HocWSNs with noisy links — part I: Distributed estimation of deterministicsignals. IEEE Transactions on Signal Processing , 56(1), 350–364.

Vidyasagar, M. (2002). Nonlinear systems analysis. Philadelphia: SIAM.

Documents

Hopfield neural networks for on-line parameter estimation