Chapter 3 D Pl Parameter Identiﬁcation: P E … Identiﬁcation: ... of the parametric model SPM, DPM, B-SPM, ... 2007 The Society for Industrial and Applied Mathematics

Ioannou F2006/11/6page 25

�

�

�

�

�

�

�

�

Online Parameter Estimator

Controller u PlantDynamics

yInput

Command

Chapter 3

Parameter Identification:Continuous Time

3.1 IntroductionThe purpose of this chapter is to present the design, analysis, and simulation of a wide classof algorithms that can be used for online parameter identification of continuous-time plants.The online identification procedure involves the following three steps.

Step 1. Lump the unknown parameters in a vector θ∗ and express them in the formof the parametric model SPM, DPM, B-SPM, or B-DPM.

Step 2. Use the estimate θ of θ∗ to set up the estimation model that has the sameform as the parametric model. The difference between the outputs of the estimation andparametric models, referred to as the estimation error, reflects the distance of the estimatedparameters θ(t) from the unknown parameters θ∗ weighted by some signal vector. Theestimation error is used to drive the adaptive law that generates θ(t) online. The adaptivelaw is a differential equation of the form

θ = H(t)ε,

where ε is the estimation error that reflects the difference between θ(t) and θ∗ and H(t) is atime-varying gain vector that depends on measured signals. A wide class of adaptive lawswith different H(t) and ε may be developed using optimization techniques and Lyapunov-type stability arguments.

Step 3. Establish conditions that guarantee that θ(t) converges to θ∗ with time. Thisstep involves the design of the plant input so that the signal vector φ(t) in the parametricmodel is persistently exciting (a notion to be defined later on), i.e., it has certain propertiesthat guarantee that the measured signals that drive the adaptive law carry sufficient infor-mation about the unknown parameters. For example, for φ(t) = 0, we have z = θ∗T φ = 0,and the measured signals φ, z carry no information about θ∗. Similar arguments could bemade for φ that is orthogonal to θ∗ leading to z = 0 even though θ �= θ∗, etc.

We demonstrate the three design steps using the following example of a scalar plant.

25

Copyright (c) 2007 The Society for Industrial and Applied Mathematics

From: Adaptive Control Tutorial by Petros Ioannou and Baris Fidan


�

�

�

�

�

�

�

�

26 Chapter 3. Parameter Identification: Continuous Time

3.2 Example: One-Parameter CaseConsider the first-order plant model

y = a

s + 2u, (3.1)

where a is the only unknown parameter and y and u are the measured output and input ofthe system, respectively.

Step 1: Parametric Model We write (3.1) as

y = a1

s + 2u = auf , (3.2)

where uf = 1s+2u. Since u is available for measurement, uf is also available for measure-

ment. Therefore, (3.2) is in the form of the SPM

z = θ∗φ, (3.3)

where θ∗ = a and z = y, φ = uf are available for measurement.

Step 2: Parameter Identification Algorithm This step involves the development of anestimation model and an estimation error used to drive the adaptive law that generates theparameter estimates.

Estimation Model and Estimation Error The estimation model has the same formas the SPM with the exception that the unknown parameter θ∗ is replaced with its estimateat time t , denoted by θ(t), i.e.,

z = θ(t)φ, (3.4)

where z is the estimate of z based on the parameter estimate θ(t) at time t . It is obviousthat the difference between z and z is due to the difference between θ(t) and θ∗. As θ(t)

approaches θ∗ with time we would expect that z would approach z at the same time. (Notethat the reverse is not true, i.e., z(t) = z(t) does not imply that θ(t) = θ∗; see Problem 1.)Since θ∗ is unknown, the difference θ = θ(t) − θ∗ is not available for measurement.Therefore, the only signal that we can generate, using available measurements, that reflectsthe difference between θ(t) and θ∗ is the error signal

ε = z − z

m2s

, (3.5)

which we refer to as the estimation error. m2s ≥ 1 is a normalization signal1 designed to

guarantee that φ

msis bounded. This property of ms is used to establish the boundedness of

the estimated parameters even when φ is not guaranteed to be bounded. A straightforwardchoice for ms in this example is m2

s = 1 +αφ2, α > 0. If φ is bounded, we can take α = 0,1Note that any m2

s ≥ nonzero constant is adequate. The use of a lower bound 1 is without loss of generality.




�

�

�

�

�

�

�

�

3.2. Example: One-Parameter Case 27

i.e., m2s = 1. Using (3.4) in (3.5), we can express the estimation error as a function of the

parameter error θ = θ(t) − θ∗, i.e.,

ε = − θφ

m2s

. (3.6)

Equation (3.6) shows the relationship between the estimation error ε and the parametererror θ . It should be noted that ε cannot be generated using (3.6) because the parametererror θ is not available for measurement. Consequently, (3.6) can be used only for analysis.

Adaptive Law A wide class of adaptive laws or parameter estimators for generatingθ(t), the estimate of θ∗, can be developed using (3.4)–(3.6). The simplest one is obtainedby using the SPM (3.3) and the fact that φ is scalar to write

θ(t) = z(t)

φ(t), (3.7)

provided φ(t) �= 0. In practice, however, the effect of noise on the measurements of φ(t),especially when φ(t) is close to zero, may lead to erroneous parameter estimates. Anotherapproach is to update θ(t) in a direction that minimizes a certain cost of the estimation errorε. With this approach, θ(t) is adjusted in a direction that makes |ε| smaller and smalleruntil a minimum is reached at which |ε| = 0 and updating is terminated. As an example,consider the cost criterion

J (θ) = ε2m2s

2= (z − θφ)2

2m2s

, (3.8)

which we minimize with respect to θ using the gradient method to obtain

θ = −γ∇J (θ), (3.9)

where γ > 0 is a scaling constant or step size which we refer to as the adaptive gain andwhere ∇J (θ) is the gradient of J with respect to θ . In this scalar case,

∇J (θ) = dJ

dθ= − (z − θφ)

m2s

φ = −εφ,

which leads to the adaptive law

θ = γ εφ, θ(0) = θ0. (3.10)

Step 3: Stability and Parameter Convergence The adaptive law should guarantee thatthe parameter estimate θ(t) and the speed of adaptation θ are bounded and that the estimationerror ε gets smaller and smaller with time. These conditions still do not imply that θ(t) willget closer and closer to θ∗ with time unless some conditions are imposed on the vector φ(t),referred to as the regressor vector.




�

�

�

�

�

�

�

�


Let us start by using (3.6) and the fact that ˙θ = θ − θ∗ = θ (due to θ∗ being constant)

to express (3.10) as

˙θ = −γ

φ2

m2s

θ , θ (0) = θ0. (3.11)

This is a scalar linear time-varying differential equation whose solution is

θ (t) = e−γ

∫ t

0φ2(τ )

m2s (τ )

dτθ0, (3.12)

which implies that for ∫ t

0

φ2(τ )

m2s (τ )

dτ ≥ α0t (3.13)

and some α0 > 0, θ (t) converges to zero exponentially fast, which in turn implies thatθ(t) → θ∗ exponentially fast. It follows from (3.12) that θ(t) is always bounded for any

φ(t) and from (3.11) that θ (t) = ˙θ(t) is bounded due to φ(t)

ms(t)being bounded.

Another way to analyze (3.11) is to use a Lyapunov-like approach as follows: Weconsider the function

V = θ2

2γ.

Then

V = θ

γ

dθ

dt= − φ2

m2s

θ2 ≤ 0

or, using (3.6),

V = − φ2

m2s

θ2 = −ε2m2s ≤ 0. (3.14)

We should note that V = −ε2m2s ≤ 0 implies that V is a negative semidefinite

function in the space of θ . V in this case is not negative definite in the space of θ because itcan be equal to zero when θ is not zero. Consequently, if we apply the stability results of theAppendix, we can conclude that the equilibrium θe = 0 of (3.11) is uniformly stable (u.s.)and that the solution of (3.11) is uniformly bounded (u.b.). These results are not as useful,as our objective is asymptotic stability, which implies that the parameter error converges tozero. We can use the properties of V , V , however, to obtain additional properties for thesolution of (3.11) as follows.

Since V > 0 and V ≤ 0, it follows that (see the Appendix) V is bounded, whichimplies that θ is bounded and V converges to a constant, i.e., limt→∞ V (t) = V∞. Let usnow integrate both sides of (3.14). We have∫ t

0V (τ )dτ = −

∫ t

0ε2(τ )m2

s (τ )dτ

or

V (t) − V (0) = −∫ t

0ε2(τ )m2

s (τ )dτ . (3.15)




�

�

�

�

�

�

�

�

3.2. Example: One-Parameter Case 29

Since V (t) converges to the limit V∞ as t → ∞, it follows from (3.15) that∫ ∞

0ε2(τ )m2

s (τ )dτ = V (0) − V∞ < ∞,

i.e., εms is square integrable or εms ∈ L2. Since m2s ≥ 1, we have ε2 ≤ ε2m2

s , which

implies ε ∈ L2. From (3.6) we conclude that θφ

ms∈ L2 due to εms ∈ L2. Using (3.10), we

write

θ = γ εms

φ

ms

.

Since φ

msis bounded and εms ∈ L2 ∩L∞, it follows (see Problem 2) that θ ∈ L2 ∩L∞.

In summary, we have established that the adaptive law (3.10) guarantees that (i) θ ∈ L∞and (ii) ε, εms, θ ∈ L2 ∩ L∞ independent of the boundedness of φ. The L2 property of ε,εms , and θ indicates that the estimation error and the speed of adaptation θ are bounded inthe L2 sense, which in turn implies that their average value tends to zero with time.

It is desirable to also establish that ε, εms , and θ go to zero as t → ∞, as such aproperty will indicate the end of adaptation and the completion of learning. Such a propertycan be easily established when the input u is bounded (see Problem 3).

The above properties still do not guarantee that θ(t) → θ∗ as t → ∞. In orderto establish that θ(t) → θ∗ as t → ∞ exponentially fast, we need to restrict φ

msto be

persistently exciting (PE), i.e., to satisfy

1

T

∫ t+T

t

φ2(τ )

m2s

dτ ≥ α0 > 0 (3.16)

∀t ≥ 0 and some constants T , α0 > 0. The PE property of φ

msis guaranteed by choosing

the input u appropriately. Appropriate choices of u for this particular example include(i) u = c > 0, (ii) u = sin ωt for any ω �= 0 and any bounded input u that is not vanishingwith time. The condition (3.16) is necessary and sufficient for exponential convergence ofθ(t) → θ∗ (see Problem 4).

The PI algorithm for estimating the constant a in the plant (3.1) can now be summa-rized as

θ = γ εφ, θ(0) = θ0,

ε = (z − z)

m2s

, z = θφ,

z = y, φ = 1

s + 2u, m2

s = 1 + φ2,

where θ(t) is the estimate of the constant a in (3.1).The above analysis for the scalar example carries over to the vector case without any

significant modifications, as demonstrated in the next section. One important difference,however, is that in the case of a single parameter, convergence of the Lyapunov-like functionV to a constant implies that the estimated parameter converges to a constant. Such a resultcannot be established in the case of more than one parameter for the gradient algorithm.




�

�

�

�

�

�

�

�


3.3 Example: Two ParametersConsider the plant model

y = b

s + au, (3.17)

where a, b are unknown constants. Let us assume that y, y, u are available for measurement.We would like to generate online estimates for the parameters a, b.

Step 1: Parametric Model Since y, y are available for measurement, we can express(3.17) in the SPM form

z = θ∗T φ,

where z = y, θ∗ = [b, a]T , φ = [u,−y]T , and z, φ are available for measurement.

Step 2: Parameter Identification Algorithm

Estimation Modelz = θT φ,

where θ(t) is the estimate of θ∗ at time t .

Estimation Error

ε = z − z

m2s

= z − θT φ

m2s

, (3.18)

where ms is the normalizing signal such that φ

ms∈ L∞. A straightforward choice for ms is

m2s = 1 + αφT φ for any α > 0.

Adaptive Law We use the gradient method to minimize the cost,

J (θ) = ε2m2s

2= (z − θT φ)2

2m2s

= (z − θ1φ1 − θ2φ2)2

2m2s

,

where φ1 = u, φ2 = −y, and setθ = −�∇J,

where

∇J =[∂J

∂θ1,∂J

∂θ2

]T,

� = �T > 0 is the adaptive gain, and θ1, θ2 are the elements of θ = [θ1, θ2]T . Since

∂J

∂θ1= − (z − θT φ)

m2s

φ1 = −εφ1,∂J

∂θ2= − (z − θT φ)

m2s

φ2 = −εφ2,

we haveθ = �εφ, θ(0) = θ0, (3.19)

which is the adaptive law for updating θ(t) starting from some initial condition θ(0) = θ0.




�

�

�

�

�

�

�

�

3.4. Persistence of Excitation and Sufficiently Rich Inputs 31

Step 3: Stability and Parameter Convergence As in the previous example, the equationfor the parameter error θ = θ − θ∗ is obtained from (3.18), (3.19) by noting that

ε = z − θT φ

m2s

= θ∗T φ − θT φ

m2s

= − θ T φ

m2s

= −φT θ

m2s

(3.20)

and ˙θ = θ , i.e.,

˙θ = �φε = −�

φφT

m2s

θ . (3.21)

It is clear from (3.21) that the stability of the equilibrium θe = 0 will very muchdepend on the properties of the time-varying matrix −�φφT

m2s

, which in turn depends on theproperties of φ. For simplicity let us assume that the plant is stable, i.e., a > 0. If wechoose m2

s = 1, � = γ I for some γ > 0 and a constant input u = c0 > 0, then at steadystate y = c1 � c0b

a�= 0 and φ = [c0,−c1]T , giving

−�φφT

m2s

= −γ

[c2

0 −c0c1

−c0c1 c21

]� A,

i.e.,˙θ = Aθ,

where A is a constant matrix with eigenvalues 0, −γ (c20 + c2

1), which implies that theequilibrium θe = 0 is only marginally stable; i.e., θ is bounded but does not necessarilyconverge to 0 as t → ∞. The question that arises in this case is what properties of φ

guarantee that the equilibrium θe = 0 is exponentially stable. Given that

φ = H(s)u,

where for this example H(s) = [1,− bs+a

]T , the next question that comes up is how tochoose u to guarantee that φ has the appropriate properties that imply exponential stabilityfor the equilibrium θe = 0 of (3.21). Exponential stability for the equilibrium point θe = 0of (3.21) in turn implies that θ(t) converges to θ∗ exponentially fast. As demonstrated abovefor the two-parameter example, a constant input u = c0 > 0 does not guarantee exponentialstability. We answer the above questions in the following section.

3.4 Persistence of Excitation and Sufficiently Rich InputsWe start with the following definition.

Definition 3.4.1. The vector φ ∈ Rn is PE with level α0 if it satisfies∫ t+T0

t

φ(τ )φT (τ )dτ ≥ α0T0I (3.22)

for some α0 > 0, T0 > 0 and ∀t ≥ 0.




�

�

�

�

�

�

�

�


Since φφT is always positive semidefinite, the PE condition requires that its integralover any interval of time of length T0 is a positive definite matrix.

Definition 3.4.2. The signal u ∈ R is called sufficiently rich of order n if it contains at leastn2 distinct nonzero frequencies.

For example, u = ∑10i=1 sin ωit , where ωi �= ωj for i �= j is sufficiently rich of order

20. A more general definition of sufficiently rich signals and associated properties may befound in [95].

Let us consider the signal vector φ ∈ Rn generated as

φ = H(s)u, (3.23)

where u ∈ R and H(s) is a vector whose elements are transfer functions that are strictlyproper with stable poles.

Theorem 3.4.3. Consider (3.23) and assume that the complex vectorsH(jω1), . . . , H(jωn)

are linearly independent on the complex space Cn ∀ω1, ω2, . . . , ωn ∈ R, where ωi �= ωj

for i �= j . Then φ is PE if and only if u is sufficiently rich of order n.

Proof. The proof of Theorem 3.4.3 can be found in [56, 87].

We demonstrate the use of Theorem 3.4.3 for the example in section 3.3, where

φ = H(s)u

and

H(s) =[

1− b

s+a

].

In this case n = 2 and

H(jω1) =[

1− b

jω1+a

], H(jω2) =

[1

− bjω2+a

].

We can show that the matrix [H(jω1),H(jω2)] is nonsingular, which implies thatH(jω1), H(jω2) are linearly independent for any ω1, ω2 different than zero and ω1 �= ω2.

Let us chooseu = sin ω0t

for some ω0 �= 0 which is sufficiently rich of order 2. According to Theorem 3.4.3, thisinput should guarantee that φ is PE for the example in section 3.3. Ignoring the transientterms that converge to zero exponentially fast, we can show that at steady state

φ =[

sin ω0t

c0 sin(ω0t + ϕ0)

],

where

c0 = |b|√ω2

0 + a2, ϕ0 = arg

( −b

jω0 + a

).




�

�

�

�

�

�

�

�

3.4. Persistence of Excitation and Sufficiently Rich Inputs 33

Now

φφT =[

sin2 ω0t c0 sin ω0t sin(ω0t + ϕ0)

c0 sin ω0t sin(ω0t + ϕ0) c20 sin2(ω0t + ϕ)

]

and ∫ t+T0

t

φ(τ )φT (τ )dτ =[a11 a12

a12 a22

],

where

a11 = T0

2− sin 2ω0(t + T0) − sin 2ω0t

4ω0,

a12 = c0T0

2cosϕ0 + c0

sin ϕ0

4ω0(cos 2ω0t − cos 2ω0(t + T0)),

a22 = c20T0

2− c2

0sin 2(ω0(t + T0) + ϕ0) − sin 2(ω0t + ϕ0)

4ω0.

Choosing T0 = πω0

it follows that

a11 = T0

2, a12 = T0c0

2cosϕ0, a22 = T0

2c2

0

and ∫ t+T0

t

φ(τ )φT (τ )dτ = T0

2

[1 c0 cosϕ0

c0 cosϕ0 c20

],

which is a positive definite matrix. We can verify that for α0 = 12(1−cos2 ϕ0)c

20

1+c20

> 0,

∫ t+T0

t

φ(τ )φT (τ )dτ ≥ T0α0I,

which implies that φ is PE.Let us consider the plant model

y = b(s2 + 4)

(s + 5)3u,

where b is the only unknown parameter. A suitable parametric model for estimating b is

z = θ∗φ,

where

z = y, θ∗ = b, φ = s2 + 4

(s + 5)3u.

In this case φ ∈ R and H(s) = s2+4(s+5)3 ; i.e., n = 1 in Theorem 3.4.3. Let us use

Theorem 3.4.3 to choose a sufficiently rich signal u that guarantees φ to be PE. In this case,




�

�

�

�

�

�

�

�


according to the linear independence condition of Theorem 3.4.3 for the case of n = 1, weshould have

|H(jω0)| = 4 − ω20

(25 + ω20)

3/2�= 0

for any ω0 �= 0. This condition is clearly violated for ω0 = 2, and therefore a sufficientlyrich input of order 1 may not guarantee φ to be PE. Indeed, the input u = sin 2t leadsto y = 0, φ = 0 at steady state, which imply that the output y and regressor φ carry noinformation about the unknown parameter b. For this example u = sin ω0t will guaranteeφ to be PE, provided ω0 �= 2. Also, u = constant �= 0 and u = ∑m

i=1 sin ωit , m ≥ 2,will guarantee that φ is PE. In general, for each two unknown parameters we need at leasta single nonzero frequency to guarantee PE, provided of course that H(s) does not lose itslinear independence as demonstrated by the above example.

The two-parameter case example presented in section 3.3 leads to the differentialequation (3.21), which has exactly the same form as in the case of an arbitrary number ofparameters. In the following section, we consider the case where θ∗, φ are of arbitrarydimension and analyze the convergence properties of equations of the form (3.21).

3.5 Example: Vector CaseConsider the SISO system described by the I/O relation

y = G(s)u, G(s) = Z(s)

R(s)= kp

Z(s)

R(s), (3.24)

where u and y are the plant scalar input and output, respectively,

R(s) = sn + an−1sn−1 + · · · + a1s + a0,

Z(s) = bmsm + · · · + b1s + b0,

and kp = bm is the high-frequency gain.2

We can also express (3.24) as an nth-order differential equation given by

y(n) + an−1y(n−1) + · · · + a1y + a0y = bmu

(m) + · · · + b1u + b0u. (3.25)

Parametric Model Lumping all the parameters in (3.25) in the vector

θ∗ = [bm, . . . , b0, an−1, . . . , a0]T ,we can rewrite (3.25) as

y(n) = θ∗T [u(m), . . . , u,−y(n−1), . . . ,−y]T . (3.26)

Filtering each side of (3.26) with 1�(s)

, where �(s) = sn +λn−1sn−1 +· · ·+λ1s +λ0

is a monic Hurwitz polynomial, we obtain the parametric model

z = θ∗T φ, (3.27)

2At high frequencies or large s, the plant behaves as kp

sn−m ; therefore, kp is termed high-frequency gain.




�

�

�

�

�

�

�

�

3.5. Example: Vector Case 35

where

z = 1

�(s)y(n) = sn

�(s)y,

θ∗ = [bm, . . . , b0, an−1, . . . , a0]T ∈ Rn+m+1,

φ =[

sm

�(s)u, . . . ,

1

�(s)u,− sn−1

�(s)y, . . . ,− 1

�(s)y

]T.

If Z(s) is Hurwitz, a bilinear model can be obtained as follows: Consider the poly-nomials P(s) = pn−1s

n−1 + · · · + p1s + p0, Q(s) = sn−1 + qn−2sn−2 + · · · + q1s + q0

which satisfy the Diophantine equation (see the Appendix)

kpZ(s)P (s) + R(s)Q(s) = Z(s)�0(s),

where �0(s) is a monic Hurwitz polynomial of order 2n−m− 1. If each term in the aboveequation operates on the signal y, we obtain

kpZ(s)P (s)y + Q(s)R(s)y = Z(s)�0(s)y.

Substituting for R(s)y = kpZ(s)u, we obtain

kpZ(s)P (s)y + kpZ(s)Q(s)u = Z(s)�0(s)y.

Filtering each side with 1�0(s)Z(s)

, we obtain

y = kp

[P(s)

�0(s)y + Q(s)

�0(s)u

].

Letting

z = y,

ρ∗ = kp,

θ∗ = [qn−2, . . . , q0, pn−1, . . . , p0]T ,

φ =[sn−2

�0(s)u, . . . ,

1

�0(s)u,

sn−1

�0(s)y, . . . ,

1

�0(s)y

]T,

z0 = sn−1

�0(s)u,

we obtain the B-SPMz = ρ∗(θ∗T φ + z0). (3.28)

We should note that in this case θ∗ contains not the coefficients of the plant transferfunction but the coefficients of the polynomials P(s), Q(s). In certain adaptive controlsystems such as MRAC, the coefficients of P(s), Q(s) are the controller parameters, andparameterizations such as (3.28) allow the direct estimation of the controller parameters byprocessing the plant I/O measurements.




�

�

�

�

�

�

�

�


If some of the coefficients of the plant transfer function are known, then the dimensionof the vector θ∗ in the SPM (3.27) can be reduced. For example, if a1, a0, bm are known,then (3.27) can be rewritten as

z = θ∗T φ, (3.29)

where

z = sn + a1s + a0

�(s)y − bms

m

�(s)u,

θ∗ = [bm−1, . . . , b0, an−1, . . . , a2]T ,

φ =[sm−1

�(s)u, . . . ,

1

�(s)u,− sn−1

�(s)y, . . . ,− s2

�(s)y

]T.

Adaptive Law Let us consider the SPM (3.27). The objective is to process the signalsz(t) and φ(t) in order to generate an estimate θ(t) for θ∗ at each time t . This estimate maybe generated as

θ (t) = H(t)ε(t),

where H(t) is some gain vector that depends on φ(t) and ε(t) is the estimation error signalthat represents a measure of how far θ(t) is from θ∗. Different choices for H(t) and ε(t)

lead to a wide class of adaptive laws with, sometimes, different convergence properties, asdemonstrated in the following sections.

3.6 Gradient Algorithms Based on the Linear ModelThe gradient algorithm is developed by using the gradient method to minimize some ap-propriate functional J (θ). Different choices for J (θ) lead to different algorithms. Asin the scalar case, we start by defining the estimation model and estimation error for theSPM (3.27).

The estimate z of z is generated by the estimation model

z = θT φ, (3.30)

where θ(t) is the estimate of θ∗ at time t . The estimation error is constructed as

ε = z − z

m2s

= z − θT φ

m2s

, (3.31)

where m2s ≥ 1 is the normalizing signal designed to bound φ from above. The normalizing

signal often has the form m2s = 1 +n2

s , where ns ≥ 0 is referred to as the static normalizingsignal designed to guarantee that φ

msis bounded from above. Some straightforward choices

for ns include

n2s = αφT φ, α > 0,

or

n2s = φT Pφ, P = PT > 0,




�

�

�

�

�

�

�

�

3.6. Gradient Algorithms Based on the Linear Model 37

where α is a scalar and P is a matrix selected by the designer.The estimation error (3.31) and the estimation model (3.30) are common to several

algorithms that are generated in the following sections.

3.6.1 Gradient Algorithm with Instantaneous Cost Function

The cost function J (θ) is chosen as

J (θ) = ε2m2s

2= (z − θT φ)2

2m2s

, (3.32)

wherems is the normalizing signal given by (3.31). At each time t , J (θ) is a convex functionof θ and therefore has a global minimum. The gradient algorithm takes the form

θ = −�∇J, (3.33)

where � = �T > 0 is a design matrix referred to as the adaptive gain. Since ∇J =− (z−θT φ)φ

m2s

= −εφ, we have

θ = �εφ. (3.34)

The adaptive law (3.34) together with the estimation model (3.30), the estimationerror (3.31), and filtered signals z, φ defined in (3.29) constitute the gradient parameteridentification algorithm based on the instantaneous cost function whose stability propertiesare given by the following theorem.

Theorem 3.6.1. The gradient algorithm (3.34) guarantees the following:

(i) ε, εms, θ ∈ L2 ∩ L∞ and θ ∈ L∞.

(ii) If φ

msis PE, i.e.,

∫ t+T0

t

φφT

m2sdτ > α0T0I ∀t ≥ 0 and for some T0, α0 > 0, then θ(t) →

θ∗ exponentially fast. In addition,

(θ(t) − θ∗)T �−1(θ(t) − θ∗) ≤ (1 − γ1)n(θ(0) − θ∗)T �−1(θ(0) − θ∗),

where 0 ≤ t ≤ nT0, n = 0, 1, 2, . . . , and

γ1 = 2α0T0λmin(�)

2 + β4λ2max(�)T

20

, β = supt

∣∣∣∣ φms

∣∣∣∣ .(iii) If the plant model (3.24) has stable poles and no zero-pole cancellations and the input

u is sufficiently rich of order n + m + 1, i.e., it consists of at least n+m+12 distinct

frequencies, then φ, φ

msare PE. Furthermore, |θ(t)− θ∗|, ε, εms , θ converge to zero

exponentially fast.

Proof. (i) Since θ∗ is constant, ˙θ = θ and from (3.34) we have

˙θ = �εφ = −�

φφT

m2s

θ . (3.35)




�

�

�

�

�

�

�

�


We choose the Lyapunov-like function

V (θ) = θ T �−1θ

2.

Then along the solution of (3.35), we have

V = θ T φε = −ε2m2s ≤ 0, (3.36)

where the second equality is obtained by substituting θ T φ = −εm2s from (3.31). Since

V > 0 and V ≤ 0, it follows that V (t) has a limit, i.e.,

limt→∞V (θ(t)) = V∞ < ∞,

andV, θ ∈ L∞, which, together with (3.31), imply that ε, εms ∈ L∞. In addition, it followsfrom (3.36) that ∫ ∞

0ε2m2

s dτ ≤ V (θ(0)) − V∞,

from which we establish that εms ∈ L2 and hence ε ∈ L2 (due to m2s = 1 + n2

s ). Now from(3.35) we have

| ˙θ | = |θ | ≤ ‖�‖|εms | |φ|ms

,

which together with |φ|ms

∈ L∞ and |εms | ∈ L2 imply that θ ∈ L2 ∩ L∞, and the proof of (i)is complete.

The proof of parts (ii) and (iii) is longer and is presented in the web resource[94].

Comment 3.6.2 The rate of convergence of θ to θ∗ can be improved if we choose the designparameters so that 1 − γ1 is as small as possible or, alternatively, γ1 ∈ (0, 1) is as close to 1as possible. Examining the expression for γ1, it is clear that the constants β, α0, T0 dependon each other and on φ. The only free design parameter is the adaptive gain matrix �. Ifwe choose � = λI , then the value of λ that maximizes γ1 is

λ∗ =(

2

2 − β4T 20

)1/2

,

provided β4T 20 < 2. For λ > λ∗ or λ < λ∗, the expression for γ1 suggests that the rate

of convergence is slower. This dependence of the rate of convergence on the value of theadaptive gain � is often observed in simulations; i.e., very small or very large values of �lead to slower convergence rates. In general the convergence rate depends on the signalinput and filters used in addition to � in a way that is not understood quantitatively.

Comment 3.6.3 Properties (i) and (ii) of Theorem 3.6.1 are independent of the boundednessof the regressor φ. Additional properties may be obtained if we make further assumptionsabout φ. For example, if φ, φ ∈ L∞, then we can show that ε, εms, θ → 0 as t → ∞ (seeProblem 6).




�

�

�

�

�

�

�

�


Comment 3.6.4 In the proof of Theorem 3.6.1(i)–(ii), we established that limt→∞ V (t) =V∞, where V∞ is a constant. This implies that

limt→∞V (θ(t)) = lim

t→∞θ T (t)�−1θ (t)

2= V∞.

We cannot conclude, however, that θ = θ − θ∗ converges to a constant vector. Forexample, take � = I , θ (t) = [sin t, cos t]T . Then

θ T θ

2= sin2 t + cos2 t

2= 1

2,

and θ (t) = [sin t, cos t]T does not have a limit.

Example 3.6.5 Consider the nonlinear system

x = af (x) + bg(x)u,

where a, b are unknown constants, f (x), g(x) are known continuous functions of x, andx, u are available for measurement. We want to estimate a, b online. We first obtain aparametric model in the form of an SPM by filtering each side with 1

s+λfor some λ > 0, i.e.,

s

s + λx = a

1

s + λf (x) + b

1

s + λg(x)u.

Then, for

z = s

s + λx, θ∗ = [a, b]T , φ = 1

s + λ[f (x), g(x)u]T ,

we have

z = θ∗T φ.

The gradient algorithm (3.34),

θ = �εφ,

ε = z − θT φ

m2s

, m2s = 1 + n2

s , n2s = φT φ,

can be used to generate θ(t) = [a(t), b(t)]T online, where a(t), b(t) are the online esti-mates of a, b, respectively. While this adaptive law guarantees properties (i) and (ii) ofTheorem 3.6.1 independent of φ, parameter convergence of θ(t) to θ∗ requires φ

msto be PE.

The important question that arises in this example is how to choose the plant input u so thatφ

msis PE. Since the plant is nonlinear, the choice of u that makes φ

msPE depends on the

form of the nonlinear functions f (x), g(x) and it is not easy, if possible at all, to establishconditions similar to those in the LTI case for a general class of nonlinearities.




�

�

�

�

�

�

�

�


Example 3.6.6 Consider the dynamics of a hard-disk drive servo system [96] given by

y = kp

s2(u + d),

where y is the position error of the head relative to the center of the track, kp is a knownconstant, and

d = A1 sin(ω1t + ϕ1) + A2 sin(ω2t + ϕ2)

is a disturbance that is due to higher-order harmonics that arise during rotation of the diskdrive. In this case, ω1, ω2 are the known harmonics that have a dominant effect and Ai , ϕi ,i = 1, 2, are the unknown amplitudes and phases. We want to estimate d in an effort tonullify its effect using the control input u.

Using sin(a + b) = sin a cos b + cos a sin b, we can express d as

d = θ∗1 sin ω1t + θ∗

2 cosω1t + θ∗3 sin ω2t + θ∗

4 cosω2t,

whereθ∗

1 = A1 cosϕ1, θ∗2 = A1 sin ϕ1,

θ∗3 = A2 cosϕ2, θ∗

4 = A2 sin ϕ2(3.37)

are the unknown parameters. We first obtain a parametric model for

θ∗ = [θ∗1 , θ

∗2 , θ

∗3 , θ

∗4 ]T .

We haves2y = kpu + kpθ

∗T ψ,

whereψ(t) = [sin ω1t, cosω1t, sin ω2t, cosω2t]T .

Filtering each side with 1�(s)

, where �(s) = (s + λ1)(s + λ2) and λ1, λ2 > 0 aredesign constants, we obtain the SPM

z = θ∗T φ,

where

z = s2

�(s)y − kp

1

�(s)u,

φ = kp1

�(s)ψ(t) = kp

1

�(s)[sin ω1t, cosω1t, sin ω2t, cosω2t]T .

Therefore, the adaptive law

θ = �εφ,

ε = z − θT φ

m2s

, m2s = 1 + n2

s , n2s = αφT φ,

where � = �T > 0 is a 4 × 4 constant matrix, may be used to generate θ(t), the onlineestimate of θ∗. In this case, φ ∈ L∞ and therefore we can take α = 0, i.e., m2

s = 1. For



Ioannou2006/11/6page 41

�

�

�

�

�

�

�

�


ω1 �= ω2, we can establish that φ is PE and therefore θ(t) → θ∗ exponentially fast. Theonline estimate of the amplitude and phase can be computed using (3.37) as follows:

tan ϕ1(t) = θ2(t)

θ1(t), tan ϕ2(t) = θ4(t)

θ3(t),

A1(t) =√θ2

1 (t) + θ22 (t), A2(t) =

√θ2

3 (t) + θ24 (t),

provided of course that θ1(t) �= 0, θ3(t) �= 0. The estimated disturbance

d(t) = A1(t) sin(ω1t + ϕ1(t)) + A2(t) sin(ω2t + ϕ2(t))

can then be generated and used by the controller to cancel the effect of the actual distur-bance d .

3.6.2 Gradient Algorithm with Integral Cost Function

The cost function J (θ) is chosen as

J (θ) = 1

2

∫ t

0e−β(t−τ)ε2(t, τ )m2

s (τ )dτ ,

where β > 0 is a design constant acting as a forgetting factor and

ε(t, τ ) = z(τ ) − θT (t)φ(τ )

m2s (τ )

, ε(t, t) = ε, τ ≤ t,

is the estimation error that depends on the estimate of θ at time t and on the values of thesignals at τ ≤ t . The cost penalizes all past errors between z(τ ) and z(τ ) = θT (t)φ(τ ),τ ≤ t , obtained by using the current estimate of θ at time t with past measurements ofz(τ ) and φ(τ). The forgetting factor e−β(t−τ) is used to put more weight on recent data bydiscounting the earlier ones. It is clear that J (θ) is a convex function of θ at each time t andtherefore has a global minimum. Since θ(t) does not depend on τ , the gradient of J withrespect to θ is easy to calculate despite the presence of the integral. Applying the gradientmethod, we have

θ = −�∇J,

where

∇J = −∫ t

0e−β(t−τ) z(τ ) − θT (t)φ(τ )

m2s (τ )

φ(τ)dτ .

This can be implemented as (see Problem 7)

θ = −�(R(t)θ + Q(t)), θ(0) = θ0,

R = −βR + φφT

m2s

, R(0) = 0,

Q = −βQ − zφ

m2s

, Q(0) = 0,




�

�

�

�

�

�

�

�


where R ∈ Rn×n, Q ∈ Rn×1; � = �T > 0 is the adaptive gain; n is the dimension of thevector θ∗; and ms is the normalizing signal defined in (3.31).

Theorem 3.6.7. The gradient algorithm with integral cost function guarantees that

(i) ε, εms, θ ∈ L2 ∩ L∞ and θ ∈ L∞.

(ii) limt→∞ |θ (t)| = 0.

(iii) If φ

msis PE, then θ(t) → θ∗ exponentially fast. Furthermore, for � = γ I , the rate of

convergence increases with γ .

(iv) If u is sufficiently rich of order n + m + 1, i.e., it consists of at least n+m+12 distinct

frequencies, and the plant is stable and has no zero-pole cancellations, then φ,φ

ms

are PE and θ(t) → θ∗ exponentially fast.

Proof. The proof is presented in the web resource [94].

Theorem 3.6.7 indicates that the rate of parameter convergence increases with in-creasing adaptive gain. Simulations demonstrate that the gradient algorithm based on theintegral cost gives better convergence properties than the gradient algorithm based on theinstantaneous cost. The gradient algorithm based on the integral cost has similarities withthe least-squares (LS) algorithms to be developed in the next section.

3.7 Least-Squares AlgorithmsThe LS method dates back to the eighteenth century, when Gauss used it to determine theorbits of planets. The basic idea behind LS is fitting a mathematical model to a sequence ofobserved data by minimizing the sum of the squares of the difference between the observedand computed data. In doing so, any noise or inaccuracies in the observed data are expectedto have less effect on the accuracy of the mathematical model.

The LS method has been widely used in parameter estimation both in recursive andnonrecursive forms mainly for discrete-time systems [46, 47, 77, 97, 98]. The method issimple to apply and analyze in the case where the unknown parameters appear in a linearform, such as in the linear SPM

z = θ∗T φ. (3.38)

We illustrate the use and properties of LS by considering the simple scalar example

z = θ∗φ + dn,

where z, θ∗, φ ∈ R, φ ∈ L∞, and dn is a noise disturbance whose average value goes tozero as t → ∞, i.e.,

limt→∞

1

t

∫ t

0dn(τ )dτ = 0.




�

�

�

�

�

�

�

�

3.7. Least-Squares Algorithms 43

In practice, dn may be due to sensor noise or external sources, etc. We examine thefollowing estimation problem: Given the measurements of z(τ ), φ(τ) for 0 ≤ τ < t , finda “good’’ estimate θ(t) of θ∗ at time t . One possible solution is to calculate θ(t) as

θ(t) = z(τ )

φ(τ)= θ∗ + dn(τ )

φ(τ)(3.39)

by using the measurements of z(τ ), φ(τ) at some τ < t for which φ(τ) �= 0. Because ofthe noise disturbance, however, such an estimate may be far off from θ∗. For example, atthe particular time τ at which we measured z and φ, the effect of dn(τ ) may be significant,leading to an erroneous estimate for θ(t) generated by (3.39).

A more intelligent approach is to choose the estimate θ(t) at time t to be the one thatminimizes the square of all the errors that result from the mismatch of z(τ ) − θ(t)φ(τ )

for 0 ≤ τ ≤ t . Hence the estimation problem above becomes the following LS problem:Minimize the cost

J (θ) = 1

2

∫ t

0|z(τ ) − θ(t)φ(τ )|2dτ (3.40)

w.r.t. θ(t) at any given time t . The cost J (θ) penalizes all the past errors from τ = 0 to t thatare due to θ(t) �= θ∗. Since J (θ) is a convex function over R at each time t , its minimumsatisfies

∇J (θ) = −∫ t

0z(τ )φ(τ)dτ + θ(t)

∫ t

0φ2(τ )dτ = 0,

which gives the LS estimate

θ(t) =(∫ t

0φ2(τ )dτ

)−1 ∫ t

0z(τ )φ(τ)dτ ,

provided of course that the inverse exists. The LS method considers all past data in an effortto provide a good estimate for θ∗ in the presence of noise dn. For example, when φ(t) = 1,∀t ≥ 0, we have

limt→∞ θ(t) = lim

t→∞1

t

∫ t

0z(τ )dτ = θ∗ + lim

t→∞1

t

∫ t

0dn(τ )dτ = θ∗;

i.e., θ(t) converges to the exact parameter value despite the presence of the noise disturb-ance dn.

Let us now extend this problem to the linear model (3.38). As in section 3.6, theestimate z of z and the normalized estimation are generated as

z = θT φ, e = z − z

m2s

= z − θT φ

m2s

,

where θ(t) is the estimate of θ∗ at time t , and m2s = 1 + n2

s is designed to guaranteeφ

ms∈ L∞. Below we present different versions of the LS algorithm, which correspond to

different choices of the LS cost J (θ).




�

�

�

�

�

�

�

�


3.7.1 Recursive LS Algorithm with Forgetting Factor

Consider the function

J (θ) = 1

2

∫ t

0e−β(t−τ) [z(τ ) − θT (t)φ(τ )]2

m2s (τ )

dτ + 1

2e−βt (θ − θ0)

TQ0(θ − θ0), (3.41)

where Q0 = QT0 > 0, β ≥ 0 are design constants and θ0 = θ(0) is the initial parameter

estimate. This cost function is a generalization of (3.40) to include possible discountingof past data and a penalty on the initial error between the estimate θ0 and θ∗. Since z

ms,

φ

ms∈ L∞, J (θ) is a convex function of θ over Rn at each time t . Hence, any local minimum

is also global and satisfies∇J (θ(t)) = 0 ∀t ≥ 0.

The LS algorithm for generating θ(t), the estimate of θ∗, in (3.38) is therefore obtainedby solving

∇J (θ) = e−βtQ0(θ(t) − θ0) −∫ t

0e−β(t−τ) z(τ ) − θT (t)φ(τ )

m2s (τ )

φ(τ)dτ = 0 (3.42)

for θ(t), which yields the nonrecursive LS algorithm

θ(t) = P(t)

[e−βtQ0θ0 +

∫ t

0e−β(t−τ) z(τ )φ(τ)

m2s (τ )

dτ

], (3.43)

where

P(t) =[e−βtQ0 +

∫ t

0e−β(t−τ) φ(τ )φ

T (τ )

m2s (τ )

dτ

]−1

(3.44)

is the so-called covariance matrix. BecauseQ0 = QT0 > 0 andφφT is positive semidefinite,

P(t) exists at each time t . Using the identity

d

dtPP−1 = P P−1 + P

d

dtP−1 = 0

and εm2s = z − θT φ, and differentiating θ(t) w.r.t. t , we obtain the recursive LS algorithm

with forgetting factor

θ = Pεφ, θ(0) = θ0,

P = βP − PφφT

m2s

P , P (0) = P0 = Q−10 .

(3.45)

The stability properties of (3.45) depend on the value of the forgetting factor β, asdiscussed in the following sections. If β = 0, the algorithm becomes the pure LS algorithmdiscussed and analyzed in section 3.7.2. When β > 0, stability cannot be established unlessφ

msis PE. In this case (3.45) is modified, leading to a different algorithm discussed and

analyzed in section 3.7.3.The following theorem establishes the stability and convergence of θ to θ∗ of the

algorithm (3.45) in the case where φ

msis PE.




�

�

�

�

�

�

�

�


Theorem 3.7.1. If φ

msis PE, then the recursive LS algorithm with forgetting factor (3.45)

guarantees that P, P−1 ∈ L∞ and that θ(t) → θ∗ as t → ∞. The convergence ofθ(t) → θ∗ is exponential when β > 0.

Proof. The proof is given in [56] and in the web resource [94].

Since the adaptive law (3.45) could be used in adaptive control where the PE propertyof φ

mscannot be guaranteed, it is of interest to examine the properties of (3.45) in the absence

of PE. In this case, (3.45) is modified in order to avoid certain undesirable phenomena, asdiscussed in the following sections.

3.7.2 Pure LS Algorithm

When β = 0 in (3.41), the algorithm (3.45) reduces to

θ = Pεφ, θ(0) = θ0,

P = −PφφT

m2s

P , P (0) = P0,(3.46)

which is referred to as the pure LS algorithm.

Theorem 3.7.2. The pure LS algorithm (3.46) guarantees that

(i) ε, εms, θ ∈ L2 ∩ L∞ and θ, P ∈ L∞.

(ii) limt→∞ θ(t) = θ , where θ is a constant vector.

(iii) If φ

msis PE, then θ(t) → θ∗ as t → ∞.

(iv) If (3.38) is the SPM for the plant (3.24) with stable poles and no zero-pole cancel-lations, and u is sufficiently rich of order n + m + 1, i.e., consists of at least n+m+1

2

distinct frequencies, then φ,φ

msare PE and therefore θ(t) → θ∗ as t → ∞.

Proof. From (3.46) we have that P ≤ 0, i.e., P(t) ≤ P0. Because P(t) is nonincreasingand bounded from below (i.e., P(t) = PT (t) ≥ 0 ∀t ≥ 0) it has a limit, i.e.,

limt→∞P(t) = P ,

where P = P T ≥ 0 is a constant matrix. Let us now consider the identity

d

dt(P−1θ ) = −P−1P P−1θ + P−1 ˙

θ = φφT θ

m2s

+ εφ = 0,

where the last two equalities are obtained using θ = ˙θ , d

dtP−1 = −P−1P P−1, and ε =

− θ T φ

m2s

= −φT θ

m2s

. Hence, P−1(t)θ (t) = P−10 θ (0) and therefore θ (t) = P(t)P−1

0 θ (0) and

limt→∞ θ (t) = P P−10 θ (0), which implies that limt→∞ θ(t) = θ∗ + P P−1

0 θ (0) � θ .




�

�

�

�

�

�

�

�


Because P(t) ≤ P0 and θ (t) = P(t)P−10 θ (0) we have θ, θ ∈ L∞, which, together

with φ

ms∈ L∞, implies that εms = − θ T φ

msand ε, εms ∈ L∞. Let us now consider the

function

V (θ, t) = θ T P−1(t)θ

2.

The time derivative V of V along the solution of (3.46) is given by

V = εθT φ + θ T φφT θ

2m2s

= −ε2m2s + ε2m2

s

2= −ε2m2

s

2≤ 0,

which implies that V ∈ L∞, εms ∈ L2, and therefore ε ∈ L2. From (3.46) we have

|θ | ≤ ‖P ‖ |φ|ms

|εms |.

Since P,φ

ms, εms ∈ L∞, and εms ∈ L2, we have θ ∈ L∞ ∩ L2, which completes the

proof for (i) and (ii). The proofs of (iii) and (iv) are included in the proofs of Theorems 3.7.1and 3.6.1, respectively.

The pure LS algorithm guarantees that the parameters converge to some constant θwithout having to put any restriction on the regressor φ. If φ

ms, however, is PE, then θ = θ∗.

Convergence of the estimated parameters to constant values is a unique property of the pureLS algorithm. One of the drawbacks of the pure LS algorithm is that the covariance matrixP may become arbitrarily small and slow down adaptation in some directions. This is dueto the fact that

d(P−1)

dt= φφT

m2s

≥ 0,

which implies that P−1 may grow without bound, which in turn implies that P may reducetowards zero. This is the so-called covariance wind-up problem. Another drawback of thepure LS algorithm is that parameter convergence cannot be guaranteed to be exponential.

Example 3.7.3 In order to get some understanding of the properties of the pure LS algo-rithm, let us consider the scalar SPM

z = θ∗φ,

where z, θ∗, φ ∈ R. Let us assume that φ ∈ L∞. Then the pure LS algorithm is given by

θ = pεφ, θ(0) = θ0,

p = −p2φ2, p(0) = p0 > 0,

ε = z − θφ = −θφ.

Let us also take φ = 1, which is PE, for this example. Then we can show by solvingthe differential equation via integration that

p(t) = p0

1 + p0t




�

�

�

�

�

�

�

�


and θ (t) = θ (0)1+p0t

, i.e.,

θ(t) = θ∗ + θ(0) − θ∗

1 + p0t.

It is clear that as t → ∞, p(t) → 0, leading to the so-called covariance wind-upproblem. Since φ = 1 is PE, however, θ(t) → θ∗ as t → ∞ with a rate of 1

t(not

exponential) as predicted by Theorem 3.7.2. Even though θ(t) → θ∗, the covariance wind-up problem may still pose a problem in the case where θ∗ changes to some other value aftersome time. If at that instance p(t) ∼= 0, leading to θ ∼= 0, no adaptation will take place andθ(t) may not reach the new θ∗.

For the same example, consider φ(t) = 11+t

, which is not PE since

∫ t+T

t

φ2(τ )dτ =∫ t+T

t

1

(1 + τ)2dτ = 1

1 + t− 1

1 + t + T

goes to zero as t → ∞, i.e., it has zero level of excitation. In this case, we can show that

p(t) = p0(1 + t)

1 + (1 + p0)t,

θ(t) = θ∗ + (θ(0) − θ∗)1 + t

1 + (1 + p0)t

by solving the differential equations above. It is clear that p(t) → p0

1+p0and θ(t) →

p0θ∗+θ(0)

1+p0as t → ∞; i.e., θ(t) converges to a constant but not to θ∗ due to lack of PE. In this

case p(t) converges to a constant and no covariance wind-up problem arises.

3.7.3 Modified LS Algorithms

One way to avoid the covariance wind-up problem is to modify the pure LS algorithm usinga covariance resetting modification to obtain

θ = Pεφ, θ(0) = θ0,

P = −PφφT

m2s

P , P (t+r ) = P0 = ρ0I,

m2s = 1 + n2

s , n2s = αφT φ, α > 0,

(3.47)

where t+r is the time at which λmin(P (t)) ≤ ρ1 and ρ0 > ρ1 > 0 are some design scalars.Due to covariance resetting, P(t) ≥ ρ1I ∀t ≥ 0. Therefore, P is guaranteed to be positivedefinite for all t ≥ 0. In fact, the pure LS algorithm with covariance resetting can be viewedas a gradient algorithm with time-varying adaptive gainP , and its properties are very similarto those of a gradient algorithm analyzed in the previous section. They are summarized byTheorem 3.7.4 in this section.

When β > 0, the covariance wind-up problem, i.e., P(t) becoming arbitrarily small,does not exist. In this case, P(t) may grow without bound. In order to avoid this phe-




�

�

�

�

�

�

�

�


nomenon, the following modified LS algorithm with forgetting factor is used:

θ = Pεφ,

P ={βP − PφφT P

m2s

if ‖P(t)‖ ≤ R0,

0 otherwise,

(3.48)

where P(0) = P0 = PT0 > 0, ‖P0‖ ≤ R0, R0 is a constant that serves as an upper bound

for ‖P ‖, and m2s = 1 + n2

s is the normalizing signal which satisfies φ

ms∈ L∞.

The following theorem summarizes the stability properties of the two modified LSalgorithms.

Theorem 3.7.4. The pure LS algorithm with covariance resetting (3.47) and the modifiedLS algorithm with forgetting factor (3.48) guarantee that

(i) ε, εms, θ ∈ L2 ∩ L∞ and θ ∈ L∞.

(ii) If φ

msis PE, then θ(t) → θ∗ as t → ∞ exponentially fast.

(iii) If (3.38) is the SPM for the plant (3.24) with stable poles and no zero-pole cancella-tions, and u is sufficiently rich of order n+m+1, thenφ, φ

msare PE, which guarantees

that θ(t) → θ∗ as t → ∞ exponentially fast.

Proof. The proof is presented in the web resource [94].

3.8 Parameter Identification Based on DPMLet us consider the DPM

z = W(s)[θ∗T ψ].This model may be obtained from (3.27) by filtering each side with W(s) and redefin-

ing the signals z, φ. Since θ∗ is a constant vector, the DPM may be written as

z = W(s)L(s)[θ∗T φ], (3.49)

where φ = L−1(s)ψ , L(s) is chosen so that L−1(s) is a proper stable transfer function, andW(s)L(s) is a proper strictly positive real (SPR) transfer function.

z = W(s)L(s)[θT φ].We form the normalized estimation error

ε = z − z − W(s)L(s)[εn2s ], (3.50)

where the static normalizing signal ns is designed so that φ

ms∈ L∞ for m2

s = 1 + n2s . If

W(s)L(s) = 1, then (3.50) has the same expression as in the case of the gradient algorithm.Substituting for z in (3.50), we express ε in terms of the parameter error θ = θ − θ∗:

ε = W(s)L(s)[−θ T φ − εn2s ]. (3.51)




�

�

�

�

�

�

�

�

3.8. Parameter Identification Based on DPM 49

For simplicity, let us assume that W(s)L(s) is strictly proper and rewrite (3.51) in theminimum state-space representation form

e = Ace + bc(−θ T φ − εn2s ),

ε = cTc e,(3.52)

where W(s)L(s) = cTc (sI − Ac)−1bc. Since W(s)L(s) is SPR, it follows that (see the

Appendix) there exist matrices Pc = PTc > 0, Lc = LT

c > 0, a vector q, and a scalar ν > 0such that

PcAc + ATc Pc = −qqT − νLc,

Pcbc = cc.(3.53)

The adaptive law for θ is generated using the Lyapunov-like function

V = eT Pce

2+ θ T �−1θ

2,

where � = �T > 0. The time derivative V of V along the solution of (3.52) is given by

V = −1

2eT qqT e − ν

2eT Lce + eT Pcbc(−θ T φ − εn2

s ) + θ T �−1 ˙θ.

Since eT Pcbc = eT cc = ε, it follows that by choosing ˙θ = θ as

θ = �εφ, (3.54)

we get

V = −1

2eT qqT e − ν

2eT Lce − ε2n2

s ≤ 0.

As before, from the properties of V , V we conclude that e, ε, θ ∈ L∞ and e, ε, εns ∈L2. These properties in turn imply that θ ∈ L2. Note that without the use of the second

equation in (3.53), we are not able to choose ˙θ = θ using signals available for measurement

to make V ≤ 0. This is because the state e in (3.52) cannot be generated since it dependson the unknown input θ T φ. Equation (3.52) is used only for analysis.

The stability properties of the adaptive law (3.54) are summarized by the followingtheorem.

Theorem 3.8.1. The adaptive law (3.54) guarantees that

(i) ε, θ ∈ L∞ and ε, εns, θ ∈ L2.

(ii) If ns, φ, φ ∈ L∞ and φ is PE, then θ(t) → θ∗ exponentially fast.

Proof. The proof for (i) is given above. The proof of (ii) is a long one and is given in [56]as well as in the web resource [94].

The adaptive law (3.54) is referred to as the adaptive law based on the SPR-Lyapunovsynthesis approach.




�

�

�

�

�

�

�

�


Comment 3.8.2 The adaptive law (3.54) has the same form as the gradient algorithm eventhough it is developed using a Lyapunov approach and the SPR property. In fact, forW(s)L(s) = 1, (3.54) is identical to the gradient algorithm.

3.9 Parameter Identification Based on B-SPMConsider the bilinear SPM described by (3.28), i.e.,

z = ρ∗(θ∗T φ + z0), (3.55)

where z, z0 are known scalar signals at each time t and ρ∗, θ∗ are the scalar and vectorunknown parameters, respectively. The estimation error is generated as

z = ρ(θT φ + z0),

ε = z − z

m2s

,

where ρ(t), θ(t) are the estimates of ρ∗, θ∗, respectively, at time t and where ms is designedto bound φ, z0 from above. An example of ms with this property is m2

s = 1 + φT φ + z20.

Let us consider the cost

J (ρ, θ) = ε2m2s

2= (z − ρ∗θT φ − ρξ + ρ∗ξ − ρ∗z0)

2

2m2s

,

whereξ = θT φ + z0

is available for measurement. Applying the gradient method we obtain

θ = −�1∇Jθ = �1ερ∗φ,

ρ = −γ∇Jρ = γ εξ,

where �1 = �T1 > 0, γ > 0 are the adaptive gains. Since ρ∗ is unknown, the adaptive law

for θ cannot be implemented. We bypass this problem by employing the equality

�1ρ∗ = �1|ρ∗| sgn(ρ∗) = � sgn(ρ∗),

where � = �1|ρ∗|. Since �1 is arbitrary any � = �T > 0 can be selected without havingto know |ρ∗|. Therefore, the adaptive laws for θ , ρ, may be written as

θ = �εφ sgn(ρ∗),ρ = γ εξ,

ε = z − ρξ

m2s

, ξ = θT φ + z0.

(3.56)

Theorem 3.9.1. The adaptive law (3.56) guarantees that

(i) ε, εms, θ , ρ ∈ L2 ∩ L∞ and θ, ρ ∈ L∞.




�

�

�

�

�

�

�

�

3.9. Parameter Identification Based on B-SPM 51

(ii) If ξ

ms∈ L2, then ρ(t) → ρ as t → ∞, where ρ is a constant.

(iii) If ξ

ms∈ L2 and φ

msis PE, then θ(t) converges to θ∗ as t → ∞.

(iv) If the plant (3.24) has stable poles with no zero-pole cancellations and u is sufficientlyrich of order n + m + 1, then φ,

φ

msare PE and θ(t) converges to θ∗ as t → ∞.

Proof. Consider the Lyapunov-like function

V = θ T �−1θ

2|ρ∗| + ρ2

2γ.

Then

V = θ T φε|ρ∗| sgn(ρ∗) + ρεξ.

Using |ρ∗| sgn(ρ∗) = ρ∗ and the expression

εm2s = ρ∗θ∗T φ + ρ∗z0 − ρθT φ − ρz0

= −ρz0 + ρ∗θ∗T φ − ρθT φ + ρ∗θT φ − ρ∗θT φ

= −ρ(z0 + θT φ) − ρ∗θ T φ = −ρξ − ρ∗θ T φ,

we have

V = ε(ρ∗θ T φ + ρξ ) = −ε2m2s ≤ 0,

which implies that V ∈ L∞ and therefore ρ, θ ∈ L∞. Using similar analysis as in the caseof the gradient algorithms for the SPM, we can establish (i) from the properties of V , V andthe form of the adaptive laws.

(ii) We have

ρ(t) − ρ(0) =∫ t

0ρdτ ≤

∫ t

0|ρ|dτ ≤ γ

∫ t

0|εms | |ξ |

ms

dτ

≤ γ

(∫ t

0ε2m2

s dτ

)1/2 (∫ t

0

|ξ |2m2

s

dτ

)1/2

< ∞,

where the last inequality is obtained using the Schwarz inequality (see (A.14)). Sinceεms,

ξ

ms∈ L2, the limit as t → ∞ exists, which implies that ρ ∈ L1 and limt→∞ ρ(t) = ρ

for some constant ρ. The proof of (iii) is long and is presented in the web resource [94].The proof of (iv) is included in the proof of Theorem 3.6.1.

The assumption that the sign of ρ∗ is known can be relaxed, leading to an adaptivelaw for θ , ρ with additional nonlinear terms. The reader is referred to [56, 99–104] forfurther reading on adaptive laws with unknown high-frequency gain.




�

�

�

�

�

�

�

�


3.10 Parameter ProjectionIn many practical problems, we may have some a priori knowledge of where θ∗ is located inRn. This knowledge usually comes in terms of upper and/or lower bounds for the elementsof θ∗ or in terms of location in a convex subset of Rn. If such a priori information isavailable, we want to constrain the online estimation to be within the set where the unknownparameters are located. For this purpose we modify the gradient algorithms based on theunconstrained minimization of certain costs using the gradient projection method presentedin section A.10.3 as follows.

The gradient algorithm with projection is computed by applying the gradient methodto the following minimization problem with constraints:

minimize J (θ)

subject to θ ∈ S,

where S is a convex subset of Rn with smooth boundary almost everywhere. Assume thatS is given by

S = {θ ∈ Rn|g(θ) ≤ 0},where g : Rn → R is a smooth function.

The adaptive laws based on the gradient method can be modified to guarantee thatθ ∈ S by solving the constrained optimization problem given above to obtain

θ = Pr(−�∇J ) =

⎧⎪⎨⎪⎩

−�∇J if θ ∈ S0

or θ ∈ δ(S) and −(�∇J )T ∇g ≤ 0,

−�∇J + �∇g∇gT

∇gT �∇g�∇J otherwise,

(3.57)where δ(S) = {θ ∈ Rn|g(θ) = 0} and S0 = {θ ∈ Rn|g(θ) < 0} denote the boundary andthe interior, respectively, of S and Pr(·) is the projection operator as shown in sectionA.10.3.

The gradient algorithm based on the instantaneous cost function with projectionfollows from (3.57) by substituting for ∇J = −εφ to obtain

θ = Pr(�εφ) =

⎧⎪⎨⎪⎩�εφ if θ ∈ S0

or θ ∈ δ(S) and (�εφ)T ∇g ≤ 0,

�εφ − �∇g∇gT

∇gT �∇g�εφ otherwise,

(3.58)

where θ(0) ∈ S.The pure LS algorithm with projection becomes

θ = Pr(P εφ) =

⎧⎪⎨⎪⎩Pεφ if θ ∈ S0

or if θ ∈ δ(S) and (P εφ)T ∇g ≤ 0,

P εφ − P∇g∇gT

∇gT P∇gP εφ otherwise,

(3.59)

where θ(0) ∈ S,

P =

⎧⎪⎨⎪⎩βP − P

φφT

m2sP if θ ∈ S0

or if θ ∈ δ(S) and (P εφ)T ∇g ≤ 0,

0 otherwise,




�

�

�

�

�

�

�

�

3.10. Parameter Projection 53

and P(0) = P0 = PT0 > 0.

Theorem 3.10.1. The gradient adaptive laws of section 3.6 and the LS adaptive laws ofsection 3.7 with the projection modifications given by (3.57) and (3.59), respectively, retainall the properties that are established in the absence of projection and in addition guaranteethat θ(t) ∈ S ∀t ≥ 0, provided θ(0) ∈ S and θ∗ ∈ S.

Proof. The adaptive laws (3.57) and (3.59) both guarantee that whenever θ ∈ δ(S), thedirection of θ is either towards S0 or along the tangent plane of δ(S) at θ . This propertytogether with θ(0) ∈ S guarantees that θ(t) ∈ S ∀t ≥ 0.

The gradient adaptive law (3.57) can be expressed as

θ = −�∇J + (1 − sgn(|g(θ)|))max(0, sgn(−(�∇J )T ∇g))�∇g∇gT

∇gT �∇g�∇J, (3.60)

where sgn |x| = 0 when x = 0. Hence, for the Lyapunov-like functionV used in section 3.6,i.e., V = θ T �−1 θ

2 , we have

V = −θ T ∇J + Vp,

where

Vp = (1 − sgn(|g(θ)|))max(0, sgn(−(�∇J )T ∇g))θT∇g∇gT

∇gT �∇g�∇J.

The term Vp is nonzero only when θ ∈ δ(S), i.e., g(θ) = 0 and − (�∇J )T ∇g > 0.In this case

Vp = θ T∇g∇gT

∇gT �∇g�∇J = 1

∇gT �∇g(θT ∇g)(∇gT �∇J )

= 1

∇gT �∇g(θT ∇g)((�∇J )T ∇g).

Since θ T ∇g = (θ − θ∗)T ∇g ≥ 0 for θ∗ ∈ S and θ ∈ δ(S) due to the convexity ofS, (�∇J )T ∇g < 0 implies that Vp ≤ 0. Therefore, the projection due to Vp ≤ 0 can onlymake V more negative. Hence

V ≤ −θ T ∇J, (3.61)

which is the same expression as in the case without projection except for the inequality.Hence the results in section 3.6 based on the properties of V and V are valid for the adaptivelaw (3.57) as well. Moreover, since �

∇g∇gT

∇gT �∇g∈ L∞, from (3.60) we have

|θ |2 ≤ c|�∇J |2

for some constant c > 0, which can be used together with (3.61) to show that θ ∈ L2. Thesame arguments apply to the case of the LS algorithms.




�

�

�

�

�

�

�

�


Example 3.10.2 Let us consider the plant model

y = b

s + au,

where a, b are unknown constants that satisfy some known bounds, e.g., b ≥ 1 and 20 ≥a ≥ −2. For simplicity, let us assume that y, y, u are available for measurement so that theSPM is of the form

z = θ∗T φ,where z = y, θ∗ = [b, a]T , φ = [u,−y]T . In the unconstrained case the gradient adaptivelaw is given as

θ = �εφ, ε = z − θT φ

m2s

,

where m2s = 1 + φT φ; θ = [b, a]T ; b, a are the estimates of b, a, respectively. Since we

know that b ≥ 1 and 20 ≥ a ≥ −2, we can constrain the estimates b, a to be within theknown bounds by using projection. Defining the sets for projection as

Sb = {b ∈ R|1 − b ≤ 0},Sla = {a ∈ R| − 2 − a ≤ 0},

Sua = {a ∈ R|a − 20 ≤ 0}

and applying the projection algorithm (3.57) for each set, we obtain the adaptive laws

˙b =

{γ1εu if b > 1 or (b = 1 and εu ≥ 0),

0 if b = 1 and εu < 0,

with b(0) ≥ 1, and

˙a =

⎧⎪⎨⎪⎩

−γ2εy if 20 > a > −2 or (a = −2 and εy ≤ 0)

or (a = 20 and εy ≥ 0),

0 if (a = −2 and εy > 0) or (a = 20 and εy < 0),

with a(0) satisfying 20 ≥ a(0) ≥ −2.

Example 3.10.3 Let us consider the gradient adaptive law

θ = �εφ

with the a priori knowledge that |θ∗| ≤ M0 for some known bound M0 > 0. In mostapplications, we may have such a priori information. We define

S ={θ ∈ Rn|g(θ) = θT θ

2− M2

0

2≤ 0

}and use (3.57) together with ∇g = θ to obtain the adaptive law with projection

θ ={�εφ if |θ | < M0 or (|θ | = M0 and φT �θε ≤ 0),

�εφ − � θθT

θT �θ�εφ if |θ | = M0 and φT �θε > 0

with |θ(0)| ≤ M0.




�

�

�

�

�

�

�

�

3.11. Robust Parameter Identification 55

3.11 Robust Parameter IdentificationIn the previous sections we designed and analyzed a wide class of PI algorithms based onthe parametric models

z = θ∗T φ or W(s)θ∗T φ.

These parametric models are developed using a plant model that is assumed to be freeof disturbances, noise, unmodeled dynamics, time delays, and other frequently encountereduncertainties. In the presence of plant uncertainties we are no longer able to express theunknown parameter vector θ∗ in the form of the SPM or DPM where all signals are measuredand θ∗ is the only unknown term. In this case, the SPM or DPM takes the form

z = θ∗T φ + η or z = W(s)θ∗T φ + η, (3.62)

where η is an unknown function that represents the modeling error terms. The followingexamples are used to show how (3.62) arises for different plant uncertainties.

Example 3.11.1 Consider the scalar constant gain system

y = θ∗u + d, (3.63)

where θ∗ is the unknown scalar and d is a bounded external disturbance due to measurementnoise and/or input disturbance. Equation (3.60) is already in the form of the SPM withmodeling error given by (3.62).

Example 3.11.2 Consider a system with a small input delay τ given by

y = b

s + ae−τsu, (3.64)

where a, b are the unknown parameters to be estimated and u ∈ L∞. Since τ is small, theplant may be modeled as

y = b

s + au (3.65)

by assuming τ = 0. Since a parameter estimator for a, b developed based on (3.65) has tobe applied to the actual plant (3.64), it is of interest to see how τ �= 0 affects the parametricmodel for a, b. We express the plant as

y = b

s + au + b

s + a(e−τs − 1)u,

which we can rewrite in the form of the parametric model (3.62) as

z = θ∗T φ + η,

where

z = s

s + λy, θ∗ = [b, a]T , φ =

[ 1s+λ

u

− 1s+λ

y

], η = b

s + λ(e−τs − 1)u,

and λ > 0. It is clear that τ = 0 implies η = 0 and small τ implies small η.




�

�

�

�

�

�

�

�


Example 3.11.3 Let us consider the plant

y = θ∗(1 + µ�m(s))u, (3.66)

where µ is a small constant and �m(s) is a proper transfer function with poles in the openleft half s-plane. Since µ is small and �m(s) is proper with stable poles, the term µ�m(s)

can be treated as the modeling error term which can be approximated with zero. We canexpress (3.66) in the form of (3.62) as

y = θ∗u + η,

whereη = µθ∗�m(s)u

is the modeling error term.

For LTI plants, the parametric model with modeling errors is usually of the form

z = θ∗T u + η,

η = �1(s)u + �2(s)y + d,(3.67)

where�1(s), �2(s) are proper transfer functions with stable poles and d is a bounded distur-bance. The principal question that arises is how the stability properties of the adaptive lawsthat are developed for parametric models with no modeling errors are affected when appliedto the actual parametric models with uncertainties. The following example demonstratesthat the adaptive laws of the previous sections that are developed using parametric modelsthat are free of modeling errors cannot guarantee the same properties in the presence ofmodeling errors. Furthermore, it often takes only a small disturbance to drive the estimatedparameters unbounded.

3.11.1 Instability Example

Consider the scalar constant gain system

y = θ∗u + d,

where d is a bounded unknown disturbance and u ∈ L∞. The adaptive law for estimatingθ∗ derived for d = 0 is given by

θ = γ εu, ε = y − θu, (3.68)

where γ > 0 and the normalizing signal is taken to be 1. If d = 0 and u, u ∈ L∞, thenwe can establish that (i) θ , θ, ε ∈ L∞, (ii) ε(t) → 0 as t → ∞ by analyzing the parametererror equation

˙θ = −γ u2θ ,

which is a linear time-varying differential equation. When d �= 0, we have

˙θ = −γ u2θ + γ du. (3.69)




�

�

�

�

�

�

�

�


In this case we cannot guarantee that the parameter estimate θ(t) is bounded for anybounded input u and disturbance d. In fact, for θ∗ = 2, γ = 1,

u = (1 + t)−1/2,

d(t) = (1 + t)−1/4

(5

4− 2(1 + t)−1/4

)→ 0 as t → ∞,

we have

y(t) = 5

4(1 + t)−1/4 → 0 as t → ∞,

ε(t) = 1

4(1 + t)−1/4 → 0 as t → ∞,

θ(t) = (1 + t)−1/4 → ∞ as t → ∞;i.e., the estimated parameter drifts to infinity with time even though the disturbance d(t)

disappears with time. This instability phenomenon is known as parameter drift. It ismainly due to the pure integral action of the adaptive law, which, in addition to integratingthe “good’’ signals, integrates the disturbance term as well, leading to the parameter driftphenomenon.

Another interpretation of the above instability is that, for u = (1 + t)−1/2, the homo-

geneous part of (3.69), i.e., ˙θ = −γ u2θ , is only uniformly stable, which is not sufficient to

guarantee that the bounded input γ du will produce a bounded state θ . If u is persistentlyexciting, i.e.,

∫ t+T0

tu2(τ )dτ ≥ α0T0 for some α0, T0 > 0 and ∀t ≥ 0, then the homoge-

neous part of (3.69) is e.s. and the bounded input γ du produces a bounded state θ (showit!). Similar instability examples in the absence of PE may be found in [50–52, 105].

If the objective is parameter convergence, then parameter drift can be prevented bymaking sure the regressor vector is PE with a level of excitation higher than the level ofthe modeling error. In this case the plant input in addition to being sufficiently rich is alsorequired to guarantee a level of excitation for the regressor that is higher than the level ofthe modeling error. This class of inputs is referred to as dominantly rich and is discussed inthe following section.

3.11.2 Dominantly Rich Excitation

Let us revisit the example in section 3.11.1 and analyze (3.69), i.e.,

˙θ = −γ u2θ + γ du, (3.70)

when u is PE with level α0 > 0. The PE property of u implies that the homogeneous partof (3.70) is e.s., which in turn implies that

|θ (t)| ≤ e−γα1t |θ (0)| + 1

α1(1 − e−γα1t ) sup

τ≤t

|u(τ)d(τ )|

for some α1 > 0 which depends on α0. Therefore, we have

limt→∞ sup

τ≥t

|θ (τ )| ≤ 1

α1limt→∞ sup

τ≥t

|u(τ)d(τ )| = 1

α1supτ

|u(τ)d(τ )|. (3.71)




�

�

�

�

�

�

�

�


The bound (3.71) indicates that the PI error at steady state is of the order of thedisturbance; i.e., as d → 0 the parameter error also reduces to zero. For this simpleexample, it is clear that if we choose u = u0, where u0 is a constant different from zero,then α1 = α0 = u2

0; therefore, the bound for |θ | is supt|d(t)|u0

. Thus the larger the value ofu0 is, the smaller the parameter error. Large u0 relative to |d| implies large signal-to-noiseratio and therefore better accuracy of identification.

Example 3.11.4 (unmodeled dynamics) Let us consider the plant

y = θ∗(1 + �m(s))u,

where �m(s) is a proper transfer function with stable poles. If �m(s) is much smaller than1 for small s or all s, then the system can be approximated as

y = θ∗u

and �m(s) can be treated as an unmodeled perturbation. The adaptive law (3.68) that isdesigned for �m(s) = 0 is used to identify θ∗ in the presence of �m(s). The parametererror equation in this case is given by

˙θ = −γ u2θ + γ uη, η = θ∗�m(s)u. (3.72)

Since u is bounded and �m(s) is stable, it follows that η ∈ L∞ and therefore theeffect of �m(s) is to introduce the bounded disturbance term η in the adaptive law. Hence,if u is PE with level α0 > 0, we have, as in the previous example, that

limt→∞ sup

τ≥t

|θ (τ )| ≤ 1

α1supt

|u(t)η(t)|

for some α1 > 0. The question that comes up is how to choose u so that the above boundfor |θ | is as small as possible. The answer to this question is not as straightforward as inthe example of section 3.11.1 because η is also a function of u. The bound for |θ | dependson the choice of u and the properties of �m(s). For example, for constant u = u0 �= 0,we have α0 = α1 = u2

0 and η = θ∗�m(s)u0, i.e., limt→∞ |η(t)| = |θ∗||�m(0)||u0|, andtherefore

limt→∞ sup

τ≥t

|θ (τ )| ≤ |�m(0)||θ∗|.

If the plant is modeled properly, �m(s) represents a perturbation that is small in thelow-frequency range, which is usually the range of interest. Therefore, for u = u0, weshould have |�m(0)| small if not zero leading to the above bound, which is independent ofu0. Another choice of a PE input is u = cosω0t for some ω0 �= 0. For this choice of u,since

e−γ∫ t

0 cos2 ω0τdτ = e− γ

2

(t+ sin 2ω0 t

2ω0

)= e

− γ

4

(t+ sin 2ω0 t

ω0

)e− γ

4 t ≤ e− γ

4 t

(where we used the inequality t + sin 2ω0tω0

≥ 0 ∀t ≥ 0) and supt |η(t)| ≤ |�m(jω0)||θ∗|,we have

limt→∞ sup

r≥t

|θ (r)| ≤ c|�m(jω0)|,




�

�

�

�

�

�

�

�


where c = 4|θ∗|γ

. This bound indicates that for small parameter error, ω0 should be chosenso that |�m(jω0)| is as small as possible. If �m(s) is due to high-frequency unmodeleddynamics, then |�m(jω0)| is small, providedω0 is a low frequency. As an example, consider

�m(s) = µs

1 + µs,

where µ > 0 is a small constant. It is clear that for low frequencies |�m(jω)| = O(µ)3

and |�m(jω)| → 1 as ω → ∞. Since

|�m(jω0)| = |µω0|√1 + µ2ω2

0

,

it follows that for ω0 = 1µ

we have |�m(jω0)| = 1√2

and for ω0 � 1µ

, �m(jω0) ≈ 1,

whereas for ω0 < 1µ

we have |�m(jω0)| = O(µ). Therefore, for more accurate PI,the input signal should be chosen to be PE, but the PE property should be achieved withfrequencies that do not excite the unmodeled dynamics. For the above example of �m(s),u = u0 does not excite �m(s) at all, i.e., �m(0) = 0, whereas for u = sin ω0t with ω0 � 1

µ,

the excitation of �m(s) is small leading to an O(µ) steady-state error for |θ |.The above example demonstrates the well-known fact in control systems that the

excitation of the plant should be restricted to be within the range of frequencies where theplant model is a good approximation of the actual plant. We explain this statement furtherusing the plant

y = G0(s)u + �a(s)u + d, (3.73)

where G0(s), �a(s) are proper and stable, �a(s) is an additive perturbation of the modeledpart G0(s), and d is a bounded disturbance. We would like to identify the coefficients ofG0(s) by exciting the plant with the input u and processing the I/O data.

Because �a(s)u is treated as a disturbance, the input u should be chosen so that ateach frequency ωi contained in u, we have |G0(jωi)| � |�a(jωi)|. Furthermore, u shouldbe rich enough to excite the modeled part of the plant that corresponds to G0(s) so that ycontains sufficient information about the coefficients of G0(s). For such a choice of u to bepossible, the spectrums of G0(s) and �a(s)u should be separated, or |G0(jω)| � |�a(jω)|at all frequencies. If G0(s) is chosen properly, then |�a(jω)| should be small relative to|G0(jω)| in the frequency range of interest. Since we are usually interested in the systemresponse at low frequencies, we would assume that |G0(jω)| � |�a(jω)| in the low-frequency range for our analysis. But at high frequencies, we may have |G0(jω)| of thesame order as or smaller than |�a(jω)|. The input signal u should therefore be designed tobe sufficiently rich for the modeled part of the plant, but its richness should be achieved inthe low-frequency range for which |G0(jω)| � |�a(jω)|. An input signal with these twoproperties is called dominantly rich [106] because it excites the modeled or dominant partof the plant much more than the unmodeled one.

The separation of spectrums between the dominant and unmodeled part of the plantcan be seen more clearly if we rewrite (3.73) as

y = G0(s)u + �a(µs)u + d, (3.74)3A function f (x) is of O(µ) ∀x ∈ � if there exists a constant c ≥ 0 such that ‖f (x)‖ ≤ c|µ| ∀x ∈ �.




�

�

�

�

�

�

�

�


where �a(µs) is due to high-frequency dynamics and µ > 0 is a small parameter referredto as the singular perturbation parameter. �a(µs) has the property that �a(0) = 0 and|�a(µjω)| ≤ O(µ) for ω � 1

µ, but �a(µs) could satisfy |�a(µjω)| = O(1) for ω � 1

µ.

Therefore, for µ small, the contribution of �a in y is small, provided that the plant is notexcited with frequencies of the order of 1

µor higher. Let us assume that we want to identify

the coefficients of G0(s) = Z(s)

R(s)in (3.74) which consist of m zeros and n stable poles, a

total of n + m + 1 parameters, by treating �a(µs)u as a modeling error. We can express(3.74) in the form

z = θ∗T φ + η,

where η = �a(µs)R(s)

�(s)u + d , 1

�(s)is a filter with stable poles, θ∗ ∈ Rn+m+1 contains the

coefficients of the numerator and denominator of G0(s), and

φ = 1

�(s)[smu, sm−1u, . . . , u,−sn−1y,−sn−2y, . . . ,−y]T .

If φ is PE, we can establish that the adaptive laws of the previous sections guaranteeconvergence of the parameter error to a set whose size depends on the bound for the modelingerror signal η, i.e.,

|θ (t)| ≤ α1e−αt (|θ (0)| + η0) + α2η0,

where η0 = supt |η(t)| and α, α1, α2 > 0 are some constants that depend on the level ofexcitation of φ. The problem is how to choose u so that φ is PE despite the presence ofη �= 0. It should be noted that it is possible, for an input that is sufficiently rich, for themodeled part of the plant not to guarantee the PE property of φ in the presence of modelingerrors because the modeling error could cancel or highly corrupt terms that are responsiblefor the PE property φ. We can see this by examining the relationship between u and φ

given byφ = H0(s)u + H1(µs, s)u + B1d, (3.75)

where

H0(s) = 1

�(s)[sm,sm−1, . . . ,s,1,−sn−1G0(s),−sn−2G0(s), . . . ,−sG0(s),−G0(s)]T ,

H1(µs, s) = 1

�(s)[0, . . . , 0,−sn−1�a(µs),−sn−2�a(µs), . . . ,−�a(µs)]T ,

B1 = 1

�(s)[0, . . . , 0,−sn−1,−sn−2, . . . ,−s,−1]T .

It is clear that the term H1(µs, s)u + B1d acts as a modeling error term and coulddestroy the PE property of φ, even for inputs u that are sufficiently rich of order n+m+ 1.So the problem is to define the class of sufficiently rich inputs of order n+m+1 that wouldguarantee φ to be PE despite the presence of the modeling error term H1(µs, s)u + B1d

in (3.75).

Definition 3.11.5. A sufficiently rich input u of order n + m + 1 for the dominant part ofthe plant (3.74) is called dominantly rich of order n+m+ 1 if it achieves its richness withfrequencies ωi , i = 1, 2, . . . , N , where N ≥ n+m+1

2 , |ωi | < O( 1µ), |ωi − ωj | > O(µ),

i �= j , and |u| > O(µ) + O(d).




�

�

�

�

�

�

�

�


Lemma 3.11.6. Let H0(s), H1(µs, s) satisfy the following assumptions:

(a) The vectors H0(jω1),H0(jω2), . . . , H0(jωn) are linearly independent on Cn for allpossible ω1, ω2, . . . , ωn ∈ R, where n � n + m + 1 and ωi �= ωk for i �= k.

(b) For any set {ω1, ω2, . . . , ωn} satisfying |ωi−ωk| > O(µ) for i �= k and |ωi | < O( 1µ),

we have | det(H )| > O(µ), where H � [H0(jω1),H0(jω2), . . . , H0(jωn)].(c) |H1(jµω, jω)| ≤ c for some constant c independent of µ and ∀ω ∈ R.

Then there exists a µ∗ > 0 such that for µ ∈ [0, µ∗), φ is PE of order n+m+ 1 withlevel of excitation α1 > O(µ), provided that the input signal u is dominantly rich of ordern + m + 1 for the plant (3.74).

Proof. Since |u| > O(d), we can ignore the contribution of d in (3.75) and write

φ = φ0 + φ1,

whereφ0 = H0(s)u, φ1 = H1(µs, s)u.

Since H0(s) does not depend on µ, φ0 is the same signal vector as in the ideal case.The sufficient richness of order n of u together with the assumed properties (a)–(b) of H0(s)

imply that φ0 is PE with level α0 > 0 and α0 is independent of µ, i.e.,

1

T

∫ t+T

t

φ0(τ )φT0 (τ )dτ ≥ α0I (3.76)

∀t ≥ 0 and for someT > 0. On the other hand, sinceH1(µs, s) is stable and |H1(jµω, jω)| ≤c ∀ω ∈ R, we have φ1 ∈ L∞,

1

T

∫ t+T

t

φ1(τ )φT1 (τ )dτ ≤ βI (3.77)

for some constant β which is independent of µ. Because of (3.76) and (3.77), we have

1

T

∫ t+T

t

φ(τ )φT (τ )dτ = 1

T

∫ t+T

t

(φ0(τ ) + µφ1(τ ))(φT0 (τ ) + µφ

T

1 (τ ))dτ

≥ 1

T

(∫ t+T

t

φ0(τ )φT0 (τ )

2dτ − µ2

∫ t+T

t

φ1(τ )φT1 (τ )dτ

)≥ α0

2I − µ2βI = α0

4I + α0

4I − µ2βI,

where the first inequality is obtained by using (x + y)(x + y)T ≥ xxT

2 − yyT . In other

words, φ has a level of PE α1 = α04 for µ ∈ [0, µ∗), where µ∗ �

√α04β .

For further reading on dominant richness and robust parameter estimation, see [56].




�

�

�

�

�

�

�

�


Example 3.11.7 Consider the plant

y = b

s + a

(1 + 2µ(s − 1)

(µs + 1)2

)u,

where a, b are the unknown parameters and µ = 0.001. The plant may be modeled as

y = b

s + au

by approximating µ = 0.001 ∼= 0. The input u = sin ω0t with 1 � ω0 � 1000 wouldbe a dominantly rich input of order 2. Frequencies such as ω0 = 0.006 rad/sec or ω0 =900 rad/sec would imply that u is not dominantly rich even though u is sufficiently rich oforder 2.

3.12 Robust Adaptive LawsIf the objective in online PI is convergence of the estimated parameters (close) to their truevalues, then dominantly rich excitation of appropriate order will meet the objective, andno instabilities will be present. In many applications, such as in adaptive control, the plantinput is the result of feedback and cannot be designed to be dominantly or even sufficientlyrich. In such situations, the objective is to drive the plant output to zero or force it tofollow a desired trajectory rather than convergence of the online parameter estimates totheir true values. It is therefore of interest to guarantee stability and robustness for theonline parameter estimators even in the absence of persistence of excitation. This can beachieved by modifying the adaptive laws of the previous sections to guarantee stability androbustness in the presence of modeling errors independent of the properties of the regressorvector φ.

Let us consider the general plant

y = G0(s)u + �u(s)u + �y(s)y + d, (3.78)

where G0(s) is the dominant part, �u(s), �y(s) are strictly proper with stable poles andsmall relative to G0(s), and d is a bounded disturbance. We are interested in designingadaptive laws for estimating the coefficients of

G0(s) = Z(s)

R(s)= bms

m + · · · + b1s + b0

sn + an−1sn−1 + · · · + a1s + a0.

The SPM for (3.78) isz = θ∗T φ + η, (3.79)

where

θ∗ = [bm, . . . , b0, an−1, . . . , a0]T ∈ Rn+m+1,

z = sn

�(s)y,




�

�

�

�

�

�

�

�

3.12. Robust Adaptive Laws 63

φ =[

sm

�(s)u, . . . ,

1

�(s)u,− sn−1

�(s)y, . . . ,− 1

�(s)y

]T,

η = �u(s)R(s)

�(s)u + �y(s)

R(s)

�(s)y + R(s)

�(s)d.

The modeling error term η is driven by both u and y and cannot be assumed to bebounded unless u and y are bounded.

The parametric model can be transformed into one where the modeling error termis bounded by using normalization, a term already used in the previous sections. Let usassume that we can find a signal ms > 0 with the property φ

ms,

η

ms∈ L∞. Then we can

rewrite (3.79) asz = θ∗T φ + η, (3.80)

where z = zms

, φ = φ

ms, η = η

ms. The normalized parametric model has all the measured

signals bounded and can be used to develop online adaptive laws for estimating θ∗. Thebounded term η can still drive the adaptive laws unstable, as we showed using an examplein section 3.11.1. Therefore, for robustness, we need to use the following modifications:

• Design the normalizing signalms to bound the modeling error in addition to boundingthe regressor vector φ.

• Modify the “pure’’ integral action of the adaptive laws to prevent parameter drift.

In the following sections, we develop a class of normalizing signals and modifiedadaptive laws that guarantee robustness in the presence of modeling errors.

3.12.1 Dynamic Normalization

Let us consider the SPM with modeling error

z = θ∗T φ + η,

whereη = �1(s)u + �2(s)y.

�1(s), �2(s) are strictly proper transfer functions with stable poles. Our objectiveis to design a signal ms so that η

ms∈ L∞. We assume that �1(s), �2(s) are analytic in

�[s] ≥ − δ02 for some known δ0 > 0. Apart from this assumption, we require no knowledge

of the parameters and/or dimension of �1(s), �2(s). It is implicitly assumed, however, thatthey are small relative to the modeled part of the plant in the frequency range of interest;otherwise, they would not be treated as modeling errors.

Using the properties of the L2δ norm (see Lemma A.5.9), we can write

|η(t)| ≤ ‖�1(s)‖2δ‖ut‖2δ + ‖�2(s)‖2δ‖yt‖2δ

for any δ ∈ [0, δ0], where the above norms are defined as

‖xt‖2δ�=(∫ t

0e−δ(t−τ)xT (τ )x(τ )dτ

)1/2

,




�

�

�

�

�

�

�

�


‖H(s)‖2δ�= 1√

2π

(∫ ∞

−∞

∣∣∣∣H(jω − δ

2

)∣∣∣∣2 dω)1/2

.

If we define

nd = ‖ut‖22δ0

+ ‖yt‖22δ0

,

which can be generated by the differential equation

nd = −δ0nd + u2 + y2, nd(0) = 0,

then it follows that

m2s = 1 + nd

bounds |η(t)| from above and

|η(t)|ms

≤ ‖�1(s)‖2δ0 + ‖�2(s)‖2δ0 .

The normalizing signal ms used in the adaptive laws for parametric models freeof modeling errors is required to bound the regressor φ from above. In the presence ofmodeling errors,ms should be chosen to bound bothφ and the modeling error η for improvedrobustness properties. In this case the normalizing signal ms has the form

m2s = 1 + n2

s + nd,

where n2s is the static part and nd the dynamic one. Examples of static and dynamic nor-

malizing signals are n2s = φT φ or φT Pφ, where P = PT > 0,

nd = −δ0nd + δ1(u2 + y2), nd(0) = 0, (3.81)

or

nd = n21, n1 = −δ0n1 + δ1(|u| + |y|), n1(0) ≥ 0, (3.82)

or

nd = n2∞, n∞ = δ1 max

(supτ≤t

|u(τ)|, supτ≤t

|y(τ)|). (3.83)

Any one of choices (3.81)–(3.83) can be shown to guarantee that η

ms∈ L∞. Since

φ = H(s)[ uy], the dynamic normalizing signal can be chosen to bound φ from above,

provided that H(s) is analytic in �[s] ≥ − δ02 , in which case m2

s = 1 + nd bounds both φ

and η from above.




�

�

�

�

�

�

�

�


3.12.2 Robust Adaptive Laws: σ -Modification

A class of robust modifications involves the use of a small feedback around the “pure’’integrator in the adaptive law, leading to the adaptive law structure

θ = �εφ − σ��θ, (3.84)

where σ� ≥ 0 is a small design parameter and � = �T > 0 is the adaptive gain, which inthe case of LS is equal to the covariance matrix P . The above modification is referred to asthe σ -modification [50, 51, 59] or as leakage.

Different choices of σ� lead to different robust adaptive laws with different properties,described in the following sections. We demonstrate the properties of these modificationswhen applied to the SPM with modeling error,

z = θ∗T φ + η, (3.85)

where η is the modeling error term that is bounded from above by the normalizing signal.Let us assume that η is of the form

η = �1(s)u + �2(s)y,

where �1(s), �2(s) are strictly proper transfer functions analytic in �[s] ≥ − δ02 for some

known δ0 > 0. The normalizing signal can be chosen as

m2s = 1 + α0φ

T φ + α1nd,

nd = −δ0nd + δ1(u2 + y2), nd(0) = 0,

(3.86)

for some design constants δ1, α0, α1 > 0, and shown to guarantee that φ/ms, η/ms ∈ L∞.

Fixed σ -Modification

In this caseσ�(t) = σ > 0 ∀t ≥ 0, (3.87)

where σ is a small positive design constant. The gradient adaptive law for estimating θ∗ in(3.85) takes the form

θ = �εφ − σ�θ,

ε = z − θT φ

m2s

,(3.88)

where ms is given by (3.86). If some a priori estimate θ0 of θ∗ is available, then the termσ�θ may be replaced with σ�(θ − θ0) so that the leakage term becomes larger for largerdeviations of θ from θ0 rather than from zero.

Theorem 3.12.1. The adaptive law (3.88) guarantees the following properties:

(i) ε, εms, θ, θ ∈ L∞.




�

�

�

�

�

�

�

�


(ii) ε, εms, θ ∈ S(σ + η2

m2s).4

(iii) If φ

msis PE with level α0 > 0, then θ(t) converges exponentially to the residual set

Dσ = {θ | |θ | ≤ c(σ + η)}, where η = supt|η(t)|ms(t)

and c ≥ 0 is some constant.

Proof. (i) From (3.88), we obtain

˙θ = �εφ − σ�θ, εm2

s = −θ T φ + η.

Consider the Lyapunov-like function

V (θ) = θ T �−1θ

2.

We have

V = θ T φε − σ θT θ

or

V = −ε2m2s + εη − σ θT (θ + θ∗) ≤ −ε2m2

s + |εms |∣∣∣∣ ηms

∣∣∣∣− σ θT θ + σ |θ ||θ∗|.

Using −a2 + ab = − a2

2 − 12 (a − b)2 + b2

2 ≤ − a2

2 + b2

2 for any a, b, we write

V ≤ −ε2m2s

2− σ

2|θ |2 + 1

2

|η|2m2

s

+ σ

2|θ∗|2.

Since η

ms∈ L∞, the positive terms in the expression for V are bounded from above

by a constant. Now

V (θ) ≤ |θ |2 λmax(�−1)

2and therefore

−σ

2|θ |2 ≤ − σ

λmax(�−1)V (θ),

which implies

V ≤ −ε2m2s

2− σ

λmax(�−1)V (θ) + c0

(σ + η2

m2s

),

where c0 = max{ |θ∗|22 , 1

2 }. Hence for

V (θ) ≥ V0 � c0(σ + η2)λmax(�

−1)

σ,

where η = supt|η(t)|ms(t)

, we have V ≤ − ε2m2s

2 ≤ 0, which implies that V is bounded and

therefore θ ∈ L∞. From εms = − θ T φ

ms+ η

ms, θ ∈ L∞, and the fact that ms bounds

4As defined in the Appendix, S(µ) = {x : [0,∞) → Rn | ∫ t+T

txT (τ )x(τ )dτ ≤ c0µT + c1 ∀t, T ≥ 0}.




�

�

�

�

�

�

�

�


φ, η from above, we have εms, ε ∈ L∞. Furthermore, using ˙θ = �εms

φ

ms− σ�θ and

θ, εms,φ

ms∈ L∞, we obtain ˙

θ = θ ∈ L∞.

(ii) The inequality

V ≤ −ε2m2s

2− σ

λmax(�−1)V (θ) + c0

(σ + η2

m2s

)

obtained above implies

V ≤ −ε2m2s

2+ c0

(σ + η2

m2s

).

Integrating both sides, we obtain

1

2

∫ t

0ε2m2

s dτ ≤ V (0) − V (t) + c0

∫ t

0

(σ + η2

m2s

)dτ ,

which implies that εms ∈ S(σ + η2

m2s). This together with θ,

φ

ms∈ L∞ implies that ε, θ ∈

S(σ + η2

m2s).

(iii) Using εms = − θ T φ

ms+ η

ms= −φT θ

ms+ η

ms, we have

˙θ = −�

φ

ms

φT

ms

θ − σ�θ + η

ms

.

Since φ

msis PE, it follows that the homogenous part of the above equation is e.s.,

which implies that

|θ (t)| ≤ α1e−α0t |θ (t)| + α1

∫ t

0e−α0(t−τ)

(σ‖�‖|θ(τ )| + |η|

ms

)dτ

for some α1, α0 > 0. Using η

ms, θ ∈ L∞, we can write

|θ (t)| ≤ α1e−α0t |θ (0)| + c(σ + η)(1 − e−α0t ),

where η = supt|η(t)|ms(t)

, c = max( α1α0

‖�‖ supt |θ(t)|, α1α0), which concludes that θ (t) converges

to Dσ exponentially fast.

One can observe that if the modeling error is removed, i.e., η = 0, then the fixed σ -modification will not guarantee the ideal properties of the adaptive law since it introduces adisturbance of the order of the design constant σ . This is one of the main drawbacks of thefixed σ -modification that is removed in the next section. One of the advantages of the fixedσ -modification is that no assumption about bounds or location of the unknown θ∗ is made.

Switching σ -Modification

The drawback of the fixed σ -modification is eliminated by using a switching σ� term whichactivates the small feedback term around the integrator when the magnitude of the parameter




�

�

�

�

�

�

�

�


vector exceeds a certain value M0. The assumptions we make in this case are that |θ∗| ≤ M0

and M0 is known. Since M0 is arbitrary it can be chosen to be high enough in order toguarantee |θ∗| ≤ M0 in the case where limited or no information is available about thelocation of θ∗. The switching σ -modification is given by

σ�(t) = σs =

⎧⎪⎪⎨⎪⎪⎩

0 if |θ | ≤ M0,(|θ |M0

− 1)q0

σ0 if M0 < |θ | ≤ 2M0,

σ0 if |θ | > 2M0,

(3.89)

where q0 ≥ 1 is any finite integer and M0, σ0 are design constants satisfying M0 > |θ∗|and σ0 > 0. The switching from 0 to σ0 is continuous in order to guarantee the existenceand uniqueness of solutions of the differential equation. If an a priori estimate θ0 of θ∗ isavailable, it can be incorporated in (3.89) to replace |θ | with |θ − θ0|. In this case, M0 is anupper bound for |θ∗ − θ0| ≤ M0.

The gradient algorithm with the switching σ -modification given by (3.89) is de-scribed as

θ = �εφ − σs�θ,

ε = z − θT φ

m2s

,(3.90)

where ms is given by (3.86).

Theorem 3.12.2. The gradient algorithm (3.90) with the switching σ -modification guaran-tees the following:

(i) ε, εms, θ, θ ∈ L∞.

(ii) ε, εms, θ ∈ S( η2

m2s).

(iii) In the absence of modeling error, i.e., for η = 0, the properties of the adaptive law(3.90) are the same as those of the respective unmodified adaptive laws (i.e., withσ� = 0).

(iv) If φ

msis PE with level α0 > 0, then

(a) θ converges exponentially fast to the residual set

Ds = {θ | |θ | ≤ c(σ0 + η)},where c ≥ 0 is some constant;

(b) there exists a constant η∗ > 0 such that for η < η∗, θ converges exponentiallyto the residual set

Ds = {θ | |θ | ≤ cη}.




�

�

�

�

�

�

�

�


Proof. We choose the same Lyapunov-like function as in the case of no modeling error, i.e.,

V = θ T �−1θ

2.

ThenV = θ T φε − σsθ

T θ.

Substituting εm2s = −θ T φ + η, we obtain

V = −ε2m2s + εη − σsθ

T θ.

Note that −ε2m2s + εη ≤ − ε2m2

s

2 + η2

2m2s

and

σsθT θ = σs(|θ |2 − θ∗T θ) ≥ σs |θ |(|θ | − M0 + M0 − |θ∗|).

Since σs(|θ | − M0) ≥ 0 and M0 > |θ∗|, it follows that

σsθT θ ≥ σs |θ |(|θ | − M0) + σs |θ |(M0 − |θ∗|) ≥ σs |θ |(M0 − |θ∗|) ≥ 0,

i.e.,

σs |θ | ≤ σsθT θ

M0 − |θ∗| . (3.91)

Therefore, the inequality for V can be written as

V ≤ −ε2m2s

2− σsθ

T θ + η2

2m2s

. (3.92)

Since for |θ | = |θ + θ∗| > 2M0 the term −σsθT θ = −σ0θ

T θ ≤ − σ02 |θ |2 + σ0

2 |θ∗|2behaves as the equivalent fixed σ term, we can follow the same procedure as in the proofof Theorem 3.12.1 to show the existence of a constant V0 > 0 for which V ≤ 0 wheneverV ≥ V0 and conclude that V, ε, θ, θ ∈ L∞, completing the proof of (i).

Integrating both sides of (3.92) from t0 to t , we obtain that ε, εms,√σsθT θ ∈ S( η2

m2s).

From (3.91), it follows thatσ 2s |θ |2 ≤ c2σsθ

T θ

for some constant c2 > 0 that depends on the bound for σ0|θ |, and therefore |θ |2 ≤c(|εms |2 +σsθ

T θ) for some c ≥ 0. Since εms ,√σsθT θ ∈ S( η2

m2s), it follows that θ ∈ S( η2

m2s),

which completes the proof of (ii).The proof of part (iii) follows from (3.92) by setting η = 0, using −σsθ

T θ ≤ 0, andrepeating the above calculations for η = 0.

The proof of (iv)(a) is almost identical to that of Theorem 3.12.1(iii) and is omitted.To prove (iv)(b), we follow the same arguments used in the proof ofTheorem 3.12.1(iii)

to obtain the inequality

|θ | ≤ β0e−β2t + β ′

1

∫ t

0e−β2(t−τ)(|η| + σs |θ |)dτ

≤ β0e−β2t + β ′

1

β2η +

∫ t

0e−β2(t−τ)σs |θ |dτ

(3.93)




�

�

�

�

�

�

�

�


for some positive constants β0, β ′1, β2. From (3.91), we have

σs |θ | ≤ 1

M0 − |θ∗|σsθT θ ≤ 1

M0 − |θ∗|σs |θ ||θ |. (3.94)

Therefore, using (3.94) in (3.93), we have

|θ | ≤ β0e−β2t + β ′

1

β2η + β ′′

1

∫ t

0e−β2(t−τ)σs |θ ||θ |dτ , (3.95)

where β ′′1 = β ′

1M0−|θ∗| . Applying the Bellman–Gronwall (B–G) Lemma 3 (Lemma A.6.3) to

(3.95), it follows that

|θ | ≤(β0 + β ′

1

β2η

)e−β2(t−t0)e

β ′′1

∫ t

t0σs |θ |ds + β ′

1η

∫ t

t0

e−β2(t−τ)eβ ′′ ∫ t

t0σs |θ |ds

dτ . (3.96)

Note from (3.91)–(3.92) that√σs |θ | ∈ S( η2

m2s), i.e.,

∫ t

t0

σs |θ |ds ≤ c1η2(t − t0) + c0

∀t ≥ t0 ≥ 0 and for some constants c0, c1. Therefore,

|θ | ≤ β1e−α(t−t0) + β2η

∫ t

t0

e−α(t−τ)dτ , (3.97)

where α = β2 − β ′′2 c1η

2 and β1, β2 ≥ 0 are some constants that depend on c0 and on the

constants in (3.96). Hence, for any η ∈ [0, η∗), where η∗ =√

β2

β ′′1 c1

, we have α > 0, and

(3.97) implies that

|θ | ≤ β2

αη + ce−α(t−t0)

for some constant c and ∀t ≥ t0 ≥ 0, which completes the proof of (iv).

Another class of σ -modification involves leakage that depends on the estimation errorε, i.e.,

σs = |εms |ν0,

where ν0 > 0 is a design constant. This modification is referred to as the ε-modificationand has properties similar to those of the fixed σ -modification in the sense that it cannotguarantee the ideal properties of the adaptive law in the absence of modeling errors. Formore details on the ε-modification, see [56, 86, 107].

The use of σ -modification is to prevent parameter drift by forcing θ(t) to remainbounded. It is often referred to as soft projection because it forces θ(t) not to deviate farfrom the bound M0 ≥ |θ |. That is, in the case of soft projection θ(t) may exceed the M0

bound, but it will still be bounded.In the following section, we use projection to prevent parameter drift by forcing the

estimated parameters to be within a specified bounded set.




�

�

�

�

�

�

�

�


3.12.3 Parameter Projection

The two crucial techniques that we used in sections 3.12.1 and 3.12.2 to develop robustadaptive laws are the dynamic normalization ms and leakage. The normalization guaranteesthat the normalized modeling error term η

msis bounded and therefore acts as a bounded

input disturbance in the adaptive law. Since a bounded disturbance may cause parameterdrift, the leakage modification is used to guarantee bounded parameter estimates. Anothereffective way to guarantee bounded parameter estimates is to use projection to constrain theparameter estimates to lie inside a bounded convex set in the parameter space that containsthe unknown θ∗. Adaptive laws with projection have already been introduced and analyzedin section 3.10. By requiring the parameter set to be bounded, projection can be usedto guarantee that the estimated parameters are bounded by forcing them to lie within thebounded set. In this section, we illustrate the use of projection for a gradient algorithm thatis used to estimate θ∗ in the parametric model

z = θ∗T φ + η.

In order to avoid parameter drift, we constrain θ to lie inside a bounded convex setthat contains θ∗. As an example, consider the set

℘ = {θ | g(θ) = θT θ − M20 ≤ 0},

where M0 is chosen so that M0 ≥ |θ∗|. Following the results of section 3.10, we obtain

θ =

⎧⎪⎪⎨⎪⎪⎩�εφ

if θT θ < M20

or if θT θ = M20 and (�εφ)T θ ≤ 0,(

I − �θθT

θT �θ

)�εφ otherwise,

(3.98)

where θ(0) is chosen so that θT (0)θ(0) ≤ M20 and ε = z−θT φ

m2s

, � = �T > 0.The stability properties of (3.98) for estimating θ∗ in (3.85) in the presence of the

modeling error term η are given by the following theorem.

Theorem 3.12.3. The gradient algorithm with projection described by (3.98) and designedfor the parametric model z = θ∗T φ + η guarantees the following:

(i) ε, εms, θ, θ ∈ L∞.

(ii) ε, εms, θ ∈ S( η2

m2s).

(iii) If η = 0, then ε, εms, θ ∈ L2.

(iv) If φ

msis PE with level α0 > 0, then

(a) θ converges exponentially to the residual set

Dp = {θ | |θ | ≤ c(f0 + η)},where η = supt

|η|m, c ≥ 0 is a constant, and f0 ≥ 0 is a design constant;




�

�

�

�

�

�

�

�


(b) there exists a constant η∗ > 0 such that if η < η∗, then θ converges exponen-tially fast to the residual set

Dp = {θ | |θ | ≤ cη}for some constant c ≥ 0.

Proof. As established in section 3.10, projection guarantees that |θ(t)| ≤ M0 ∀t ≥ 0,provided |θ(0)| ≤ M0. Let us choose the Lyapunov-like function

V = θ T �−1θ

2.

Along the trajectory of (3.98), we have

V =

⎧⎪⎨⎪⎩

−ε2m2s + εη

if θT θ < M20

or if θT θ = M20 and (�εφ)T θ ≤ 0,

−ε2m2s + εη − θ T θ

θT �θθT �εφ if θT θ = M2

0 and (�εφ)T θ > 0.

For θT θ = M20 and (�εφ)T θ = θT �εφ > 0, we have sgn{ θ T θ

θT �θθT �εφ} = sgn{θ T θ}.

For θT θ = M20 , we have

θ T θ = θT θ − θ∗T θ ≥ M20 − |θ∗||θ | = M0(M0 − |θ∗|) ≥ 0,

where the last inequality is obtained using the assumption that M0 ≥ |θ∗|. Therefore, it

follows that θ T θθT �εφ

θT �θ≥ 0 when θT θ = M2

0 and (�εφ)T θ = θT �εφ > 0. Hence the termdue to projection can only make V more negative, and therefore

V = −ε2m2s + εη ≤ −ε2m2

s

2+ η2

2m2s

.

Since V is bounded due to θ ∈ L∞, which is guaranteed by the projection, it followsthat εms ∈ S( η2

m2s), which implies that ε ∈ S( η2

m2s). From θ ∈ L∞ and φ

ms,

η

ms∈ L∞, we have

ε, εms ∈ L∞. Now, for θT θ = M20 , we have ‖�θθT ‖

θT �θ≤ c for some constant c ≥ 0, which

implies that|θ | ≤ c|εφ| ≤ c|εms |.

Hence θ ∈ S( η2

m2s), and the proof of (i) and (ii) is complete. The proof of (iii) follows

by setting η = 0 and has already been established in section 3.10.The proof for parameter error convergence is completed as follows: Define the func-

tion

f �{

θT �εφ

θT �θif θT θ = M2

0 and (�εφ)T θ > 0,

0 otherwise.(3.99)

It is clear from the analysis above that f (t) ≥ 0 ∀t ≥ 0. Then (3.98) may be written as

θ = �εφ − �f θ. (3.100)




�

�

�

�

�

�

�

�


We can establish that f θT θ has properties very similar to σsθT θ , i.e.,

f |θ ||θ | ≥ f θT θ ≥ f |θ |(M0 − |θ∗|), f ≥ 0,

and |f (t)| ≤ f0 ∀t ≥ 0 for some constant f0 ≥ 0. Therefore, the proof of (iv)(a)–(b) can becompleted following exactly the same procedure as in the proof of Theorem 3.12.2.

The gradient algorithm with projection has properties identical to those of the switch-ing σ -modification, as both modifications aim at keeping |θ | ≤ M0. In the case of theswitching σ -modification, |θ | may exceed M0 but remain bounded, whereas in the case ofprojection |θ | ≤ M0 ∀t ≥ 0, provided |θ(0)| ≤ M0.

3.12.4 Dead Zone

Let us consider the estimation error

ε = z − θT φ

m2s

= −θ T φ + η

m2s

(3.101)

for the parametric modelz = θ∗T φ + η.

The signal ε is used to “drive’’ the adaptive law in the case of the gradient and LSalgorithms. It is a measure of the parameter error θ , which is present in the signal θ T φ, andof the modeling error η. When η = 0 and θ = 0, we have ε = 0 and no adaptation takes

place. Since η

ms,

φ

ms∈ L∞, large εms implies that θ T φ

msis large, which in turn implies that θ

is large. In this case, the effect of the modeling error η is small, and the parameter estimatesdriven by ε move in a direction which reduces θ . When εms is small, however, the effect ofη on εms may be more dominant than that of the signal θ T φ, and the parameter estimatesmay be driven in a direction dictated mostly by η. The principal idea behind the dead zoneis to monitor the size of the estimation error and adapt only when the estimation error islarge relative to the modeling error η, as shown below.

We consider the gradient algorithm for the linear parametric model (3.85). We considerthe same cost function as in the ideal case, i.e.,

J (θ, t) = ε2m2s

2= (z − θT φ)2

2m2s

,

and write

θ ={

−�∇J (θ) if |εms | > g0 >|η|ms,

0 otherwise,(3.102)

where g0 is a known upper bound of the normalized modeling error |η|ms

. In other words, wemove in the direction of the steepest descent only when the estimation error is large relativeto the modeling error, i.e., when |εms | > g0. In view of (3.102) we have

θ = �φ(ε + g), g ={

0 if |εms | > g0,

−ε if |εms | ≤ g0.




�

�

�

�

�

�

�

�


0

s

gm−

0

s

gm

( )f gε ε= +

ε

Figure 3.1. Normalized dead zone function.

To avoid any implementation problems which may arise due to the discontinuity in(3.102), the dead zone function is made continuous as shown in Figure 3.1; i.e.,

θ = �φ(ε + g), g =

⎧⎪⎨⎪⎩

g0

msif εms < −g0,

− g0

msif εms > g0,

−ε if −g0 ≤ εms ≤ g0.

(3.103)

Since the size of the dead zone depends onms , this dead zone function is often referredto as the variable or relative dead zone.

Theorem 3.12.4. The adaptive law (3.103) guarantees the following properties:

(i) ε, εms, θ, θ ∈ L∞.

(ii) ε, εms, θ ∈ S(g0 + η2

m2s).

(iii) θ ∈ L1 ∩ L2.

(iv) limt→∞ θ(t) = θ , where θ is a constant vector.

(v) If φ

msis PE with level α0 > 0, then θ (t) converges to the residual set

Dd = {θ ∈ Rn | |θ | ≤ c(g0 + η)},where η = supt

|η(t)|ms(t)

and c ≥ 0 is a constant.

Proof. The proof can be found in [56] and in the web resource [94].

The dead zone modification has the following properties:

• It guarantees that the estimated parameters always converge to a constant. This isimportant in adaptive control employing an adaptive law with dead zone because atsteady state the gains of the adaptive controller are almost constant, leading to an LTIclosed-loop system when the plant is LTI.




�

�

�

�

�

�

�

�

3.13. State-Space Identifiers 75

• As in the case of the fixed σ -modification, the ideal properties of the adaptive laware destroyed in an effort to achieve robustness. That is, if the modeling error termbecomes zero, the ideal properties of the adaptive law cannot be recovered unless thedead zone modification is also removed.

The robust modifications that include leakage, projection, and dead zone are analyzedfor the case of the gradient algorithm for the SPM with modeling error. The same modifica-tions can be used in the case of LS and DPM, B-SPM, and B-DPM with modeling errors. Forfurther details on these modifications the reader is referred to [56]. One important propertyof these modifications is that ε, εms, θ ∈ S(λ0 + η2

m2s) for some constant λ0 ≥ 0. This means

that the estimation error and speed of adaptation are guaranteed to be of the order of themodeling error and design parameter in the mean square sense. In other words intervals oftime could exist at steady state where the estimation error and speed of adaptation couldassume values higher than the modeling error. This phenomenon is known as “bursting,’’and it may take considerable simulation time to appear [105]. The dead zone exhibits nobursting as the parameters converge to constant values. Bursting can be prevented in all theother modifications if they are combined with a small-size dead zone at steady state. Thevarious combinations of modifications as well as the development of new ones and theirevaluation using analysis and simulations are a good exercise for the reader.

3.13 State-Space IdentifiersLet us consider the state-space plant model

x = Apx + Bpu, (3.104)

where x ∈ Rn is the state, u ∈ Rm is the input vector, and Ap ∈ Rn×n, Bp ∈ Rn×m areunknown constant matrices. We assume that x, u are available for measurement. One wayto estimate the elements of Ap, Bp online is to express (3.104) as a set of n scalar differentialequations and then generaten parametric models with scalar outputs and apply the parameterestimation techniques covered in the previous sections. Another more compact way ofestimating Ap, Bp is to develop online estimators based on an SSPM model for (3.104) asfollows.

We express (3.104) in the form of the SSPM:

x = Amx + (Ap − Am)x + Bpu,

where Am is an arbitrary stable matrix. The estimation model is then formed as

˙x = Amx + (Ap − Am)x + Bpu = Am(x − x) + Apx + Bpu,

where Ap(t), Bp(t) are the estimates ofAp, Bp at time t , respectively. The above estimationmodel has been referred to as the series-parallel model in the literature [56]. The estimationerror vector is defined as

ε = x − x − (sI − Am)−1εn2

s

or

ε = x − x − w,




�

�

�

�

�

�

�

�


w = Amw + εn2s , w(0) = 0,

where n2s is the static normalizing signal designed to guarantee x√

1+n2s

, u√1+n2

s

∈ L∞. A

straightforward choice for ns is n2s = xT x + uT u. It is clear that if the plant model (3.104)

is stable and the input u is bounded, then ns can be taken to be equal to zero.It follows that the estimation error satisfies

ε = Amε − Apx − Bpu − εn2s ,

where Ap � Ap − Ap, Bp � Bp − Bp are the parameter errors.The adaptive law for generating Ap, Bp is developed by considering the Lyapunov

function

V (ε, Ap, Bp) = εT P ε + tr

(AT

pP Ap

γ1

)+ tr

(BTp P Bp

γ2

), (3.105)

where tr(A) denotes the trace of matrixA; γ1, γ2 > 0 are constant scalars; and P = PT > 0is chosen as the solution of the Lyapunov equation

PAm + AmPT = −Q (3.106)

for some Q = QT > 0, whose solution is guaranteed by the stability of Am (see theAppendix). The time derivative V is given by

V = εT P ε + εT P ε + tr

⎛⎝ ˙A

T

pP Ap

γ1+ AT

pP˙Ap

γ1

⎞⎠+ tr

⎛⎝ ˙B

T

pP Bp

γ2+ BT

p P˙Bp

γ2

⎞⎠ .

Substituting for ε, using (3.106), and employing the equalities tr(A + B) = tr(A) +tr(B) and tr(AT ) = tr(A) for square matrices A, B of the same dimension, we obtain

V = −εTQε−2εT P Apx−2εT P Bpu+2 tr

⎛⎝ AT

pP˙Ap

γ1+ BT

p P˙Bp

γ2

⎞⎠−2εT P εn2

s . (3.107)

Using the equality vT y = tr(vyT ) for vectors v, y of the same dimension, we rewrite(3.107) as

V = −εTQε+2 tr

⎛⎝ AT

pP˙Ap

γ1− AT

pP εxT + BTp P

˙Bp

γ2− BT

p P εuT

⎞⎠−2εT P εn2

s . (3.108)

The obvious choice for ˙Ap = ˙

Ap, ˙Bp = ˙

Bp to make V negative is

˙Ap = γ1εx

T ,˙Bp = γ2εu

T , (3.109)

which gives usV = −εTQε − 2εT P εn2

s ≤ 0.




�

�

�

�

�

�

�

�

3.13. State-Space Identifiers 77

This implies that V , Ap, Bp, ε are bounded. We can also write

V ≤ −|ε|2λmin(Q) − 2|εns |2λmin(P ) ≤ 0

and use similar arguments as in the previous sections to establish that ε, εns ∈ L2. From(3.109) we have that

‖ ˙Ap‖ ≤ γ1|εms | |x

T |ms

, ‖ ˙Bp‖ ≤ γ2|εms | |u

T |ms

,

where m2s = 1 + n2

s . Since |x|ms,

|u|ms

∈ L∞ and |εms | ∈ L2, we can also conclude that

‖ ˙Ap‖, ‖ ˙

Bp‖ ∈ L2.We have, therefore, established that independent of the stability of the plant and

boundedness of the input u, the adaptive law (3.109) guarantees that

• ‖Ap‖, ‖Bp‖, ε ∈ L∞,

• ‖ ˙Ap‖, ‖ ˙

Bp‖, ε, εns ∈ L2.

These properties are important for adaptive control where the adaptive law is used aspart of the controller and no a priori assumptions are made about the stability of the plant andboundedness of the input. If the objective, however, is parameter estimation, then we haveto assume that the plant is stable and the input u is designed to be bounded and sufficientlyrich for the plant model (3.104). In this case, we can take n2

s = 0.

Theorem 3.13.1. Consider the plant model (3.104) and assume that (Ap, Bp) is controllableand Ap is a stable matrix. If each element ui , i = 1, 2, . . . , m, of vector u is bounded,sufficiently rich of order n+1, and uncorrelated, i.e., each ui contains different frequencies,then Ap(t), Bp(t) generated by (3.109) (where ns can be taken to be zero) converge to Ap,Bp, respectively, exponentially fast.

The proof of Theorem 3.13.1 is long, and the reader is referred to [56] for the details.

Example 3.13.2 Consider the second-order plant

x = Ax + Bu,

where x = [x1, x2]T , u = [u1, u2]T is a bounded input vector, the matrices A, B areunknown, and A is a stable matrix. The estimation model is generated as

˙x =[−am 0

0 −am

](x − x) +

[a11 a12

a21 a22

]x +

[b11 b12

b21 b22

]u,

where x = [x1, x2]T and am > 0. The estimation error is given by

ε = x − x,

where ns = 0 due to the stability of A and boundedness of u. The adaptive law (3.109) canbe written as ˙aij = γ1εixj ,

˙bij = γ2εiuj




�

�

�

�

�

�

�

�


for i = 1, 2, j = 1, 2, and adaptive gains γ1, γ2. An example of a sufficiently rich input forthis plant is

u1 = c1 sin 2.5t + c2 sin 4.6t,

u2 = c3 sin 7.2t + c4 sin 11.7t

for some nonzero constants ci , i = 1, 2, 3, 4.

The class of plants described by (3.104) can be expanded to include more realisticplants with modeling errors. The adaptive laws in this case can be made robust by usingexactly the same techniques as in the case of SISO plants described in previous sections,and this is left as an exercise for the reader.

3.14 Adaptive ObserversConsider the LTI SISO plant

x = Ax + Bu, x(0) = x0,

y = CT x,(3.110)

where x ∈ Rn. We assume that u is a piecewise continuous bounded function of time andthat A is a stable matrix. In addition, we assume that the plant is completely controllableand completely observable. The problem is to construct a scheme that estimates both theplant parameters, i.e., A, B, C, as well as the state vector x using only I/O measurements.We refer to such a scheme as the adaptive observer.

A good starting point for designing an adaptive observer is the Luenberger observerused in the case where A, B, C are known. The Luenberger observer is of the form

˙x = Ax + Bu + K(y − y), x(0) = x0,

y = CT x,(3.111)

where K is chosen so that A − KCT is a stable matrix, and guarantees that x → x

exponentially fast for any initial condition x0 and any input u. For A − KCT to be stable,the existence of K is guaranteed by the observability of (A,C).

A straightforward procedure for choosing the structure of the adaptive observer isto use the same equation as the Luenberger observer (3.111), but replace the unknownparameters A, B, C with their estimates A, B, C, respectively, generated by some adaptivelaw. The problem we face with this procedure is the inability to estimate uniquely the n2+2nparameters of A, B, C from the I/O data. The best we can do in this case is to estimatethe parameters of the plant transfer function and use them to calculate A, B, C. Thesecalculations, however, are not always possible because the mapping of the 2n estimatedparameters of the transfer function to the n2 +2n parameters of A, B, C is not unique unless(A,B,C) satisfies certain structural constraints. One such constraint is that (A,B,C) is in




�

�

�

�

�

�

�

�

3.14. Adaptive Observers 79

the observer form, i.e., the plant is represented as

xα =⎡⎢⎣−ap

||||

In−1−−0

⎤⎥⎦ xα + bpu,

y = [1, 0, . . . , 0]xα,(3.112)

where ap = [an−1, an−2, . . . , a0]T and bp = [bn−1, bn−2, . . . , b0]T are vectors of dimensionn, and In−1 ∈ R(n−1)×(n−1) is the identity matrix. The elements of ap and bp are thecoefficients of the denominator and numerator, respectively, of the transfer function

y(s)

u(s)= bn−1s

n−1 + bn−2sn−2 + · · · + b0s

sn + an−1sn−1 + an−2sn−2 + · · · + a0s(3.113)

and can be estimated online from I/O data using the techniques presented in the previoussections.

Since both (3.110) and (3.112) represent the same plant, we can focus on the plantrepresentation (3.112) and estimate xα instead of x. The disadvantage is that in a practicalsituation x may represent some physical variables of interest, whereas xα may be an artificialstate vector.

The adaptive observer for estimating the state xα of (3.112) is motivated from theLuenberger observer structure (3.111) and is given by

˙x = A(t)x + bp(t)u + K(t)(y − y), x(0) = x0,

y = [1, 0, . . . , 0]x, (3.114)

where x is the estimate of xα ,

A(t) =⎡⎢⎣−ap(t)

||||

In−1−−0

⎤⎥⎦ , K(t) = a∗ − ap(t),

ap(t) and bp(t) are the estimates of the vectors ap and bp, respectively, at time t , anda∗ ∈ Rn is chosen so that

A∗ =⎡⎢⎣−a∗

||||

In−1−−0

⎤⎥⎦ (3.115)

is a stable matrix that contains the eigenvalues of the observer.A wide class of adaptive laws may be used to generate ap(t) and bp(t) online. As in

Chapter 2, the parametric modelz = θ∗T φ (3.116)

may be developed using (3.113), where

θ∗ = [bTp , aTp ]T




�

�

�

�

�

�

�

�


and z, φ are available signals, and used to design a wide class of adaptive laws to generateθ(t) = [bTp (t), aTp (t)]T , the estimate of θ∗. As an example, consider the gradient algorithm

θ = �εφ, ε = z − θT φ

m2s

, (3.117)

where m2s = 1 + αφT φ and α ≥ 0.

Theorem 3.14.1. The adaptive observer described by (3.114)–(3.117) guarantees the fol-lowing properties:

(i) All signals are bounded.

(ii) If u is sufficiently rich of order 2n, then the state observation error |x − xα| and theparameter error |θ − θ∗| converge to zero exponentially fast.

Proof. (i) Since A is stable and u is bounded, we have xα, y, φ ∈ L∞ and hence m2s =

1+αφT φ ∈ L∞. The adaptive law (3.117) guarantees that θ ∈ L∞ and ε, εms, θ ∈ L2∩L∞.The observer equation may be written as

˙x = A∗x + bp(t)u + (A(t) − A∗)xα.

Since A∗ is a stable matrix and bp, A, u, xα are bounded, it follows that x ∈ L∞,which in turn implies that all signals are bounded.

(ii) The state observation error x = x − xα satisfies

˙x = A∗x + bpu − apy, (3.118)

where bp = bp −bp, ap = ap −ap are the parameter errors. Since for u sufficiently rich wehave that θ(t) → θ∗ as t → ∞ exponentially fast, it follows that bp, ap → 0 exponentiallyfast. Since u, y ∈ L∞, the error equation consists of a homogenous part that is exponentiallystable and an input that is decaying to zero. This implies that x = x − xα → 0 as t → ∞exponentially fast.

For further reading on adaptive observers, the reader is referred to [56, 86, 108–110].

3.15 Case Study: Users in a Single Bottleneck LinkComputer Network

The congestion control problem in computer networks has been identified as a feedbackcontrol problem. The network users adjust their sending data rates, in response to congestionsignals they receive from the network, in an effort to avoid congestion and converge to astable equilibrium that satisfies certain requirements: high network utilization, small queuesizes, small delays, fairness among users, etc. Many of the proposed congestion controlschemes require that at each link the number of flows, N say, utilizing the link is known.Since the number of users varies with time, N is an unknown time-varying parameter,




�

�

�

�

�

�

�

�

3.15. Case Study: Users in a Single Bottleneck Link Computer Network 81

Figure 3.2. Network topology.

which needs to be estimated online. Estimation algorithms, which have been proposed inthe literature, are based on pointwise time division, which is known to lack robustness andmay lead to erroneous estimates. In this study, we consider a simple estimation algorithm,which is based on online parameter identification.

We consider the single bottleneck link network shown in Figure 3.2. It consists ofN users which share a common bottleneck link through high bandwidth access links. Atthe bottleneck link we assume that there exists a buffer, which accommodates the incomingpackets. The rate of data entering the buffer is denoted by y, the queue size is denoted byq, and the output capacity is denoted by C. At the bottleneck link, we implement a signalprocessor, which calculates the desired sending rate p. This information is communicatedto the network users, which set their sending rate equal to p. The desired sending rate p isupdated according to the following control law:

p =

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩

1N

[ki(C − y) − kqq] if 1 < p < C,1N

[ki(C − y) − kqq] if p = 1, 1N

[ki(C − y) − kqq] > 0,1N

[ki(C − y) − kqq] if p = C, 1N

[ki(C − y) − kqq] < 0,

0 otherwise,

(3.119)

where N is an estimate of N which is calculated online and ki , kq are design parameters.Since N is changing with time, its estimate N has to be updated accordingly. In this studywe use online parameter estimation to generate N . Since the sending rate of all users isequal to p, it follows that

y = Np. (3.120)

Since y and p are measured at the bottleneck link, (3.120) is in the form of an SPMwith N as the unknown parameter. We also know that N cannot be less than 1. Using the




�

�

�

�

�

�

�

�


Figure 3.3. Time response of the estimate of the number of flows.

results of the chapter, we propose the following online parameter estimator:

˙N =

{γ εp if N > 1 or N = 1 and εp ≥ 0,

0 otherwise,

ε = y − Np

1 + p2,

(3.121)

where N(0) ≥ 1. We demonstrate the effectiveness of the proposed algorithm using sim-ulations, which we conduct on the packet-level simulator ns-2. We consider the networktopology of Figure 3.2 in our simulations. The bandwidth of the bottleneck link is set to155 Mb/s, and the propagation delay of each link is set to 20 ms. The design parametersare chosen as follows: γ = 0.1, ki = 0.16, kq = 0.32. Initially 30 users utilize the net-work. The estimator starting with an initial estimate of 10 converges to 30. After t = 30seconds 20 of these users stop sending data, while an additional 20 users enter the networkat t = 45 seconds. The output of the estimator at the bottleneck link is shown in Figure 3.3.We observe that the estimator accurately tracks the number of flows utilizing the network.In addition we observe good transient behavior as the responses are characterized by fastconvergence and no overshoots. The estimator results are obtained in the presence of noiseand delays which were not included in the simple model (3.120). Since the number ofparameters to be estimated is 1, the PE property is satisfied for p �= 0, which is always thecase in this example.

See the web resource [94] for examples using the Adaptive Control Toolbox.

Problems1. Consider the SPM

z = θ∗T φ




�

�

�

�

�

�

�

�

Problems 83

and the estimation modelz = θT (t)φ.

Find values for θ(t), φ(t) such that z = z but θ(t) �= θ∗.

2. Consider the adaptive lawθ = γ εφ,

where θ, φ ∈ Rn, φ

ms∈ L∞, εms ∈ L2 ∩ L∞, and ms ≥ 1. Show that θ ∈ L2 ∩ L∞.

3. Show that if u ∈ L∞ in (3.1), then the adaptive law (3.10) with m2s = 1+αφ2, α ≥ 0,

guarantees that ε, εms, θ ∈ L2 ∩ L∞ and that ε(t), ε(t)ms(t), θ (t) → 0 as t → ∞.

4. (a) Show that (3.16) is a necessary and sufficient condition for θ(t) in the adaptivelaw (3.10) to converge to θ∗ exponentially fast.

(b) Establish which of the following choices for u guarantee that φ in (3.10) is PE:

(i) u = c0 �= 0, c0 is a constant.

(ii) u = sin t .

(iii) u = sin t + cos 2t .

(iv) u = 11+t

.

(v) u = e−t .

(vi) u = 1(1+t)1/2 .

(c) In (b), is there a choice for u that guarantees that θ(t) converges to θ∗ but doesnot guarantee that φ is PE?

5. Use the plant model (3.24) to develop the bilinear parametric model (3.28). Show allthe steps.

6. In Theorem 3.6.1, assume that φ, φ ∈ L∞. Show that the adaptive law (3.34) withm2

s = 1 + n2s , n2

s = αφT φ, and α ≥ 0 guarantees that ε(t), ε(t)ms(t), θ (t) → 0 ast → ∞.

7. Consider the SPM z = θ∗T φ and the cost function

J (θ) = 1

2

∫ t

0e−β(t−τ) (z(τ ) − θT (t)φ(τ ))2

m2s (τ )

dτ ,

where m2s = 1 + φT φ and θ(t) is the estimate of θ∗ at time t .

(a) Show that the minimization of J (θ) w.r.t. θ using the gradient method leads tothe adaptive law

θ (t) = �

∫ t

0e−β(t−τ) z(τ ) − θT (t)φ(τ )

m2s (τ )

φ(τ)dτ , θ(0) = θ0.

(b) Show that the adaptive law in part (a) can be implemented as

θ (t) = −�(R(t)θ(t) + Q(t)), θ(0) = θ0,

R(t) = −βR(t) + φ(t)φT (t)

m2s (t)

, R(0) = 0,




�

�

�

�

�

�

�

�


Q(t) = −βQ(t) − z(t)φ(t)

m2s (t)

, Q(0) = 0,

which is referred to as the integral adaptive law.

8. Consider the second-order stable system

x =[a11 a12

a21 0

]x +

[b1

b2

]u,

where x, u are available for measurement, u ∈ L∞, and a11, a12, a21, b1, b2 areunknown parameters. Design an online estimator to estimate the unknown parameters.Simulate your scheme using a11 = −0.25, a12 = 3, a21 = −5, b1 = 1, b2 = 2.2, andu = 10 sin 2t . Repeat the simulation when u = 10 sin 2t + 7 cos 3.6t . Comment onyour results.

9. Consider the nonlinear system

x = a1f1(x) + a2f2(x) + b1g1(x)u + b2g2(x)u,

where u, x ∈ R; fi , gi are known nonlinear functions of x; and ai , bi are unknownconstant parameters and i = 1, 2. The system is such that u ∈ L∞ implies x ∈ L∞.Assuming that x, u can be measured at each time t , design an estimation scheme forestimating the unknown parameters online.

10. Design and analyze an online estimation scheme for estimating θ∗ in (3.49) whenL(s) is chosen so that W(s)L(s) is biproper and SPR.

11. Design an online estimation scheme to estimate the coefficients of the numeratorpolynomial

Z(s) = bn−1sn−1 + bn−2s

n−2 + · · · + b1s + b0

of the plant

y = Z(s)

R(s)u

when the coefficients of R(s) = sn + an−1sn−1 + · · · + a1s + a0 are known. Repeat

the same problem when Z(s) is known and R(s) is unknown.

12. Consider the mass–spring–damper system shown in Figure 3.4, where β is the damp-ing coefficient, k is the spring constant, u is the external force, and y(t) is the dis-placement of the mass m resulting from the force u.

(a) Verify that the equations of motion that describe the dynamic behavior of thesystem under small displacements are

my + βy + ky = u.

(b) Design a gradient algorithm to estimate the constants m, β, k when y, u can bemeasured at each time t .

(c) Repeat (b) for an LS algorithm.

(d) Simulate your algorithms in (b) and (c) assuming m = 20 kg, β = 0.1 kg/sec,k = 5 kg/sec2, and inputs u of your choice.




�

�

�

�

�

�

�

�

Problems 85

Figure 3.4. The mass–spring–damper system for Problem 12.

Figure 3.5. The mass–spring–damper system for Problem 13.

(e) Repeat (d) when m = 20 kg for 0 ≤ t ≤ 20 sec and m = 20(2 − e−0.01(t−20)) kgfor t ≥ 20 sec.

13. Consider the mass–spring–damper system shown in Figure 3.5.

(a) Verify that the equations of motion are given by

k(y1 − y2) = u,

k(y1 − y2) = my2 + βy2.

(b) If y1, y2, u can be measured at each time t , design an online parameter estimatorto estimate the constants k, m, and β.

(c) We have the a priori knowledge that 0 ≤ β ≤ 1, k ≥ 0.1, and m ≥ 10. Modifyyour estimator in (b) to take advantage of this a priori knowledge.

(d) Simulate your algorithm in (b) and (c) when β = 0.2 kg/sec, m = 15 kg,k = 2 kg/sec2, and u = 5 sin 2t + 10.5 kg · m/sec2.

14. Consider the block diagram of a steer-by-wire system of an automobile shown inFigure 3.6, where r is the steering command in degrees, θp is the pinion angle indegrees, and θ is the yaw rate in degree/sec.The transfer functions G0(s), G1(s) are of the form

G0(s) = k0ω20

s2 + 2ξ0ω0s + ω20(1 − k0)

,




�

�

�

�

�

�

�

�


Σ 0 ( )G s 1( )G sr θ

.

pθ

pθ

Figure 3.6. Block diagram of a steer-by-wire system for Problem 14.

G1(s) = k1ω21

s2 + 2ξ1ω1s + ω21

,

where k0, ω0, ξ0, k1, ω1, ξ1 are functions of the speed of the vehicle. Assuming thatr , θp, θ can be measured at each time t , do the following:

(a) Design an online parameter estimator to estimate ki, ωi, ξi , i = 0, 1, using themeasurements of θp, θ , r .

(b) Consider the values of the parameters shown in the following table at differentspeeds:

Speed V k0 ω0 ξ0 k1 ω1 ξ1

30 mph 0.81 19.75 0.31 0.064 14.0 0.36560 mph 0.77 19.0 0.27 0.09 13.5 0.505

Assume that between speeds the parameters vary linearly. Use these values tosimulate and test your algorithm in (a) when

(i) r = 10 sin 0.2t + 8 degrees and V = 20 mph.(ii) r = 5 degrees and the vehicle speeds up from V = 30 mph to V = 60 mph

in 40 seconds with constant acceleration and remains at 60 mph for 10seconds.

15. Consider the equation of the motion of the mass–spring–damper system given inProblem 12, i.e.,

my + βy + ky = u.

This system may be written in the form

y = ρ∗(u − my − βy),

where ρ∗ = 1k

appears in a bilinear form with the other unknown parameters m, β.Use the adaptive law based on the bilinear parametric model to estimate ρ∗, m, βwhen u, y are the only signals available for measurement. Since k > 0, the sign ofρ∗ may be assumed to be known. Simulate your adaptive law using the numericalvalues given in (d) and (e) of Problem 12.

16. The effect of initial conditions on the SPM can be modeled as

z = θ∗T φ + η0,




�

�

�

�

�

�

�

�

Problems 87

ω0 = �ω0,

η0 = CT ω0,

where � is a transfer matrix with all poles in �[s] < 0 and ω0 ∈ Rn, η0 ∈ R. Showthat the properties of an adaptive law (gradient, LS, etc.) with η0 = 0 are the same asthose for the SPM with η0 �= 0.

17. Consider the system

y = e−τs b

(s − a)(µs + 1)u,

where 0 < τ � 1, 0 < µ � 1, and a, b, τ , µ are unknown constants. We want toestimate a, b online.

(a) Obtain a parametric model that can be used to design an adaptive law to esti-mate a, b.

(b) Design a robust adaptive law of your choice to estimate a, b online.

(c) Simulate your scheme for a = −5, b = 100, τ = 0.01, µ = 0.001, and differentchoices of the input signal u. Comment on your results.

18. The dynamics of a hard-disk drive servo system are given by

y = kp

s2

6∑i=1

b1i s + b0i

s2 + 2ζiωis + ω2i

u,

where ωi , i = 1, . . . , 6, are the resonant frequencies which are large, i.e., ω1 =11.2π × 103 rad/sec, ω2 = 15.5π × 103 rad/sec, ω3 = 16.6π × 103 rad/sec, ω4 =18π×103 rad/sec, ω5 = 20π×103 rad/sec, ω6 = 23.8π×103 rad/sec. The unknownconstants b1i are of the order of 104, b0i are of the order of 108, and kp is of the orderof 107. The damping coefficients ζi are of the order of 10−2.

(a) Derive a low-order model for the servo system. (Hint: Take αωi

∼= 0 and henceα2

ω2i

∼= 0 for α of the order of less than 103.)

(b) Assume that the full-order system parameters are given as follows:

b16 = −5.2 × 104, b01 = 1.2 × 109, b02 = 5.4 × 108,

b03 = −7.7 × 108, b04 = −1.6 × 108, b05 = −1.9 × 108,

b06 = 1.2 × 109, kp = 3.4 × 107, ζ1 = 2.6 × 10−2,

ζ2 = 4.4 × 10−3, ζ3 = 1.2 × 10−2, ζ4 = 2.4 × 10−3,

ζ5 = 6.8 × 10−3, ζ6 = 1.5 × 10−2.

Obtain a Bode plot for the full-order and reduced-order models.

(c) Use the reduced-order model in (a) to obtain a parametric model for the unknownparameters. Design a robust adaptive law to estimate the unknown parametersonline.




�

�

�

�

�

�

�

�


19. Consider the time-varying plant

x = −a(t)x + b(t)u,

where a(t), b(t) are slowly varying unknown parameters; i.e., |a|, |b| are very small.

(a) Obtain a parametric model for estimating a, b.

(b) Design and analyze a robust adaptive law that generates the estimates a(t), b(t)of a(t), b(t), respectively.

(c) Simulate your scheme for a plant with a(t) = 5 + sin µt , b(t) = 8 + cos 2µtfor µ = 0, 0.01, 0.1, 1, 5. Comment on your results.

20. Consider the parameter error differential equation (3.69), i.e.,

˙θ = −γ u2θ + γ du.

Show that if the equilibrium θe = 0 of the homogeneous equation

˙θ = −γ u2θ

is exponentially stable, then the bounded input γ du will lead to a bounded solutionθ (t). Obtain an upper bound for |θ (t)| as a function of the upper bound of thedisturbance term γ du.

21. Consider the system

y = θ∗u + η,

η = �(s)u,

where y, u are available for measurement, θ∗ is the unknown constant to be estimated,and η is a modeling error signal with �(s) being proper and analytic in �[s] ≥ −0.5.The input u is piecewise continuous.

(a) Design an adaptive law with a switching σ -modification to estimate θ∗.

(b) Repeat (a) using projection.

(c) Simulate the adaptive laws in (a), (b) using the following values:

θ∗ = 5 + sin 0.1t,

�(s) = 10µµs − 1

(µs + 1)2

for µ = 0, 0.1, 0.01 and u = constant, u = sin ω0t , where ω0 = 1, 10, 100.Comment on your results.

22. The linearized dynamics of a throttle angle θ to vehicle speed V subsystem are givenby the third-order system

V = bp1p2

(s + a)(s + p1)(s + p2)θ + d,

where p1, p2 > 20, 1 ≥ a > 0, and d is a load disturbance.




�

�

�

�

�

�

�

�

Problems 89

(a) Obtain a parametric model for the parameters of the dominant part of the system.

(b) Design a robust adaptive law for estimating these parameters online.

(c) Simulate your estimation scheme when a = 0.1, b = 1, p1 = 50, p2 = 100, andd = 0.02 sin 5t , for different constant and time-varying throttle angle settings θof your choice.

23. Consider the parametric model

z = θ∗T φ + η,

whereη = �u(s)u + �y(s)y

and �u, �y are proper transfer functions analytic in �[s] ≥ − δ02 for some known

δ0 > 0.

(a) Design a normalizing signal ms that guarantees η

ms∈ L∞ when (i) �u, �y are

biproper, (ii) �u, �y are strictly proper. In each case specify the upper boundfor |η|

ms.

(b) Calculate the bound for |η|ms

when (i)�u(s) = e−τs−1s+2 ,�y(s) = µ s2

(s+1)2 , (ii)�u(s) =µs

µs+2 , �y(s) = µs

(µs+1)2 , where 0 < µ � 1 and 0 < τ � 1.

(c) Design and simulate a robust adaptive law to estimate θ∗ for the followingexample:

θ∗ = [1, 0.1]T ,φ = [u, y]T ,z = s

s + 5y,

�u(s) = e−τs − 1

s + 2, �y(s) = µ(s − 1)

(µs + 1)2,

where τ = 0.01, µ = 0.01.



Documents

Chapter 3 D Pl Parameter Identiﬁcation: P E … Identiﬁcation: ... of the parametric model SPM, DPM, B-SPM, ... 2007 The Society for Industrial and Applied Mathematics