8/9/2019 Adaptive Control Using Multiple Models, Switching, And
1/6
Adaptive Control
using
Multiple Models, Switching, and
Tuning
Kumpat i S. Narendra and Osvaldo
A.
Driollet
Center for Systems Science
Department of Electrical Engineering
Yale University, New Haven , C T 06520, USA
Abstract
The paper describes the efforts in recent years to in-
crease substantially the scope of adaptive control by
using multiple models, switching, and tuning. Both
deterministic and stochastic systems are considered
and stability and convergence problems are discussed.
While deeply rooted in conventional adaptive control
theory, the multiple model approach provides greater
flexibility in design, and in many cases may result in
faster, more accurate, and more robust systems. Prac-
tical applications in flight control, process control, the
control of
a
flexible transmission system
as
well
as
a so-
lar power plant, where the approach has already proved
to be successful, are also described.
1 Introduction
Speed, accuracy and robustness are the prerequisites
of any good control system. Achieving them in the
presence of complexity, nonlinearity, uncertainty, and
time-variations is the challenge for the control theorist
today. The proof of stability of the ideal adaptive
control problem was given in 1980, and during the fol-
lowing ten years numerous algorithms were developed
to make the adaptive systems robust in the presence of
bounded disturbances, unmodeled dynamics, and time-
varying parameters. In recent years nonlinear adaptive
control has attracted much a ttention, and methods for
coping with nonlinearities, albeit in special structures,
have also been developed. In spite of these impressive
achievements, conventional adaptive control based on a
single model suffers from some basic limitations. When
the dynamics of the controlled process changes rapidly
or even discontinuously with time, conventional a d a p
tive control may not be able t o cope with it, since large
changes in system dynamics can result in large and un-
acceptable transients. Further, while algorithms tai-
lored to specific disturbances have been developed, in
practice the nature
of
the disturbance may vary, calling
for rapid changes in the algorithm used. Finally, it is
well known, that even in
a
single environment, many
different designs of adaptive controllers are possible,
0-7803-5800-7/00$10.0002000 IEEE
based on different assumptions concerning the system
as well as the overall objective. Which of them would
prove superior in a given situation is always not ev-
ident, and simulation studies may have to be carried
out to make
a
proper choice. However, such
a
proce-
dure does not lend itself for use in real-time control
situations. All the above problems call for a different
approach to adaptive control.
It is well known tha t the great efficiency of control
strategies in the biological world was responsible for
the creation of terms in systems theory such as Adap-
tat ion, Learning, Patt ern Recognition and Ar-
tificial Intelligence. Biological systems are known to
create a wide repertoire of behaviors for different situ-
ations, and choose the one that is most appropria te for
a given situation. They are also known to combine dif-
ferent actions to cause novel behaviors. The adaptive
systems described in this paper, first introduced in
111
may be considered as a first step in the mathematical
representation and practical design of such systems.
In the past fifteen years there has also been a dramatic
increase
in
research into the computational capabili-
ties of neural networks, which are highly interconnected
networks of simple processing units. By their very na-
ture artificial neural networks can cope with complex-
ity, uncertainty, and nonlinearity, and neural networks
have been used successfully to identify and control non-
linear dynamical systems. Towards the end of this pa-
per, we will comment briefly on how an integration of
neural networks and control concepts involving switch-
ing and tuning is leading to
a
new class of adaptive
systems that can perform satisfactorily in more com-
plex environments.
2
Multiple
Models
To implement the approach described in the previous
section in engineering systems we need
a
multiplicity
of mathematical models. It is well known that every
model of the system is based on some simplifying as-
sumptions, and that different models may be useful
159
8/9/2019 Adaptive Control Using Multiple Models, Switching, And
2/6
Esf imf ion
Model
l
Ident error 1
-
M h
.................
-
r
ContmllerC,
DesiredOutput
Figure 1: Adaptive control using multiple models.
for understanding different aspects of the same phe-
nomenon. Multiple models are needed to describe the
plant characteristics when they change rapidly with
time,
as
for example, when there is
a
fault or
a
sub-
system failure. In some cases, two or more models may
be included t o realize their combined advantages.
2.1
Switching and Tuning
Switching and tuning are mutually exclusive and logi-
cally exhaustive methods of adjusting parameters. In
conventional adaptive control, parameters are tuned to
improve a performance criterion. When the tuning set
is discrete, e.g.
a
finite number of models from which
one has to be chosen
as
an approximation of the plant,
there is no alternative to switching. Both switching
and tuning arise in control of systems in the presence
of dynamical systems, and both have been extensively
studied.
2.2 Adaptive Control Us ing Switching and Tun-
ing
The control of a plant (or dynamical system) using mul-
tiple models is best described using Figure
l .
At every
instant t he error between the outputs predicted by the
different models
li
and the actual plant are computed,
and using some performance criterion, one of the mod-
els is chosen at t hat instant as being the best. Corre-
sponding to each model li,a controller
i
is designed,
and when one model is selected, the corresponding con-
troller is used. This is the switching part of the system.
Since a match between the plant and the model is rarely
perfect, adapta tion or tuning the parameters of one or
more of the models (and controllers) may be needed to
improve performance. If switching and tuning are to be
used to improve the performance of a system, it must
be first demonstrated that the processes will converge.
Questions that naturally arise are the criterion to be
used for switching, and the model to which the system
should switch, as well as how switching might affect
the stability and the performance of the system. Since
both accuracy and speed are of interest, the factors that
govern both quantities have also to be investigated.
As the system grows in complexity, th e number of dif-
ferent situations that can exist also grows, and corre-
sponding to each of these situations the system must
have a model and
a
controller. Dealing with such a
situation in
a
computationally efficient manner is one
of the challenges facing the control theorist.
2.3
The Problem
A
linear time-invariant discrete-time plant of order
n
has input
U
and output y . Its transfer function is
W,(z) =
k p B , where k p and the
272 - 1
coeffi-
cients of the monic polynomials
Z, z)
and
Rp(z)
on-
stitute the elements of the unknown parameter vector
p
of the plant.
S
is a compact set in the, parameter
space and
p
E S C The plant is assumed to have
a known relative degree (delay) d and the polynomial
Z, z) is stable. The objective is to determine
a
control
input
U*
such that limk_,w[y(k)
y* k)]
=
0
where
y* lc)
=
Wm(z)r(k) s
a
desired output, and r lc) is
an arbitrary bounded signal. For continuous-time sys-
tems the problem can be stated in terms of the transfer
function of the plant and the reference model.
The Structure of the Controller In Figure
1
4 , 2,-.-,N are the N predictive models used to
estimate the parameters of the plant.
CI
2,
.. CN
are
N
controllers corresponding to the
N
predictive
models, and
as
mentioned earlier, the controller
Ci
is
used
to control the
plant at any
instant if the model
Ii
is judged to be the best at that instant.
If 3ji
is the
output of model
Ii, = ei
is the output prediction
error, and
a
performance criterion
J ( e i ) i
=
1 , 2 ,
...
N
is used to determine which of the pairs
[ I j Cj]
is used
at
any specific instant. The choice of
J ( e i )
determines
how rapidly the system can switch from one model
to another and hence the overall performance of the
system.
3 Deterministic and Stochastic Adaptive
Control
The deterministic adaptive control problem of a
continuous-time plant, along the lines described in the
previous section was first introduced in
[l].
n
[2]
differ-
ent schemes for switching and tuning were considered,
and the conditions for the stability of the correspond-
ing algorithms were discussed. In [3] the same prob-
lems were considered in a stochastic setting when
a
disturbance affects the output of a discrete-time plant.
The principal contribution of [3] was the demonstration
that in spite
of
the switching and tuning, all the signals
would be mean square bounded and that the adaptive
160
8/9/2019 Adaptive Control Using Multiple Models, Switching, And
3/6
controller would evolve to
a
minimum-variance con-
troller. From the analysis of the stochastic adaptive
control problem contained in [3], the importance of the
parameter estimation scheme in proving convergence
of the control problem became evident. This was made
use of in
[4]
o demonstrate that different estimation
schemes, which result asymptotically in similar inequal-
ities involving outputs and parameter errors, could be
used simultaneously to improve the performance of a
stochastic adaptive system in the presence of a random
disturbance.
In this paper we shall consider both continuous-time
and discrete-time systems and examine critically the
results presented in all four papers [1]-[4], which repre-
sent the s tate of the a rt in thi s area. Following this we
will discuss how some of these ideas can be extended
to more complex cases. Towards the end of the paper,
applications based on the Multiple Models Switching
and Tuning (MMST) approach are discussed.
3.1 Deterministic Adaptive Control
Consider the equation
Y k)
=
4 v
Q O
where
q5(k)
E Rn s a regression vector
at
time k ,
80 E
R
is an unknown constant parameter vector of
the plant, and
y k)
is the output of the plant at time
k.
Equation (1) is an input-output description
of
the
deterministic plant t o be controlled. For simplicity it is
assumed that t he delay of the plant is unity. Numerous
recursive algorithms have been proposed to estimate
the parameter
80.
In one such algorithm (the recursive
least squares) the estimation model is described by the
equation
(IC) =
@ ( k
1)8(k
1)
d k)
=
d k
1
+ ~ ( k1)4(k )e(k)
(2)
where the parameter estimate is updated
as
(3)
and
P(k
) , the matrix of adaptive gains, is defined
as
e(k) = y k)
( k )
s the prediction error in the output
estimate.
3.1.1 Adaptive Control
using a Single
Mo de l: At very instant
k ,
assuming that the param-
eter estimate
e ( k )
s known, the input
u ( k )
s computed
using the certainty equivalence principle, so that
Q(k)
=
y* k)
=
p ( k @ 1
It has been shown
[3]
that this procedure results in all
the signals remaining bounded and the output error
tending to zero asymptotically, i.e. limk,,
e ( k ) = 0.
In [3] it is also shown that the same result holds when
the delay of the system is
d
> 1 so that
y(k +
d )
=
4 T k ) w ) .
3.1.2 Adaptive Control
using
Multiple
Models : In section 3.1.1 a single prediction model
was used to estimate the parameter vector 80. For rea-
sons that will be described later in this section, we
consider now the case where
N
predictive models of
identical structure are used to estimate 80. While all
the estimation models use the same adaptation law (3),
the initial conditions of th e models may be chosen ran-
domly. We denote the estimate at instant
k
of the
model
Ii
by
ei(k).
The controller
Ci
uses the certainty
equivalence principle to compute
ui k),
ts estimate of
the input
u ( k )
o be used
at
tha t instant. Let the con-
trollers
Ci
be chosen at random at every instant.
It
can be shown
[3]
that the overall system is globally
stable, that all the signals are bounded, and that the
plant estimation error e(k), and the plant controller
error e,(k) =
y(k)
* k) tend t o zero.
Note:
The above fact implies tha t stability and perfor-
mance can be decoupled. Hence, switching algorithms
based on heuristics can be chosen to improve the per-
formance of the system while retaining stabili ty. In
the deterministic case, performance criteria
Ji i =
1,.
N )
based on the estimation error can be cho-
sen to switch from one controller to another.
J i ( k ) =
e i 2 ( k ) , J i ( k )=
&Cfef(k)and Ji(k)= $Cf-T+le?(k)
which correspond t o instantaneous, time-averaged, and
average error over
a
finite interval have been used.
A t
every instant
k ,
the identifier-controller pair
[Ii,C i ]
is
chosen where J i ( k ) = mini J j ( k ) and the correspond-
ing input
ui k)
s applied t o the plant.
3.1.3 Fixed and Adaptive Models:
The ap-
proach described above is fast and accurate, but is
computationally intensive. In view of this, numerous
schemes involving fixed and adaptive models were sug-
gested in [l ]. Th e scheme tha t was recommended over
all the others consists of
N
2 fixed models, a free
running adaptive model, and an adaptive model that
is initialized at every instant
k
at precisely the point
in parameter space where the fixed model is found to
be optimal at that instant. This reinitialized adaptive
model is used only so long
as
no other fix model is
chosen. If at a later instant another fixed model Ij is
chosen, the previous reinitialized adaptive model at Ii
is replaced by the one
at I j .
The proof of convergence
of this scheme is given in [2].
161
8/9/2019 Adaptive Control Using Multiple Models, Switching, And
4/6
3.2 Stochastic Adaptive Control
In the stochastic adaptive control problem, the linear
system with unknown parameter 00 has an additional
input w ( k ) besides the control input
u k).
A natural
description of the system is
A ( q - ' ) y ( k ) = q-dB(q- )u(k) C(q- )w(k)
where A and
C
are monic polynomials of degrees n and
1
in
q-
and B(q- ) is a stable polynomial of degree
m.
w k ) is
a
white noise sequence which is mean squared
bounded, with zero conditional mean and finite vari-
ance (i.e. E[w k ) lk
11 =
0; E[w2 k)lk
11 =
c2 .
If
a
single estimation model is used to control the sys-
tem, the unknown parameters of the polynomials A ,
B
and
C
are estimated as in the deterministic case us-
ing the output prediction error and t he Extended Least
Squares method (ELS). The forms of the equations de-
scribing the plant and the estimation model are the
same as in th e deterministic case, i.e.
(IC)
=
f )*(k )e^(k
-
1)
where
+ T ( k
- d )
= [y(k
d) ,
. .,y k +
1 ) , u k
-
d ) ,
u(k
- m d +
1)
-&(k
) , .
,&(k + l ]
60 = [ Q O , Q l , . . ,Qn7h00,.
.
7 P m + d - - l , C l ,
c27
. cn]
where g(k) = 4 T ( k
- d)e^(k
- 1) is the
a
posteriori
estimate
of
y k)
at time k . It is shown in
[3],
that if
an ELS method is used to estimate
80,
nd certainty
equivalence control u(k) is chosen at instant k based
on the estimate
&(k),
the overall system converges to
a minimum variance controller.
3.2.1 Adaptive Control using Multiple
Models: The MMST approach can be extended to
the case of multiple models along the lines described
for the deterministic problem. However, it must be
noted that while the regression vector in the determin-
istic case is identical for
all
the models, it is not the
same in the stochastic case.
In
particular, the a poste-
riori estimates are different for the N models. Hence,
as
the system switches from one controller to another,
the regression vector also changes. This makes stabil-
ity analysis quite complex. However, in
[3]
it is shown
using a modified Kronecker lemma that all the regres-
sion vectors (if unbounded) can only grow
at
the same
rate. This in turn permits arguments similar to those
in the deterministic case to be used in the stochastic
case as well.
3.2.2 Adaptive Control using Multiple Es-
The proof of convergence
imation Algorithms:
given in
[3]
has strong consequences for the use of
the MMST approach in other contexts. It is well
known that many recursive methods have been devel-
oped in the system identification literature which have
advantages under suitable assumptions about the dis-
turbance. In particular, if the estimates given by dif-
ferent schemes can only grow at the same rate (if an
unstable controller is used) it follows that the MMST
method will result in bounded minimum variance con-
trol . Using multiple estimation schemes to improve
performance in the presence of time-varying distur-
bances has been treated in
[4].
4 Adaptive Control
of
Nonlinear Systems
using Neural Networks
The MMST approach developed for linear systems at-
tains its full potential when dealing with nonlinear sys-
tems. The structure of the overall system is the same as
tha t shown in Figure
1,
but
Ii
and
Ci
are approximated
using neural networks.
From a system theoretic point of view neural net-
works can be considered as practically implementable
parametrizations of nonlinear maps from one finite di-
mension space to another. In identification and control
problems our objective is to first demonstrate theoret-
ically tha t certain nonlinear maps exist, and later use
neural network to approximate them. For example, it
can be shown that an input-output (NARMA) repre-
sentation
of
the form
exists for
a
nonlinear plant of degree
n
in the neighbor-
hood of its equilibrium state, and tha t the correspond-
ing input
u(k)
o the system can be expressed in terms
of the past values of the inputs and outputs as well
as
the desired output
r ( k )
as
Hence, neural network models having the form
and
6 ( k
+
d ) = n i ~ [ y k ) , k ) , .
y k
n
+
l ,
T ( k ) , U ( k ) ,
k
),
.
. .
u(k
1 I ]
162
8/9/2019 Adaptive Control Using Multiple Models, Switching, And
5/6
can be used to approximate them. Using such represen-
tations, methods for tracking bounded desired outpu ts
have been studied extensively in th e lite rature
[5],
and
such methods have also been extended to multivari-
able systems aswell
as
disturbance rejection problems.
The success of neural networks in t he above case pro-
vided the incentive to attempt their use in control prob-
lems using the MMST approach. While the methodol-
ogy is the same, the implementation raises many ques-
tions concerned with the location of different models,
creation and deletion of models, speed of adaptation,
stabili ty, and robustness. Considerable advances have
been made in each of the above areas. In particular,
it has been recently shown [6] that under suitable as-
sumptions concerning the nonlinear plant to be con-
trolled, the MMST approach using
a
combination of
linear and neural network based identifiers and con-
trollers can improve performance of the system while
assuring stability. In
a
similar fashion, under appro-
priate assumptions, it also appears possible to com-
bine advanced information processing capabilities such
as
pattern recognition, adaptation, learning, and opti-
mization to perform satisfactorily in complex systems
in the presence of nonlinearity and uncertainty.
5 Applications
The MMST approach discussed in the previous section
combines fixed and adaptive models in novel ways and
results in
fast
and accurate adaptive control even while
remaining computationally efficient. As such, it is find-
ing
a
wide following in industry in areas where large
changes in system dynamics are anticipated and adap-
tive control using
a
single model is found to be inade-
quate. Four specific applications in which the method
has proved particularly successful are described below.
Flight Control: Fast and accurate flight control recon-
figuration is of paramount importance for increasing
aircraft survivability in the presence of subsystem fail-
ures and struc tural damage. The MMST approach ap-
pears to be eminently suited to cope with such situa-
tions. BoSkoviC and his colleagues at Scientific Systems
Inc. have been systematically applying the approach
to flight control problems [7]. They have proposed new
parametrizations for the modeling of control effector
failures of different types, which naturally lead to a
multiple model formulation of the problem. The most
complex case considered by them is one in which one of
the effectors undergoes float, lock-in-place, or hard-over
failure, while all others lose effectiveness. Computer
simulations of a linearized model of the F/A - 18A air-
craft during carrier landing maneuvers using five mod-
els demonstrated that excellent performance under
a
critical effector failure could be achieved. The large
transien ts resulting from adaptive control of the same
system using
a
single model made the la tter unaccept-
able. A t the present time BoSkovi6 and his group are
investigating the robustness of the MMST approach.
Process Control: Another area where the MMST ap-
proach may prove attractive is in process control.
Chemical companies, interested in the applicability of
advanced control theories to industrial process control
have presented benchmark problems that include many
difficulties encountered by the process control indus-
tries. These include nonlinearities, product transitions,
constrains and delays. Recently, Gundala and
Hoo [8]
have applied the MMST approach to control a MIMO
nonlinear stable two-phase chemical reactor. The au-
thors conclude that the approach is particularly attrac-
tive when the process is known to transition to un-
known operating states. The response obtained was
faster and more accurate compared to a multiple
PI
controller strategy.
A
Flexible Transmission System: An application of the
MMST approach to
a
flexible transmission system is
presented En [9]. Flexible systems with low damping
factor which occur in aerospace applications are in gen-
eral very difficult to control, particularly in the pres-
ence of large load variations. Experiments have shown
that a fixed high performance controller designed for
one loading may lead to instability for another load-
ing. While several k e d parameter controllers have
been designed to assure robust stability, very high per-
formance could not be achieved by any of them for
all loadings. Adaptive control using a single model
of the plant resulted in large transients with sudden
changes in load, and parameter drifts, when the in-
put signals are not persistently exciting, resulting in
instability due to unmodeled dynamics. The MMST
approach together with a closed loop output error pa-
rameter estimation technique was applied to the above
problem and resulted in improved stability and perfor-
mance of the overall system.
Solar Power Plant: The adaptive control problem of a
solar power plant is the subject of [lo ]. The plant , em-
ploying a distributed collector field has to collect solar
energy and transfer it to oil being piped through the
system. To use the heated oil for power production,
the main control requirement is to maintain the outlet
temperature of the field at a constant value. This is
achieved by adjusting the flow of oil. Since solar ra-
diation changes substantially during operation due to
the daily solar cycle, as well as atmospheric conditions,
there are significant variations in the dynamic charac-
teristics (e.g. response rate, time delays, etc.) of the
field. To cope with such fast variations a switching
controller, using different ARMAX models of the plant
for different operating points, was used to control the
system. In contrast to t he previous applications, only
switching between multiple models was used. The mul-
163
8/9/2019 Adaptive Control Using Multiple Models, Switching, And
6/6
timodel scheme was found to be superior to the con-
ventional adaptive control scheme both in regard to
robustness in the presence of disturbances, as well
as
the amount of prior information needed to design the
controller.
6 Conclusions
The paper discusses recent developments in the area
of adaptive control using MMST. The main idea of
the approach is to use multiple models, each of which
corresponds to a different environment in which the
plant may have to operate. To reduce the compu-
tational burden, only two models are adaptive while
all of the others are fixed and provide better initial
conditions for one of the adaptive models. The proofs
of convergence for both, continuous-time [ l , 21 and
discrete-time stochastic system [3] have been given.
Simulation studies of numerous practical systems have
also demonstrated the superiority of the approach
over conventional adaptive control. Applications in
flight control, process control, the control of
a
flexible
transmission system and
a
solar power plant reveal
that the approach is practically feasible. However,
many issues remain which must be investigated before
the MMST approach can be used with confidence.
These include the selection of reference models, a
detailed analysis of the robustness of the scheme under
different types of perturbations , and the extension t o
nonlinear systems.
Acknowledgment
The research reported here
w s
supported by Contract
N00014-97-1-0948 from the Office of Naval Research.
References
[l]
K.
S. Narendra and J. Balakrishnan, Improving
Transient Response of Adaptive Control Systems using
Multiple Models and Switching,
IEEE Trans. on Auto-
matic Control, vol. 39, no. 9, pp.1861-1866, September
1994.
[2]
K. S.
Narendra and J. Balakrishnan, Adaptive
Control Using Multiple Models and Switching,
IEEE
Trans.
on Automatic Control, vol. 42, no. 2, pp.171-
187, February 1997.
[3]
K. S.
Narendra and C. Xiang, Adaptive Control
of Discrete-time S yste ms Using Multiple Models, IEEE
Trans. on Automatic Control (To appear in September
2000). Also, Technical Reports 9801 and 9803, Center
for
Systems Science, Yale University, 1998.
[4]
K. S.
Narendra and
0.
Driollet, Stochastic Adap-
tive Control Using Multiple Estimation Models, Tech-
nical Report 2002, Center for Systems Science, Yale
University, 2000. To be published in The International
Journal of Adaptive Control and Signal Processing,
Special Issue on Switching and Logic in Adaptive Con-
trol.
[5]
J.
Cabrera and K.
S
Narendra,
Issues
in
the Ap-
plication of Neural Networks for Tracking Based on In-
verse Control, IEEE Trans. on Automatic Control, vol.
44, no. 11, pp.2007-2027, November 1999.
[6] L. Chen and
K . S.
Narendra, Nonlinear Adaptive
Control Using Neural Networks and Multiple Models,
To appear in the Proceedings of the lg th American
Control Conference, 2000.
[7] J. BoSkoviC and R. Mehra, Stable Multiple Model
Adaptive Flight Control for Accomodation of a Large
Class
of
Control Effector Failures, Proceedings of the
18thAmerican Control Conference 1999, pp.1920-1924.
[8] It. Gundala and
K.
Hoo, Multiple Model Adap-
t ive Control Design
for
a MIMO Chemical Reactor,
To
appear in Industrial and Engineering Chemistry Re-
search.
191 A . Karimi and I. Landau, Robust Adaptive Con-
trol of a Flexible Transmission System Using Multi-
ple Models, Control Systems Technology, vo1.8, n0.2,
pp.321-331, March 2000.
[ lo] R. Pickhardt, Application
of
Adaptive Con-
trollers to a Solar Power Plant using Multiple Model
Descript ion, Proceedings of the 17th American Control
Conference 1998, pp.2895-2899.
.
164