Adaptive Control Using Multiple Models, Switching, And

8/9/2019 Adaptive Control Using Multiple Models, Switching, And

1/6

Adaptive Control

using

Multiple Models, Switching, and

Tuning

Kumpat i S. Narendra and Osvaldo

A.

Driollet

Center for Systems Science

Department of Electrical Engineering

Yale University, New Haven , C T 06520, USA

Abstract

The paper describes the efforts in recent years to in-

crease substantially the scope of adaptive control by

using multiple models, switching, and tuning. Both

deterministic and stochastic systems are considered

and stability and convergence problems are discussed.

While deeply rooted in conventional adaptive control

theory, the multiple model approach provides greater

flexibility in design, and in many cases may result in

faster, more accurate, and more robust systems. Prac-

tical applications in flight control, process control, the

control of

a

flexible transmission system

as

well

as

a so-

lar power plant, where the approach has already proved

to be successful, are also described.

1 Introduction

Speed, accuracy and robustness are the prerequisites

of any good control system. Achieving them in the

presence of complexity, nonlinearity, uncertainty, and

time-variations is the challenge for the control theorist

today. The proof of stability of the ideal adaptive

control problem was given in 1980, and during the fol-

lowing ten years numerous algorithms were developed

to make the adaptive systems robust in the presence of

bounded disturbances, unmodeled dynamics, and time-

varying parameters. In recent years nonlinear adaptive

control has attracted much a ttention, and methods for

coping with nonlinearities, albeit in special structures,

have also been developed. In spite of these impressive

achievements, conventional adaptive control based on a

single model suffers from some basic limitations. When

the dynamics of the controlled process changes rapidly

or even discontinuously with time, conventional a d a p

tive control may not be able t o cope with it, since large

changes in system dynamics can result in large and un-

acceptable transients. Further, while algorithms tai-

lored to specific disturbances have been developed, in

practice the nature

of

the disturbance may vary, calling

for rapid changes in the algorithm used. Finally, it is

well known, that even in

a

single environment, many

different designs of adaptive controllers are possible,

0-7803-5800-7/00$10.0002000 IEEE

based on different assumptions concerning the system

as well as the overall objective. Which of them would

prove superior in a given situation is always not ev-

ident, and simulation studies may have to be carried

out to make

a

proper choice. However, such

a

proce-

dure does not lend itself for use in real-time control

situations. All the above problems call for a different

approach to adaptive control.

It is well known tha t the great efficiency of control

strategies in the biological world was responsible for

the creation of terms in systems theory such as Adap-

tat ion, Learning, Patt ern Recognition and Ar-

tificial Intelligence. Biological systems are known to

create a wide repertoire of behaviors for different situ-

ations, and choose the one that is most appropria te for

a given situation. They are also known to combine dif-

ferent actions to cause novel behaviors. The adaptive

systems described in this paper, first introduced in

111

may be considered as a first step in the mathematical

representation and practical design of such systems.

In the past fifteen years there has also been a dramatic

increase

in

research into the computational capabili-

ties of neural networks, which are highly interconnected

networks of simple processing units. By their very na-

ture artificial neural networks can cope with complex-

ity, uncertainty, and nonlinearity, and neural networks

have been used successfully to identify and control non-

linear dynamical systems. Towards the end of this pa-

per, we will comment briefly on how an integration of

neural networks and control concepts involving switch-

ing and tuning is leading to

a

new class of adaptive

systems that can perform satisfactorily in more com-

plex environments.

2

Multiple

Models

To implement the approach described in the previous

section in engineering systems we need

a

multiplicity

of mathematical models. It is well known that every

model of the system is based on some simplifying as-

sumptions, and that different models may be useful

159


2/6

Esf imf ion

Model

l

Ident error 1

-

M h

.................

-

r

ContmllerC,

DesiredOutput

Figure 1: Adaptive control using multiple models.

for understanding different aspects of the same phe-

nomenon. Multiple models are needed to describe the

plant characteristics when they change rapidly with

time,

as

for example, when there is

a

fault or

a

sub-

system failure. In some cases, two or more models may

be included t o realize their combined advantages.

2.1

Switching and Tuning

Switching and tuning are mutually exclusive and logi-

cally exhaustive methods of adjusting parameters. In

conventional adaptive control, parameters are tuned to

improve a performance criterion. When the tuning set

is discrete, e.g.

a

finite number of models from which

one has to be chosen

as

an approximation of the plant,

there is no alternative to switching. Both switching

and tuning arise in control of systems in the presence

of dynamical systems, and both have been extensively

studied.

2.2 Adaptive Control Us ing Switching and Tun-

ing

The control of a plant (or dynamical system) using mul-

tiple models is best described using Figure

l .

At every

instant t he error between the outputs predicted by the

different models

li

and the actual plant are computed,

and using some performance criterion, one of the mod-

els is chosen at t hat instant as being the best. Corre-

sponding to each model li,a controller

i

is designed,

and when one model is selected, the corresponding con-

troller is used. This is the switching part of the system.

Since a match between the plant and the model is rarely

perfect, adapta tion or tuning the parameters of one or

more of the models (and controllers) may be needed to

improve performance. If switching and tuning are to be

used to improve the performance of a system, it must

be first demonstrated that the processes will converge.

Questions that naturally arise are the criterion to be

used for switching, and the model to which the system

should switch, as well as how switching might affect

the stability and the performance of the system. Since

both accuracy and speed are of interest, the factors that

govern both quantities have also to be investigated.

As the system grows in complexity, th e number of dif-

ferent situations that can exist also grows, and corre-

sponding to each of these situations the system must

have a model and

a

controller. Dealing with such a

situation in

a

computationally efficient manner is one

of the challenges facing the control theorist.

2.3

The Problem

A

linear time-invariant discrete-time plant of order

n

has input

U

and output y . Its transfer function is

W,(z) =

k p B , where k p and the

272 - 1

coeffi-

cients of the monic polynomials

Z, z)

and

Rp(z)

on-

stitute the elements of the unknown parameter vector

p

of the plant.

S

is a compact set in the, parameter

space and

p

E S C The plant is assumed to have

a known relative degree (delay) d and the polynomial

Z, z) is stable. The objective is to determine

a

control

input

U*

such that limk_,w[y(k)

y* k)]

=

0

where

y* lc)

=

Wm(z)r(k) s

a

desired output, and r lc) is

an arbitrary bounded signal. For continuous-time sys-

tems the problem can be stated in terms of the transfer

function of the plant and the reference model.

The Structure of the Controller In Figure

1

4 , 2,-.-,N are the N predictive models used to

estimate the parameters of the plant.

CI

2,

.. CN

are

N

controllers corresponding to the

N

predictive

models, and

as

mentioned earlier, the controller

Ci

is

used

to control the

plant at any

instant if the model

Ii

is judged to be the best at that instant.

If 3ji

is the

output of model

Ii, = ei

is the output prediction

error, and

a

performance criterion

J ( e i ) i

=

1 , 2 ,

...

N

is used to determine which of the pairs

[ I j Cj]

is used

at

any specific instant. The choice of

J ( e i )

determines

how rapidly the system can switch from one model

to another and hence the overall performance of the

system.

3 Deterministic and Stochastic Adaptive

Control

The deterministic adaptive control problem of a

continuous-time plant, along the lines described in the

previous section was first introduced in

[l].

n

[2]

differ-

ent schemes for switching and tuning were considered,

and the conditions for the stability of the correspond-

ing algorithms were discussed. In [3] the same prob-

lems were considered in a stochastic setting when

a

disturbance affects the output of a discrete-time plant.

The principal contribution of [3] was the demonstration

that in spite

of

the switching and tuning, all the signals

would be mean square bounded and that the adaptive

160


3/6

controller would evolve to

a

minimum-variance con-

troller. From the analysis of the stochastic adaptive

control problem contained in [3], the importance of the

parameter estimation scheme in proving convergence

of the control problem became evident. This was made

use of in

[4]

o demonstrate that different estimation

schemes, which result asymptotically in similar inequal-

ities involving outputs and parameter errors, could be

used simultaneously to improve the performance of a

stochastic adaptive system in the presence of a random

disturbance.

In this paper we shall consider both continuous-time

and discrete-time systems and examine critically the

results presented in all four papers [1]-[4], which repre-

sent the s tate of the a rt in thi s area. Following this we

will discuss how some of these ideas can be extended

to more complex cases. Towards the end of the paper,

applications based on the Multiple Models Switching

and Tuning (MMST) approach are discussed.

3.1 Deterministic Adaptive Control

Consider the equation

Y k)

=

4 v

Q O

where

q5(k)

E Rn s a regression vector

at

time k ,

80 E

R

is an unknown constant parameter vector of

the plant, and

y k)

is the output of the plant at time

k.

Equation (1) is an input-output description

of

the

deterministic plant t o be controlled. For simplicity it is

assumed that t he delay of the plant is unity. Numerous

recursive algorithms have been proposed to estimate

the parameter

80.

In one such algorithm (the recursive

least squares) the estimation model is described by the

equation

(IC) =

@ ( k

1)8(k

1)

d k)

=

d k

1

+ ~ ( k1)4(k )e(k)

(2)

where the parameter estimate is updated

as

(3)

and

P(k

) , the matrix of adaptive gains, is defined

as

e(k) = y k)

( k )

s the prediction error in the output

estimate.

3.1.1 Adaptive Control

using a Single

Mo de l: At very instant

k ,

assuming that the param-

eter estimate

e ( k )

s known, the input

u ( k )

s computed

using the certainty equivalence principle, so that

Q(k)

=

y* k)

=

p ( k @ 1

It has been shown

[3]

that this procedure results in all

the signals remaining bounded and the output error

tending to zero asymptotically, i.e. limk,,

e ( k ) = 0.

In [3] it is also shown that the same result holds when

the delay of the system is

d

> 1 so that

y(k +

d )

=

4 T k ) w ) .

3.1.2 Adaptive Control

using

Multiple

Models : In section 3.1.1 a single prediction model

was used to estimate the parameter vector 80. For rea-

sons that will be described later in this section, we

consider now the case where

N

predictive models of

identical structure are used to estimate 80. While all

the estimation models use the same adaptation law (3),

the initial conditions of th e models may be chosen ran-

domly. We denote the estimate at instant

k

of the

model

Ii

by

ei(k).

The controller

Ci

uses the certainty

equivalence principle to compute

ui k),

ts estimate of

the input

u ( k )

o be used

at

tha t instant. Let the con-

trollers

Ci

be chosen at random at every instant.

It

can be shown

[3]

that the overall system is globally

stable, that all the signals are bounded, and that the

plant estimation error e(k), and the plant controller

error e,(k) =

y(k)

* k) tend t o zero.

Note:

The above fact implies tha t stability and perfor-

mance can be decoupled. Hence, switching algorithms

based on heuristics can be chosen to improve the per-

formance of the system while retaining stabili ty. In

the deterministic case, performance criteria

Ji i =

1,.

N )

based on the estimation error can be cho-

sen to switch from one controller to another.

J i ( k ) =

e i 2 ( k ) , J i ( k )=

&Cfef(k)and Ji(k)= $Cf-T+le?(k)

which correspond t o instantaneous, time-averaged, and

average error over

a

finite interval have been used.

A t

every instant

k ,

the identifier-controller pair

[Ii,C i ]

is

chosen where J i ( k ) = mini J j ( k ) and the correspond-

ing input

ui k)

s applied t o the plant.

3.1.3 Fixed and Adaptive Models:

The ap-

proach described above is fast and accurate, but is

computationally intensive. In view of this, numerous

schemes involving fixed and adaptive models were sug-

gested in [l ]. Th e scheme tha t was recommended over

all the others consists of

N

2 fixed models, a free

running adaptive model, and an adaptive model that

is initialized at every instant

k

at precisely the point

in parameter space where the fixed model is found to

be optimal at that instant. This reinitialized adaptive

model is used only so long

as

no other fix model is

chosen. If at a later instant another fixed model Ij is

chosen, the previous reinitialized adaptive model at Ii

is replaced by the one

at I j .

The proof of convergence

of this scheme is given in [2].

161


4/6

3.2 Stochastic Adaptive Control

In the stochastic adaptive control problem, the linear

system with unknown parameter 00 has an additional

input w ( k ) besides the control input

u k).

A natural

description of the system is

A ( q - ' ) y ( k ) = q-dB(q- )u(k) C(q- )w(k)

where A and

C

are monic polynomials of degrees n and

1

in

q-

and B(q- ) is a stable polynomial of degree

m.

w k ) is

a

white noise sequence which is mean squared

bounded, with zero conditional mean and finite vari-

ance (i.e. E[w k ) lk

11 =

0; E[w2 k)lk

11 =

c2 .

If

a

single estimation model is used to control the sys-

tem, the unknown parameters of the polynomials A ,

B

and

C

are estimated as in the deterministic case us-

ing the output prediction error and t he Extended Least

Squares method (ELS). The forms of the equations de-

scribing the plant and the estimation model are the

same as in th e deterministic case, i.e.

(IC)

=

f )*(k )e^(k

-

1)

where

+ T ( k

- d )

= [y(k

d) ,

. .,y k +

1 ) , u k

-

d ) ,

u(k

- m d +

1)

-&(k

) , .

,&(k + l ]

60 = [ Q O , Q l , . . ,Qn7h00,.

.

7 P m + d - - l , C l ,

c27

. cn]

where g(k) = 4 T ( k

- d)e^(k

- 1) is the

a

posteriori

estimate

of

y k)

at time k . It is shown in

[3],

that if

an ELS method is used to estimate

80,

nd certainty

equivalence control u(k) is chosen at instant k based

on the estimate

&(k),

the overall system converges to

a minimum variance controller.

3.2.1 Adaptive Control using Multiple

Models: The MMST approach can be extended to

the case of multiple models along the lines described

for the deterministic problem. However, it must be

noted that while the regression vector in the determin-

istic case is identical for

all

the models, it is not the

same in the stochastic case.

In

particular, the a poste-

riori estimates are different for the N models. Hence,

as

the system switches from one controller to another,

the regression vector also changes. This makes stabil-

ity analysis quite complex. However, in

[3]

it is shown

using a modified Kronecker lemma that all the regres-

sion vectors (if unbounded) can only grow

at

the same

rate. This in turn permits arguments similar to those

in the deterministic case to be used in the stochastic

case as well.

3.2.2 Adaptive Control using Multiple Es-

The proof of convergence

imation Algorithms:

given in

[3]

has strong consequences for the use of

the MMST approach in other contexts. It is well

known that many recursive methods have been devel-

oped in the system identification literature which have

advantages under suitable assumptions about the dis-

turbance. In particular, if the estimates given by dif-

ferent schemes can only grow at the same rate (if an

unstable controller is used) it follows that the MMST

method will result in bounded minimum variance con-

trol . Using multiple estimation schemes to improve

performance in the presence of time-varying distur-

bances has been treated in

[4].

4 Adaptive Control

of

Nonlinear Systems

using Neural Networks

The MMST approach developed for linear systems at-

tains its full potential when dealing with nonlinear sys-

tems. The structure of the overall system is the same as

tha t shown in Figure

1,

but

Ii

and

Ci

are approximated

using neural networks.

From a system theoretic point of view neural net-

works can be considered as practically implementable

parametrizations of nonlinear maps from one finite di-

mension space to another. In identification and control

problems our objective is to first demonstrate theoret-

ically tha t certain nonlinear maps exist, and later use

neural network to approximate them. For example, it

can be shown that an input-output (NARMA) repre-

sentation

of

the form

exists for

a

nonlinear plant of degree

n

in the neighbor-

hood of its equilibrium state, and tha t the correspond-

ing input

u(k)

o the system can be expressed in terms

of the past values of the inputs and outputs as well

as

the desired output

r ( k )

as

Hence, neural network models having the form

and

6 ( k

+

d ) = n i ~ [ y k ) , k ) , .

y k

n

+

l ,

T ( k ) , U ( k ) ,

k

),

.

. .

u(k

1 I ]

162


5/6

can be used to approximate them. Using such represen-

tations, methods for tracking bounded desired outpu ts

have been studied extensively in th e lite rature

[5],

and

such methods have also been extended to multivari-

able systems aswell

as

disturbance rejection problems.

The success of neural networks in t he above case pro-

vided the incentive to attempt their use in control prob-

lems using the MMST approach. While the methodol-

ogy is the same, the implementation raises many ques-

tions concerned with the location of different models,

creation and deletion of models, speed of adaptation,

stabili ty, and robustness. Considerable advances have

been made in each of the above areas. In particular,

it has been recently shown [6] that under suitable as-

sumptions concerning the nonlinear plant to be con-

trolled, the MMST approach using

a

combination of

linear and neural network based identifiers and con-

trollers can improve performance of the system while

assuring stability. In

a

similar fashion, under appro-

priate assumptions, it also appears possible to com-

bine advanced information processing capabilities such

as

pattern recognition, adaptation, learning, and opti-

mization to perform satisfactorily in complex systems

in the presence of nonlinearity and uncertainty.

5 Applications

The MMST approach discussed in the previous section

combines fixed and adaptive models in novel ways and

results in

fast

and accurate adaptive control even while

remaining computationally efficient. As such, it is find-

ing

a

wide following in industry in areas where large

changes in system dynamics are anticipated and adap-

tive control using

a

single model is found to be inade-

quate. Four specific applications in which the method

has proved particularly successful are described below.

Flight Control: Fast and accurate flight control recon-

figuration is of paramount importance for increasing

aircraft survivability in the presence of subsystem fail-

ures and struc tural damage. The MMST approach ap-

pears to be eminently suited to cope with such situa-

tions. BoSkoviC and his colleagues at Scientific Systems

Inc. have been systematically applying the approach

to flight control problems [7]. They have proposed new

parametrizations for the modeling of control effector

failures of different types, which naturally lead to a

multiple model formulation of the problem. The most

complex case considered by them is one in which one of

the effectors undergoes float, lock-in-place, or hard-over

failure, while all others lose effectiveness. Computer

simulations of a linearized model of the F/A - 18A air-

craft during carrier landing maneuvers using five mod-

els demonstrated that excellent performance under

a

critical effector failure could be achieved. The large

transien ts resulting from adaptive control of the same

system using

a

single model made the la tter unaccept-

able. A t the present time BoSkovi6 and his group are

investigating the robustness of the MMST approach.

Process Control: Another area where the MMST ap-

proach may prove attractive is in process control.

Chemical companies, interested in the applicability of

advanced control theories to industrial process control

have presented benchmark problems that include many

difficulties encountered by the process control indus-

tries. These include nonlinearities, product transitions,

constrains and delays. Recently, Gundala and

Hoo [8]

have applied the MMST approach to control a MIMO

nonlinear stable two-phase chemical reactor. The au-

thors conclude that the approach is particularly attrac-

tive when the process is known to transition to un-

known operating states. The response obtained was

faster and more accurate compared to a multiple

PI

controller strategy.

A

Flexible Transmission System: An application of the

MMST approach to

a

flexible transmission system is

presented En [9]. Flexible systems with low damping

factor which occur in aerospace applications are in gen-

eral very difficult to control, particularly in the pres-

ence of large load variations. Experiments have shown

that a fixed high performance controller designed for

one loading may lead to instability for another load-

ing. While several k e d parameter controllers have

been designed to assure robust stability, very high per-

formance could not be achieved by any of them for

all loadings. Adaptive control using a single model

of the plant resulted in large transients with sudden

changes in load, and parameter drifts, when the in-

put signals are not persistently exciting, resulting in

instability due to unmodeled dynamics. The MMST

approach together with a closed loop output error pa-

rameter estimation technique was applied to the above

problem and resulted in improved stability and perfor-

mance of the overall system.

Solar Power Plant: The adaptive control problem of a

solar power plant is the subject of [lo ]. The plant , em-

ploying a distributed collector field has to collect solar

energy and transfer it to oil being piped through the

system. To use the heated oil for power production,

the main control requirement is to maintain the outlet

temperature of the field at a constant value. This is

achieved by adjusting the flow of oil. Since solar ra-

diation changes substantially during operation due to

the daily solar cycle, as well as atmospheric conditions,

there are significant variations in the dynamic charac-

teristics (e.g. response rate, time delays, etc.) of the

field. To cope with such fast variations a switching

controller, using different ARMAX models of the plant

for different operating points, was used to control the

system. In contrast to t he previous applications, only

switching between multiple models was used. The mul-

163


6/6

timodel scheme was found to be superior to the con-

ventional adaptive control scheme both in regard to

robustness in the presence of disturbances, as well

as

the amount of prior information needed to design the

controller.

6 Conclusions

The paper discusses recent developments in the area

of adaptive control using MMST. The main idea of

the approach is to use multiple models, each of which

corresponds to a different environment in which the

plant may have to operate. To reduce the compu-

tational burden, only two models are adaptive while

all of the others are fixed and provide better initial

conditions for one of the adaptive models. The proofs

of convergence for both, continuous-time [ l , 21 and

discrete-time stochastic system [3] have been given.

Simulation studies of numerous practical systems have

also demonstrated the superiority of the approach

over conventional adaptive control. Applications in

flight control, process control, the control of

a

flexible

transmission system and

a

solar power plant reveal

that the approach is practically feasible. However,

many issues remain which must be investigated before

the MMST approach can be used with confidence.

These include the selection of reference models, a

detailed analysis of the robustness of the scheme under

different types of perturbations , and the extension t o

nonlinear systems.

Acknowledgment

The research reported here

w s

supported by Contract

N00014-97-1-0948 from the Office of Naval Research.

References

[l]

K.

S. Narendra and J. Balakrishnan, Improving

Transient Response of Adaptive Control Systems using

Multiple Models and Switching,

IEEE Trans. on Auto-

matic Control, vol. 39, no. 9, pp.1861-1866, September

1994.

[2]

K. S.

Narendra and J. Balakrishnan, Adaptive

Control Using Multiple Models and Switching,

IEEE

Trans.

on Automatic Control, vol. 42, no. 2, pp.171-

187, February 1997.

[3]

K. S.

Narendra and C. Xiang, Adaptive Control

of Discrete-time S yste ms Using Multiple Models, IEEE

Trans. on Automatic Control (To appear in September

2000). Also, Technical Reports 9801 and 9803, Center

for

Systems Science, Yale University, 1998.

[4]

K. S.

Narendra and

0.

Driollet, Stochastic Adap-

tive Control Using Multiple Estimation Models, Tech-

nical Report 2002, Center for Systems Science, Yale

University, 2000. To be published in The International

Journal of Adaptive Control and Signal Processing,

Special Issue on Switching and Logic in Adaptive Con-

trol.

[5]

J.

Cabrera and K.

S

Narendra,

Issues

in

the Ap-

plication of Neural Networks for Tracking Based on In-

verse Control, IEEE Trans. on Automatic Control, vol.

44, no. 11, pp.2007-2027, November 1999.

[6] L. Chen and

K . S.

Narendra, Nonlinear Adaptive

Control Using Neural Networks and Multiple Models,

To appear in the Proceedings of the lg th American

Control Conference, 2000.

[7] J. BoSkoviC and R. Mehra, Stable Multiple Model

Adaptive Flight Control for Accomodation of a Large

Class

of

Control Effector Failures, Proceedings of the

18thAmerican Control Conference 1999, pp.1920-1924.

[8] It. Gundala and

K.

Hoo, Multiple Model Adap-

t ive Control Design

for

a MIMO Chemical Reactor,

To

appear in Industrial and Engineering Chemistry Re-

search.

191 A . Karimi and I. Landau, Robust Adaptive Con-

trol of a Flexible Transmission System Using Multi-

ple Models, Control Systems Technology, vo1.8, n0.2,

pp.321-331, March 2000.

[ lo] R. Pickhardt, Application

of

Adaptive Con-

trollers to a Solar Power Plant using Multiple Model

Descript ion, Proceedings of the 17th American Control

Conference 1998, pp.2895-2899.

.

164

Documents

Adaptive Control Using Multiple Models, Switching, And