Brian DO Anderson The Australian National University and National ICT Australia Limited

Two Decades of Adaptive Control Pitfalls

KUL System Identification and Data Modelling

Lennart Ljung Symposium

Brian DO Anderson

The Australian National University

and

National ICT Australia Limited

Lennart Ljung Symposium Oct 20042

Outline

Adaptive Control

MIT Rule

Bursting

Good Models, Bad Models and Changing the Controller

Multiple Model Approach to Adaptive Control

Conclusions


Plant is initially unknown or partially known, or is slowly varying.

There is an underlying performance index, eg

—Reference

r

+Controller

Input

uPlant

Output

y

Disturbanced

Adaptive Control

minimize T → ∞lim 1

T ∫ 0T u2 + y−r( )2

[ ]dt


A non-adaptive controller maps the error signal r-y into u in a causal, time-invariant way eg

An adaptive controller is one where parameters are adjusted.

Adaptive Control (continued)

˙ x c =Acxc +bc r −y( )

u=ccxc

with Ac,bc,cc, constant, xc a vector

—Reference

r

+Controller

Input

uPlant

Output

y

Disturbanced


Other ways of doing this exist Often 3 time scales:

Underlying plant dynamics (with fixed parameters) Time scale for identifying plant Time scale of plant parameter variation

ControlLaw Calculation

IdentifierPlant Parameters

Controller Plant

disturbanceControllerParameters

yu

One Formof Adaptive Controller


Outline

Adaptive Control

MIT Rule

Bursting



Conclusions


MIT Rule Problem

k m(t)

+

-

• Zp(s) is known, km is known, k p is positive and unknown, but kc(t) is known and adjustable

• Problem is to find a rule using e(t) to adjust kc(t) to cause e(t) to go to zero

• Problem source: k p depends on dynamic air pressure for aircraft.

kc(t) kpZp(s)

kmZp(s)

r(t)yp(t)

e(t)

ym(t)


MIT Rule Intuition and Performance

Use gradient descent to try to drive e(t) to 0:

with g a gain constant

€

˙ k c = −g∂[

1

2e2(t)]

∂kc

€

˙ k c = −g y p − ym[ ]ym

Equivalently,

Sometimes this worked, sometimes it did not work.Why?


Example of performance

•Unshaded regionis stable

•Sine wave input at frequency

•Plant is (s+1)-1

g


Underlying differential equation for kc is Mathieu equation. Solution regions of this equation are depicted.

One instability mechanism is interaction of excited plant dynamics with adaptive dynamics, made worse at high gain g

Explaining Instability

High (adaptive) gain instability for some Zp(s) : consider a

constant input R to display phenomenon. The MIT rule

leads to a characteristic equation; high g may give RHP zero

zero:€

˙ k c = −g y p − ym[ ]ym

s + gkmkpR2Zp(s) = 0From derivative of

kc

From PlantDynamics


Explaining Instability II

k m(t)kmZm(s)

+

-

Zm(s) is known, km is known, k p is positive and unknown, but kc(t) is known and adjustable

A second instability mechanism comes from modelling errors, here errors between Zp(s) and Zm(s)

Following two figures show case where plant and model are the same and where they are different.

kc(t) kpZp(s)r(t)

yp(t)e(t)

ym(t)


Example of performance

•Unshaded regionis stable•Sine wave input at frequency •Plant and model are (s+1)-1

g


Performance: another example

g



•Plant is e-s(s+1)-1

while model is still(s+1)-1


Averaging theory is the general analysis tool usable given separation of time scales of the plant dynamics and the learning/adaptation rate

Rescuing the MIT Rule: Averaging

€

˙ k c = −g y p − ym[ ]ymZp(s)kpkc(t)r(t)

Averaging theory treats kc slowly-varying : kc* is approximately kc for

small g where .

kc*=-g{Zp(s)kp r(t)}{Zm(s)kmr(t)} kc* + terms indep of kc* Stability is ensured if the average value of

{Zp(s)kp r(t)}{Zm(s)kmr(t)}

is positive--and if Zp is like Zm at frequencies where r(t) is concentrated, then stability is achieved.


Performance: another example

g



•Plant is e-s(s+1)-1 iswhile model is still

(s+1)-1


Rescuing the MIT Rule: Averaging

Lennart Ljung used averaging when he explained how to analyse the behaviour of a discrete time adaptive algorithm with the aid of an ordinary differential equation

The adaptation rate became slower and slower as time evolved--achieving the time scale separation.


Adaptive Control

MIT Rule

Bursting



Conclusions

Outline


Bursting Phenomenon


Bursting Phenomenon (continued)

Bursting phenomena were seen in an experimental adaptive control system - sometimes after 1 week of successful operation

Why do they occur? How could they be stopped?

bs+c

u(t) y(t)

˙ y +cy=bu

From measurements of u(•), y(•), one should be able to identify b and c If u = constant, can only identify b/c-the DC gain Adaptive controllers contain adaptive identifier of b and c


Bursting Phenomenon (Explanation)

Control law is designed based on estimates of b,c. Hence could accidentally implement unstable closed loop.

Instability then enriches the signals, giving improved identification.

α2I > ss+Tφ t( )φT t( )dt>α1I

Identification process is robust if T such that for all s and

some positive 1, 2, regression vector (t) obeys:

or ∑ ss+Tφ k( )φT k( ) in place of integral[ ]

normally involves inputs and outputs. Need to convert to input-only condition


Rich Excitation

If there are p scalar parameters to be identified, input needs to have a complexity related to p: (p/2 sinusoidal frequencies).

Practical issue: unless adaptation is turned off, must drive the system with “rich” input. [Some algorithms turn adaptation off at 1/t rate]


Adaptive Control

MIT Rule

Bursting



Conclusions

Outline



In adaptive control, at each time instant

• There is a model of the plant (which may be a good model)

• There is a certain controller attached to the plant

• If the plant model is a good one, a simulation of the model and controller will perform like the actual plant and controller

In adaptive control

• The controller may be changed to better reflect a control objective

• The calculation of the new controller is based on the current model--applying with the current controller


Good Models, Bad Models and Changing the Controller (continued)

This presents a fundamental challenge in adaptive control Consider:

True plant:

Model:

[Transfer functions are and ]

0.1̇ ̇ y +1.1˙ y +y=u

˙ y +y=u

1s+1( ) 0.1s+1( )

1

s+1


P1 =1

s+1Similar open-loop behaviours: and P2 =

1(s+1)(0.1s+1)

open-loop closed-loop

K =100(−)/ K =1(−−−)



P1 =1

s+1Different open-loop behaviours: and P2 =

1s

Plants in open-loop Plants in closed-loop with K =100




Moral: changing the controller may turn a good model into a bad one, or vice versa

Changing the controller is like changing the experimental condition--and Lennart Ljung always told us to watch the experimental conditions!

“Goodness of fit of a model” is a term which only makes sense for a particular set of experimental conditions OR

Don’t overgeneralise what you have learnt

If you change the controller significantly, you might produce instability with the real plant, while it works fine with the model (=estimate of plant)


A frequently advanced approach to adaptive control design is iterative identification and controller redesign.

One iteration comprises (re) identifying the plant with the current controller redesigning the controller to achieve the design

objective on the basis of the identified model, and implementing it on the real plant

Iterative Identification and Controller Redesign

This can lead to instability!

One needs algorithms which will move performance with the model and the new controller towards the design objective--but not change the controller too much.Same issue for IFT, VRFT Safe adaptive control.


Adaptive Control

MIT Rule

Bursting



Conclusions

Outline


Multiple Model Adaptive Control

Imagine a bus on a city street. The equations of motion of the bus have parameters depending on

• the load• The friction between tyres and road

Many plants have equations in which a (frequently small) number of physically-originating parameters are changeable/unknown. Call such a plant p(). Here = physical parameter vector

Learning from measurements with an equation of the form

may be too hard, especially for nonlinear plants

ˆ ˙ λ = f ˆ λ ,measurements( )ˆ λ → λ

true


Multiple Model Adaptive Control (continued)

An alternative approach (MMAC) is as follows:

• Suppose that the unknown parameter lies in a bounded simply connected region. Call the unknown plant .

• Choose a set of values in this region, with associated plants P1,.......,PN.

• Design (in advance) nice controllers for P1,.......,PN.

• Call them C1,......,CN .

• Run an algorithm which at any instant of time estimates (via the measurements) the particular Pi which is the best model to explain the measurements from . Call the associated parameter

• Connect up

P

P C

λ ˆ i

λ1,λ2,....λN

λˆ i


Supervisor

noisereference + +

Controller i u Unknown or + y partially known

input - Plant P

Supervisor studies effect of using present controller and decides whether or not to switch controller

Desirable outcome: after a finite number of switchings, the best controller for the plant is obtained.

Unknown or Partially known

Plant P

Supervisor

Controller i

Multiple Model Adaptive Control(continued)


Why the name “multiple model”?

Underlying precept is that the plant coincides with or is near one of N nominal plants P1,.......,PN

P

Controller i, denoted Ci, is a good controller for Pi

(and possibly plants “near” Pi)


Deciding the Best Model Pi for P

u y1

Multi-y estimator yN

+r + Controller k Plant P -

Multiestimator is a device which produces N outputs if (and only if with complicated signals)

(The controller is irrelevant)

Controller k

Multi-estimator

yi =y

P =Pi

_Plant P


Multi- y1

estimatory yN

r + Controller J Plant P

-

Idea of algorithm: study

for some small a > 0, and k=1,…,N. If the smallest occurs for

k = I, say that P is best modelled by

Switch in

_Plant P

Early Approach to Supervision: Using Multiestimator

Controller J

Multi-estimator

CI

PI

0t y−yk( )

2dt or

0te

−at−s( )2

y−yk( )2ds

u

∫

This may lead to switching in a destabilising controller!


Example

Plant is 3rd order, stable, with non-minimum phase zero in [1,10] and DC gain in [.2,2].

Control objective is to extend bandwidth beyond open loop plant, with closed loop transfer function close to 1 in magnitude. Non-minimum phase zero is a limiter.

441 plant models chosen, with DC gain and non-minimum phase zero each in 20 logarithmically space intervals

Reference signal is wideband noise Measurement noise and process (input) noise are present


Example of Temporary Instability

Figure 7a: Example of Temporary Instability (without safety)


Example of Temporary Instability

Figure 7b: Example of Temporary Instability (without safety)


Multiple Model Adaptive Control-Difficulties

How can one avoid the instabilities?Should there be 7, 70 or 7000 models? How

should one actually choose the models?

These questions are actually linked.


How does one choose N? How does one choose 1,.........., N?

Idea of solution: Pick 1 . Design C1 for P( 1). Figure the plant set P() around P1 = P( 1) such that C1 is a good controller. Pick 2 near the boundary. Figure the plant set P() around P2 = P( 2) such that C2 is a good controller. Pick 3 near the boundary of union of these two sets, etc.

The set is then covered by a set of balls indexed by

1,.........., N and this determines N.

Choosing the Multiple Models

Metrics (Vinnicombe) help with this in the linear case


Safe Switching

Difficulty is that P may be best modelled by PI when CJ is connected, but may be best modelled by PK when CI is connected. PI may be a terrible model of P when CI is connected.

The index of the best model of P (out of P1,........PN) with controller CJ connected is NOT NECESSARILY the index

of the best controller to connect to P. Nontrivial fact: using crude estimation techniques one can obtain set of controllers {CK } which, when used to replace CJ, are guaranteed to retain stability (and even retain similarity of performance). Vinnicombe metric is used.

Even if PI is the best model of P when CJ is connected, CI

may not be in the safe set of {CK } . Only switch if it is safe.


OVERVIEW OF RESULTS

With safety constraint, controller switching is less frequent, convergence to the “best” controller was slower.

With safety constraint, performance could be poor but never unstable.

Without safety constraint, most runs exhibited poor performance, some yielded instability

With use of possibly more nominal plants and controllers, one can probably get “performance safety” as well as “stability safety”


Safe Controller Switching

Figure 3a:(Safe) Controller Switching


Safe Controller Switching

Figure 3b:(Safe) Controller Switching


Adaptive Control

MIT Rule

Bursting



Conclusions

Outline


Conclusions

Keeping adaptation and plant time scales different is good practice

Modelling as well as you can is a good idea--even with an adaptation capability.

Having lots more parameters than you need could be dangerous

• Bursting• Satisfactory learning occurs only for a limited

set of experimental conditions

If you want to be able to keep learning (accurately) , you need to continue excitation


Conclusions (continued)

A good model is only good for a particular set of experimental conditions. If you change the controller, it may cease to be good.

Picking representative models from an infinite set can often be done scientifically

Abrupt changes of a controller can introduce instability -even if on the basis of having a good model, the new controller looks good.

Safe adaptive control should be contemplated--to avoid temporary connection of a controller which can destabilise the (unknown) plant

Need Vinnicombe metric ideas for nonlinear problems

Documents

Brian DO Anderson The Australian National University and National ICT Australia Limited