FREEMAN2013_Survey on Iterative Learning Control, Repetitive Control and Run-To-run Control

8/16/2019 FREEMAN2013_Survey on Iterative Learning Control, Repetitive Control and Run-To-run Control

1/13

604 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 3, MAY 2013

Iterative Learning Control With Mixed Constraintsfor Point-to-Point Tracking

Chris T. Freeman and Ying Tan

Abstract— Iterative learning control (ILC) is concerned withtracking a reference trajectory defined over a finite time duration,and is applied to systems which perform this action repeatedly.

However, in many application domains the output is not critical

at all points over the task duration. In this paper the facility totrack an arbitrary subset of points is therefore introduced, and theadditional flexibility this brings is used to address other controlobjectives in the framework of iterative learning. These comprise

hard and soft constraints involving the system input, output andstates. Experimental results using a robotic arm confirm that

embedding constraints in the ILC framework leads to superiorperformance than can be obtained using standard ILC and an a

priori specified reference.

Index Terms— Iterative learning control (ILC), iterative

methods, learning control systems, linear systems, motion control,optimization methods, robot motion, test facilities.

I. I NTRODUCTION

I TERATIVE learning control (ILC) is a methodology appli-cable to systems which repeatedly track a reference, ,defined over a finite interval . The aim is to use past

experience to sequentially improve tracking performance over

repeated trials of the task. Over the last 25 years it has been

an area of intense research interest in both theoretical and application domains, for recent overviews of the literature see [1]

and [2]. However, rather than follow a motion profile defined at

all points, in many applications the system output is only crit-

ical at a finite set of prescribed time instants. Examples include

production line automation, crane control, satellite positioning,

and robotic ’pick and place’ tasks in which the critical points

correspond to the location of the payloads. Furthermore, ILC

has recently been used to great effect within stroke rehabilita-

tion [3], where motion control is naturally specifiedin terms ofa

point-to-point optimization problem in order to correspond with

results from human motor learning [4].

The standard ILC framework is able to tackle the

point-to-point problem simply by employing an arbitrary

Manuscript received August 11, 2011; revised November 17, 2011; acceptedJanuary 22, 2012. Manuscript received in final form February 08, 2012. Dateof publication March 14, 2012; date of current version nulldate. This work wassupported by Australian Research Council Future Fellow Grant FT0991385.Recommended by Associate Editor S. S. Saab.

C. T. Freeman is with the School of Electronics and Computer Sci-ence, University of Southampton, Southampton SO17 1BJ, U.K. (e-mail:[email protected]).

Y. Tan is with the Electrical and Electronic Engineering Department, Univer-sity of Melbourne, Parkville VIC 3010, Australia (e-mail: [email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TCST.2012.2187787

reference, , which passes through the desired points.

However superior results follow if this is coupled with strate-

gies such as Input Shaping in order to suppress vibrations that

occur between the critical points. This approach is taken in [5]

for a high-acceleration positioning table. An alternative is to

use a simpler feedback controller to track and to employ

ILC to update parameters within the input sha ping filter applied

to the reference, as proposed by [6] for control of an industrial

robot. Another approach is to develop ILC algorithms which

have two separate components; one which ensures tracking of

, and another which reduces the amplitude of residualvibrations occurring after the point-to-point location is reached

[7]. The drawback to all these methods is that they fail to utilize

the extra freedom available in ILC design to address additional

performance demands. Furthermore, if is designed a

priori to meet such performance objectives, these will not be

met in practice due to the presence of model uncertainty and

noise.

Other approaches to point-to-point motion control have

broken away from the standard ILC framework of tracking a

static reference defined over , but have only con-

sidered the case where a specified position must be reached at

time , as in [8]–[13], or the case of a movement betweentwo equilibrium points [14]. While these approaches dispense

with tracking unnecessary output points, they do not use the

resulting freedom to tackle additional performance objectives

which may be of critical concern. A further limitation is that

they only consider a single point-to-point movement, rather

than a sequence of actions needed to build up complex move-

ments, such as is required in robotic automation and production

line assembly.

This paper addresses current drawbacks by providing

a framework that can deal with an arbitrary number of

point-to-point movements, while also addressing a general

form of perfor mance objective which encompasses a wide

variety of pr actical performance concerns. Along with soft

performance constraints, this includes hard constraints which

are needed to address actuator saturation, physical workspace

limitations, or imposed safety restrictions. This framework

significantly increases the flexibility and functionality of

point-to-point ILC compared with approaches currently avail-

able. Moreover, the action of embedding both the performance

and tracking objectives within the framework of iterative

learning yields algorithms which are capable of reaching op-

timal solutions in the presence of model uncertainty and noise.

To achieve this, ILC is employed as an iterative optimization

paradigm which uses experimental data to tackle a general form

of cost function involving the input, output and states. A similar

1063-6536/$31.00 © 2012 IEEE


2/13

FREEMAN AND TAN: ILC WITH MIXED CONSTRAINTS FOR POINT-TO-POINT TRACKING 605

approach has recently been applied to constrained ILC in [15],

which addresses both soft and hard constraints, but without

explicit reference to, or analysis of, the point-to-point tracking

problem. In addition, the current paper’s focus on point-to-point

tracking requires it to be embedded in the form of an additional

equality constraint, in order that the soft constraints may not be

allowed to degrade tracking at the critical time locations. Such

separation is not addressed in [15], with tracking and additional

constraints being combined in the same soft constraint.

The paper is organized as follows. Section II introduces

and motivates the point-to-point tracking control problem.

Section III develops gradient descent ILC laws which address

both hard and mixed constraints. In Section IV, ILC algorithms

based on the Newton method are presented. Experimental

results are provided in Section V and conclusions and future

work are given in Section VI.

II. PROBLEM FORMULATION

Denote the set of real numbers as , and the set of integersas . The symbol denotes the trial number and . For

any vector , . For any matrix ,

is the induced norm of the vector norm, denotes

the eigenvalue of , is the minimum singular value

of , is the maximum singular value of , and

is the spectral radius of . The notation is

the pseudoinverse of and . The notation

is the orthogonal projection onto the nullspace

of . The identity and zero matrices are denoted by

and , respectively.

In order to simplify presentation, the following linear time-

invariant (LTI) system is considered:

(1)

defined over the finite time interval

where the number of samples . Here ,

, are the state, input and output vectors,

respectively, and the input and output sequences are given by

Remark 1: The analysis framework provided in this work

can be extended to general nonlinear time-varying discrete-time

systems with a proper linearization. With a slight modification,

similar (local) results can be obtained.

The standard ILC framework constructs a series of inputs

which drives the system to track a reference sequence

Let and be the input and output vectors respectively on

the th trial, with the tracking error. Then it is

necessary to find a sequence of control inputs that satisfies

(2)

where is the unknown desired input sequence corresponding

to . This leads to

Over the trial the relationship between the input and output

time-series can be expressed by where the

matrix is

......

. . . ...

(3)

Here is the response to initial conditions whose effect can

be absorbed into the reference trajectory, so that without loss of

generality it is assumed , or equivalently .

For some , an ILC update of the form

(4)

can be considered as an iterative numerical method to solve the

tracking problem, and the derivation of a suitable matrix has

been the focus of significant research effort. Since

(5)

the update (4) is convergent to a solution satisfying (2) if and

only if

(6)

The convergence speed is determined by the magnitude of and is maximum when .

A. Point-to-Point ILC Formulation

Now suppose that the th plant output is only required to track

a reference trajectory at a fixed number, , of sample

instants along the trial duration. These sample instants are given

by . To define the point-to-

point tracking problem it is first necessary to remove the points

that do not need to be tracked from the original reference .

This yields a reduced reference vector whose length

is given by

(7)

It is then necessary to define a matrix transformation

such that . This is achieved by

first introducing a row vector whose th element

is 1 if the th element of is required to be tracked, and 0

otherwise. The formal definition for is

if ,

otherwise(8)

where and denotes the “floor” func-

tion. The matrix is then produced as follows: 1) set ,


3/13


2) starting at the first element, increment along the bottom row

of , and whenever a non-zero element is encountered move all

subsequent bottom row entries into a newly created bottom row

that is appended to , maintaining their position along the row

and padding the remaining entries of both rows with zeros. For-

mally this is defined by

if

otherwise.

(9)

As seen by the relation , when any output vector is

pre-multiplied by , it extracts the components that correspond

to prescribed point-to-point locations, while retaining the order

in which they appear.

Remark 2: Suppose that at each point-to-point location each

component of the output is required to track a reference point,

that is if . In this case the matrix

has a simpler form given by the block-wise components

if ,otherwise (10)

and the reference has the form

(11)

where is the prescribed output vector at sample ,

and .

ILC can be re-formulated for the point-to-point case by de-

riving an iterative numerical solution to the problem of finding

a control input which minimizes the point-to-point error norm.

The control objective is to find a sequence of control inputs

such that

(12)

which replaces the standard requirement (2). The ILC update

(4) now assumes the form

(13)

so that the point-to-point error evolution is

(14)

and the convergence condition (6) becomes

(15)

which guarantees zero point-to-point error.

In Sections III and IV learning operators are derived to

satisfy (15), but first further motivation is provided to support

the utility of point-to-point ILC over the standard framework.

B. Point-to-Point ILC Motivation

The first result shows point-to-point ILC can enlarge the fea-

sible region of the solution. That is, it confirms that some prob-

lems cannot be solved by the standard ILC framework, but are

feasible for point-to-point ILC.

Theorem 1: Let denote the rank deficiency of the plant ma-

trix (the number of linearly dependent rows). If the

standard ILC update (4) cannot force the plant to track an arbi-

trary reference trajectory . The point-to-point update (13) can

enforce tracking of an arbitrary reference if and only if the

tracked points are chosen such that

(16)

Proof: A necessary and suf ficient condition for an oper-ator to exist satisfying the convergence condition (15) is that

. For the standard ILC case ,

and hence , leading to

having eigenvalues at unity. Now the row of

is the row of , hence if and the

point-to-point samples are chosen to correspond to any subset

of linearly independent rows of , the convergence condition

(15) can be satisfied. If then the additional condition

is imposed.

Remark 3: Let system (1) be written as discrete transfer-func-

tion matrix with component

the transfer-function linking the th output with the th

input. If the relative degree of is , then we have

.

The ability of point-to-point ILC to employ a modified stan-

dard reference to recover feasibility is extremely important, es-

pecially as in practice due to the delay action of a zero-

order hold. However many tasks are naturally defined only at a

small number of points, and hence additional benefits may also

be expected by not enforcing tracking of unnecessary points.

The next lemma shows how the space of feasible inputs expands

as the number of tracked points, , reduces.

Lemma 1: Assuming , the feasible input

space which forces the system (1) to track is of dimension

, and is given by .

The nullspace of has an orthogonal basis given by the rows

of , where is such that the

matrix is full rank.

It is next illustrated how this enlarged space of feasible inputs

can be used to increase performance. In particular, the practi-

cally relevant case is addressed in which a weighted input norm

is required to be small. However, before this can be considered

a preliminary proposition is required.

Proposition 1: Let comprise point-to-point locations

satisfying . Let equal but with the

row removed, and hence correspond to tracking all but the point-to-point location. Let the eigenvalues of the matrix

be denoted ,

which also equal the singular values since is Normal. Simi-

larly, let the eigenvalues of the matrix

be denoted , which also equal

the singular values since is Normal. Then the following

relationship holds:

(17)

In particular, let equal the th column of with the th ele-

ment removed. Then if the eigenvalues of are distinct and no

eigenvector of is orthogonal to then

(18)


4/13


Proof: First note that is a Hermitian matrix of order ,

and that is a principal submatrix of of order . Then

(17) follows as an application of Cauchy’s Interlace Theorem

for eigenvalues of Hermitian matrices [16]. It is further proven

in [16] that (18) holds provided: 1) the eigenvalues of satisfy

and 2) the vector

(19)

has non-zero elements, where is a unitary matrix of order

such that , with .

To satisfy 2) a suitable choice for has columns that are the

eigenvectors of , and hence only if is orthogonal

to an eigenvector of .

With the help of Proposition 1, the following result is ob-

tained.

Theorem 2: Consider the system (1) and a point-to-point

tracking task which has a corresponding matrix satisfying

. There then exists an input which achieves

tracking of and has a weighted norm with upper bound

(20)

whose right-hand side reduces as the number of points is

reduced. Here the operator has full rank.

Proof: Suppose the input solves the standard tracking

problem, so that , where contains the desired

points, , along with additional ’free components’ that are

not associated with the point-to-point objective. Then exchange

rows in matrix and to group the stipulated, , and free

components, , as

(21)

where is such that is full rank.

The optimal cost associated with the problem

subject to

is the norm of the orthogonal projection of , onto the range

of , and it follows that:

(22)

Now insert the relationship

into (22) to obtain the solution

The relationship

leads to the weighted input bound (20). It follows that the input

norm is small when point-to-point locations are selected which

maximize the smallest eigenvalue of . Application

of Proposition 1 means that increases as each

point-to-point location is removed, and hence the right-hand

side of (20) decreases.

Remark 4: If each component of the plant output is only re-

quired to track a reference position at a single sample instant,

that is , then (20) becomes

(23)

This also holds if the temporal distance between point locations

exceeds the time taken for the impulse response to approxi-

mately go to zero (assuming asymptotic stability).

Theorem 2 provides an example of the benefit obtained com-

pared with the bound corresponding to

standard ILC (if it exists). This benefit increases as the number

of tracked points is reduced, or their temporal spacing is in-

creased.

Reductionin the number of tracked points, , hence expands

the set of feasible inputs and enables them to be chosen to ad-dress infeasibility in the tracking of a reference defined at all

samples along the trial, as well as addi-

tional performance objectives. In the next section these objec-

tives will be embedded in the ILC framework to enable optimal

solutions to be arrived at in the presence of model uncertainty

and noise.

III. GRADIENT DESCENT POINT-TO-POINT ILC

The gradient descent method is one of most popular nu-

merical algorithms used to tackle nonlinear optimization

problems, and has previously been applied to the single-input,

single-output (SISO) case within the standard ILC framework [17]. Unlike alternative approaches, it is straightforward to

embed experimental data, directly yielding updates in the ILC

framework which, through suitable step size selection, have

favorable convergence and robustness properties that can be

manipulated in a simple and transparent manner. Motivated by

(12) and the accompanying discussion, the gradient descent

method is applied to solve

(24)

leading to the iterative update for the control input

(25)

where is the gradient operator with respect to and is

a positive scalar. Note that the experimental plant output,

has replaced the nominal value, , so that the optimization

is robustly achieved within the ILC framework.

Theorem 3: Provided the point-to-point locations are chosen

such that , the choice of gain

(26)

guarantees convergence of the update (25) to the reference .

In particular, the maximum convergence rate corresponds to

(27)


5/13


The convergence rate using (27) increases as the number of

point locations, , is reduced.

Proof: The convergence condition for (25) is given by

(28)

which ensures a linear convergence rate to zero error [18]. This

yields (26) since

where the inequality since

is positive definite. The solution to

corresponds to the choice (27) and the convergence rate is

(29)

Application of Proposition 1 guarantees that each point removed

from increases and reduces .

Hence the convergence rate (29) increases.

Having shown point-to-point ILC increases the convergence

rate, robustness margins are next established.

Theorem 4: Let there exist a multiplicative uncer-

tainty on each element of the plant model , such that

. Here is the actual plant and the

model corresponds to the matrix used in the update

law (25). A suf ficient condition for monotonic convergence is

that each lies in the open interval ,

demonstrating an allowable phase margin uncertainty of 90 . Proof: This is an extension of robustness analysis for

the standard gradient algorithm ( ) in [19] for the

SISO case. Suppose that the uncertainty can be expressed in

the matrix form , and that point locations are such

that . Then using (25) the point-to-point error

satisfies

where . If is positive, the first term on the right-hand

side is strictly positive for an arbitrary non-zero and ,and of . Similarly the second term is of and strictly

negative, and hence there always exists a which en-

sures monotonic reduction in error norm. This also holds if the

components of are reordered so that the elements corre-

sponding to the same input are grouped, resulting in a reordering

of the matrix such that . The stip-

ulation that the components of associated with the same input

have the same uncertainty then results in having the block

diagonal structure , where corre-

sponds to the th input. A suf ficient condition for to be posi-

tive definite is that each is positive definite. This is the same

condition as that given in [19] which goes on to show that a

suf ficient condition is that each is positive-real. There-

fore a suf ficient condition for monotonic convergence is that

lies in the open interval . Note

that any gain uncertainty can be tolerated through use

of a suf ficiently small .

Remark 5: The term in (25) can be ef-

ficiently generated using the co-state representation of system

(1). More specifically, it is equal to the output of the system

(30)

with the input and terminal state

.

Use of (30) therefore avoids calculation of the large matrix

appearing in (25) and the algorithms which follow.

Remark 6: It is shown in [20] that the gradient point-to-point

algorithm (25) applied to a linear system always converges to a

solution which minimizes . Hence, using Theorem 2, (25)

converges to a solution with a norm satisfying

(31)

whose upper bound decreases as the number of tracked points

is reduced.

A. Inequality Constrained Gradient Point-to-Point ILC

Consider vector inequality constraints on the system input of

the form , where and , where is

the number of imposed constraints. The point-to-point problem

(24) now becomes

subject to (32)

This can be tackled using an interior-point approach to in-

equality constrained minimization, termed the barrier function

[21]. A logarithmic barrier function is employed, producing the

auxiliary problem

(33)

where , are the rows of , , respectively. The scalar

is used to weight the action of the barrier and should

be gradually increased to result in a solution which satisfies the

tracking requirement. The solution via the gradient method is

(34)

where the elements of are given by ,

and is the value of on trial .

With appropriately chosen scalars and , this converges

to the zero error solution as provided there exists

which satisfies [21], [22]. In the context

of ILC the increase in must not be too rapid in order to ensure

that the barrier component effectively engages with the ILC up-

date. Conversely it must be fast enough not to significantly re-

duce overall convergence speed. The selection of and must

therefore ensure:

1) the input (34) remains feasible ( );


6/13


2) the constraint term in (34) comprises a significant propor-

tion of the input over samples which are required to adapt

to the imposed hard constraint;

3) 1) and 2) are satisfied without reducing .

It follows that an appropriate update strategy is to select the

highest value of that results in a feasible input without the

constraint term, that is, on trial choose a value which sat-

isfies

s.t. (35)

and then update according to

if

otherwise (36)

and use in the update (34). The multiplier is chosen

as a compromise between convergence to the hard constraint,

and robustness [21]. By satisfying the hard constraint and en-

suring is updated slow enough to engage with the ILC up-date, (34) converges to a input which solves the tracking re-

quirement as , provided such an input exists. If it does

not exist the procedure converges to a local minimizer of (33).

In practice the designer can gain insight into the feasibility of

the problem through simulations using the nominal plant model.

Note that the simple structure of the gradient ILC update allows

transparent control over convergence in practical conditions in-

volving model uncertainty and noise that is not possible using

many of the alternative inequality constrained minimization ap-

proaches available.

Remark 7: The convergence of iterative algorithm (34) with

respect to the given constrained optimization problem (32) be-longs to a class which has been extensively studied in the opti-

mization literature, see, for example, [22], and hence the proof

of convergence is omitted.

B. Incorporation of Additional Objectives

Having achieved point-to-point tracking with inequality con-

straints on the input, a wide range of other performance in-

dices that are important in practice can be addressed. These may

comprise reducing the input or output energy, or reducing the

output derivative at critical times to provide smoother move-

ments. Theorem 2 illustrates how the expanded feasibility space

provides scope to achieve such objectives. Consider the generalcase of minimizing a linear function, , of the input,

output, and states. If the point-to-point tracking requirement is

expressed as an equality constraint, the problem can be written

as

subject to (37)

where is a weighting matrix. It is

worthwhile highlighting that the formulation of point-to-point

tracking as an equality constraint that must be satisfied at each

iteration is a much stronger performance requirement compared

with that of standard ILC. Moreover, it is a requirement that

must be satisfied in the face of additional performance demands.

The equality constraint reflects the fact that the point-to-point

tracking requirement is an essential element of the task (e.g.,

a robot must reach the required positions during the assembly

task, whilst the soft constraint is merely desirable).

To remove the equality constraint express the vector quantity

in terms of the input vector, , using

(38)

in which

......

... . . .

...

(39)

and .

For notational simplicity, and without loss of generality, it is

assumed that . Now take as any solution satisfying and

. Providing it is feasible, that is , such an input is found

in practice through application of the approach of Section III-B.

Also introduce as a matrix with columns

that form an orthogonal basis of the nullspace of . From

Lemma 1, a suitable candidate is

Denote , then the minimization (37)

is equivalent to the inequality constrained problem

s.t.

(40)

with . The solution is then

(41)

This has split the solution into components and that min-

imize the soft and tracking constraints, respectively. The use

of means that updating does not affect the plant output at

the point-to-point locations which have already been forced to

follow the prescribed reference. Applying the barrier function

method to solve (40) yields the auxiliary problem

This has iterative solution via the gradient method

(42)

where the elements of are given by

. With suitable updating of the step-sizes

and , (42) is guaranteed to reach a local solution of

(37). As discussed in Section III-B, for the barrier function to


7/13


effectively engage with the minimization component of (42)

is chosen to satisfy

s.t.

on trial and then (36) is applied to update . Convergence

properties of the barrier method have been extensively studied,

see, for example [22], and therefore a convergence proof for

update (42) is omitted here.

The equality constraint in (37) ensures minimization of the

soft objective does not conflict with the point-to-point tracking

task. However in practice must also be updated to ensure it

continues to be satisfied in the presence of model uncertainty

and noise. This is done following the update of and hence

it must ensure the plant input satisfies

, and hence is given by

subject to (43)

with the resulting update

(44)

where the elements of are given by

. To ensure that the step-sizes and

engage productively, the procedure (35), (36) is again

employed, now with elements .

The final update sequence on each trial is

(45a)

(45b)

(45c)

where is the experimentally obtained performance function

.

Remark 8: In the absence of inequality constraints the soft

constraints reach a global minimum with convergence criterion

(46)

which corresponds to the choice of step-size

(47)

This guarantees existence of a solution provided that

is full, which requires . In the

absence of inequality constraints the input to the problem (37)

converges to a solution satisfying (20) with the substitution

. The upper bound reduces as points are removed from the

tracking task.

Remark 9: Rather than using the update of Section III-B to

initially solve the equality constraint, the updates (45a)–(45c)

may be applied directly to solve (37) starting from an arbitrary

initial input. However the presence of soft constraints influences

the action of the hard constraint on the point-to-point tracking

component, and may mean convergence to zero point-to-point

tracking error is no longer achieved.

In order to illustrate how to convert some performance in-

dices into the standard formulation in (37), two examples are

provided.

1) Example 1—Derivative Constraints: It is often desired

that the plant output velocity be zero at certain time instants. In

many applications this is important for vibration suppression,

or to ensure the system is momentarily stationary in order to

carry out a task (such as picking up or placing a component

on a manufacturing line). In addition, constraints on the input

velocity are useful for reducing actuator wear. This leads to (37)

becoming

subject to (48)

where the diagonal matrices ,select the points at which the respective derivatives

are required to be zero. Since , where

is the differential operator of appropriate dimension, this

gives , and (45b) becomes

(49)

In the case where vibrations are suppressed at the point-to-point

locations, is given by , where is defined in (8).

2) Example 2—Energy Constraints: Suppose instead a com-

bination of the input and output signal norms are required to be

minimal, giving rise to the cost

subject to

(50)

This form of constraint may be used to reduce either the input

or output norm, by setting the other weight to zero. This may

lead to an excessively impulsive action, however, which can be

addressed by instead using a small, non-zero value multiplied

by the identity matrix (or alternatively minimizing the output

derivative). The cost (50) corresponds to

and . This gives , and the

update (45b) becomes

(51)

within the ILC framework. Here the signals can be read directly

or observed using a suitable estimator.

IV. NEWTON METHOD-BASED ILC

While providing a high level of robustness to plant uncer-

tainty, the gradient descent approach has only a linear conver-

gence rate. This section shows how the previous algorithms can

be extended to deliver quadratic convergence. Consider again

the point-to-point tracking problem

(52)


8/13


and now apply the Newton method [18] to solve it, yielding

(53)

where is the Hessian matrix. From (15) the neces-

sary and suf ficient condition for convergence to zero error in a

single trial is now

(54)

which is satisfied if and only if . Since its com-

putation involves inverse and derivative operations, the update

(53) is dif ficult to implement, especially for large values of . It

may contain excessive amplitudes and high frequencies which

increase learning transients, and, depending on point-to-point

locations, it may be singular. However, it is shown in [23] that

is the solution, , to

subject to (55)

which is further shown to be equal to the solution to

(56)

via the unconstrained gradient descent method of Section III

which always yields the minimum input energy solution. Sub-

stituting and in (24) and (25) yields

an update of

(57)

which converges to the required solution provided (26) is sat-

isfied with the scalar gain . Between trials and of

ILC, some techniques: updates ( ) of (57) are applied

in simulation to the plant , to yield a suitable approxima-

tion to . The number of inter-trial updates is

chosen to affect a compromise between excessively high am-

plitudes/frequencies in the update, robustness, and subsequent

performance. The total update sequence is as follows.

Algorithm 1

(a) apply input to the real plant and record output

(b) solve (56) through repeated application of (57) to the

plant model to obtain a suitable approximation to

(c) use the resulting input to form the next descent direction

in the Newton update (53). Go to (a)

The next theorem establishes how the number of inter-trial

updates influences the convergence rate of the overall ILC law.

Theorem 5: For some , if inter-trials updates of (57)

are performed, the error evolution is given by

(58)

and the necessary and suf ficient convergent criterion for Newton

method based point-to-point ILC (54) is replaced by

(59)

Proof: Application of cycles of (57) to the plant matrix

produces the signal

(60)

where . Since , the resulting

operator which replaces in (53) is

The argument of the convergence criteria with this value substi-tuted is

(61)

If this relation is applied times, (61) simplifies to

This corresponds to the convergence rate (58), and directly

yields the convergence criterion given by (59).

The necessary and suf ficient convergence criterion for the

gradient algorithm (25) is given by (28), and is satisfied with

a scalar gain, , satisfying (26). Since

(62)

the faster Newton based method is guaranteed to converge if

the gradient method is convergent, with a rate that increases by

the power . Hence increasing the number of inter-trial updates

provides a smooth transition between the convergence rate of

the gradient algorithm (28), and the more rapid convergence rate

of the Newton method based algorithm (54). In particular, the

choice of means that the two algorithms are equivalent.

In order to approximate the Hessian matrix term in (53) a sig-

nificant number of inter-trial updates may be needed. Each how-

ever can be implemented using a state-space system of order

and hence is not computationally intensive. The parameter is

a tuning parameter chosen to affect a compromise between con-

vergence speed, amplitude/frequency content of the input, and

resulting robustness.


9/13


A. Inequality Constrained Newton Method-Based

Point-to-Point ILC

Consider again the case in which hard constraints alone are

required by the application, yielding the constrained problem

subject to (63)

This can be solved via the Newton method by imposing the in-

equality constraint in the inter-trial calculation of the descent di-

rection, , given b y the s olution t o (55). Within

this inter-trial problem, the inequality constraint translates to

and the descent direction is thus generated using

subject to (64)

This is equivalent to applying the gradient method to solve

subject to

This is the form addressed in Section III-B, with corresponding

inter-trial update

(65)

applied in simulationto the plant model , where the elements

of are given by .

The full update sequence is therefore as follows.

Algorithm 2


(b) between trial and , construct suitable

approximation to satisfying

when used in Newton update (53), through

repeated application of (65) to the simulated plant

(c) use the resulting input to form the next descent direction

in the Newton update (53). Go to (a)

The number, , of inter-trial updates is chosen heuristically

to affect a compromise between the amplitude of the descent

direction , and the overall convergence of the ILC scheme,

dictated by (58). In practice this is application specific, and is

achieved by decreasing in response to excessive amplitudes/

frequencies in the input signal, levels of fluctuation in the error

norm, and the overall convergence rate achieved.

B. Incorporation of Additional Objectives

As previously considered, having satisfied the point-to-point

tracking requirement using the algorithms of Section III-B or

Section IV-B, an additional objective function may be intro-

duced. This is required to be minimized while continuing to sat-

isfy the point-to-point tracking requirement with an inequality

constrained input. The problem is given by

s.t. (66)

Following the procedure of Section III-C, this is equivalent to

the inequality constrained problem (40). The control input ap-

plied to the plant is

(67)

Temporarily omitting the constraint from (40), the solution

using the Newton method is

(68)

with convergence criterion

(69)

which is satisfied if the matrix has full rank, requiring

. The term is calculated between

each trial by solving

s.t. (70)

via the gradient method. Hence to solve (40) via the Newton

method, now impose the inequality constraint on (70). In terms

of the control input, on trial , the constraint enforces

, which, assuming has not yet been updated,

can be written as . In terms of the Newton

descent direction, this translates to .

Hence, (70) becomes

s.t.

This is equivalent to

s.t.

with corresponding update

(71)

applied to the plant , where the elements of are

given by . Similar analysis to

that used in Theorem 5 relates the number of inter-trial updates

of (71) to the convergence of the Newton update (68) whose

descent direction it approximates.Although separation of the soft constraint and tracking error

objective is ensured by the inequality constraint in (66), in prac-

tice must also beupdated to ensure it continues to besatisfied

in the presence of model uncertainty and noise. Therefore in

(67) is also updated using the Newton ILC update

(72)

with the constraint where it has been

assumed that has just been updated via (68) as discussed.

The unconstrained Newton ILC descent direction,

, in (72) is the solution, , to

s.t. (73)


10/13


so that the corresponding required constraint is

. This gives

s.t.

which is equivalent to

s.t.

with the corresponding inter-trial gradient descent update

(74)

applied to the simulated plant model , where the elements of

are given by .

The total update sequence is therefore as follows.

Algorithm 3


(b) construct a suitable approximation to the term

which satisfies

when used in Newton update (68), through repeated

application of (71)

(c) use the resulting input to form the next update (68)

(d) construct suitable approximation to

which satisfies when used in

Newton update (72), through repeated application of (74)

(e) use the resulting input to form the next update (72)

(f) use the new and values to form the next

control input using (67). Go to (a)

The number of inter-trial updates of (71) and (74) is chosen

heuristically to provide a compromise between excessive am-

plitudes/frequencies present in the update , robustness, and

the subsequent convergence governed by (62). In practice is

treated as a tuning parameter which is adjusted by monitoring

plant input, output and error signals between trials.

Remark 10: Instead of initially solving the equality con-

straint through application of the algorithms in Section III-B or

Section IV-B, the mixed constraint updates may be applied di-

rectly to solve (66) starting from an arbitrary initial input. As in

the gradient approach, however, the soft constraints may influ-ence the action of the hard constraint so that the point-to-point

tracking component may not be solved as accurately as when it

is tackled in the absence of the soft constraint.

Similar to Section III-C, Example 1 and Example 2 are again

used to show how to incorporate some performance indices into

the objective function (66).

1) Example 1—Derivative Constraints: Again consider the

output derivative constrained problem for vibration suppression

at prescribed time instants. From Example 1 in Section III-C

and the soft constraint component (68)

becomes

(75)

Fig. 1. Robotic manipulator system showing output angle, .

Using (71), the descent direction in (75) is produced after trial

by inter-trial updates of the input

(76)

to the system

2) Example 2—Energy Constraints: Consider again the

mixed input and output constrained problem (Example 2 in

Section III-C) which reduces signal bounds while ensuring a

non-impulsive action. From Section II this cost corresponds to

, and the update (68) is

(77)

From (71), the descent direction in (77) is produced after trial by inter-trial updates of the input

(78)

to the simulated system

V. EXPERIMENTAL R ESULTS

The ILC approaches developed have been tested on a six de-

gree of freedom anthropomorphic robotic arm whose five rotary

joints are composed of PowerCubes (Schunk GmbH & Co.) in-

corporating brushless servomotors with integrated power elec-

tronics and transmission. These communicate with a dSPACE

ds1103 control board via a CAN bus at a rate of 500 kbit/s.

Results are presented for the first joint which is aligned in the

horizontal plane as shown in Fig. 1. Each servomotor includes

cascaded current and velocity control loops, and frequency re-

sponse tests have established that the linear model (79), shown

at the bottom of the next page, adequately represents the system

dynamics, with input and output in degrees. A sampling time of

200 Hz has been used in all experimental tests.


11/13


Fig. 2. Unconstrained point-to-point ILC experimental results: a) output; b)input; and c) point-to-point tracking error. Final trial output and input are shownfor the gradient update with . The output produced in simulation isshown in a) and denoted “standard ILC reference”.

The task replicates an industrial process and comprises

moving to three angles , at corresponding

samples , and . First the un-

constrained gradient approach of Section III has been applied.

The update (25) is employed for both andvalues, and error norm and tracking results are shown in Fig. 2.

Larger values of produce overly oscillatory behavior and

ultimately error divergence. For comparison, results using the

Newton update of Section IV are also given. Here Algorithm

1 is performed using inter-trial updates of (57) with

used to produce each Newton descent direction em-

ployed in (53). These values have been chosen heuristically to

affect a compromise between convergence and excessive input

amplitudes/frequencies which give rise to learning transients

and ultimately error divergence.

The point-to-point framework is now compared to the stan-

dard ILC framework in terms of error tracking and ability tomeet performance objectives. The unconstrained point-to-point

algorithms correspond to a minimum input energy performance

objective. Hence standard ILC implementations ( )

of both gradient and Newton-based algorithms have been con-

ducted using a reference that is designed a priori based on the

nominal plant model, to minimize the same criterion [shown in

Fig. 3. Unconstrained point-to-point compared against standard ILC with a priori designed optimal reference (denoted “standard ILC reference” in Fig. 2),for both gradient and Newton based algorithms, using optimal .

Fig. 2(a)]. Fig. 3 shows the corresponding point-to-point error

norm and input energy using the optimal gain choice (27). From

the input energy plot it is clear that embedding performance ob-

jectives leads to superior values since the updates reach a min-

imum through learning from experimental data, rather than one

purely relying on the nominal model. It is also important to note

that forcing tracking along the full trial duration also creates ad-

ditional learning transients which degrade convergence as con-

firmed by the plot of . These reflect the reduced con-

vergence rate of standard gradient ILC shown in Theorem 3. Next inequality constraints of have

been introduced to represent actuator saturation, through

selection of and

within (32) or (63). Results are

shown in Fig. 4 where 150 trials of both the gradient approach

of Section III-B, and the Newton approach of Section IV-B

have been performed. The gradient scheme uses (34) with

updated using (35), (36) and . It can be seen that the

input satisfies the inequality constraints while meeting the

point-to-point tracking requirement. The inequality constrained

Newton update is implemented using Algorithm 2 with

trials of (65) using to produce each Newton update.These values have been chosen heuristically to provide a com-

promise between convergence speed and excessive amplitudes,

frequencies and learning transients. To compare point-to-point

ILC with the standard ILC framework, a reference has first been

generated by solving the constrained point-to-point tracking

problem in simulation using the nominal plant [the converged

(79)


12/13


Fig. 4. Experimental results for point-to-point ILC with hard input constraints:a) output; b) input; and c) point-to-point tracking error. Final trial output andinput are shown for the gradient update with . The output producedin simulation is also shown in a) and denoted “standard ILC reference”. Whenthis is used as a reference in standard ILC, the input produced violates the hardconstrains, as shown in b).

output is shown in Fig. 4(a)]. This output is then used as the ref-

erence for standard ILC ( ) applied to the experimental

system using the optimal gain choice (27). The input producedover 150 trials is shown in Fig. 4(b) and clearly does not

satisfy the hard constraint. This confirms that the standard ILC

framework is unable to satisfy constraints since a predefined

reference is not robust to model uncertainty and noise.

Finally the case of a soft constraint in conjunction with

inequality constraints is considered. The soft constraint

comprises minimizing the output derivative over the period

, as may be required when fragile payloads or

open top containers containing liquid are handled. This cor-

responds to the weight

and the function in (37) or (66). Inequality

constraints of have also been employedthrough selection of and

. For the gradient case, up-

dates (45a) and (45c) are used in conjunction with (49). Results

are shown in Fig. 5 using , . Results using the

Newton update with mixed constraints are also shown, where

Algorithm 3 of Section IV-C has been implemented using 10

iterations of (76) with to construct the descent direction

in (75), and 20 iterations of (74) with to approximate

the descent direction in (72). As with previous results, the

Newton approach converges to very similar input and output

signals as the gradient approach, but requires significantly

fewer trials. To facilitate comparison with the standard ILC

framework, the constrained point-to-point tracking problem

has been solved in simulation using the nominal plant [with

Fig. 5. Experimental results for point-to-point ILC with mixed constraints: a)output; b) input; c) point-to-point tracking error; and d) derivative norm. Finaltrial output and input are shown for the gradient update with ,

. The output produced in simulation is also shown in a) anddenoted “standard

ILC reference”. When this is used as a reference in standard ILC, the input produced violates the hard constrains, as shown in b).

converged output shown in Fig. 5(a)]. Using the optimal gain

choice (27), this output is then used as the reference over 150

experimental trials of standard ILC ( ). The converged

input is shown in Fig. 5(b) and clearly violates the required

inequality constraints, again illustrating that using a predefined

reference with standard ILC framework cannot satisfy con-

straints in practice.

In all tests parameters are chosen to affect a compromise be-

tween convergence, input amplitudes/frequencies and learning

transients. The experimental results confirm the ability of point-to-point ILC to robustly address both hard and soft

constraints. Results also confirm that the proposed algorithms

significantly improve on the standard ILC framework which is

not robust with respect to the imposed constraints.

VI. CONCLUSION AND FUTURE WORK

The requirement for point-to-point motion control arises in

many practical applications, including industrial automation,

robotics, and rehabilitation engineering. However, there are no

available approaches to address general point-to-point tasks and

performance objectives in a framework which uses learning to

attain optimal solutions in the presence of model uncertainty

and noise. This paper addresses this deficit, enabling multiple


13/13


point-to-point tasks to be achieved while simultaneously tack-

ling both hard and soft constraints of wide relevance. Experi-

mental results confirm the practical utility and performance of

the proposed approaches and illustrate the benefit gained over

using the standard framework with an a priori generated refer-

ence. They also clearly show the benefit of point-to-point ILC

over the standard ILC framework.

Future work will consider the inclusion of prescribed

variation in the temporal point-to-point locations to provide

more flexibility and faster convergence properties. Constraints

linking two or more outputs will also be considered, allowing

coordinated movements to be performed while relaxing unnec-

essary temporal constraints.

R EFERENCES

[1] D. A. Bristow, M. Tharayil, and A. G. Alleyne, “A survey of itera-tive learning control a learning-based method for high-performancetracking control,” IEEE Control Syst. Mag., vol. 26, no. 3, pp. 96–114,2006.

[2] H. S. Ahn, Y. Chen, and K. L. Moore, “Iterative learning control: Brief survey and categorization,” IEEE Trans. Syst., Man, Cybern. C, Appl.

Rev., vol. 37, no. 6, pp. 1099–1121, Nov. 2007.[3] C. T. Freeman, E. Rogers, A. M. Hughes, J. H. Burridge, and K. L.

Meadmore, “Iterative learning control in healthcare: Electrical stimula-tion and robotic-assisted upper limb stroke rehabilitation,” IEEE Con-trol Syst. Mag., vol. 32, no. 1, pp. 18–43, Feb. 2012.

[4] D. M. Wolpert, Z. Ghahramani, and J. R. Flanagan, “Perspectives and problems in motor learning,” Trends in Cognitive Sci., vol. 5, no. 11, pp. 487–494, 2001.

[5] H. Ding and J. Wu, “Point-to-point control for a high-acceleration po-sitioning table via cascaded learning schemes,” IEEE Trans. Ind. Elec-tron., vol. 54, no. 5, pp. 2735–2744, Oct. 2007.

[6] J. Park, P. H. Chang, H. S. Park, and E. Lee, “Design of learning inputshaping technique for residual vibration suppression in an industrialrobot,” IEEE/ASME Trans. Mechatron., vol. 11, no. 1, pp. 55–65, Feb.2006.

[7] J. van de Wijdeven and O. Bosgra, “Residual vibration suppression

using hankel iterative learning control,” Int. J. Robust Nonlinear Con-trol , vol. 18, pp. 1034–1051, 2008.

[8] G. Gauthier and B. Boulet, “Robust design of terminal ILC withmixed sensitivity approach for a thermoforming oven,” J. Manuf. Sci.

Eng., vol. 2008, 2008, Article ID 289391.[9] Y. Wang and Z. Hou, “Terminal iterative learning control based station

stop control of a train,” Int. J. Control , vol. 84, no. 7, pp. 1263–1274,Jul. 2011.

[10] J.-X. Xu and D. Huang, “Initial state iterative learning for final statecontrol in motion systems,” Automatica, vol. 44, pp. 3162–3169, 2008.

[11] Y. Chen and J.-X. Xu, “A high-order terminal iterative learning controlscheme,” in Proc. 36th Conf. Decision Control , 1997, pp. 3771–3772.

[12] J.-X. Xu, Y. Chen, T. Lee, and S. Yamamoto, “Terminal iterativelearning control with an application to RTPCVD thickness control,”

Automatica, vol. 35, pp. 1535–1542, 1999.[13] G. Gauthier and B. Boulet, “Terminal iterative learning control de-

sign with singular value decomposition decoupling for thermoformingovens,” in Proc. Amer. Control Conf., 2009, pp. 1640–1645.

[14] P. Lucibello, S. Panzieri, and G. Ulivib, “Repositioning control of a two-link flexible arm by learning,” Automatica, vol. 33, no. 4, pp.579–590, 1997.

[15] S. Mishra, U. Topcu, and M. Tomizuka, “Optimization-based con-strained iterative learning control,” IEEE Trans. Control Syst. Technol.,vol. 19, no. 6, pp. 1613–1621, Nov. 2011.

[16] S.-G. Hwang, “Cauchy’s interlace theorem for eigenvalues of Hermi-tian matrices,” Amer. Math. Monthly, vol. 111, pp. 157–159, 2004.

[17] D. H. Owens, J. J. Hätönen, and S. Daley, “Robust monotone gradient- based discrete-time iterative learning control,” Int. J. Robust Nonlinear Control , vol. 19, pp. 634–661, 2009.

[18] J. M. Ortega and W. C. Rheinboldt , Iterative Solution Of Nonlinear Equations In Several Variables, 1st ed. New York: Academic Press,1970.

[19] J. J. Hätönen, “Issues of algebra and optimality in iterative learningcontrol,” Ph.D. dissertation, Dept. Process Environmental Eng., Univ.Oulu, , Oulu, Finland, 2004.

[20] C. T. Freeman, “Constrained point-to-point iterative learning controlwith experimental verification,” Control Eng. Practice, vol. 20, no. 5, pp. 489–498, May 2012.

[21] S. Boyd and L. Vandenberghe , Convex Optimization. Cambridge,MA: Cambridge Univ. Press, 2005.

[22] A. Ben-Tal and M. Zibulevsky, “Penalty/barrier multiplier methods for convex programming problems,” SIAM J. Optim., vol. 7, pp. 347–366,1997.

[23] C. T. Freeman, “Constrained point-to-point iterative learning control,”in Proc. 18th IFAC World Congr., 2011, pp. 3611–3616.

Chris T. Freeman received the B.Eng. degreein electromechanical engineering and the Ph.D.degree in applied control from the University of Southampton, Southampton, U.K., in 2000 and 2004,respectively, and the B.Sc. degree in mathematicalsciences from the Open University, Milton Keynes,U.K., in 2006.

His research currently focuses on the devel-opment, application and assessment of iterativelearning and repetitive controllers within both the biomedical engineering domain and for application

to industrial systems.

Ying Tan received the Bachelor’s degree fromTianjin University, Tianjin, China, in 1995, andthe Ph.D. degree from the National University of Singapore, Singapore, in 2002.

She joined McMaster University in 2002 as a post-doctoral fellow with the Department of ChemicalEngineering. She has worked with the Department of Electrical and Electronic Engineering, the Universityof Melbourne, Parkville, Australia, since 2004. Her research interests include intelligent systems, non-linear control systems, model predictive control, real

time optimization, sampled-data distributed parameter systems and formationcontrol.

Dr. Tan is a Future Fellow (2010–2013), which is a research position funded by the Australian Research Council.

Documents

FREEMAN2013_Survey on Iterative Learning Control, Repetitive Control and Run-To-run Control