Optimal Feedback JOTA 10 MP

8/8/2019 Optimal Feedback JOTA 10 MP

http://slidepdf.com/reader/full/optimal-feedback-jota-10-mp 1/17

Optimal Feedback Nonlinear Control and Two Point

Boundary Value Problem for Systems with Parameter

M. POPESCU∗

Communicated by

Abstract. We consider the problem of optimum concerning to the minimization quadratic functionals of Bolza

type with constraints represented diff erential systems with parameter. One demonstrates the uniqueness of optimal

feedback nonlinear control obtained, by utilizing that the U Hilbert space control domain is rotund. The solution for

two point boundary value problem implies the determination of the solution of the system in variations associated

to the linearized system. The construction of the solution use of an iterative procedure, yielding the initial value

results of the adjoint variable. By presenting the state vectors x ∈ X and the control vectors u ∈ U in orthonormal

basis, one develops a numerical approximation method of the solution to the optimum problem.

Key words: Semigroup, variation system, two point boundary value problem, rotund space, orthonormal basis.

1. Introduction

An extended class of dynamic phenomena has for its objective the optimization of the performance indices ex-

pressed by Bolza type quadratic functionals. The quadratic linear problem to pertains of this class, for which con-

straints are represented by the controlled linear diff erential systems. Let one consider an undetermined quadratic

form of the solution to the Bellman equation. Thus, the Riccati matrix diff erential and algebraic equation is associ-

∗Institute of Mathematical Statistics and Applied Mathematics of the Romanian Academy, P.O. Box 1-24, RO-010145, Bucharest, Romania,

e-mail: ima [email protected]

1



ated to the quadratic linear problem which corresponds to the finite and infinite time respectively. The solution to

the Riccati equations is a positively semi defined matrix, being used in determining optimal control; the minimum

value of the functional and the stabilization of the linear systems (Riccati algebraic equation), [1], [9], [4], [20].

The extremization of the quadratic functional with constraints represented by nonlinear affine and bilinear systems

has been treated by the author on the hypothesis that the algebra generated by the vector field that defines the

nonlinear system form Lie algebras. At the same time, one considers that the operator defined by the set of Lie

parentheses is nilpotent [5], [7], [12], [13], [14]. The calculus of variation off ers analytical methods for analysis

necessary and sufficient conditions of optimality. Thus, there have been determined the conditions of positivity of

the second variation of the Bolza functional of the final constraint manifold and the sufficient optimality conditions

consisting in the inexistence of points conjugated with the extremities along the optimum trajectory. The results

obtained are extended, for nonlinear systems with parameters, for the case of nonsingular and singular control [6],

[10], [10], [15], [16], [17].

The optimization of some dynamic processes determine the nonlinear feedback control by Taylor series devel-

opment, in the hypothesis that the state and control variables lies in the neighborhood of the origin [2], [3], [8],

[16]. The feedback nonlinear control can be obtained by eliminating the adjoint vector through repeated deriva-

tions with respect to time of the Hamiltonian. This procedure involves the solution to the quasi-linear system with

partial derivatives of the first order for the case of equal dimensions of the state and control vector [2], [16].

By considering the adjoint vector function to the state vector, the author has determined the fundamental solu-

tion in two points boundary value problem of the linear control systems and the existence sufficiently conditions

for the solution [19].

The present study treats the two points boundary value problem for the quasi-linear control systems with a

parameter. One writes the Boltza functionals in the Hilbert space L2 as a function of the quasi-linear system

solutions considered. Utilizing the minimum condition of these functional with respect to the control parameter u

one obtains the feedback nonlinear optimal control u.

Because the optimal feedback control belongs to the rotund space it follows that it is unique. One demonstrates

a fundamental result for the variations system used in determining the solution of the two points boundary value

2



problem and the construction of a field of extremals in the neighborhood of the optimal solution.

We consider {ei} n orthonormal basis for X and {d i} be an orthonormal basis for U.

By writing the state vector function of the control u ∈ U and the orthonormal bases, one obtains an algorithm

which represents a numerical method of approximation to the solution for the optimum problem formulated. This

method was used by the author in minimizing energy in of orbital rendezvous problems [28].

2. The optimal control problem

We are concerned with a dynamical system governed by the following diff erential equation

x = A x + B u + F(λ, t ) , x(0, λ) = x0(λ) ∈ H , t ∈ [0, t f ] (1)

where A ∈ D(A) ⊂ H −→ H , B : U −→ H are linear operators defined on the Hilbert spaces H (state space), U

(control space) λ ∈ [λ0, λ f ] ⊂ Λ (parameter space), F function of the parameter and time.

Let X, Z be Hilbert spaces. We denote by L(X, Z) the Banach space of all linear bounded operators T : X −→ Z

endowed with the norm T = sup { |T x| : x ∈ X , |x| ≤ 1 }.

Definition 2.1. A strongly continuous semi group on X is a mapping: T : [0, ∞) −→ L(X), t −→ T(t ) such

that

(i) T(t + s) = T(t ) T(s) , ∀ t , s ≥ 0 T(0) = I,

(ii) T( · ) x is continuous for all x ∈ X.

Remark 2.1. T( ·) is locally bounded by the uniform boundedness theorem.

The infinitesimal generator A of T( · ) is defined by

D(A) =

x ∈ X : ∃ lim

h→0+

T(h) − I

h; A (x) = lim

h→0+

T(h) − I

h

(2)

We now consider a particular case for the system equation (1) defined by:

A ∈ M(n, n) , B ∈ M(n, m) are constant matrices, the state space E = Rn, the control space U = Rm, function

F with n components.

We assume that the cost functional; is of the form

J =1

2

t f

0

(Q x(s, λ), x(s, λ)) + (R u(s, λ), u(s, λ)) ds +1

2G(t f ) x(t f , λ), x(t f , λ) (3)

3



where Q ∈ M(n, n) , R ∈ M(m, m) , G(t f ) ∈ M(n, n). are symmetric, nonnegative matrices and R is a positive

definite matrix.

We have

x(t , λ) = L2[0, t f ] × L2[λ0, λ f ] = X (4)

Jt f can be written in terms of inner products in L2.

Jt f =

1

2

Q x, x

+

1

2

R u, u

(5)

The optimal control problem is formulated.

Determine u ∈ U defined on [t 0, t f ] × [λ0, λ f ] such that the functional Jt f is minimized subject to the constraint

imposed by the model system (1).

3. Optimal non-linear feedback control

A generates a strongly continuous semi group T(t ) = e t A on X. The solution of the system (1) is given by

x(t , λ) = T(t ) x0(λ) +

t 0

T(t − s) B u(λ, s) d s +

t 0

T(t − s) F(λ, s) d s t ∈ [0, t f ] (6)

Let Et : X → X defined by

Et F =

t 0

T(t − τ) F(λ, τ) d τ t ∈ [0, t f ] (7)

while

P(λ, t ) = T(t ) x0(λ) + Et F t ∈ [0, t f ] (8)

so that the equation (6) can by written

x(λ, t ) = Et B u + P t ∈ [0, t f ] (9)

Substituting this equation into the expression for Jt f gives

Jt f =

1

2

Et B u + P

, Q

Et B u + P

+

u, R u

(10)

and by re-arrangement

Jt f =

1

2

B∗ E∗t Q Et B + R

u, u+ 2

B∗ E∗t Q P, u

+

P, Q P

(11)

4





or

B ≤

n

i, k =1

b2i k

1/2

= M 0 (18)

In the same way one obtains

Q ≤

ni, k =1

q2i k

1/2

= M 2 (19)

One assumes that there exists ρ = const so that F(λ, t ) < ρ for |λ| < h. Thus relation (14) becomes

˜ x ≤ N e ν t x0 + N M e ν t

t 0

e −ν t ˜ x d s +

ρ

ν−

ρ

νe −ν t

N e ν t (20)

where we wrote M 0 M 1 M 2 = M .

By multiplying (20) by e −ν t , we have

e −ν t ˜ x ≤ N x0 + N M

t 0

e −ν t ˜ x d s +ρ

νN 1 − e −ν t

(21)

For the evaluation of ˜ x one uses the following

Lemma 4.1. If functions ψ(t ), g(t ) ≥ 0 and k (t ) ≥ 0, continuous over interval [0, t f ] satisfy the inequality in

this interval

ψ(t ) ≤ g(t ) +

t 0

k (s) ψ(s) d s (22)

then in the same interval, there occurs relation

ψ(t ) ≤ g(t ) +

t 0

g(s) k (s) e

t 0

k (τ) d τ

d s (23)

We take

ψ(t ) = e −ν t ˜ x

g(t ) = N x0 − N ρ

ν

e −ν t + N

ρ

ν

k = N M

(24)

By applying the Lemma 4.1 and carrying out the calculations, it results

˜ x ≤N ρ

−(ν + N M )(25)

We assume condition ν + N M < 0 to be fulfilled. Thus out of (25) one obtains an upper limit of the optimal

state x. By using the expression (12) of the optimal control we have

˜ u ≤ − R−1 B∗ E∗t Q ˜ x (26)

6





5. Fundamental matrix of solutions for variation system

We consider the system

d y(t )

d t = f (y(t )) (31)

where y is the state variable, f the function defined and continuous in D satisfying Lipschitz condition

| f (y) − f (y) | < K | y − y | (32)

with K a constant independent of y and defined on D.

In accordance with a known theorem, if the condition (32) is fulfilled from yn → y0, it results y(t ; t 0, yn) →

y(t ; t 0, y0), where y(t ; t 0, y0) represent the solution of system (31).

Taking into account (32) one obtained

| y(t ; t 0, yn) − y(t ; t 0, y0) | ≤ | yn − y0 | e K (t −t 0) (33)

It results the uniform convergence with t , within the interval [t 0, t f ] on which the solutions are defined.

Let f (y(t )) diff erentiable in D for t ∈ [t 0, t f ]. For y0 ∈ D, t ∈ [t 0, t f ] we have

f (y) − f (y0) =∂ f

∂ y

y= y0

( y − y0) + O(| y − y0|) (34)

Theorem 5.1. If f is diff erentiable in D for t ∈ [t 0, t f ] and y(t ; t 0, y0) ∈ D for t ∈ [t 0, t f ] is diff erentiable with

respect y0 and∂ y(t ; t 0, y0)

∂ y0is a fundamental matrix of solutions for the linear system

d δ y

d t =

∂ f (y(t ; t 0, y0))

∂ yδ y (35)

which represent the variation system corresponding to y(t ; t 0, y0) solution.

Proof

Let Y(t , t 0) a fundamental matrix for (35) where the columns of the matrix Y(t , t 0) are solutions for the variation

system and Y(t 0, t 0) = I (I is identifying matrix).

We consider y0 and y1 respectively the initial condition for (31), y0, y1 ∈ D, t ∈ [t 0, t f ]. The systems of

8



solutions for system (31) can be written

y(t ; t 0, y0) = y0 +

t t 0

f (y(τ; t 0, y0)) d τ

y(t ; t 0, y1) = y1 +

t t 0

f (y(τ; t 0, y0)) d τ

(36)

Because Y(t , t 0) is a solution of the variation system we have

Y((t , t 0) ( y1 − y0) = y1 − y0 +

t t 0

∂ f (y(τ; t 0, y0))

∂ yY(t , t 0) ( y1 − y0) d τ (37)

From (36) and (37) are obtained

y(t ; t 0, y1) − y(t ; t 0, y0) − Y(t , t 0) ( y1 − y0) =

t t 0

f [y(τ; t 0, y1)] − f [y(τ; t 0, y0)] −

∂ f [y(τ; t 0, y0)]

∂ yY(τ, t 0) ( y1 − y0)

d τ

(38)

Utilizing the diff erentiability assumption of f (y(t)) in D expressed by the relation (34) we obtain

y(t ; t 0, y1) − y(t ; t 0, y0) − Y(t , t 0) ( y1 − y0) =

t t 0

∂ f [y(τ; t 0, y0)]

∂ y

y(τ; t 0, y1) − y(τ; t 0, y0) − Y(τ, t 0) ( y1 − y0)

d τ+

t t 0

Oy(τ; t 0, y1) − y(τ; t 0, y0)

d τ

(39)

From the relation (33) it result

Oy(τ; t 0, y1) − y(τ; t 0, y0)

= O y1 − y0

(40)

for t fixed, so that relation (39) leads to

y(t ; t 0, y1) − y(t ; t 0, y0) − Y(t , t 0) ( y1 − y0) ≤

t t 0

Ky(τ; t 0, y1) − y(τ; t 0, y0) − Y(τ, t 0) ( y1 − y0)

d τ + O

y1 − y0 (41)

From relation (41) using a known inequality we have

y(t ; t 0, y1) − y(t ; t 0, y0) − Y(t , t 0) ( y1 − y0) = O

y1 − y0

(42)

The theorem 5.1 has been demonstrated.

By considering t 0 = 0 we obtain

Y(t , 0) = ∂ yi(t )

∂ y j(0) (i, j = 1, 2, . . . , n) (43)

9





Because x0 = const. for λ = λ0 = const. given and p0 is unspecified it result δ x0 = 0 (admissible trajectories),

respectively δ p0 = I such that it result

Y(0) = Y0 =

0

· · ·

I

↔ n

n

· · ·

n

(52)

The solution y(t , λ) is obtained from

y =

A B R−1 B∗

−Q −A∗

y(t ) +

F(t , λ)

0

(53)

y(0, λ) =

x0(λ)

· · ·

0

(54)

We obtain

y(t , λ) = Y(t )

x0

(λ)

· · ·

0

+ Y(t )

t 0

Y−1(s)

F(s, λ)

· · ·

0

d s (55)

A general solution x and p to equation (48) that satisfies the initial boundary condition (49) can be written asx(t , λ)

p(t , λ)

= Y(t ) p0 + y(t , λ) (56)

where p0 is a set of n-constants that correspond to a certain choice of initial conditions for p(t , λ).

They are determined at the final time from the terminal boundary condition equation (50). The relation (50)

can be written as

p∗(t f , λ) + x∗(t f , λ) G(t f ) =

x∗(t f , λ) p∗(t f , λ)

G(t f )

I

∗

= 0 (57)

such that from (56) the boundary condition (50) became

I

−G(t f )

Y(t f ) p0 + y(t f ) = 0 (58)

11



The solution of this equation (58) determines the missing initial condition on p0. A solution to two point bound-

ary value problem may be obtained by reintegrating the diff erential equations forward with the initial condition

x0, p0.

7. Field of extremals

If the right member of the diff erential system(48) allows continuous partial derivatives not only with respect to

variables x and p, but also with respect to parameter λ for λ ∈ [λ0, λ f ], then its solution allows for continuous

partial derivatives with respect to initial values λ.

This case amounts to a system without an absent parameter which admits continuous partial derivatives with

respect to the initial values. Thus one completes system (48) of the 2n order with equation (2n + 1), d λd t = 0 and

the initial value λ = λ0 for t = t 0 = 0. Then the solution of the system (48) is derivable with respect to the initial

values, particularly with respect to λ0 for any λ ∈ [λ0, λ f ]. Since d λd λ0

=d λ0

d λ0= 1 the system in variations which

corresponds to (35) is

d δ yi

d t =

2 j=1

∂ f i

∂ y jδ yi +

∂ f i

∂ λ i = 1, 2 (59)

where

y = ( y1, y2)∗= (x, p)∗

= ( x1, . . . , xn, p1, . . . , pn)∗

∂ yi

∂ λ= δ yi

(60)

In section 6 one presented a method of determining the initial value p of the adjoint variable for the two

point boundary value problem. Let a particular solution y(t , λ) of system (48) obtained for the initial values

t 0 = 0, x(0, λ) = x0(λ), p(0, λ) = p0(λ), λ = λ0 on which we write

y1 = ϕ1(t , λ) y2 = ϕ2(t , λ) (61)

The variations are defined by the values of the derivatives with respect to parameter λ so that we get (

δ yi =∂ ϕi(t , τ)

∂ λ=

∂ yi

∂ λ

t =0, x0(λ), p0(λ)

i = 1, 2 (62)

At the same time, variations δ yi satisfy the system (59) where one writes

d δ yi

d t =

2

j=1

∂ f i [ϕ1(t , λ), ϕ2(t , λ)]

∂ y j

δ yi+

∂ f i [ϕ1(t , λ), ϕ2(t , λ)]

∂ λi = 1, 2 (63)

12



The initial conditions become

(δ y1)0 =∂ x0(λ)

∂ λ= x

0(λ) (δ y2)0 =∂ p0(λ)

∂ λ= p

0(λ) (64)

By integrating system (63) with the initial conditions (64) one obtains

δ yi = Y(t )

x0

(λ)

· · ·

p0

(λ)

+ Y(t )

t

0

Y−1(s)

∂ F(t , λ)

∂ λ

· · ·

0

d s (65)

Thus if one knows a particular solution (61) one can determine the variations δ yi.

Proposition 7.1 If λ0 is the value of the parameter for which we have solution (61), then the solution corre-

sponding to the value λ0 + d λ of the parameter is obtained by Taylor series development, for which one retains

only the first degree terms in d λ.

yi(t , λ0 + d λ) = yi(t , λ0) +

∂ yi

∂ λ

λ=λ0

d λ = yi(t , λ0) + δ yi(t , λ0) d λ i = 1, 2 (66)

Hence, the diff erence between solution (61) and a neighboring solution is written through the product between

variations δ yi and the diff erential of parameter λ. The results of this study can be extended without difficulty for

an arbitrary number of parameters (λ, µ . . .).

8. Numerical method for approximate solution

Let the state space X = Rn, the control space U be Hilbert space and linear transformation S : U → X. The

mathematical model can be postulated as x = S u where S is a linear operator. We consider {ei} n orthonormal

basis for X and {d i} be an orthonormal basis for U. Define

u j =

u, d j

so that u =

n j=1

u j d j (67)

and define similarly

xi = x, ei (68)

Then

x(λ) = S n

j=1

u j

d j =

n

j=1

u j

S d j

(69)

13



Taking inner products with ei one obtained

x, ei =

n j=1

u j S d j, ei

(70)

which can be written

xi =

n j=1

u j

S d j, ei

(71)

or

x =

ni=1

xi ei =

ni=1

n j=1

u j

S d j, ei

ei (72)

Denoting the scalars

S d j, ei

= α j i (73)

Then the mathematical model is defined by

x =

ni=1

n

j=1

u j α j i

ei (74)

In the problem of the optimal control considered, we have

x(t f , λ) = Φ(t f , 0) x0(λ) +

t f

0

Φ(t f , τ) B u(λ, τ) d τ +

t f

0

Φ(t f , τ) F(λ, τ) d τ

where the matrix Φ(t f , τ) = T (t − τ) so that the control problem is to ensure that the following equation is satisfied:

x(t f , λ) = x(t f , λ) − Φ(t f , 0) x0(λ) −

t f

0

Φ(t f , τ) F(λ, τ) d τ =

t f

0

Φ(t f , τ) B u(λ, τ) d τ = S u (75)

with the control u that satisfies the equation and minimizes the given cost functional (3)

Each element (S u)i

can be written

(S u)i =

nk =1

t f

0

Φ(t f , τ) B

i k

uk (τ, λ) d τ (76)

Let {ei} be the usual coordinate basis for X then

x = S u =

ni=1

ei(S u)i =

ni=1

ei

nk =1

t f

0

Φ(t f , τ) B

i k

uk (τ, λ) d τ (77)

The rows of Φ(t f , τ) B

i = 1, . . . , n are the functionals that form a basis {d i} for space U.

14



9. Conclusions

The results of this study refer to the minimization of the Bolza quadratic functional, sunmitted restrictions repre-

sented but diff erential systems controlled with parameter. The feedback control determined is function by bounded

linear operator. One demonstrates that the control space is rotund which implies the need for optimal control.

One demonstrates that the control space is rotund, which involves the uniqueness of optimal control. Through the

mathematical methods used in this study one verifies the ” Banach - Steinhaus theorem” and ” The open mapping

theorem” referring to the bounding of the operator norm and existence of optimal control respectively. The de-

termination of the initial value of the adjoint variable through an interactive procedure allows the integration of

the system out of which one obtains the solution for the two-point value problem. The construction of the field

of extremals in the neighborhood of the optimal trajectory is achieved by solving the associated variations system.

The neighboring trajectory extremal class represents the relative extremal in the optimumum problem formulated.

The absolute extremum results from verifying the optimal sufficient conditions defined by the fact that there are no

points conjugated with the endemicities of the trajectory analyzed. This issue was treated by the author in [10] [15]

[16] [ 17]. Finally, there is presented a numerical method for determining an approximate solution to the optimum

control problem where the elements from the state and control spaces respectively are expressed in orthonormal

bases.

References

[1] B, R.: Dynamic Programming, Princeton University press (1977).

[2] B - S, H.: On applications of a new method for computing optimal nonlinear feedback

controls, Optimal Control Applications & Methods, Vol. 8, pp. 397 - 409 (1987).

[3] B - S, H. and F, M.: Optimal feedback control of nonlinear systems, Automatica,

Vol. 23, No. 3 pp. 365 - 372 (1987).

[4] D P, G.: Linear Control Theory for Infinite Dimensional Systems, A Agrachev Ed. ICTP Lecture Notes

pp 63 - 105 (2009).

15



[5] H, H.: Nilpotent and high order approximations of vector field systems, SIAM Reveiw vol. 33, pp 238

- 264 (1991).

[6] H, D. G., Optimal control theory for application, Springer-Verlag New York (2003).

[7] N, H.: Controlled invariance for affine control systems, Int. J. Constr. vol. 34 pp. 824 - 833 (1981).

[8] S, SM. and R, P.: Optimal nonlinear feedback regulation of spacecraft angular momentum, Optim

control appl. methods vol. 5, pp. 101-110 (1984).

[9] Z, J.: Mathematical Control Theory, An introduction Birkhauser (1996).

[10] P, M.: Singular normal extremals and conjugate point for Bolza functionals, Journal of Optimization

Theory and Applications Vol. 115, No. 2, pp 267-282 (2002).

[11] P, M.: Optimal singular control for controlled dynamical systems, Academic Ed. Bucharest (2002).

[12] P, M.: On minimum quadratic functional control of affine nonlinear systems, Nonlinear Analysis, vol.

36, pp 1165-1173 (2004).

[13] P, M.: Control of affine nonlinear systems with nilpotent structure in singular problems, Journal of

Optimization Theory and Application, Vol. 124, No. 2, pp 455 - 466 (2005).

[14] P, M. and P, F.: Courbes optimales pour une distribution affine, Bull. Sci. Math. vol. 129, pp.

701-725 (2005).

[15] P, M.: Sweep method in analysis optimal control for rendez-vous problems, Journal of Applied Math-

ematics & Computing, vol. 23, No. 1-2, pp. 243-256 (2007).

[16] P, M.: Variational and transithory proceses, nonlinear analysis in optimal control, Technical Ed,

Bucharest (2007).

[17] P, M.: Optimal control for quasi-linear systems with small parameter, Journal of Applied Mathematics

& Computing, vol. 27, No. 1-2, pp. 393 - 409 (2008).

16



[18] P, M.: Advances in Mathematical Problems in Engineering Aerospace and Sciences, Cambridge Sci-

entific Publishers, England vol. 2, (2008) Edited by S. Sivasundaram.

[19] P, M.: Fundamental solotion for linear two point boundary value problem, Journal of Applied Mathe-

matics & Computing, vol. 23, No. 5-6, pp. 385 - 394 (2009).

[20] P, M.: Stability and Stabilization of Dynamical Systems, Technical Ed., Bucharest (2009).

17

Documents

Optimal Feedback JOTA 10 MP