Numerical Optimizationoptimal control problem derivation of similar results based on the concept of Lyapunov stability will also be demonstrated. 2. Focus on numerical methods for

1

6-Aug-09 Vira ChankongEECS, CWRU

1

Numerical Optimization

Instructor: Vira ChankongElectrical Engineering and Computer ScienceCase Western Reserve University

Phone: 216 368 4054, Fax: 216 368 3123E-mail: [email protected]

A WorkshopAt

Department of Mathematics Chiang Mai University

August 4-15, 2009


2Vira ChankongEECS. CWRU

2

Session:Optimization of Dynamic Systems

Vira ChankongVira ChankongCase Western Reserve UniversityCase Western Reserve University

Electrical Engineering and Computer ScienceElectrical Engineering and Computer Science

2


3

Module Objective Module Objective

To develop functional knowledge and skills in

• modeling dynamic systems and/or optimal control problems

• determining a state trajectory and/or a control of dynamic systems that optimizes a performance functional subject to constraints on control and/or states.


4

Module HighlightsModule HighlightsTwo pronged approach:

1. Focus on the two traditional approaches for dealing with dynamicoptimization/optimal control problems

• the variational approach based on calculus of variations leading to the maximum principle

• the dynamic programming approach and its corresponding Hamilton-Jacobi-Bellman (HJB) equation. When applied to linear optimal control problem derivation of similar results based on the concept of Lyapunov stability will also be demonstrated.

2. Focus on numerical methods for solving large real-world dynamic optimization and optimal control problems with complex constraints. The two numerical approaches are the indirect approach and the direct approach. MATLAB will be used to implement methods discussed.

Applications to engineering and economic problems will be illustrated throughout

3


5

ReferencesReferencesText:

Optimal Control Theory, D.E. Kirk, Dover Publications, ISBN: 0486434842, 2004

References:

1. Practical Methods for Optimal Control Using Nonlinear Programming, J.T. Betts, SIAM, ISBN: 0898714885, 2001

2. Applied Dynamic Programming for Optimization of Dynamical Systems, R.D. Robinett III, D.G. Wilson, G. Richard Eisler and J. E. Hurtado, SIAM, ISBN: 089715865, 2006


6

Dynamic OptimizationDynamic Optimization00

( )

: [ , ]

( ) ( )0 0 0 0 0 0 0

( ) ( )

min ( , ,.., , )

subject to , ( ) , ( ) ,.., ( )

and possibly , ( ) , ( ) ,.., ( )

f

f

t n

tx t t R

n n

n nf f f f f f f

g x x x t dt

t x t x x t x x t x

t x t x x t x x t x

J→

=

= = =

= = =

∫

Note:1) The final time may or may not be specified.

If is specified problem

(or

fixed-end-time

fixed-terminal-time)If is not specified variable- en

f

f

f

t

t

t

⇒

⇒ problem

d-time

open-terminal-time, open ( horor izon)2) If is specified, and if

( ) is also fixed problem

( ) is constraine

fixed-end-point

constrained-terminald (i.e. ( ) ) problem

( ) is not fixed or has no restric

-point

frtion

f

f

f f

f

t

x t

x t x t S

x t

⇒

∈ ⇒

⇒ ee-end-poin prot blem

4


7

Optimal Control Problems

00: [ , ]

0

0 0

System Dynamics

min ( ( ), )+ ( ( ), ( ), )

subject to ( ) ( ( ), ( ), ); where state : [ , ]

( )

:

Initial conditions:Terminal or final

f

mf

t

f f tt t R

nf

h t t g t t t dt

t t t t

J

t t R

t

→=

= →

=

∫ux x u

x a x u x

x x

0 0

Find a control ( ) to take the system from the initial state( ) at the initial time to the final

(

state ( ) at the final tim

cond

e

i )tions:

.

f

f f

f

tt t

t

t t

=

x x

x x

u


8


0

min max 0

0

Possible additional constraints:

( ) for all [ , ]

e.g. ( ) for all [ , ]

( ) for all [ , ]

Contraints on control:

Contraints o

e.g.

n state:

t f

f

t f

t U t t t

t t t t

t X t t t

∈ ∈

≤ ≤ ∈

∈ ∈

u

u u u

x

min max 0 ( ) for all [ , ]ft t t t≤ ≤ ∈x x x

5


9

Optimal Control ProblemsNote:1) As before, may be specified (

or may not specififixed-end-time)

variable-end-time or open term

ei l

d

(na

ft

time)

2) If is specified,

( ) may be fixed ( or

fixed-end-point or hard terminal constraint)softconstrained ( ) (

or not fixed or nterminal con

o restrictiostraint)

fren ( e-end-point)

f

f

f

t

x tx t S∈


10


g = 0,

Special Mayer Problems: (

: i.e. ( ( ), ) ------------------

( )

Various types of Mayer Problem

(Li

nea) ar

(

r

)

)

b

Maye

f f

f

JJ h t t

J c t

• =

=

x

x

( ) ( )( )- ( ) ( )- ( ) (Terminal control proble m)T

f f f fJ t t t t= x r H x r

0

00

= 0,

Special Lagrange Problems:

= 1, i.e.

Lagrange Problem

i.e. ( ( ), ( ), ) --------

1 -

= ( ) , i.e.

(a) (Mini

mum time)

(b ) (

f

f

t

t

t

ft

J g t t t dt

J dt t t

h

g

t tJg

• =

= =

=

∫

∫u u

x u

0

0

2

)

=

(Minimum fuel)

(c) (Mi ( ) , i.e. nimum energy( ) ( )

)

f

f

t

t

t

t

Tg t t

t

dtt

d

J =

∫

∫u u u

6


11


( ) ( )( ) ( )

0

Tr

ac

(

k

(

iSpecial Bolza Problems:

(a)

), )+ ( ( ), ( ), ) --------

( )- ( ) ( )- ( )

=

ng probl

( )- ( )

B

( )- ( )

olza Proble

=

m:

ms

e

ft

f f t

T

f f f f

T

J h t t g t t t dt

t t t t

t t t

h

g t

• = ∫x x u

x r H x r

x r Q x r

( ) ( ) ( ) ( )0

( ) ( )

( )- ( ) ( )- ( ) ( )- ( ) ( )- ( ) ( ) ( )

Note: , and are w eighting matri es

c

f

T

tT T Tf f f f t

t t

J t t t t t t t t t t dt

+

= + +⇒ ∫

u Ru

x r H x r x r Q

Q

x r u Ru

H R

0

(b) =

=

Regulator problem ( ) ( )

( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( ) ( )

:

f

Tf f

T T

tT T Tf f t

t t

t t t

h

t

J t t t t t t dt

g +

⇒ = + +∫

x Hx

x Qx u Ru

x Hx x Qx u Ru


12

Example 1: (Brachistochrome) A bead of mass 1 unit descends along a wire joining two fixed points (x0,y0) and (xf,yf). We wish to find the shape of the wire so that the bead completes its slide in minimum time.

(x0,y0)

(xf,yf)

t = t0

7


13

Example 1: (Brachistochrome) Model:

(x0,y0)

(xf,yf)

t = t0

y = y(x), y: [x0,xf]→R, y ∈ C1 ( )0 0

0 0

1 ( )min 2 ( )

subject to ( )

Minimum timeFixed terminal point problem(Hard terminal constr

Dynamic Optimization Mod

a

e

i

)

l:

n )

(

t

fx

f x

f f

y xt dxg y y x

y x yy x y

′+=

−

=

=

∫


14

Example 2: (River-Crossing 1) A boat travels with constant velocity Vwith respect to the water. In the region the velocity of the current is parallel to x-axis but varies with y. Given the destination (xf,yf) on the other side of the river, find the path to be taken by the boat to minimize the travel time,

(x0,y0)=

(0,0)

(xf,yf)

t = t0

t = tf

vc(y)

x

y

θ

V0

0 0 0 0

0min 1

subject to(Equation of M otion):

cos ( ) sin

Optimal Control M odel:

Dynamics

Initial conditions: Final c

( ) ; ( ) ( )

M inim

onditions: ; )

(

um

ft

f t

c

f f f f

t dt

x V v yy V

x t x y t yx t x y t y

t

θθ

=

= +

+

== =

= =

∫

timeFixed terminal point problem

8


15

Example 3: (River-Crossing 2) A boat travels with constant velocity Vwith respect to the water. In the region the velocity of the current is parallel to x-axis but varies with y. Given the final time tf, find the path to be taken by the boat to maximize the landing distance on the other side of the river,

(x0,y0)=

(0,0)

(xf,yf)

t = t0

t = tf

vc(y)

x

y

θ

V

0 0 0 0

max = ( )

subject to(Equation of M otion):

cos ( )

Optimal Control M odel:

Dynamics

Initial conditions: Final con

di

sin ( ) ; ( )

Linear M ayer

ti

problem

(: )

w

ons

f

c

f f

J x t

x V v yy V

x t x y t yy t y

θθ

= +=

= ==

ith semi-hard terminal constraint


16

Example 4: (Braking and Acceleration): Want to move a car of mass m from 0 to xf in minimum time.

x(0) = 0

0

0

1 2

2

1 2

1

min = 1

subject to(Equation of Motion):

( ) ( ) ( )

Optimal Control:

Dynamics

Initial conditio (ns: Terminal co

0)nditions:

0; (0 0( )

)

ft

f

f f

J t dt

x x tx t t

x xx t x

α β

=

== +

= ==

∫

2

1 1 2 2

1 2

; ( ) 0

For each [0, ] :

On control: 0 ( ) ; 0 ( )

Contraints:

On state:

0 ( ) ;

0 ( )

f

f

f

x t

t t

u t M u t Mx t x x t

=

∈

≤ ≤ ≤ ≤≤ ≤ ≤

tf

x(tf) = xfx(t)

α (t) +β (t) t

1

2 1

( ) ( ) ( ) ( ) ( )

( ) ( )

EOM:Stat

(e :

)s

x t t tx t x tx t x t x t

α β= +== =

1 1

2 2

: ( ) ( )

0 1 0 0,

0 0

Compact Dynamic

1 1

( ) ( ) ( )( ) , ( )

( )

s

) ( ) (

t t

x t u t tt t

x t u t tαβ

= +

⎛ ⎞ ⎛ ⎞= =⎜ ⎟ ⎜ ⎟⎝ ⎠ ⎝ ⎠⎛ ⎞ ⎛ ⎞ ⎛ ⎞

= = =⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎝ ⎠⎝ ⎠ ⎝ ⎠

x Ax Bu

A B

x u

Minimum timeHard terminal constraint with bound constraints

9


17

Example 5: (Taking-off) An aircraft of mass m (assumed point mass) is to be lifted by a constant trust T to reach the cruising altitude H at time tf, at maximum speed (along x-direction).

H

t0= 0

t = tf

v

x

y

β(t)

T2

1 2

2

3 4

4

1

max = ( )

(Equation of Motion):

cos

Opt

imal Control Mod

el:

Dynamics

sin

(

Initial conditions:

fJ x t

x xTxm

x xTx gm

x

β

β

=

=

=

= −

2 3 4

3 4

3

0) (0) (0) (0) 0 ( ) ; ( ) 0

On control 0 ( ) /2, [0, ]

On state: ( ) 0, 1, .

Fin

.4; (

al conditions:

Constraints:

)

f f

f

i

x x xx t H x t

u t t t

x t i x t H

π

= = = == =

≤ ≤ ∈

≥ = ≤1 2 1

3 4 3

( ) cos ( ) ( ) sin ( )

( ) ( ); ( ) ( ) ( )

EOM:

States:

Contr ( ) ( ); ( ) ( ) ( )

)ol ( ) (

mx t T tmy t T t mgx t x t x t x t x t

x t y t x t x t y tu t t

ββ

β

== −= = == = =

= Linear Mayer problem with semi-hard terminal constraint


18

Example 6: (Rover Control) Want to control the speed of a Mariner Mars rover at about 5 mph using as little energy as possible. The controller is the output voltage of a battery-operated voltage regulating system.

1 2

1 2

( ) ( ) ( )

( ) ( )

( ) ( ) ( ) ( )

( ) (

Dynamics:

States:

Controls:

); ( ) ( )

( ) ( ); ( ) ( )

ff f f

t f

L

f

L

di tR i t L e t

dtt K i t

t I t B t t

x t i t x t t

u t e t u t

torque

t

λ

λ θ θ λ

θ

λ

+ =

= − − −

= + +

= =

= =

Voltage Regulating System

Rf

Battery

Lf

if

e(t)Ra

La

Ia (constant)

λ(t)

θ(t)

λL(t)

Viscous friction coefficient B

10


19

Example 6: (Rover Control)( )2

2 1 10

1 1 1

2 1 2 2

1 2

Min ( ( ) 5) ( ) ( )

:

1 ( ) ( )

1

Dynamics

Initial condi

( ) - ( ) ( )

(0) (0) 0

tions: Final con None On

ditions: Constrai t :

n s

ft

f

f f

t

J k x t wx t u t dt

Rx x t u t

L L

K Bx x t x t u tI I I

x x

= − +

= +

= −

= =

∫

1 max

2 max

1 max

2 max

control: ( ) , [0, ]

( ) , [0, ]

On state: ( ) , [0, ]

( ) , [0, ]

f

f

f

f

u t e t t

u t t t

x t I t t

x t t t

λ

≤ ∈

≤ ∈

≤ ∈

≤Ω ∈Minimum energy problem with free terminal conditions but with bound constraints

Voltage Regulating System

Rf

Battery

Lf

if

e(t)

Ra

La

Ia (constant)

λ(t)

θ(t)

λL(t)

Viscous friction coefficient B


20

Example 7: (Mixture) Want to find inflow rate of fresh water (control) to the two-tank system so that the salt concentration in tanks A and B are equal using minimum amount of fresh water.

u(t)

50gal tank A

u(t) u(t)

50gal tank B

80lb salt at

t=0

Fresh water at t=0

1

2

( ) lbs of salt in A at time

Assume: 1) Well-mixed at all time

( ) lbs of salt in B at

2) I

time

ncompressible fluid States:

Co ( ) gal/m of fresh watentrols: r

x t tx t t

u t

===

passing thru at time t

0

1 1

2 1 2

1 2

1 2

min = ( )

:( ) ( )

50( ) ( ) ( ) ( )

50 5

Optimal Control Model:

Dynamics

Initial conditions: Terminal conditions:

Contrai

0(0) 80; (0) 0

( )- ( ) = 0

For cnts: ea

ft

f f

J u t dt

u tx x t

u t u tx x t x t

x xx t x t

= −

= −

= =

∫

max

1 2

h :On control: 0 ( ) , [0, ]

On state: 0 ( ) 80; 0 ( )

fu t u t t

x t x t

≤ ≤ ∈

≤ ≤ ≤

Minimum control effort withSoft terminal constraint and bound constraints

11


21

Optimizing Yeast or Ethanol Production in a Bioreactor

Bioreactors are large vessels that serve as an environment for biochemical reactions to occur. Typical uses include the growth of microorganisms and the breakdown of products.

The environment within the vessel is controlled to optimize performance. Typical control variables include nutrient feed rate, oxygen air flow rate, and temperature. There is large economic incentive to develop control strategies to maximize the production of baker’s yeast and ethanol, two important commercial products produced in bioreactors. Yeast is typically grown off a solution containing glucose and other nutrients essential for cellular growth. When glucose concentration in the medium is high or when there is a limited supply of oxygen, the yeast microorganism excretes ethanol.

Source: D. Moore, MS Thesis, EECS, CWRU, 2007


22

Optimizing Yeast or Ethanol Production in a Bioreactor

X = yeast concentration (g/l)S = substrate (glucose) concentration (g/l)E = ethanol concentration (g/l)O = dissolved oxygen (O2) concentration (g/l)C = dissolved carbon dioxide (CO2) concentration (g/l)V = liquid volume (l)Fin = Substrate feed rate (l/h)Sin = Influent Substrate Concentration (g/l)D = Fin/V = Dilution rate (1/h)OTR = kLa(OS – O) = O2 transfer rate (g/L h-1)CER = kVkLaC = CO2 evolution rate (g/L h-1)m = Maintenance term (g of S /g of X h-1)

Model of Growth Dynamics

7-Aug-09

We wish to maximize production of yeast, or ethanol or both by controlling the substrate feed rate, airflow (O2) and temperature

12


23

Two Ways to Solve Dynamic

Optimization/Optimal Control Problems

1. Indirect Method:

• Solve the necessary conditions for optimality derived through variationalprinciples rooted in Calculus of Variations

• That is: Solve two-point boundary value problems (TPBVP)


24

Two Ways to Solve DynamicOptimization/Optimal Control Problems

2. Direct Method:• Optimize the functional directly as

constrained optimization• Require conversion to nonlinear programs

through transcription of the ODEs(dynamics of system)

• Often possesses high sparsity and special structure

13


25

Direct Method for Dynamic Optimization

• Convert to nonlinear programs through direct transcription of the ODEs (dynamics of system) or the corresponding DAEs

• Use nonlinear optimizer such SQP to solve the resulting nonlinear programs (Software such as SNOPT by Boeing, etc.

• Fine-tune the result through mesh-refinement techniques


26

Direct Method: Transcription methodsDirect Method: Transcription methods

( )

1

1

1 2 3 4

1

2 1 1

( , , )

1 wh

Euler:

ere

2 26

( , , )1

Classical Runge-Katta:

( , ,2

k k k k k k

k k k k

k

k k k k

kk k k k

h th

h th

h t

+

+

+

= += +

= + + +

=

= + +

x x a x ux x s

s k k k k

k a x u

k a x k u

( )

3 2 1

4 3 1 1

1 1 1

1

Trapezoidal: Hermit-Simp

)2

1 ( , , )2 2

( , , )( , , ) ( ( , , ), , )

( , ,

son:

) 6

kk k k k

k k k k

k k k k k k k k k k k k k

kk k k k k

hh t

h th t h t t

ht

+

+ +

+ + +

+

= + +

= +

= + + +

= + +

k a x k u

k a x k ux x a x u a x a x u u

x x a x u a( )

( ) ( )

1 1 1

1 1 1

1 1 1

( ( , , ), , )

1 where ( , , ) ( , , ) ( ( , , ), , )2 8

and ( , , )2

k k k k k k k k

kk k k k k k k k k k k k k k k k k

kk k k k

h t t

hh t t h t t

ht

+ + +

+ + +

+ + +

+ +

= + + + − +

= +

x a x u u a

x x x a x u a x u a x a x u u

a a x u

14


27

Direct Method: Transcription methods

Band, Stair-case Structure and Sparsity of resulting matrix:

Employ special numerical tricks to take advantage of the special structure and sparsity of the resulting problem


28

Indirect Method: Derivation of Optimality Conditions

•• EulerEuler--Lagrange Equations and all Boundary Lagrange Equations and all Boundary ConditionsConditions

•• HamiltonHamilton--JacobiJacobi--Bellman ConditionsBellman Conditions•• PontryaginPontryagin’’ss Minimum PrincipleMinimum Principle

15


29

Indirect Method: Fundamentals

1 2 1 22

var

2

( ( )) : where [ , ] and :[ , ]

( *, ) ( *, ) ( )

where ( ) 0 as

Path optimization: Functional

0Op

Variations of a Functional:

n

increment iation error term

J x x x x x x R

J Jδ δ δ ο δ

ο δ δ

∈ →

Δ = +

→ →

y y

y y y y y

y y

0 0

2

var

Function ( ( )) or ( ( ), ) : where [ , ] and :[ , ]

( *, ) ( *, ) ( )

timal control:

( *, , ) ( *, ,

nf f f f f

increment iation error term

f f

increment

J t J t t t t t t t R

J J

J t J t

δ δ δ ο δ

δ δ δ δ δ

∈ →

Δ = +

Δ =

x x x

x x x x x

x x x x22

var

) ( , )f

iation error term

tο δ δ+ x


30

0

( )

( )(

1 21 2

)

1) Key variation formular: ( , )

( ( )) ( , ,.., , )

Then ( , ) ..

For example:

..

f

T

n

t nf t

nn

n

JJ

J J Jx

J t g x x x t dt

J J JJ x x

x x

x

x x x

xx x

δ δ δ

δ δ δ δ δ

δ δ δ

∂=∂

∂ ∂ ∂= + + +∂ ∂

=

∂ ∂ ∂= + + +∂ ∂ ∂

∂

∫x

x x

x x xx

0

0

00

( )( )

2) Term like are dealt with thru integrati

on by par

.

t

i.e.

.

f

ff

f

t

t

tt

tt

t nnt

g g g

g x dtx

g g gx dt

x x x dtx x

xx x t x

x

δ

δ δ δ

δ δ

∂ ∂ ∂⎛ ⎞= + + +⎜ ⎟∂ ∂ ∂⎝∂⎛ ⎞

⎜ ⎟∂⎝ ⎠

⎛∂ ∂ ∂ ∂⎛ ⎞ ⎛ ⎞ ⎛ ⎞= −⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜∂ ∂ ∂ ∂⎝ ⎠ ⎝ ⎠ ⎝⎝

⎠

⎠

∫

∫

∫0

ft

txdtδ⎞⎟

⎠∫


16


31

* is an extremal of ( ) only if ( *, ) 0 for all admissible3) Fundamental Theorem of Calculus of Variations:

4) Fundamental Lemmas of Calculus of Variation

a) Let ( ) [ , ].

s

.

:

J J

x C a bα

δ δ δ=

∈

x x x x x

a

1

b 1

a

b

b)

If ( ) ( )

Let (

0 for

) [ , ].

If ( ) ( ) 0 for all ( ) [ , ] with ( ) ( ) 0

then ( ) fo

all ( ) [ , ] with ( ) ( ) 0

then ( ) 0 for all ]

[ ,x C a b

x h x d

x h x dx h x C a b h a h b

x

x h x C a b h a h b

b

x c

x aα

α

α

α

α

= ∈ = =

=

∈

′ = ∈ = =

=

∈

∫

∫

( )

1

b 1

a

c) Let ( ) and (x) [ , ].

If ( ) ( ) ( ) ( ) 0 for all ( ) [ , ] with ( ) ( ) 0

then ( ) ( )

r all [

for all

, ]

[ , ]

x C a b

x h x x h x dx h x C a b h a h b

x x

x b

a b

a

x

α β

α β

β α

∈

′+ = ∈ = =

= ∈

∈

′∫



32

00( ) ( , , ) , :[ , ]

(fixed end point)

* is an extre

Now consider

a) Case: and ( ) are fixed

0 ( *,

mal of ( ) only if ( *, ) 0 for all admissibl

)

e .

ft

ft

f f

J x g x x t d

t x t

J JJ x x x

J x J x x

t R

x

x t t

xδ

δ

δ

δ

δ

δ∂ ∂

⇒

=

∂

→

= =∂

=

+

∫

x

0

00

for

=

all admissible

for all admissib

(integration by part)

le

ff

f

tt

tt

t

t

g

x xx

g gx x dt xx x

g x gx x dtx tx x

δ δ

δ δ δ

δδ δ⎛ ⎞∂ ∂ ∂⎛ ⎞ ⎛ ⎞+⎜ ⎟ ⎜ ⎟⎜ ⎟∂ ∂ ∂⎝

∂ ∂⎛ ⎞= +⎜ ⎟∂ ∂⎝ ⎠

∂⎝∂⎠ ⎠⎝ ⎠

−∫

∫

0

0

for all admissible

(since ( ) 0 and ( ) 0)

Fundamental Lemm

=

as of Calcu

0 (b s y lu o

ft

t

f

g x xx

x t x t

g

g dtt x

gt xx

δ δ

δ δ

⎛ ⎞∂ ∂⎛ ⎞⎜ ⎟⎜ ⎟∂ ∂⎝ ⎠⎝ ⎠

∂ ∂⎛

∂−

∂

= =

⇒∂ ⎞ =⎜ ⎟−

∂ ⎝∂ ∂ ⎠

∫

f Variations 4a)

This is Euler-Lagrange Equation

Indirect Method: Necessary Conditions

17


33

00For

with and ( ) fixed (fixed en

* is an ext

( ) ( , , ) , :

remal

d point)

0

[ , ]

Euler-L

of ( ) only if

agrange Equati

(secon

on

d-

Necessary Conditions:

f

f f

t

ft

t x t

gJ x

J x g x x t

gx t

t t t

x

d x R

∂ ∂ ∂⎛ ⎞− =⎜ ⎟∂ ∂ ∂⎝

= →

⎠

∫

x

0 0

order ODE)

with 2-boundary point ( ) and ( )

Solved by shooting method (for example)

f f

x t xx t x

=

=



34

( )

( )

( , )

2 2 2

0

1 2

0

with (0) 0, ( 2) 1 (fixed end point)

( ) ( ) ( ) , : [ , ]

Euler-Lagrange Equatio

Exam

2 2 2 2 0

( ) cos sinWi

ple:

(

n

th 0)

g x x

f

x x

g g x x x xx t x t

x t c t c tx

J x x t x t dt x t t Rπ

π= =

∂ ∂ ∂ ∂⎛ ⎞− = − − = + =⎜ ⎟∂ ∂ ∂

= − →

∂⎝ ⎠= +⇒

=

∫

0 and ( 2) 1 ( ) 0cos 1sin = sin

Solved numerically by the shooting method

xx t t t t

π⇒

== +


18


35

00( ) ( , , ) , :[ , ]

(free end point)

* is an extremal o

For

b) Case: is fixed and ( ) is free

f

0 ( *, )

( ) only if ( *, , ) 0 for all admissible ( . , )

ft

ft

f f

f f

J

t x t

JJ x x

J x g x x t dt x t t

x

x J x x

R

x xxδ δ δδ

δ δ

δ=

⇒ =∂

= →

∂=

∫

x

0

00

for all admissible

for all admissible

= (integration by part)

f

f

ft

t

tt

t

t

Jx x xx

g gx x dtx t x

g gx x dt xx x

g xx

δ δ δ

δ δ

δ δ

δ

δ ⎛ ⎞∂ ∂ ∂⎛ ⎞ ⎛ ⎞+⎜ ⎟ ⎜ ⎟⎜ ⎟∂ ∂ ∂⎝ ⎠

∂+∂

∂ ∂⎛ ⎞=

⎝ ⎠

+⎜ ⎟∂ ∂

⎠−

∂⎝

⎝ ⎠

∂

∫

∫

0

0

*,

for all admissible , and ( )

(since

0 (

( ) 0 and ( )

=

E

0)

( ) f

f

t

f tx

f

f

t

g gx t dtx t

g g

g x x x tx

x t x

x x

t

t

xδ δ δ

δ δ

δ ⎛ ⎞∂

∂ ∂ ∂⎛ ⎞− =⎜ ⎟∂

∂ ∂⎛ ⎞+ ⎜ ⎟⎜ ⎟∂ ∂ ∂⎝ ⎠

∂ ∂⎝⇒

∂

⎠−

⎝

≠

⎠

∂

=

∫

*,

0

uler-Lagrange)

0

( ) 0fx t

gx

x t

∂=

∂

=



36

00Now consider

and is free (variable end time, free terminal time or horizon)

a) Case: (

( , ) ( , , ) , : [ , ]

(free termina) is free and is indepe l condindent of

0 (

tions)

f

f

f

t

f ft

f

J x t g x x t dt

t

x t t

x t t R

J xδ⇒ =

= →∫

0

*,

*, , , ) for all admissible ( , , )

for all admissible , ( , , ) ( *, *, )

=

)

an

(

df f f

f

f

f

t t t

t t

T

fx t

f f f

f fg x x

x x t x x t

x x t

g

t dt g x x t dt

g gxx

xx

tt x

δδ

δ δ δ δ δ δ

δ δ δ

δ δ

++

⎛ ⎞∂ ∂ ∂⎛ ⎞+ ⎜ ⎟⎜ ⎟∂ ∂ ∂⎝ ⎠⎝

=

∂−

∂ ⎠

∫ ∫

( )

0

0*,

*,

(since ( ) - * ( ) )

( *, *, )

= - * ( ) ( *, *, )

=

f

f

f

f

f f

t

f

f

ft

T t

f

f

f f f ftx t

T

fx t

dt g x x t t

g gx x t t dt g x x t tx t x

x t

g xx

gxx

x t

g x

x tδ δ

δ

δ

δ

δ

δδ δ

+

⎛ ⎞∂ ∂ ∂⎛ ⎞+ +⎜ ⎟⎜ ⎟∂ ∂ ∂⎝ ⎠⎝

=

⎠

∂

∂−

∂

∂∂

+∂

∫

∫

0*,

*,

for all admissibl

0 (Euler-Lagr

(

e , an

*, *, ) * ( )

( *, *, )

ange)

0,

d

f

f

f

Tt

f f ftx t

T

f

f

x

f

x t

x

x

g gdt g x x

g gx t

t x t tt x x

gg xgx

x t

x

x

t

x

δ

δ δ

δ

δ

⎛ ⎞⎛ ⎞∂ ∂ ∂⎛ ⎞ ⎜ ⎟

∂ ∂ ∂⎛ ⎞− =⎜ ⎟∂ ∂ ∂⎝ ⎠

∂=

+ −⎜ ⎟⎜ ⎟ ⎜ ⎟∂ ∂ ∂⎝ ⎠⎝ ⎠ ⎝ ⎠

∂−

∂∂

−

⇒

∫

*,0* ( ) = 0, and ( ) 0

f

ft

x tx t =


19


37

00Now consider

and is free (variable end time, free terminal time or horizo

( , ) ( , , ) ,

n)

b) Cas

0 (Eu

e:

: [ , ]

(Hard terminal co( ) =

l

ntraint)

er-Lagra

f

f

f

f

f

t

ft

g

J x t g x x t dt x t t R

t

x t x

gx t x∂ ∂ ∂⎛ ⎞− =⎜ ⎟∂ ∂ ∂

→

⎝ ⎠

=

⇒

∫

*,

*

0

,

( *, *, ) * ( ) = 0, and

(Soft terminal contraint

( ) =

c) Case: ( ) =

nge)

, ( ) 0

0 (Euler-Lagrange)

) -- ( )

( ) (

( )

*,

f

f

T

f fx t

f f f

T

fx

f f

f f

t

gg x x t x tx

x t t

g t g

x t

g gx t

x t x

x t t

xx

x

θ δ θ δ

θ

∂−

∂

=

=

∂ ∂ ∂⎛ ⎞− =⎜ ⎟∂ ∂⇒

∂

∂+

⎠

∂

⎝

*

0

,

*, ) * ( ) = 0,

and ( ) = ( )( ) 0,f

f

f fx t

f

T

x t x t t

gx t x tx

θ=

∂−

∂



38

00Now consider

and is free (variable end time, free terminal time or horizon)

c) Case: ( ) = ( )

0 ( *, , , )

( , ) ( , , ) , : [ , ]

(Soft terminal contraint

for a

)

ll a

f

f

f f

t

t

f f

f fJ x t g x

t

x

x

t t

J x x x t

t dt x t t R

θ

δ δ δ δ⇒ =

= →∫

0

0*,

( *, *, ) ( *, *, )

dmissible

for all admissible

= ( ) ( *, *, )

=

f f f

f

f

f

t t t

t t

T t

f f ftx t

T

g x x t dt g x x t dt

g gx t dt g x x t tx t x

g

x

x

g xx

x

δ

δ

δδ

δδ δ

++

⎛ ⎞∂ ∂ ∂⎛ ⎞+ +⎜ ⎟⎜ ⎟∂ ∂ ∂⎝ ⎠⎝

∂

=

∂−

∂ ⎠

∂

∫ ∫

∫

( )0

0

*,

*,

(since ( ) - * ( ) )

- * ( ) ( *, *, )

= ( *, *, )

f

f

f

f

t

f f f f ftx t

T t

f ftx t

f f f fx t

g xx

gx x t t dt g x x t tt x

g gx d

x x t t

t g x x tg xxx t x

δ δ δ

δ

δ

δ δ δ

δ

∂−

∂

∂

⎛ ⎞∂ ∂⎛ ⎞+ +⎜ ⎟⎜ ⎟∂ ∂⎝ ⎠⎝ ⎠

⎛ ⎞∂ ∂ ∂ ∂⎛ ⎞+ + −⎜ ⎟⎜ ⎟∂ ∂ ∂⎝ ⎠⎝ ⎠−

∂

=

∫

∫*,

*,0

*,

for all admissible , an

0 (Euler-Lagrange)

0

* ( )

( *, *, ) * ( ) =, ( 0, and

0

d

)

f

ff

T

f fx t

T

fx t

fx t

f f

g gx t x

g x

g x t tx

gg

x x

x

t

xt x t

xtx

δ δ

δ

δ

∂ ∂ ∂⎛ ⎞− =⎜ ⎟∂ ∂

⎛ ⎞⎜ ⎟⎜ ⎟∂⎝ ⎠

∂−

∂

∂⎝ ⎠

∂

⇒

∂= =

Example Derivation of Case (c)

20


39

Weistrass Erdman Corner Co1) Piecewise Continuous Solution: Require additional or 2) Con

Further

straint

nditions

s onTranve

( ) rrsatility

equire the

extensions

use of mulConditions

tipliers M

:

tx

ost important cases are in optimal control problems:



40

0

0 0

( , , ) ( ( ), ) ( , , )

( , , )Boundary condi

Optimal Control Problems:

Optimize

I) is fixed and (

tions: ( )

) is fixeCas :

ds

II

e

ft

f f t

f f

fJ t h t t g t dt

t

t t

t

= +

==

∫x u x x u

s.t. x a x ux x

x

) is fixed and ( ) is free

III) is fixed, and m( ( )) 0

IV) is free and ( ) is free

V) is free and ( ) is fixed

VI) is free and ( ) = ( )

VII) is free,

f f

f f

f f

f f

f f f

f

t t

t t

t t

t t

t t t

t

=

x

x

x

x

x θ

and m( ( )) 0

VIII) is free, and m( ( ), ) 0f

f f f

t

t t t

=

=

x

x


21


41


0

0

0 0

0

( , , ) ( ( ), ) ( , , )

( , , )Boundary conditions: ( )

( ( ), ) ( ( ),Case IV:

Optimal Co

)

ntrol Problems:

Optimize

( ( ),

f

f

t

f f f t

t

f f t

J t h t t g t dt

tt

h t t h t t dt h t

= +

==

= +

∫

∫

x u x x u

s.t. x a x ux x

x x x

( )

0

0

a

( , , , )

) ignored (constant)

( ( ), ) ( ( ), ) =

( ( g ( , , , , ) ( , , ) ( , , )

fT

t

t

T

H t Hamiltonian

t

h t t h t t dtt

ht g t t−−

− −

⎛ ⎞∂ ∂⎛ ⎞ +⎜ ⎟⎜ ⎟⎜ ⎟∂ ∂⎝ ⎠⎝ ⎠

∂= + + − +

∫

x u p

x xxx

xx x u p x u p a x u x

f

0

0a

t

( *, , *, , *, , , )

* *

), ) ( ( ), )

( , , , , ) g ( , , , , )

=

( *, ) + ( ) * ( *, *,

f

a f f

t

Tf T

f f

T

t

a f t

f

J t

H H dt

h t

t t h t tt

J t t d

g t

t

t

δ δ δ δ δ δ

δ δ

δ

⎛ ⎞∂ ∂⎛ ⎞+ +⎜ ⎟⎜ ⎟∂ ∂⎝ ⎠⎝ ⎠

∂⎛ ⎞− +

∂⎛ ⎞⎜ ⎟

+⎜ ⎟∂⎝

∂ ∂⎝ ⎠

⇒ =

⇒

⎠

∫

∫x x u u p p x

u p xu x

xp x p a x u

x

xxx

x x u p x x u p

( * ( ),)

= 0 for all admissible , , ,

f ff

f f

h t tt

t

δ

δ δ δ δ

∂⎛ ⎞+⎜ ⎟∂⎝ ⎠

xx

x u x


42

free and ( ) free

Hamilto

Necessary Conditions fo

nian-Jacobi Conditions:

r Optimal Control Problems:Case IV: Free-end point problems:

*, *)is extremal, then there exists co-states *such that:

(

f ft t

⇒

x

x u p* = 0 (1)

* (2)

( , , )

H

t

H

∂∂

∂=

∂=

−

u

px

x a x u

0 0

( *, ) ( *( ),

OTHER CASES CAN BE SIMILARILY

(3)

+ (4)

and ( )DERIVED

( ) * = 0

f f ff f f

h t h t t

t

t H tδ δ∂ ∂⎛ ⎞ ⎛ ⎞

− +⎜ ⎟ ⎜ ⎟∂ ∂⎝ ⎝=

⎠ ⎠

x xp x

xx x

x


Documents

Numerical Optimizationoptimal control problem derivation of similar results based on the concept of Lyapunov stability will also be demonstrated. 2. Focus on numerical methods for