Introduction to Calculus of Variations

Contents

1 Euler-Lagrange Equation 2

1.1 Functionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Euler-Lagrange Equation . . . . . . . . . . . . . . . . . . . . . 4

1.2.1 Transversality conditions . . . . . . . . . . . . . . . . . 61.2.2 Functionals involving several independent variables . . 9

1.3 Constrained minimization of functionals . . . . . . . . . . . . 111.3.1 Point constraints . . . . . . . . . . . . . . . . . . . . . 111.3.2 Differential equation constraints . . . . . . . . . . . . . 12

2 Variational approach for optimal control 16

2.1 Necessary conditions for optimal control . . . . . . . . . . . . 162.1.1 Problems with fixed final time . . . . . . . . . . . . . . 182.1.2 Problems with free final time . . . . . . . . . . . . . . 19

1

1Euler-Lagrange Equation

1.1 Functionals

The calculus of variations deals with minimization or maximization of func-tions of functions, or functionals. A functional is a mapping or a transfor-mation that depends on one or more functions and the values of functionalsare numbers.

Example 1.1.1 Suppose x(t) is a continuous function of t defined in theinterval [t0, tf ] and:

J(x) =∫ tf

t0x(t)dt (1.1)

The real number assigned by the functional J is the area under the x(t) curve.

t

x(t)

t0 tf

t

x(t)

t0 tf

x1(t)

x2(t)

Figure 1.1: Area under a curve

As shown in Figure 1.1, for various functions x(t), the functional hasdifferent real values.

2

Example 1.1.2 The length l, of an arc connecting two given points A(x0, y0),B(x1, y1) in the x−y plane as depicted in Figure 1.2 is an example of a func-tional. We know that the length can be calculated as, (***, 2001):

y

x

B(x 1 ,y 1 )

A(x 0 ,y 0 )

dx dy(x) dl

Figure 1.2: An arc connecting two points

l = l(y(x)) =∫ x1

x0

dl(x) =∫ x1

x0

√

dx2 + dy(x)2 =∫ x1

x0

√

√

√

√1 +

(

dy(x)

dx

)2

dx

subject to the constraints:

y(x0) = y0, y(x1) = y1 (1.2)

Example 1.1.3 The Brachistochrone problem. The problem originallygave rise to the calculus of variations (Johann Bernoulli posed it in 1696).The term derives from the Greek brachistos ”the shortest” and chronos”time, delay”.

We wish to find the curve that minimizes the time necessary for a beadattached to a wire to go from (0,y1) to (x1,0) under the influence of a constantgravitational field

y

x

gravity (0,y 1 )

(x 1 , 0 )

Figure 1.3: Brachistochrone problem

The time to travel from point (0,y1) to (x1,0) is:

T =∫ T

0

dt (1.3)

3

If the length of the curve is l, and ds is an infinitely small part of the curve,we obtain:

T =∫ T

0

dt =∫ l

0

ds

v(1.4)

where v is the speed of the bead. Since ds =√dx2 + dy2, the equation (1.4)

is written as:

T =∫ x1

0

√dx2 + dy2

v=∫ x1

0

1

v

√

√

√

√1 +

(

dy

dx

)2

dx =∫ x1

0

1

v

√

1 + y′2dx (1.5)

The speed at any point can be obtain from energy conservation, equatingkinetic energy to gravitational potential energy:

1

2mv2 = 2gy, ⇒ v =

√

2gy (1.6)

The functional to be minimized then gives:

T =∫ x1

0

√

1 + y′2

2gydx =

∫ x1

0

L(y, y′)dx (1.7)

where y = y(x).

1.2 Euler-Lagrange Equation

Problem: Suppose we are given two points (t0, x0) and (tf , xf) in the (t, x)plane. We wish to find a curve, trajectory, joining the given points such thatthe functional:

J =∫ tf

t0g(t, x, x)dt

along this trajectory can achieve its extremal, that is maximal or minimalvalue, (Kirk, 2004).

A fundamental equation of calculus of variations states that J has astationary value if the Euler-Lagrange equation :

∂g

∂x− d

dt

(

∂g

∂x

)

= 0 (1.8)

is satisfied.The curves x = x(t) that are solutions to the Euler-Lagrange equation

are called extremals. We shall write the above equation also as:

gx −d

dtgx = 0

4

x(t)

t

x=x(t)

x*=x*(t)

t f t 0

x 0

x f

Figure 1.4: Trajectories between two points

Example 1.2.1 Find the extremal for the functional:

J =∫ π/2

0

(x(t)2 − x(t)2)dt =∫ π/2

0

(x2 − x2)dt

where x(0) = 0, x(π/2) = 1. The Euler-Lagrange equation for the problemtakes the form:

gx −d

dtgx =

∂

∂x(x2 − x2)− d

dt

(

∂

∂x(x2 − x2)

)

= 0

−2x− d

dt(2x) = −2x− 2x = 0

orx+ x = 0

Obs. ax+ bx+ cx = 0 has the characteristic equation as2+ bs+ c = 0. If theroots of the characteristic equation are real: s1,2 ∈ ℜ, the general solution is:x(t) = c1e

s1t + c2es2t. If the roots of the characteristic equation are complex

s1,2 = α± jβ, the general solution is: x(t) = c1eαt cos βt+ c2e

αt sin βtThe characteristic equation s2 + 1 = 0 has two complex roots s1,2 = ±j

and the solution is:x(t) = c1 cos t + c2 sin t

Taking into account the end-point conditions:

x(0) = c1 cos 0 + c2 sin 0 = c1 = 0

x(π/2) = c1 cosπ/2 + c2 sin π/2 = c2 = 1

Thus we have the extremal x(t) = sin t. The functional J can be minimizedor maximized only on the extremal x(t) = sin t. The extremal value of thefunctional is: J∗(sin t) = 1− π/4 = 0.214.

5

Example 1.2.2 Find the extremal for the functional

J =∫

1

0

(x(t)2 + 12tx(t))dt

where x(0) = 0, x(1) = 1.The Euler-Lagrange equation for this functional is:

gx −d

dtgx =

∂

∂x(x2 + 12tx)− d

dt

(

∂

∂x(x2 + 12tx)

)

= 0

12t− d

dt(2x) = 12t− 2x = 0

or

x(t) = 6t, x(t) = 6t2

2+ c1

x(t) =3t3

3+ c1t+ c2 = t3 + c1t + c2

x(0) = c2 = 0, x(1) = 1 + c1 + c2 = 1 ⇒ c1 = 0

The extremal has the form: x(t) = t3.

1.2.1 Transversality conditions

Final time specified, x(tf ) free

Problem: Find a necessary condition for a function to be extremal for afunctional

J =∫ tf

t0g(x(t), x(t), t)dt

where t0, x(t0) and tf are specified and x(tf ) is free.The asmissible curves all begin at the same point and terminate on a

vertical line as for example in the case presented in Figure 1.5. We shall call

x(t)

t

x=x(t)

x*=x*(t)

t f t 0

x 0

Figure 1.5: Free final state

the necessary condition:

6

∂

∂xg(x, x, t)|t=tf = 0 (1.9)

the natural boundary condition or the first transversality condition.

Example 1.2.3 Determine the extremal for the functional

J =∫

2

0

[x(t)2 + 2x(t)x(t) + 4x(t)2]dt

where x(0) = 1, x(2) is free.The Euler-Lagrange equation:

gx −d

dtgx =

∂

∂x(x2 + 2xx+ 4x2)− d

dt

(

∂

∂x(x2 + 2xx+ 4x2)

)

= 0

2x+ 8x− d

dt(2x+ 2x) = 2x+ 8x− 2x− 2x = 0

or x− 4x = 0.The characteristic equation is s2 − 4 = 0 with the solutions: s1,2 = ±2.

The general solution is:

x(t) = c1e2t + c2e

−2t

Evaluating the transversality condition (1.9) with t = 2:

∂g

∂x|t=2 = (2x(t) + 2x(t))t=2 = x(2) + x(2) = 0

wherex(t) = c1e

2t + c2e−2t, x(t) = 2c1e

2t − 2c2e−2t

We obtain:

2c1e4 − 2c2e

−4 + c1e4 + c2e

−4 = 0, 3c1e4 − c2e

−4 = 0

From x(0) = 1:c1e

2·0 + c2e−2·0 = 1, ⇒ c1 + c2 = 1

The constants c1, c2 result:

c1 =1

3e4 + e−4, c2 =

3e4 + e−4 − 1

3e4 + e−4

Exercise 1.1 . Determine the smooth curve of smallest length connectingthe point x(0) = 1 to the line t = 5, (Figure 1.6).

Hint. Find the extremal for the functional:

J(x) =∫ tf

t0

√

√

√

√1 +

(

dx

dt

)2

dt =∫

5

0

√

1 + x(t)dt

—————–

7

x(t)

t

x=x(t)

5 0

1

Figure 1.6: The curve connecting a point to a line

Final time free, x(tf ) specified

Problem: Find a necessary condition for a function to be extremal for afunctional

J =∫ tf

t0g(x(t), x(t), t)dt

where t0, x(t0) = x0 and x(tf ) are specified and tf is free.The second transversality condition that is the necessary condition needed

is:

(

g − x∂g

∂x

)

|t=tf = 0, or (g − xgx) |t=tf = 0 (1.10)

Example 1.2.4 Find an extremal for the functional:

J =∫ tf

1

(2x(t) +1

2x(t)2)dt

with x(1) = 4, x(tf ) = 4 and tf > 1 free.

gx −d

dtgx = 2− d

dt

(

∂

∂x(2x− 1

2x2)

)

= 2− x = 0

orx = 2

with the solution x(t) = 2t+ c1, x(t) = t2 + c1t+ c2.The final time tf is unspecified so, from (1.10):

(g − xgx) |t=tf = (2x(t) +1

2x(t)2 − x(t) · x(t))t=tf = 0

2x(tf )−1

2x(tf )

2 = 0 ⇒ 2(t2f + c1tf + c2)−1

2(2tf + c1) = 0

8

From4t2f + 4c1tf + 4c2 − 2tf − c1 = 0

together with the boundary conditions:

x(1) = 1 + c1 + c2 = 4, x(tf ) = t2f + c1tf + c2 = 4

the constants and the final time can be calculated and the extremal result as:

x(t) = t2 − 6t + 9, and tf = 5

1.2.2 Functionals involving several independent vari-

ables

In the n+1 dimensional space: (t, x1, x2, ..., xn) the functional

J =∫ tf

t0g(t, x1, x2, ..., xn, x1, x2, ..., xn)dt

can only be extremized on the trajectories that satisfy the n Euler-Lagrangeequations:

∂g

∂xi− d

dt

(

∂g

∂xi

)

= 0, or gxi− d

dtgxi

= 0, i = 1, ..., n

Using matrix-vector notation the functional, the Euler-Lagrange equationand the transversality conditions are stated as:

J =∫ tf

t0g(x(t), x(t), t)dt (1.11)

∂g

∂x− d

dt

(

∂g

∂x

)

= 0 (1.12)

∂g

∂x|t=tf = 0 (1.13)

g −[

∂g

∂x

]T

x

t=tf

= 0 (1.14)

where:

x = x(t) =

x1(t)...

xn(t)

, x = x(t) =

ddtx1(t)...

ddtxn(t)

9

Example 1.2.5 Find an extremal for the functional

J(x =∫ π/4

0

[x2

1(t) + x1(t)x2(t) + x2

2(t)]dt

The functions x1 and x2 are independent and the boundary conditions are:

x1(0) = 1; x1(π/4) = 2; x2(0) = 3/2; x2(π/4) free

The Euler-Lagrange equations are:

gx1− d

dtgx1

= 2x1 −d

dt(x2) = 2x1 − x2 = 0 (1.15)

gx2− d

dtgx2

= − d

dt(x1 + 2x2) = −x1 − 2x2 = 0 (1.16)

From (1.15): x2(t) = 2x1(t). By replacing in (1.16) we obtain:

x1 + 4x1 = 0

which has the solution:

x1(t) = c1 cos 2t+ c2 sin 2t

thereforex2(t) = 2c1 cos 2t+ 2c2 sin 2t

Integrating twice yields:

x2(t) = −c12cos 2t− c2

2sin 2t+ c3t+ c4

From the first transversality condition (tf is specified and x2(tf) is free:(

∂g

∂x2

)

t=π/4

= 0

gx2= x1 + 2x2 = 2c3 = 0, ⇒ c3 = 0

From the specified boundary conditions we have:

x1(0) = c1 · 1 + c2 · 0 = 1 ⇒ c1 = 1

x2(0) = −c12· 1− c2

2· 0 + c3 · 0 + c4 =

3

2, ⇒ c4 = 1.5 +

c12

= 2

x1(π/4) = c1 · 0 + c2 · 1 = 2, ⇒ c2 = 2

The extremals are then:

x1(t) = cos 2t+ 2 sin 2t, x2(t) = −1

2cos 2t− sin 2t+ 2

10

1.3 Constrained minimization of functionals

1.3.1 Point constraints

Let us determine a set of necessary conditions for a function x∗(t) to be anextremal for a functional of the form:

J =∫ tf

t0g(x(t), x(t), t)dt (1.17)

where x(t) is a n × 1 vector of functions, that is required to satisfy m rela-tionships of the form:

fi(x(t), t) = 0, i = 1, 2, ..., m < n (1.18)

which are called point constraints. Constraints of this type would be presentif, for example, the admissible trajectories were required to lie on a specifiedsurface in the n + 1 dimensional (x(t), t) space. The presence of these mconstraining relations means that only n −m of the n components of x areindependent.

One way to attack this problem might be to solve equations (1.18) form of the components of x(t) in terms of the remaining n−m components -which can the be regarded as n −m independent functions - and use theseequations to eliminate the m dependent components of x(t) and x(t) fromJ .

Unfortunately, the constraining equations are generally nonlinear alge-braic equations, which may be quite difficult to solve.

As an alternative approach we can use Lagrange multipliers. The first stepis to form an augmented functional by adjoining the constraining relationsto J , which yields:

Ja(x, λ) =∫ tf

t0[g(x(t), x(t), t) + λ1(t)f1(x(t), t) + .... + λm(t)fm(x(t), t)] dt

=∫ tf

t0

[

g(x(t), x(t), t) + λT (t)f(x(t), t)]

dt (1.19)

Since the constraints must be satisfied for all t ∈ [t0, tf ], the Lagrangemultipliers λ1, ..., λm are assumed to be functions of time. This allows us theflexibility of multiplying the constraining relations by a different real numberfor each value of t.

If we define the augmented integrand function as:

ga(x(t), x(t), λ(t), t) = g(x(t), x(t), t) + λT (t)f(x(t), t)

11

The Euler-Lagrance equation is written:

∂ga∂x

(x, x, λ, t)− d

dt

(

∂ga∂x

(x, x, λ, t)

)

= 0 (1.20)

Equations (1.20) are a set of n second-order differential equations, and theconstraining relations (1.18) are a set of m algebraic equations. Togetherthese n+m equations constitute a set of necessary conditions for x(t) to bean extremal.

Example 1.3.1 Find necessary conditions that must be satisfied by the curveof smallest length which lies on the sphere x2

1(t) + x2

2(t) + t2 = R2, for t ∈

[t0, tf ], and joins the specified points x0, t0 and xf , tf .The functional to be minimized is:

J(x) =∫ tf

t0

√

1 + x21(t) + x2

2(t)dt

The augmented integrand function is:

ga(x, x, λ, t) =√

1 + x21(t) + x2

2(t) + λ(t)(x2

1(t) + x2

2(t) + t2 − R2)

Performing the operations indicated by (1.20) gives:

∂ga∂x1

− d

dt

(

∂ga∂x1

)

= 2x1λ− d

dt[x1(1 + x2

1+ x2

2)−1/2] = 0

∂ga∂x2

− d

dt

(

∂ga∂x2

)

= 2x2λ− d

dt[x2(1 + x2

1+ x2

2)−1/2] = 0

In addition, of course, it is necessary that the constraint relation:

x2

1(t) + x2

2(t) + t2 = R2

is satisfied.

1.3.2 Differential equation constraints

Let us now find necessary conditions for a function x(t) to be an extremalfor the functional

J(x) =∫ tf

t0g(x(t), x(t), t)dt

x is a (n + m) × 1 vector of functions which must satisfy the n differentialequations:

fi(x(t), x(t), t) = 0, i = 1, 2, ..., n

12

Because of the n differential equations constraints, only m components of xare independent. Constraints of this type may represent the state equationconstraints in optimal control problems where x corresponds to the n + mvector [x|u]T .

As with point constraints, it is generally not feasible to eliminate n de-pendent functions and their derivatives from the functional J so we shall usethe method of Lagrange multipliers.

If we define the augmented integrand fuction as:

ga(x(t), x(t), λ(t), t) = g(x(t), x(t), t) + λ(t)T f(x(t), x(t), t)

where λ(t) is a vector of n functions of time (Lagrange multipliers).The Euler- Lagrange equation can be written:

∂ga∂x

(x(t), x(t), t)− d

dt

[

∂ga∂x

(x(t), x(t), t)

]

= 0 (1.21)

and the constraints can be written in the vector form:

f(x(t), x(t), t) = 0 (1.22)

Equations (1.21) and (1.22) compose a set of 2n+m second-order differentialequations.

Example 1.3.2 Find the equations that must be satisfied by an extremal forthe functional

J(x) =∫ tf

t0

1

2[x2

1(t) + x2

2(t)]dt

where x1(t) and x2(t) are related by:

x1(t) = x2(t)

There is one constraint, so the function f in (1.22) is:

f(x(t), x(t), t) = x2(t)− x1(t)

and one Lagrange multiplier is required.The augmented function ga is:

ga(x(t), x(t), t) =1

2[x2

1(t) + x2

2(t)] + λ(x2(t)− x1(t))

From equation (1.21) we have:

∂ga∂x1

− d

dt

[

∂ga∂x1

]

= x1(t)−d

dt(λ(t)) = x1(t) + λ(t) = 0 (1.23)

∂ga∂x2

− d

dt

[

∂ga∂x2

]

= x2(t) + λ(t) = 0 (1.24)

13

We need also:x1(t) = x2(t) (1.25)

Equations (1.23),(1.24), (1.25) are necessary conditions for x∗(t) = [x1(t) x2(t)]T

to be an extremal for the functional J .The solution of system (1.23, 1.24, 1.25) can be found as follows: from

(1.24) we have λ(t) = −x2(t) which, if differentiated with respect to t andreplaced in (1.23) gives: x2(t) = x1(t), or (using (1.25)) x2(t) = x2(t).

The solution result as:

x1(t) = C1 · et − C2 · e−t, x2(t) = C1 · et + C2 · e−t (1.26)

For a set of initial conditions x1(0) = x10, x2(0) = x20, the constants C1

and C2 can be calculated and the solution result as:

x1(t) =x10 + x20

2·et− x20 − x10

2·e−t, x2(t) =

x10 + x20

2·et+ x20 − x10

2·e−t

(1.27)

Example 1.3.3 Suppose that the system

x1(t) = x2(t)− x1(t)

x2(t) = −2x1(t)− 3x2(t) + u(t)

is to be controlled to minimize the performance measure

J(x,u) =∫ tf

t0

1

2[x2

1(t) + x2

2(t) + u2(t)]dt

Find a set of necessary conditions for optimal control.The differential equation constraints are then:

f1(x, u) = x2(t)− x1(t)− x1(t)

f2(x, u) = −2x1(t)− 3x2(t) + u(t)− x2(t)

The augmented function ga:

ga(x, u, λ) =1

2[x2

1(t) + x2

2(t) + u2(t)] + λ1(x2(t)− x1(t)− x1(t)) +

+λ2(−2x1(t)− 3x2(t) + u(t)− x2(t))

The Euler equations:

∂ga∂x1

− d

dt

[

∂ga∂x1

]

= x1 − λ1 − 2λ2 + λ1 = 0

∂ga∂x2

− d

dt

[

∂ga∂x2

]

= x2 + λ1 − 3λ2 + λ2 = 0

∂ga∂u

− d

dt

[

∂ga∂u

]

= u+ λ2 = 0

14

We obtain the differential equations:

λ1 = −x1 + λ1 + 2λ2

λ2 = −x2 − λ1 + 3λ2

and the algebraic equation (since u does not appear in ga)

u+ λ2 = 0

The two additional equations that must be satisfied by an extremal are theconstraints:

x1 = x2 − x1

x2 = −2x1 − 3x2 + u

15

2Variational approach for optimal control

2.1 Necessary conditions for optimal control

In this section we shall apply variational methods to optimal control prob-lems.

We consider the problem of minimizing the performance measure

J = h(x(tf ), tf) +∫ tf

t0g(x(t),u(t), t)dt (2.1)

subject to:x = f(x(t),u(t), t), x(t0) = x0 (2.2)

The problem is to find an admissible control u∗(t) that causes the system(2.2) to follow an admissible trajectory x∗(t) that minimizes the performancemeasure (2.1).

We shall initially assume that the admissible state and control are notbounded and that the initial conditions x(t0) = x0 and the initial time t0are specified. x(t) is the n × 1 state vector and u(t) is the m × 1 vector ofcontrol inputs.

In the terminology of the previous section (Euler-Lagrange equation withdifferential equation constraints) we have a problem involving n+m functionswhich must satisfy the n differential equation constraints. The m controlinputs are the independent functions. The only difference between (2.1) andthe functionals considered in the previous section is the term involving thefinal states and final time. Assuming that h is a differentiable function wecan write:

h(x(tf ), tf ) =∫ tf

t0

d

dt[h(x(t), t)]dt+ h(x(t0), t0)

16

so that the performance measure can be expressed as:

J =∫ tf

t0

[

g(x(t),u(t), t) +d

dt(h(x(t), t))

]

dt+ h(x(t0), t0)

Since x0 and t0 are fixed, the minimization does not affect the h(x(t0), t0)term, so we need consider only the functional:

J(x,u) =∫ tf

t0

[

g(x(t),u(t), t) +d

dt(h(x(t), t))

]

dt

Using the chain rule of differentiation, we find that this becomes:

J(x,u) =∫ tf

t0

g(x(t),u(t), t) +

[

∂h

∂x(x(t), t)

]T

x(t) +∂h

∂t(x(t), t)

dt

Obs. The derivative of h(x(t), t) with respect to time was calculated:

d

dth(x(t), t) =

∂h

∂x

dx

dt+

∂h

∂t=

∂h

∂xx +

∂h

∂t

To include the differential equation constraints we form the augmentedfunction ga by introducing the Lagrange multipliers λT (t) = [λ1(t) ... λn(t)]

T :

ga(x,u, λ, t) = g(x,u, t) +

[

∂h

∂x(x, t)

]T

x+∂h

∂t(x, t) + λT (f(x,u, t)− x)

The Euler-Lagrange equations are:

∂ga∂x

− d

dt

(

∂ga∂x

)

= 0

∂ga∂u

− d

dt

(

∂ga∂u

)

= 0

∂ga∂x

− d

dt

(

∂ga∂x

)

=∂

∂x

[

g +∂h

∂xx+

∂h

∂t+ λT (f − x)

]

− d

dt(∂h

∂x− λ) =

=∂

∂x(g + λTf) + λ+

∂

∂x

(

∂h

∂xx+

∂h

∂t

)

− d

dt

(

∂h

∂x

)

=

=∂

∂x(g + λTf) + λ+

∂2h

∂x2x+

∂2h

∂x∂t− ∂2h

∂x2

∂x

∂t− ∂2h

∂x∂t

=∂

∂x(g + λTf) + λ = 0 (2.3)

17

∂ga∂u

− d

dt

(

∂ga∂u

)

=∂ga∂u

=∂

∂u(g + λTf) (2.4)

In the following we shall find it convenient to use the function H calledthe hamiltonian, defined as:

H(x(t),u(t), λ(t), t) = g(x(t),u(t), t) + λT (t)f(x(t),u(t), t) = g + λTf

Using this notation, from (2.3) and (2.4) we can write the necessary condi-tions as follows:

λ = −∂H

∂x(x(t),u(t), λ(t), t) (2.5)

0 =∂H

∂u(x(t),u(t), λ(t), t) (2.6)

together with the equations (2.2), which are constraints for the optimal con-trol problem:

x = f(x(t),u(t), t)

The differential equation (2.5) is called the costate equation and λ(t) isthe costate vector.

2.1.1 Problems with fixed final time

If the final time tf is specified, the final state x(tf) may be free or specified.

Case I Final state specified.x(tf) = xf

are n relations which are boundary conditions for differential equations(2.5) and (2.2). Together with the initial conditions x(t0) = x0 willensure a unique solution of the system of differential equations

Case II Final state free. The final condition can be calculated from the firsttransversality condition:

(

∂ga∂x

(x(t),u(t), λ(t), t)

)

t=tf

=∂h

∂x(x(tf ), tf)− λ(tf ) = 0

Thus:

λ(tf) =∂h

∂x(x(tf), tf) (2.7)

18

2.1.2 Problems with free final time

Case I Final state x(tf ) = xf fixed, tf free. The second transversalitycondition calculated for ga is:

(

ga − x∂ga∂x

)

t=tf

= 0

[

g +∂h

∂xx+

∂h

∂t+ λT (f − x)− x

(

∂h

∂t− λ

)]

t=tf

=

(

g + λTf +∂h

∂t

)

t=tf

= 0

The 2n+ 1 relationship is:

H(x(tf),u(tf), λ(tf ), tf) +∂h

∂t(x(tf), tf ) = 0

Case II Final state free. The two boundary conditions are derived from thetwo transversality conditions:

λ(tf ) =∂h

∂x(x(tf), tf )

H(x(tf),u(tf), λ(tf), tf ) +∂h

∂t(x(tf), tf ) = 0

Example 2.1.1 The system

x1(t) = x2(t)

x2(t) = −x2(t) + u(t)

is to be controlled so that its control effort is conserved; that is the perfor-mance measure

J(u) =∫ tf

t0

1

2u2(t)dt

is to be minimized. The admissible states and controls are not bounded. Findnecessary conditions that must be satisfied for optimal control.

The first step is to form the hamiltonian

H(x, u, λ) =1

2u2(t) + λ1(t)x2(t) + λ2(t)(−x2(t) + u(t))

From equations (2.5) and (2.6) the necessary conditions for optimality are:

λ1(t) = −∂H

∂x1

= 0

λ2(t) = −∂H

∂x2

= −λ1(t) + λ2(t) (2.8)

19

and

0 =∂H

∂u= u(t) + λ2(t) (2.9)

If (2.9) is solved for u∗(t) and substituted into the state equations, we have

u∗(t) = −λ2(t)

x1(t) = x2(t)

x2(t) = −x2(t)− λ2(t) (2.10)

Equations (2.10) and (2.8) are a set of 4 first-order constant-coefficient dif-ferential equations. From (2.8):

λ1(t) = c1; λ2(t) = −c1 + λ2(t) ⇒ λ2(t) = c2et + c1

u∗(t) = −c2et − c1

x1 = x2

x2 = −x2 − λ2 = −x2 − c2et − c1; ⇒ x2 = c3e

−t + c4et;

The constant c4 from the particular solution of x2 is calculated by replacingthe general solution into the equation

−c3e−t + c4e

t + c3e−t + c4e

t = −c2et − c1 ⇒ c4 = −c2

2− c1

2e−t

The general solution of the state equation result:

x2(t) = c3e−t − c2

2et − c1

2

x1(t) = −c3e−t − c2

2et − c1

2t + c4

Suppose the initial and final states are specified:

x(0) = [0 0]T ; x(2) = [5 2]T

We calculate the constants from:

x1(0) = −c3 −c22+ c4 = 0

x2(0) = c3 −c22− c1

2= 0

x1(2) = −c3e−2 − c2

2e2 − c1 + c4 = 5

x2(2) = c3e−2 − c2

2e2 − c1

2

20

Example 2.1.2 For the system

x1(t) = x2(t)

x2(t) = −x2(t) + u(t)

find the necessary conditions for an optimal control u∗(t) that minimizes theperformance measure:

J(u) =1

2[x1(2)− 5]2 +

1

2[x2(2)− 2]2 +

1

2

∫

2

0

u2(t)dt

The initial conditions are x(0) = 0, and x(2) are unspecified. The hamilto-nian is:

H(x, u, λ) =1

2u2(t) + λ1(t)x2(t) + λ2(t)(−x2(t) + u(t))

∂H

∂u= u+ λ2 = 0 ⇒ u∗(t) = −λ2(t)

The costate equations:

λ1(t) = −∂H

∂x1

= 0

λ1(t) = −∂H

∂x2

= −λ1(t) + λ2(t)

have the final conditions calculated from (2.7):

λ1(tf) =∂h

∂x1

|t=tf=2 = x1(2)− 5 = λ1(2)

λ2(tf) =∂h

∂x2

|t=tf=2 = x2(2)− 2 = λ2(2)

Comparing to the previous example, the modified performance measure affectsonly the boundary conditions at t = 2. The general results obtained in theprevious example are valid here too. Only the constants will result different.

21

Bibliography

*** (2001). Optimal control. online, Purdue University.

Kirk, D. E. (2004). Optimal Control Theory. An Introduction. Dover Publi-cations, Inc.

22