Optimal Control of Heat Equation

Optimal Control of Heat Equation

Mythily RamaswamyTIFR Centre for Applicable Mathematics, Bangalore, India

CIMPA Pre-School,I.I.T Bombay

22 June - July 4, 2015

Optimal Control 2nd July, 2015 1 / 38

contents

1 Introduction

2 Optimal Control of ODE systemsMathematical FrameworkConstrained OptimizationPontryagin Maximum Principle

3 Optimal Control of PDE

4 Tools for studying Evolutionary PDESobolev Spaces involving timeDifferential equation in Banach space

5 Weak Solution

6 Optimal control problem for Heat equationExistence of a unique optimal controlOptimality conditions

7 Abstract result


Introduction

Optimal control problems

In many practical problems, a differential equation describes the evolutionof the state of the system.Sometimes the dynamics involves some control parameters also.

Our aim is to find the best control which maximizes a certain payoffcriterion or cost functional.Major Questions :

Does an optimal control exist?

How can we characterize such an optimal control?

How to build an optimal control?


Introduction

Optimal control problems

In many practical problems, a differential equation describes the evolutionof the state of the system.Sometimes the dynamics involves some control parameters also.

Our aim is to find the best control which maximizes a certain payoffcriterion or cost functional.Major Questions :

Does an optimal control exist?

How can we characterize such an optimal control?

How to build an optimal control?


Optimal Control of ODE systems

Moon Landing Problem

Question:How to bring the spacecraft to land softly on the lunar surface usingminimum amount of fuel?

The system can be modelled by

v(t) = −g +α(t)

m(t)

h(t) = v(t)

m(t) = −kα(t)

Here h(t) is the height ; v(t) is the velocitym(t) is the mass of the spacecraft ; α(t) is the thrust


Optimal Control of ODE systems

Moon Landing Problem

Problem is to minimize the fuel or to maximize the remaining amountwhen it lands.

Minimizing fuel is equivalent to maximising the mass when it lands.

Define

J(α(.)) = m(T ),

where T is the first time when v(T ) = 0, that is it lands on the surface.

Then the problem is to maximize J(α(.)) .

Other constraints are

h(t) ≥ 0, m(t) ≥ 0, 0 ≤ α(t) ≤ 1


Optimal Control of ODE systems Mathematical Framework

Mathematical Framework

Control problem:Solve the minimization problem

Min

∫ T

0L(t, x(t), u(t))dt + g(x(T ))

subject todx(t)

dt= f(t, x(t), u(t)),

β(x(0), x(T )) = 0 = γ(x(0), x(T )) inRd

Control u(t) is in Ω ⊂ Rm,J is cost functional ; L is running cost; g is the terminal cost.


Optimal Control of ODE systems Constrained Optimization

Augmented Functional

Question:Can we use Lagrange Multiplier idea in optimal control problems too?Assuming g = 0, define augmented functional, with multipliers λ0 ≥ 0, µ, λ

J(x, x, u, p, λ0, µ, λ) = λ0 J(x, u)

+

∫ T

0pT (f(x, u)− x) + µT β(x(0), x(T )) + λTγ(x(0), x(T ))

At a minimum, vanishing of the derivative in each of the variable gives theoptimality conditions.Then the integrand is :

L = λ0 L(x, u) + pT (f(x, u)− x)

Introduce the Control Hamiltonian

H(x, p) = λ0 L(x, u) + pT (f(x, u)


Optimal Control of ODE systems Constrained Optimization

Optimality Conditions

Formally, calculus of variations applied to J to characterize the extremalgives :

d

dtLx − Lx = p + Hx = 0

This gives the equation for the adjoint vector p :

p = −p(t)T Df(t, x∗(t), u∗(t)) − ∇xL(t, x∗(t), u∗(t))

with Transversality conditions, obtained by variations in the parameters

Lx|t=0 = ∇1(µTβ + λTγ)

Lx|t=T = ∇2(µTβ + λTγ)

Partial gradient with respect to the first group of variables, x(0) is ∇1 ; forx(T ) is ∇2.


Optimal Control of ODE systems Pontryagin Maximum Principle

Pontryagin Principle

Theorem

Under the usual smoothness assumptions, let ( x∗(.), u∗(.) ) be theoptimal process.Then there exist λ0 ≥ 0, λ, µ ∈ Rd and a vector function p(.)satisfying the adjoint equation together with boundary conditions given bytransversality relations, such that for a.e. t ∈ [0, T ],

minu

H(t, x∗(t), p∗(t), u) = H(t, x∗(t), p∗(t), u∗(t))


Optimal Control of PDE

Optimal control for elliptic equation : an example

Ω is a domain with boundary Γ.

y = Electrical potential and u = Current density,Or

y = Temperature distribution and u = Thermal flux.

y satisfies

−∆y + y = f in Ω,∂y

∂n= u on Γ.

Problem: Minimize the distance between y and a given distribution yd∫Ω|y − yd|2.

The consumed energy is ∫Γ|u|2.



Control Problem:

Minimize J(y, u) = 12

∫Ω |y − yd|

2 + β2

∫Γ |u|

2,

−∆y + y = f in Ω, ∂y∂n = u on Γ,

u ∈ L2(Γ), f ∈ L2(Ω),

yd ∈ L2(Ω), β > 0.



Optimal control for parabolic equation : an example

Cooling process in metallurgy:

Ω is a domain with boundary Γ.

T = Temperature in Ω, c(T ) = Specific heat capacity, ρ(T ) =Density,

K(T ) = The conductivity of steel at the temperature T ,

R = a nonlinear function related to radiation law.

The problem is described by nonlinear heat equation of the form:

ρ(T )c(T )∂T∂t = div(K(T )∇T ) in Ω× (0, tf ),

K(T )∂T∂n = R(T, u) on Γ× (0, tf )

Here u is the control variable.



Control problem:

Minimize J(y, u) = β1

∫Ω |T (tf )− T |2 + β2

∫ tf0 |u|

q,

ρ(T )c(T )∂T∂t = div(K(T )∇T ) in Ω× (0, tf ),

K(T )∂T∂n = R(T, u) on Γ× (0, tf ),

β1 > 0, β2 > 0,

Here tf is the terminal time of the process ;T is a desired profile of temperature;exponent q is chosen in function of the radiation law R.


Tools for studying Evolutionary PDE Sobolev Spaces involving time

Sobolev spaces involving time

Search for solutions u(x, t) in Sobolev spaces involving time.

Let Y be a Banach space.

Definition

The spaceLp(0, T ;Y )

consists of all strongly measurable functions u : [0, T ]→ Y with

i) ‖u‖Lp(0,T ;Y ) :=(∫ T

0 ‖u(t)‖pY dt) 1p<∞, for 1 ≤ p <∞.

ii) ‖u‖L∞(0,T ;Y ) := ess supt∈[0,T ]‖u(t)‖Y <∞.


Tools for studying Evolutionary PDE Sobolev Spaces involving time

Definition

The spaceC([0, T ];Y )

consists of all continuous functions u : [0, T ]→ Y with

‖u‖C([0,T ];Y ) := maxt∈[0,T ]

‖u(t)‖Y <∞.


Tools for studying Evolutionary PDE Differential equation in Banach space

Differential equation in Banach space

Consider the equation

y′ = Ay + f, y(0) = y0,

with f ∈ C(R;Y ) and y0 ∈ Y , Y is a Banach space.

If A ∈ L(Y ) , then this equation admits a unique solution in C1(R;Y )given by

y(t) = etAy0 +

∫ t

0e(t−s)Af(s) ds,

etA =∞∑n=0

tn

n!An, t ∈ R.



When A ∈ L(Y ), the family (etA)t∈R satisfies:

i) e0A = I,

ii) etA ∈ L(Y ), ∀ t ∈ R,

iii) e(s+t)A = esA etA, ∀ s ∈ R, ∀ t ∈ R,

iv) limt→0‖etA − I‖L(Y ) = 0,

v) Ay = limt→0(etAy−y)

t , ∀ y ∈ Y.

Qn : What about unbounded operator A ?



One Dimensional Heat equation

The heat equation in (0, L)× (0, T )

y ∈ L2(0, T ;H10 (0, L)) ∩ C([0, T ];L2(0, L)),

yt − yxx = 0 in (0, L)× (0, T ),

y(0, t) = y(L, t) = 0 in (0, T ),

y(x, 0) = y0(x) in (0, L),

where T > 0, L > 0 and y0 ∈ L2(0, L).

Here A = d2

dx2is defined on a suitable subspace, D(A) = H2 ∩H1

0 ofL2(0, L)) into L2(0, L)) but is unbounded .



To find an expression for etA, introduce

φk(x) =

√2

Lsin

(kπx

L

), ∀ k ∈ N, ∀ x ∈ (0, L).

The family (φk)k∈N is an orthonormal basis of the Hilbert space L2(0, L).φk is an eigenfunction of the operator (A,D(A)):

φk ∈ D(A), Aφk = λkφk, λk = −k2π2

L2.

Search for a solution y in the form

y(x, t) =

∞∑k=1

gk(t)φk(x).



If

y0(x) = y(x, 0) =

∞∑k=1

gk(0)φk(x),

and if the PDE is satisfied in the sense of distributions in (0, L)× (0, T ),then gk obeys

g′k + k2π2

L2 gk = 0 in (0, T ),

gk(0) = y0k = (y0, φk)L2(0,L).

We have

gk(t) = y0ke− k

2π2tL2 , ∀ t ∈ (0, T ).

The function y ∈ L2(0, T ;H10 (0, L)) ∩ C([0, T ];L2(0, L)),

y(x, t) =∞∑k=1

y0ke− k

2π2tL2 φk(x), ∀ x ∈ (0, L), t ∈ (0, T ),

is the solution of the heat equation.



Remark : y is not defined for t < 0.Let us define

S(t)y0 =

∞∑k=1

(y0, φk)L2(0,L)e− k

2π2tL2 φk(x), ∀ x ∈ (0, L), t ∈ (0, T ),

Then we have

i) S(0) = I,

ii) S(t) ∈ L(L2(0, L)), ∀ t > 0,

iii) S(s+ t)y0 = S(s) S(t)y0, ∀ s ≥ 0, ∀ t ≥ 0, ∀ y0 ∈ L2(0, L),

iv) limt↓0‖S(t)y0 − y0‖L2(0,L) = 0, ∀ y0 ∈ L2(0, L).


Weak Solution

Weak solutions to evolution equations

The differential equation is

y′ = Ay + f, y(0) = y0,

with f ∈ C(R;Y ) and y0 ∈ Y , Y a Banach space.

Definition (Weak solution)

A weak solution to the equation in Lp(0, T ;Y ), for 1 ≤ p <∞, is afunction y ∈ Lp(0, T ;Y ) such that, for all z ∈ D(A∗), the mapping

t→ 〈y(t), z〉Y,Y ′

belongs to W 1,p(0, T ) and obeys

ddt〈y(t), z〉 = 〈y(t), A∗z〉+ 〈f(t), z〉, ∈ (0, T ),

〈y(0), z〉 = 〈y0, z〉.


Weak Solution

Theorem (Existence and Uniqueness of Weak solution)

If y0 ∈ Y and if f ∈ Lp(0, T ;Y ), then the equation admits a unique weaksolution in Lp(0, T ;Y ).Moreover, this solution belongs to C([0, T ];Y ) and it satisfies

y(t) = S(t)y0 +

∫ t

0S(t− s)f(s), ∀ t ∈ [0, T ].

From the expression for the solution

‖y(t)‖C([0,T ];Y ) ≤ C(‖y0‖Y + ‖f‖L1(0,T ;Y )

),

for some positive constant C.


Optimal control problem for Heat equation


Let Ω be a bounded domain in Rn, with a boundary Γ of class C2.Let T > 0, set Q = Ω× (0, T ) and Σ = Γ× (0, T ).

The heat equation with a distributed control is

∂y∂t −∆y = f + χωu in Q,

y = 0 on Σ, y(x, 0) = y0(x), in Ω.

The optimal control problem is

infJ(y, u) | u ∈ L2(ω × (0, T ))

,

for some suitable J .



Optimal control

Let us take

J(y, u) = 12

∫Q |y − yd|

2 + 12

∫Ω |y(T )− yd(T )|2 + β

2

∫ω×(0,T ) |u|

2.

Here β > 0 and yd ∈ C([0, T ];L2(Ω)) and (y, u) solves the controlled heatequation .

Then we have estimate for the state variable:

‖y‖C([0,T ];L2(Ω)) ≤ C(‖y0‖L2(Ω) + ‖f‖L2(Q) + ‖u‖L2(ω×(0,T ))

).


Optimal control problem for Heat equation Existence of a unique optimal control

Existence of a unique optimal control

Set F (u) = J(y(u), u).

Let (un)n∈N be a minimizing sequence in L2(ω × (0, T )),

limn→∞

F (un) = infu∈L2(ω×(0,T ))F (u).

Let yn be the solution corresponding to un. Suppose that (un)n∈N isbounded in L2(ω × (0, T )) and that

un u, weakly in L2(ω × (0, T )).

Set y = y(u).


Optimal control problem for Heat equation Existence of a unique optimal control

Showyn y, weakly in L2(Q),

yn(T ) y(T ), weakly in L2(Ω).

Using the weakly lower semicontinuity of ‖ · ‖2L2(Q), ‖ · ‖2L2(Ω),

‖ · ‖2L2(ω×(0,T )), show that u is a solution of the minimizing problem.

Uniqueness of the solution follows from the strict convexity of F .


Optimal control problem for Heat equation Optimality conditions

Derivative of the state variable

IntroduceEquations satisfied by zλ = y(u+ λv)− y(u) is

∂zλ∂t −∆zλ = λχωv in Q,

zλ = 0 on Σ, zλ(x, 0) = 0, in Ω.

From the estimate it follows that

‖zλ‖C([0,T ];L2(Ω)) ≤ C|λ|‖v‖L2(ω×(0,T )).

This yields that as λ tends to zero,

y(u+ λv)→ y(u) in C([0, T ];L2(Ω)).



Derivative of F

Recall that

F ′(u) = limλ0

F (u+ λv)− F (u)

λ.

By a classical calculation, we have

F ′(u)v =∫Q(y(u)− yd)z(v)

+∫

Ω(y(u)(T )− yd(T ))z(v)(T ) + β∫ω×(0,T ) uv,

where z satisfies

∂z∂t −∆z = χωv in Q,

z = 0 on Σ, z(x, 0) = 0, in Ω.



Identification of F ′(u)

Our aim: To find q such that∫Q

(y(u)− yd)z(v) +

∫Ω

(y(u)(T )− yd(T ))z(v)(T ) =

∫ω×(0,T )

qv

Let p be any regular function defined in Q.

Using an integration by parts between z(v) and p we have∫ω×(0,T ) pv =

∫Q(zt −∆z)p

=∫Q z(−pt −∆p) +

∫Ω z(T )p(T )−

∫Σ∂z∂np



Consider the adjoint problem

−∂p∂t −∆p = y(u)− yd in Q,

p = 0 on Σ, p(x, T ) = (y(u)− yd)(T ), in Ω.

Then we have

F ′(u)v =

∫ω×(0,T )

(p+ βu)v,

if the above equations are justified.

For that we need to study the existence and the regularity of the solutionof the adjoint problem.



Adjoint problem

Theorem

Let g ∈ L2(Q) and pT ∈ L2(Ω).The terminal boundary value problem

−∂p∂t −∆p = g in Q,

p = 0 on Σ, p(x, T ) = pT , in Ω

is wellposed and p satisfies

‖p‖C([0,T ];L2(Ω)) ≤ C(‖pT ‖L2(Ω) + ‖g‖L2(Q)

).



Integration by parts between z and p

Theorem

Suppose that φ ∈ L2(Q), g ∈ L2(Q), pT ∈ L2(Ω).Then the solution z of the equation

∂z∂t −∆z = φ in Q,

z = 0 on Σ, z(x, 0) = 0, in Ω.

and the solution p of the adjoint equation satisfy the following formula∫Qφp =

∫Qzg +

∫Ωz(T )pT .



Outline of the proof

If pT ∈ H10 (Ω), then z and p belong to

L2(0, T ;D(A)) ∩H1(0, T ;L2(Ω)).

Green’s formula gives∫Ω−∆z(t)p(t)dx =

∫Ω−∆p(t)z(t)dx,

for almost every t ∈ [0, T ], and∫ T

0

∫Ω

∂z

∂tp = −

∫ T

0

∫Ω

∂p

∂tz +

∫Ωz(T )pT .

Thus IBP formula is established for pT ∈ H10 (Ω).

Then by density argument and using the estimate, we obtain theequation for any pT ∈ L2(Q).



Theorem

If (y, u) is the solution, then

u = − 1

βp|ω×(0,T ),

where p is the solution of the adjoint problem corresponding to y:

−∂p∂t −∆p = y − yd in Q,

p = 0 on Σ, p(x, T ) = y(T )− yd(T ), in Ω.

Next theorem is the converse of the above one.



Theorem

If a pair (y, p) ∈ C([0, T ];L2(Ω))× C([0, T ];L2(Ω)) obeys the system

∂y∂t −∆y = f − 1

βχωp in Q,

y = 0 on Σ, y(x, 0) = y0(x), in Ω,

−∂p∂t −∆p = y − yd in Q,

p = 0 on Σ, p(x, T ) = y(T )− yd(T ), in Ω,

Then the pair (y,− 1β p) is the optimal solution to the problem.


Abstract result

Abstract result

Consider the problem P (0, T, y0)

infJT (y, u), (y, u) | u ∈ L2(0, T ;U),

JT (y, u) =1

2

∫ T

0‖y(t)‖2Y dt+

1

2

∫ T

0‖u(t)‖2u dt.

and (y, u) satisfies

y′(t) = Ay(t) +Bu(t), ∀ t ≥ 0,

y(0) = y0,

y0 ∈ Y, u ∈ U,

where

Y and U are two Hilbert spaces.

The unbounded operator (A,D(A)) is the infinitesimal generator of astrongly continuous semigroup on Y .

The control operator B ∈ L(U ;Y ).


Abstract result

Theorem

This problem P (0, T, y0) admits a unique solution (y, u) and (y, u)satisfies the system

y′(t) = Ay(t)−BB∗p(t), y(0) = y0,

−p′(t) = A∗p(t) + y(t), p(T ) = 0,

u(t) = −B∗p(t).


Abstract result

A. Bensoussan, G. Da Prato, M. Delfour, S. K. Mitter, Representationand control of infinite dimensional systems. Second edition. Systemsand Control: Foundations & Applications. Birkhauser Boston, Inc.,Boston, MA, 2007.

L.C. Evans, An Introduction to Mathematical Optimal Control Theory,Lecture Notes.

Fleming and Rishel, Deterministic and Stochastic Optimal Control,Springer

J-L Lions, Optimal Control of systems governed by PDEs

Donald Kirk, Optimal Control Theory - An introduction, DoverPublications.

Jean-Pierre Raymond, Optimal Control of PDEs, FICUS Course Notes.

Troltzsch, Optimal Control of Partial Differential Equations, Theory,Methods and Applications, AMS Vol 112.


Documents

Optimal Control of Heat Equation