Optimal Control Problems with Directional Sparsity fileGerd Wachsmuth Optimal Control Problems with Directional Sparsity. ... Gerd Wachsmuth Optimal Control Problems with Directional

Optimal Control Problemswith Directional Sparsity

Gerd Wachsmuth Roland GriesseGeorg Stadler (ICES, Austin)

Numerical Mathematics

Skiseminar Optimierung

Gerlosberg, 7.–14. Marz 2009

Gerd Wachsmuth Optimal Control Problems with Directional Sparsity

Outline

1 Introduction to Sparsity

2 Sparsity in Optimal Control

3 Directional Sparsity


Introduction to Sparsity

Applications of Sparsity

Identification of relevant mechanisms (systems biology, chemicalengineering)

Image/signal reconstruction

Actuator placement

How to obtain Sparsity

Using µ(u 6= 0) =⇒ combinatorial complexity (in finite problems)

Using a L1 norm: [Donoho (2006)] ”For most Large UnderdeterminedSystems of Linear Equations the Minimal L1-Norm Solution is alsothe Sparsest Solution”


Introduction to Sparsity

Applications of Sparsity

Identification of relevant mechanisms (systems biology, chemicalengineering)

Image/signal reconstruction

Actuator placement

How to obtain Sparsity

Using µ(u 6= 0) =⇒ combinatorial complexity (in finite problems)

Using a L1 norm: [Donoho (2006)] ”For most Large UnderdeterminedSystems of Linear Equations the Minimal L1-Norm Solution is alsothe Sparsest Solution”


Application: Signal Reconstruction

Least-squares reconstruction of a noisy signal

minimize ‖x − xnoisy‖22 + δ ‖Dx‖2

2

Total-variation reconstruction of the same signal

minimize ‖x − xnoisy‖2 + γ ‖Dx‖1





2







2




Solutions of Underdetermined Systems

Smooth minimization problem

minimize ‖x‖22 s.t. Ax = b

Convex minimization problem


Histogram (solution components’ sizes)


Solutions of Underdetermined Systems

Smooth minimization problem


Convex minimization problem


Histogram (solution components’ sizes)


Sparsity by the L1 Norm?

Natural sparsity measure

‖x‖0 = # 1 ≤ i ≤ n : |xi | 6= 0

leads to combinatorial complexity

[Donoho(2006)]

”For most large underdetermined systems of linear equations the minimalL1-norm solution is also the sparsest solution”

Other sparsity promoting ”norms” in use

‖x‖pp for 0 < p < 1


Sparsity in Optimal Control

standard opt. control problem

Minimize 12‖y − yd‖2

L2 + α2 ‖u‖

2L2

s.t.

−4y = u in Ω

y = 0 on Γ

and ua ≤ u ≤ ub

‖u‖2L2 =

ZΩ

|u(x)|2 dx

, ‖u‖L1 =

ZΩ

|u(x)| dx

Features

optimal control u is 6= 0 everywhere

differentiable optimization problem



standard opt. control problem


L2 + α2 ‖u‖

2L2

s.t.

−4y = u in Ω

y = 0 on Γ

and ua ≤ u ≤ ub

‖u‖2L2 =

ZΩ

|u(x)|2 dx

, ‖u‖L1 =

ZΩ

|u(x)| dx

Features

optimal control u is 6= 0 everywhere

differentiable optimization problem



problem with sparsity term


L2 + β ‖u‖L1

s.t.

−4y = u in Ω

y = 0 on Γ

and ua ≤ u ≤ ub

‖u‖2L2 =

ZΩ

|u(x)|2 dx , ‖u‖L1 =

ZΩ

|u(x)| dx

Features

optimal control u is sparse

non-differentiable (convex) optimization problem


[Vossen, Maurer (2006), Stadler (2008)]

Optimality for Sparsity Problems

Regularized formulation


L2 + α2 ‖u‖

2L2 + β ‖u‖L1

s.t. PDE and ua ≤ u ≤ ub

Lemma (Optimality conditions)

There exist λ ∈ ∂‖ · ‖L1(Ω)(u) and adjoint state p = S?(yd − Su) s.t.

〈u − u, −p + β λ+ α u〉 ≥ 0 for all u ∈ Uad


Optimality for Sparsity Problems



L2 + α2 ‖u‖

2L2 + β ‖u‖L1


Lemma (Optimality conditions)

There exist λ ∈ ∂‖ · ‖L1(Ω)(u) and adjoint state p = S?(yd − Su) s.t.

〈u − u, −p + β λ+ α u〉 ≥ 0 for all u ∈ Uad


Structural Properties

Adjoint equation for example problem

−4p = y − yd in Ω

p = 0 on Γ

Problem with α2‖u‖2

L2

p

u

Problem with β ‖u‖L1

p

u


Structural Properties

Adjoint equation for example problem

−4p = y − yd in Ω

p = 0 on Γ

Problem with α2‖u‖2

L2

p

u

Problem with β ‖u‖L1

p

u


Numerical Solution of Problems with Sparsity



L2 + α2 ‖u‖

2L2 + β ‖u‖L1


Optimality conditions

µ = p − α u

u = max0, u + c (µ− β)

+ min0, u + c (µ+ β)

−max0, u − ub + c (µ− β)

−min0, u − ua + c (µ+ β)

Newton differentiable for α > 0

Sparsity is maintained

p

u





L2 + α2 ‖u‖

2L2 + β ‖u‖L1



µ = p − α u

u = max0, u + c (µ− β)

+ min0, u + c (µ+ β)

−max0, u − ub + c (µ− β)

−min0, u − ua + c (µ+ β)



p

u





L2 + α2 ‖u‖

2L2 + β ‖u‖L1



µ = p − α u

u = max0, u + c (µ− β)

+ min0, u + c (µ+ β)

−max0, u − ub + c (µ− β)

−min0, u − ua + c (µ+ β)



p

u


Convergence of Regularised Solutions

Convergence rate

Suppose ua, ub ∈ L∞(Ω) and∣∣x ∈ Ω :

∣∣|p0| − β∣∣ ≤ ε∣∣ ≤ C ε. Then for

all d < 1/3,‖uα − u0‖L2(Ω) ≤ Cd α

d .

A-priori FEM error

Assume ua, ub ∈ L∞(Ω). Then,

‖uα,h − uα‖2 ≤ C (hα−1 + h2 α−3/2).

A-posteriori FEM error, α > 0

Local error estimates are obtained =⇒ adaptive algorithms can be used


Convergence of Regularised Solutions

Convergence rate

Suppose ua, ub ∈ L∞(Ω) and∣∣x ∈ Ω :

∣∣|p0| − β∣∣ ≤ ε∣∣ ≤ C ε. Then for

all d < 1/3,‖uα − u0‖L2(Ω) ≤ Cd α

d .

A-priori FEM error

Assume ua, ub ∈ L∞(Ω). Then,

‖uα,h − uα‖2 ≤ C (hα−1 + h2 α−3/2).

A-posteriori FEM error, α > 0

Local error estimates are obtained =⇒ adaptive algorithms can be used


[Wachsmuth, Wachsmuth (to appear)]

Outline

1 Introduction to Sparsity

2 Sparsity in Optimal Control

3 Directional Sparsity


Can we do Better Than Just Sparse?

Sparsity

vs. directional sparsity

Objective function

12‖y − yd‖2

L2 + β ‖u‖L1



Sparsity vs. directional sparsity

Objective function

12‖y − yd‖2

L2 + β ‖u‖L1

Objective function

12‖y − yd‖2

L2 + β ‖u‖L1(L2)



Sparsity vs. directional sparsity

Properties

no structural assumptionsmade

Properties

exploits known or desiredgroup sparsity structure


Directional Sparsity: Basic Results

Problem formulation

min1

2‖Su − yd‖2

H +α

2‖u‖2

L2(Ω)

+ β ‖u‖L1(L2)

s.t. ua ≤ u ≤ ub a.e. in Ω x1

x2

Ω2(x

1)

x1Ω1

Problem data

ua, ub ∈ L2(Ω), S ∈ L(L2(Ω),H), Ω ⊂ RN = Rn × RN−n

Lemma

For α > 0 (or α = 0 and S injective) and β ≥ 0, there exists a uniquesolution u.



Problem formulation

min1

2‖Su − yd‖2

H +α

2‖u‖2

L2(Ω)

+ β

∫Ω1

(∫Ω2(x1)

u(x1, x2)2 dx2

)1/2dx1


x2

Ω2(x

1)

x1Ω1

Problem data


Lemma




Problem formulation

min1

2‖Su − yd‖2

H +α

2‖u‖2

L2(Ω)

+ β

∫Ω1

(∫Ω2(x1)

u(x1, x2)2 dx2

)1/2dx1


x2

Ω2(x

1)

x1Ω1

Problem data


Lemma




Problem formulation

min1

2‖Su − yd‖2

H +α

2‖u‖2

L2(Ω)

+ β

∫Ω1

(∫Ω2(x1)

u(x1, x2)2 dx2

)1/2dx1


x2

Ω2(x

1)

x1Ω1

Problem data


Lemma



Optimality for Directional Sparsity Problems

Fixed point property

u is optimal ⇔ u solves

Minimize1

2‖Su − yd‖2

H +1

2

ZΩ+

„α +

β

‖u(x1, ·)‖L2

«u(x)2 dx


and u = 0 in Ω \ Ω+

and it satisfies the complementarity property

‖p(x1, ·)‖L2 ≤ β where u(x1, ·) = 0

First solution approach

fixed-point iteration for x1 7→ ‖u(x1, ·)‖L2

semi-smooth Newton method inside


Optimality for Directional Sparsity Problems

Fixed point property

u is optimal ⇔ u solves

Minimize1

2‖Su − yd‖2

H +1

2

ZΩ+

„α +

β

‖u(x1, ·)‖L2

«u(x)2 dx


and u = 0 in Ω \ Ω+

and it satisfies the complementarity property

‖p(x1, ·)‖L2 ≤ β where u(x1, ·) = 0

First solution approach

fixed-point iteration for x1 7→ ‖u(x1, ·)‖L2

semi-smooth Newton method inside


Numerical Results

α = 10−6

β = 10−3


Convergence History

Fixed point iteration

0 100 200 300 400 500 600 700 800 900 100010

−5

10−4

10−3

10−2

Number of solved systems of equations

Resid

ual ||

νk −

Φ(ν

k)|

| 1

m=4

m=5

m=6

m=7

Newton?

Newton step for fixed point iteration coupled with inner optimalitysystem

Newton differentiability: partial results (to be cont’d)


Convergence History

Switching to Newton’s method

1 2 3 4 5 6 7

10−4

10−2

100

102

CPU time, seconds

Resid

uum

, ||

ν−

Φ(ν

)|| 1


Newton method

Newton?




Convergence History

Switching to Newton’s method

1 2 3 4 5 6 7

10−4

10−2

100

102

CPU time, seconds

Resid

uum

, ||

ν−

Φ(ν

)|| 1


Newton method

Newton?




Sparsity in Parabolic Problems

Parabolic example

minimize 12‖y − yd‖2

L2(Ω) + β ‖u‖L1(L2)

s.t.

yt − 1

10 ∆y = u in Ω = Ω1 × (0,T )

y = 0 on Γ× (0,T )

y(·, 0) = 0 in Ω1

and ua ≤ u ≤ ub a.e. in Ω

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

00.10.20.30.40.50.60.70.80.91

sparsity pattern of u

n = 2 sparse directions (space)

N − n = 1 non-sparse direction (time)


Summary and Bibliography

Summary

optimal control problems with directional sparsity terms

comprises problems with elliptic and parabolic PDEs

applications in actuator placement problems

related to joint sparsity data reconstruction

D. Donoho.

For most large underdetermined systems of linear equations the minimal l1-norm solutionis also the sparsest solution.Communications in Pure and Applied Mathematics, 59:797–829, 2006.

R. Griesse, G. Stadler, and G. Wachsmuth.Optimal control problems with directional sparsity terms.in preparation.

G. Stadler.

Elliptic optimal control problems with L1-control cost and applications for the placementof control devices.Computational Optimization and Applications, 2007.doi: http://dx.doi.org/10.1007/s10589-007-9150-9.


Summary and Bibliography

Summary

optimal control problems with directional sparsity terms

comprises problems with elliptic and parabolic PDEs

applications in actuator placement problems

related to joint sparsity data reconstruction

D. Donoho.

For most large underdetermined systems of linear equations the minimal l1-norm solutionis also the sparsest solution.Communications in Pure and Applied Mathematics, 59:797–829, 2006.

R. Griesse, G. Stadler, and G. Wachsmuth.Optimal control problems with directional sparsity terms.in preparation.

G. Stadler.

Elliptic optimal control problems with L1-control cost and applications for the placementof control devices.Computational Optimization and Applications, 2007.doi: http://dx.doi.org/10.1007/s10589-007-9150-9.


Documents

Optimal Control Problems with Directional Sparsity fileGerd Wachsmuth Optimal Control Problems with Directional Sparsity. ... Gerd Wachsmuth Optimal Control Problems with Directional