Noname manuscript No.(will be inserted by the editor)

An Indirect Shooting Method using Proper OrthogonalDecomposition for Distributed Optimal Control of theWave Equation

Z. Sabeh · M. Shamsi · I.M. Navon

Received: date / Accepted: date

Abstract This paper presents a fast numerical method, based on the indirectshooting method and Proper Orthogonal Decomposition (POD) technique, forsolving distributed optimal control of the wave equation. To solve this problem,we consider the first order optimality conditions and then by using finite elementspatial discretization and shooting strategy, the solution of the optimality con-ditions is reduced to the solution of a series of Initial Value Problems (IVPs).Generally, these IVPs are high-order and thus their solution is time-consuming.To overcome this drawback, we present a POD indirect shooting method, whichutilizes the POD technique to approximate IVPs with smaller ones and faster runtimes. Moreover, in the presence of the nonlinear term, to reduce the order of thenonlinear calculations, a Discrete Empirical Interpolation Method (DEIM) is ap-plied and a POD/DEIM indirect shooting method is developed. We investigate theperformance and accuracy of the proposed methods by means of three numericalexperiments. We show that the presented POD and POD/DEIM indirect shootingmethods dramatically reduce the CPU time compared to the full indirect shootingmethod, whereas, there is no significant difference between the accuracy of thereduced and full indirect shooting methods.

Keywords Optimal control · Proper Orthogonal Decomposition (POD) · Indirectshooting method · Wave equation ·

Z. SabehDepartment of Applied Mathematics, Faculty of Mathematics and Computer Science, Amirk-abir University of Technology, Tehran, Iran E-mail: z sabeh@aut.ac.irM. ShamsiDepartment of Applied Mathematics, Faculty of Mathematics and Computer Science, Amirk-abir University of Technology, Tehran, Iran E-mail: m shamsi@aut.ac.irTel.: +98-21-64542538Fax: +123-45-678910I. M. NavonDepartment of Scientific Computing, Florida State University, Tallahassee, FL, 32306-4120,USA E-mail: inavon@fsu.edu

2 Z. Sabeh et al.

1 Introduction

In this paper, we consider the distributed optimal control problems governed bylinear and nonlinear wave-type equations [48,20,27]. Let Ω ⊆ Rd, d ∈ 1, 2, 3 be abounded spatial domain with boundary ∂Ω, T ≥ 0 be the final time, Q := (0, T )×Ωand Σ := (0, T ) × ∂Ω. In the distributed optimal control problem for the waveequation, the aim is finding control u(t,x) and state y(t,x) such that the followingwave equation:

ytt −∆y +N1(y) = f + u, inQ (1.1a)

with initial conditions

y(0,x) = g(x), inΩ

yt(0,x) = h(x), inΩ

(1.1b)

(1.1c)

and boundary condition

y(t,x) = 0, onΣ (1.1d)

are satisfied and furthermore the following performance index is minimized:

J =κ12

∫∫Q

∣∣y(t,x)− z(t,x)∣∣2dxdt+

κ22

∫Ω

∣∣y (T,x)− w(x)∣∣2dx +

1

2

∫∫Q

∣∣u(t,x)∣∣2dxdt.

(1.1e)

In this problem z(t,x) and w(x) are target functions, g(x) and h(x) are initialconditions, f(t,x) is a given force function, κ1, κ2 ≥ 0 are constants and N1(y) isa generally nonlinear function of y.

Control problems for the wave equations arise in several applications, e.g. med-ical applications [6], acoustic problems as noise suppression [1] and linear elasticity[37]. As a consequence, several papers have been presented in this area. In [30,49,10] and the references therein, the controllability of the wave equation is inves-tigated. Optimal boundary control of the wave equation is considered in variouspublications. See for instance [12,14,35,36,34,24]. Recently, several papers havebeen presented for the numerical solution of the distributed control problem forthe wave equation. In [31], the finite volume element method is applied to thehyperbolic optimal control problems and a priori error estimation is derived. Aspace-time finite element discretization, with a priori error estimation, is consid-ered in [19], for the optimal control of nonlinear wave equation. In [27] a newcentral finite difference scheme and in [22] a Gautschi time-stepping scheme isproposed for optimal control problem of wave equations. The semi-smooth New-ton methods are investigated in [21], where the control is restricted by pointwiselower and upper bounds. For state constrained optimal distributed control of thewave equation, the Lavrentiev regularization is considered in [15].

In this work, we propose another method based on the shooting method andproper orthogonal decomposition method for solving the distributed optimal con-trol problem (1.1). Our main aim is to develop an efficient algorithm with lowcomputational complexity for solving (1.1).

Shooting methods are an important class of numerical methods for solving op-timal control problems which are applicable in the framework of the direct and

An Indirect Shooting Method using Proper Orthogonal Decomposition 3

indirect approach. Direct shooting approach reformulates the optimal control prob-lem to an optimization problem, which can be solved by well-developed solvers. Incontrast, indirect shooting methods peruse first-optimize-then-shooting approach[17] and apply the shooting method to the optimality conditions of optimal con-trol problem, which forms an Initial Terminal Boundary Value Problem (ITBVP).Indirect shooting method converts the solution of ITBVP into an Initial Bound-ary Value Problem (IBVP) by considering guess values for some initial conditions.These guess values are iteratively updated until the terminal conditions of theoriginal ITBVP are met.

In general, the solution is found by indirect methods satisfies the optimalityconditions and usually is more accurate than the solution of direct methods. Onthe other hand, the region of convergence for indirect methods may be smallerthan the region of convergence for direct methods, therefore, they require betterinitial guess [2]. We note that a common feature of the two classes of shootingmethods is that they are time-consuming and this is more significant for large-scale optimization problems. To overcome this drawback, it is necessary to use thetechniques for reduction of computation time for these methods.

Proper Orthogonal Decomposition (POD) is a well-known model reductiontechnique for nonlinear PDEs. This method generates an optimal set of basisfunctions (so-called POD basis functions) that each of them has a global supportand involves information about the system obtained out of a computational orexperimental database (snapshots). After this, the solution is obtained as a linearcombination of the POD basis functions by means of Galerkin projection method.We can refer to [41,23] for basic concepts in the POD method, [42,7,46] for usingPOD technique on numerical solution of PDEs, [13] for application of POD in PDEoptimal control, [18,38,40] for error estimation of the POD method for optimalcontrol problems and to [8] for other engineering applications.

In this paper, for solving the problem (1.1), the optimality conditions of thisproblem are considered. Then, by Finite Element (FE) spatial discretization andshooting strategy, the solution of the optimality conditions reduces to the solutionof a series of Initial Value Problems (IVPs). However, the dimension of these IVPsdepends on the number of elements in the FE spatial discretization. On the otherhand, to get a reasonable accuracy, the needed size of IVPs sometimes is of theorder of hundred thousands of variables. Consequently, the method is very time-consuming. To resolve this high dimensionality and CPU usage, we use the PODmethod for reducing the order of IVPs. To do this, the constructing of POD basisand applying the POD technique to the problem (1.1) are presented in details andan algorithm, for solving this problem, is developed.

In the absence of the nonlinear term N1, the POD shooting method is largelysatisfactory. However, when the problem is nonlinear, in the evaluation of non-linear term, the mentioned computational savings in the POD technique are notattained. As a result, the efficiency of POD strategy is reduced in the presenceof the nonlinear term. Several approaches have been proposed to overcome thiscomputational inefficiency [3]. One of these approaches is Discrete Empirical In-terpolation Method (DEIM), which is used in this paper. DEIM technique approx-imates a nonlinear function by combining projecting with interpolation [3]. DEIMis a discrete variant of Empirical Interpolation Method that is based on the greedyalgorithms [41].

4 Z. Sabeh et al.

The structure of this paper is outlined as follows: In Section 2, the distributedoptimal control problem of the wave equation and it’s necessary optimality con-ditions are introduced. A discussion of the shooting algorithm for solving theoptimality system is also placed in this section. In Section 3, an algorithm basedon the indirect shooting method and POD method is developed for solving theoptimal control of the wave equation. In Section 4, an efficient algorithm, basedon DEIM strategy, is presented for the optimal control of nonlinear wave equation.Numerical examples are given in Section 5 to demonstrate the effectiveness of theproposed method.

2 Indirect shooting method and finite element spatial discretization

The first order necessary optimality conditions for optimal control problem (1.1)are well established in [28,29]. We briefly recall them here. Suppose that y(t,x)and u(t,x) are the optimal state and control, then there exists an adjoint functionp(t,x) such that (y(t,x), u(t,x), p(t,x)) satisfies the following first order necessaryoptimality conditions:

ytt −∆y +N1(y) = f + u, inQ,

y(0,x) = g(x), inΩ,

yt(0,x) = h(x), inΩ,

y(t,x) = 0, onΣ,

ptt −∆p+N2(y, p) = κ1 (y − z) , inQ,

p(T,x) = 0, inΩ,

pt(T,x) = −κ2(y(T,x)− w(x)), in Ω,

p(t,x) = 0, onΣ,

p+ u = 0, inQ,

(2.1a)

(2.1b)

(2.1c)

(2.1d)

(2.1e)

(2.1f)

(2.1g)

(2.1h)

(2.1i)

where N2(y, p) = p∂N1

∂y (y). After eliminating u from (2.1a) by (2.1i), this systemcan be converted to a standard two point boundary value problem with respectto t. We note that the resulted boundary value system contains both initial andterminal conditions in time. Accordingly, it can’t be solved by marching methodsin time [27,16]. However, the well-known shooting method can be used for thenumerical solution of it.

In the first step of shooting method for the numerical solution of (2.1), weconsider a corresponding Initial Boundary Value Problem (IBVP) with unknowninitial conditions for the adjoint variable. If the unknown initial conditions are

An Indirect Shooting Method using Proper Orthogonal Decomposition 5

denoted by η, γ : Ω → R, then the corresponding IBVP is considered as:

ytt −∆y +N1(y) = f − p, inQ,

ptt −∆p+N2(y, p) = κ1 (y − z) , inQ,

y(0,x) = g(x), yt(0,x) = h(x), inΩ,

p(0,x) = η(x), pt(0,x) = γ(x), inΩ,

y(t,x) = 0, p(t,x) = 0, onΣ.

(2.2a)

(2.2b)

(2.2c)

(2.2d)

(2.2e)

It must be mentioned that the solution of IBVP (2.2) depends not only on t andx but also on the initial functions η(x) and γ(x). To emphasize this dependence,we use notations y(t,x; η, γ) and p(t,x; η, γ) for the solution of IBVP (2.2). Inthe second step of shooting method, we seek the initial functions η(x) and γ(x)such that the solutions y(t,x; η, γ) and p(t,x; η, γ) satisfy the following terminalconditions

F(x; η, γ) =

[p(T,x; η, γ)

pt(T,x; η, γ) + κ2 (y (T,x; η, γ)− w(x))

]= 0 (2.3)

Therefore, solving the optimality system (2.1) is equivalent to finding initial func-tions η(x) and γ(x) such that the solution of IBVP (2.2) satisfies equation (2.3).This strategy can be summarized as the following infinite shooting problem:

Infinite Shooting Problem (ISP): Find η, γ : Ω → R such that the solutions

y(t, x; η, γ) and p(t, x; η, γ) of the equation (2.2) fulfill the terminal condition (2.3).

In the following, for the sake of simplicity in notation, we will omit η and γ fromy(t,x; η, γ) and p(t,x; η, γ) and easily write y(t,x) and p(t,x).

We note that, ISP is an infinite dimensional problem and at first we need tomake a spatial discretization for its numerical solution. In this paper, we considerthe finite element method for spatial discretization of ISP. Suppose that functionsϕ1(x), . . . , ϕm(x) be linearly independent nodal basis functions and

Xh = spanϕ1(x), . . . , ϕm(x) ⊂ L2(Ω). (2.4)

We approximate state y(t,x) and adjoint functions p(t,x) in the space Xh byyh(t,x) and ph(t,x) as:

yh(t,x) =m∑j=1

yj(t)ϕj(x), ph(t,x) =m∑j=1

pj(t)ϕj(x), (2.5)

as well as the functions η(x) and γ(x) are approximated as:

ηh(x) =m∑j=1

ηjϕj(x), γh(x) =m∑j=1

γjϕj(x), (2.6)

where yj ,pj , ηj , γj , j = 1, . . . ,m are unknown finite element coefficients and weset:

y(t) = [y1(t), . . . ,ym(t)]T, p(t) = [p1(t), . . . ,pm(t)]

T,

η = [η1, . . . , ηm]T , γ = [γ1, . . . , γm]T .

6 Z. Sabeh et al.

By substituting approximations (2.5) and (2.6) in the weak form of (2.2), finally,we obtain the following IVP:

My(t) + Ky(t) + N1(t) = f(t)−Mp(t),

Mp(t) + Kp(t) + N2(t) = κ1 [My(t)− z(t)] ,

My(0) = g, My(0) = h,

p(0) = η, p(0) = γ,

(2.7a)

(2.7b)

(2.7c)

(2.7d)

where

[M]ij = 〈ϕj , ϕi〉, [K]ij = 〈∇ϕj ,∇ϕi〉, [f(t)]i = 〈ϕi, f〉,

[z(t)]i = 〈ϕi, z〉, [g]i = 〈ϕi, g〉, [h]i = 〈ϕi, h〉,

[N1(t)]i = 〈ϕi,N1(yh)〉, [N2(t)]i = 〈ϕi,N2(yh, ph)〉,

(2.8a)

(2.8b)

(2.8c)

and 〈·, ·〉 stands for the L2 inner product. In a similar manner, the shooting function(2.3) is discretized to

Fh(η,γ) =

[Mp(T )

Mp(T ) + κ2 (My (T )−w)

]= 0, (2.9)

where

[w]i = 〈ϕi, w〉. (2.10)

In summary, by using finite element method, ISP can be discretized to the follow-ing finite dimensional shooting problem

Finite Shooting Problem (FSP): Find η,γ ∈ Rm such that the solution of

IVP (2.7) fulfills the discrete terminal condition (2.9).

Based on the above discussions, we summarize the indirect shooting method, forsolving the problem (1.1), in Algorithm 1.

In Algorithm 1, for solving IVP (2.7), generally an appropriate Runge-Kuttatime marching method can be used. The dimension of IVP (2.7) depends on m,the number of considered basis functions in the finite element method. We notethat m should be extremely large when high accuracy is required. Moreover, IVP(2.7) must be iteratively solved with different initial values during the solutionof terminal equations (2.9). Accordingly, the solution of the problem (1.1) by theindirect shooting algorithm is very time-consuming. This is the motivation for usto utilize a reduced-order approach based on POD technique for improving theefficiency of the method.

3 Proper Orthogonal Decomposition(POD) method for model reduction

of ISP

In this section, we first review POD technique and develop a POD Galerkin schemefor discretization of ISP. By this, we have a reduced-order initial value problemthat must be iteratively solved in the shooting method.

An Indirect Shooting Method using Proper Orthogonal Decomposition 7

Algorithm 1 : Indirect Shooting (IS) Algorithm

Inputs:m ∈ N as the number of finite element basis.

1: Begin2: Compute matrices M,K and vectors f(t), z(t),g,h,w as (2.8) and (2.10).3: Solve the system of equations EQF(η`,γ`) = 0, by a solver and denote the solution by

(η∗`,γ∗`).4: Solve IVP (2.7) with (η`,γ`) = (η∗`,γ∗`) and denote the solutions by y∗`

j (t) and p∗`j (t).

5: Use y∗`j (t) and p∗`

j (t) in (2.5) to generate an approximation for the solution of the problem

(1.1).6: End Algorithm

Function r = EQF(a,b)Global: M,Q,K`, f `(t), z`(t),g`,h`,w`.1: Solve IVP (2.7) to obtain the approximate values of p`(T ), p`(T ) and y`(T ), where a and

b are considered as initial values for p`(0) and p`(0), respectively.

2: return r =

[Mp(T )

Mp(T ) + κ2 (My (T )−w)

].

End Function

3.1 Computation of POD basis for discretizing ISP

As mentioned in the last section, by using the finite element method with basisfunctions ϕ1(x), . . . , ϕm(x), IBVP (2.2) is discretized to IVP (2.7). For a given

η and γ, let y(t) = [y1(t), . . . ,ym(t)]T and p(t) = [p1(t), . . . ,pm(t)]

T be the ob-tained solutions of IVP (2.7). Thus, in view of (2.5) and (2.6), yh(t,x) and ph(t,x)are approximate solutions for IBVP (2.2) with η(x) = ηh(x) and γ(x) = γh(x).Indeed yh(t,x) and ph(t,x) are approximate solutions of the problem (2.2) in them-dimensional space Xh. It is noted that by increasing m, the accuracy of theobtained yh(t,x) and ph(t,x) is improved but the computational time is increased.In the POD method by combining the basis functions ϕ1(x), . . . , ϕm(x), anotherorthonormal set of functions ψ1(x), . . . , ψ`(x) are constructed that well expressthe main properties of the underlying problem and in addition ` is small in com-parison with m.

To construct ψi`i=1, we need information about the solution of the problem(2.2). For this purpose, at first 0 ≤ t1 < t2 < · · · < tn ≤ T are selected as apartition of [0, T ] and we set

yhj (x) = yh(tj ,x), phj (x) = ph(tj ,x), j = 1, . . . , n. (3.1)

For j = 1, . . . , n, yhj (x) and phj (x) are called snapshots of the state and adjoint

functions, respectively. Note that yh(tj ,x) and ph(tj ,x) are the solutions obtainedby finite element method and from (2.5) we have

yhj (x) =m∑i=1

yi(tj)ϕi(x), phj (x) =m∑i=1

pi(tj)ϕi(x). (3.2)

Let V be the spanned space by all snapshots, i.e.

V = spanyh1 , . . . , yhn, ph1 , . . . , phn ⊆ Xh ⊂ L2(Ω).

8 Z. Sabeh et al.

Clearly, V is a finite dimensional subspace of Xh and if d = dim(V) then d ≤minm, 2n. Thus, there are orthonormal basis sets with cardinality d for space V.In the POD technique, the aim is to find a complete orthonormal basis ψidi=1,such that the mean square error between the snapshots and their orthonormalprojection onto the first ` functions ψi`i=1 is minimized. Mathematically speak-ing, in the POD method, the POD basis functions ψi`i=1 are obtained from thefollowing optimization problem

minψ1,...,ψ`

n∑j=1

αj

∥∥∥yhj −∑i=1

〈yhj , ψi〉ψi∥∥∥2L2(Ω)

+ αj

∥∥∥phj −∑i=1

〈phj , ψi〉ψi∥∥∥2L2(Ω)

s.t. 〈ψi, ψj〉 = δij , 1 ≤ i, j ≤ `,

(3.3)

where, αj , j = 1, . . . , n are the trapezoidal weights [41,13], i.e.

α1 =t2 − t1

2, αj =

tj+1 − tj−1

2, for j = 2, . . . , n− 1, αn =

tn − tn−1

2.

The solution set ψi`i=1 to (3.3) is called a POD basis of rank `. We note that thisoptimization problem is not a classical optimization problem. However, in [39], itis proved that the solution of the optimization problem (3.3) is obtained by solvingthe following eigenvalue problem:

Rnψi :=n∑j=1

αj〈yhj , ψi〉yhj + αj〈phj , ψi〉p

hj = λiψi, i = 1, . . . , `. (3.4)

Note that Rn : Xh → V is a linear, bounded, nonnegative, compact and selfadjointoperator [39]. Thus, there exist nonnegative eigenvalues λ1 ≥ λ2 ≥ · · · ≥ λd > 0and an associated orthonormal set ψidi=1 of eigenfunctions that solve (3.4). It isproved that [13,39,41], the set of first ` eigenfunctions ψ1, . . . , ψ` is the solution ofthe problem (3.3).

For solving (3.4), by noting that ψj ∈ V = spanϕ1, . . . , ϕm, we can expand

ψj`j=1 based on ϕ1, . . . , ϕm as follows:

ψj (x) =m∑i=1

qijϕi (x) , j = 1, . . . , `. (3.5)

Let q ∈ Rm and Q ∈ Rm×` be defined as:

qj = [q1j , . . . , qmj ]T , Q =

q11 · · · q1`......

qm1 · · · qm`

. (3.6)

Note that, qj is the jth column of Q and from (3.5), the components of qj can beinterpreted as the finite element coefficients of ψj(x), for j = 1, . . . , `. By substitut-ing (3.2) and (3.5) in the eigenvalue problem (3.4) and by using of delta Kroneckerproperty of ϕi(x)mi=1 at the nodes of spatial discretization, finally the eigenvalueproblems (3.4) are converted to the following matrix eigenvalue problem:

ZDZTMqj = λjqj , j = 1, . . . , `, (3.7)

An Indirect Shooting Method using Proper Orthogonal Decomposition 9

where

Z = [y(t1), . . . ,y(tn),p(t1), . . . ,p(tn)],

D = diag(α1, . . . , αn, α1, . . . , αn),

(3.8a)

(3.8b)

and M is introduced in (2.8). To efficiently solve the above eigenvalue problemand some numerical consideration, we refer to [39].

By solving the eigenvalue problem (3.7), the coefficients qij are obtained andby using them in (3.5), the POD basis functions are obtained.

3.1.1 On choosing the value of `

In the POD method, to reduce the computational cost, the number of the PODbasis functions ` must be chosen several orders lower than the number of finiteelement basis function m. However, this decreases the accuracy. The crucial ques-tion is how to choose `? Indeed, we need to make a balance between accuracy andefficiency. For this purpose, following [44], we consider the following ratio of theeigenvalues:

ε(`) =Σ`i=1λi

Σdi=1λi(3.9)

By this ratio, we can measure the quality of the approximation. The value ofε(`) will tend to 1 as ` increased to d. So, this value can be used to provide anappropriate choice of ` [9]. Indeed, we have to choose the smallest ` such that ε(`)is still sufficiently close to 1 (e. g. ε(`) ' 0.99).

3.2 The presented POD Galerkin projection method

Suppose that ψi`i=1 be the POD basis functions which are obtained by thementioned procedure. We use this basis to approximate the state y and adjoint pin the space V` = spanψ1, . . . , ψ` as:

y`(t,x) =∑j=1

y`j(t)ψj(x), p`(t,x) =∑j=1

p`j(t)ψj(x), (3.10)

as well as the functions η(x) and γ(x) are approximated as:

η`(x) =∑j=1

η`jψj(x), γ`(x) =∑j=1

γ`jψj(x). (3.11)

Moreover, we set

y`(t) =[y`1(t), . . . ,y``(t)

]T, p`(t) =

[p`1(t), . . . ,p``(t)

]T,

η` =[η`1, . . . , η

``

]T, γ` =

[γ`1, . . . , γ

``

]T.

(3.12a)

(3.12b)

10 Z. Sabeh et al.

By substituting (3.10) and (3.11) in the weak form of (2.2), finally we obtain thefollowing IVP:

y`(t) + K`y`(t) + N`1(t) = f `(t)− p`(t),

p`(t) + K`p`(t) + N`2(t) = κ1

[y`(t)− z`(t)

],

y`(0) = g`, y`(0) = h`,

p`(0) = η`, p`(0) = γ`,

(3.13a)

(3.13b)

(3.13c)

(3.13d)

where [K`]ij

= 〈∇ψj ,∇ψi〉,[f `(t)

]i

= 〈ψi, f〉,[z`(t)

]i

= 〈ψi, z〉, [g`]i = 〈ψi, g〉, [h`]i = 〈ψi, h〉,[N`

1(t)]i

= 〈ψi,N1(y`)〉,[N`

2(t)]i

= 〈ψi,N2(y`, p`)〉.

(3.14a)

(3.14b)

(3.14c)

In the same way, the terminal condition (2.3) is discretized to:

F`(η`,γ`) =

[p`(T )

p`(T ) + κ2

(y` (T )−w`

)] = 0, (3.15)

where

[w`]i = 〈ψi, w〉. (3.16)

In short, by the POD Galerkin projection method, ISP is discretized to the fol-lowing finite shooting problem

POD-Shooting Problem (POD-SP): Find η`,γ` ∈ R` such that the solution

of the IVP (3.13) satisfies the terminal condition (3.15).

In order to assemble IVP (3.13), we need to compute the matrix and vectors in(3.14) and (3.16). However, note that obtaining the matrix and vectors in (3.14)and (3.16) by the inner product is not straightforward. Since in contrast to thefinite element basis functions, POD basis functions typically have global supportand are not cardinal. However, it can be seen that there exist the following rela-tion between the finite element matrices (2.8) and (2.10) and matrices in (3.14a),(3.14b) and (3.16):

K` = QTKQ, f `(t) = QT f(t), z`(t) = QT z(t),

g` = QTg, h` = QTh, w` = QTw,

(3.17a)

(3.17b)

where Q is defined in (3.6). Thank the above relations, the matrix K` and vectorsf `(t), z`(t), g`, h` and w` can be obtained by simple matrix multiplication. How-ever, for calculating N`

1(t) and N`2(t), we need more considerations. In general,

the inner products (3.14c), for computing N`1(t) and N`

2(t), cannot be expressed interms of finite element matrices (2.8) and (2.10) and are commonly approximatedby numerical quadratures such as Gauss quadrature. However, in [43] a more effi-cient approach for computation of the inner products in (3.14c) is proposed. This

An Indirect Shooting Method using Proper Orthogonal Decomposition 11

approach uses an interpolation operator Ih : C(Ω)→ Xh relative to the nodes offinite element discretization and approximate N1 and N2 as:

IhN1 (y (t,x)) =m∑i=1

N1 (y (t,xi))ϕi(x),

IhN2 (y (t,x) , p (t,x)) =m∑i=1

N2 (y (t,xi) , p (t,xi))ϕi(x).

(3.18)

(3.19)

This approximation also is known as the product approximation technique [5] orgroup finite element method [11]. For evaluation of the inner products in (3.14c),the nonlinear functions Ni is replaced by IhNi for i = 1, 2. For instance, the ithrow of nonlinear terms N`

1(t) is approximated as:

[N`

1(t)]i≈⟨ψi(x), IhN1(y`(t,x))

⟩=

⟨m∑k=1

qkiϕk(x),m∑j=1

N1(y`(t,xj))ϕj(x)

⟩

=m∑k=1

m∑j=1

qki

[N1(Qy`(t))

]j

⟨ϕk(x), ϕj(x)

⟩= qTi MN1(Qy`(t)),

(3.20)

so we have:

N`1(t) ≈ QTMN1(Qy`(t)). (3.21)

Similarly, we can get:

N`2(t) ≈ QTMN2(Qy`(t),Qp`(t)) (3.22)

In summary, by using (3.17), (3.21) and (3.22), we can assemble IVP (3.13) toutilize it in the POD-SP. If (η∗`,γ∗`) be the solution of POD-SP, then let y∗`j (t)

and p∗`j (t) be the obtained solutions of IVP (3.13) with η` = η∗` and γ` = γ∗`.Now, from (3.10) and (2.1i), we can obtain the following approximations for thecontrol and state functions

u(t,x) '∑j=1

p∗`j (t)ψj(x), (3.23)

y(t,x) '∑j=1

y∗`j (t)ψj(x). (3.24)

In the final, the presented POD method in this section is summarized in Algorithm2.

12 Z. Sabeh et al.

Algorithm 2 : POD-Indirect Shooting (POD-IS) Algorithm

Inputs:m ∈ N as the number of finite element basis.n ∈ N as the number of snapshots.ηs,γs ∈ Rm as initial conditions for IVP (2.7) in order to generate snapshots.ε > 0 for estimating the appropriate number of the POD basis.

1: Begin2: Compute matrices M,K and vectors f(t), z(t),g,h,w as (2.8) and (2.10).3: Solve IVP (2.7) with initial conditions ηs, γs and denote the solutions by y(t) and p(t).4: Construct the snapshot matrix Z and diagonal matrix D as (3.8).5: Set d = rank(Z), solve the matrix eigenvalue problem (3.7) and obtain q1, . . . ,qd.6: Use (3.9) to find the smallest integer number ` such that ε(`) < 1− ε.7: Use (3.17) to construct matrix K` and vectors f `(t), z`(t),g`,h`,w`.8: Solve the system of equations EQF(η`,γ`) = 0, by a solver and denote the solution by

(η∗`,γ∗`).9: Solve IVP (3.13) with (η`,γ`) = (η∗`,γ∗`) and using (3.21) and (3.22). Denote the solu-

tions by y∗`j (t) and p∗`

j (t).

10: Use y∗`j (t) and p∗`

j (t) in (3.23) and (3.24) to generate an approximation for the solution

of the problem (1.1).11: End Algorithm

Function r = EQF(a,b)Global: M,Q,K`, f `(t), z`(t),g`,h`,w`.1: Solve IVP (3.13) to obtain the approximate values of p`(T ), p`(T ) and y`(T ), where

I a and b are considered as initial values for p`(0) and p`(0),I For evaluation of N`

1(t) and N`2(t), Eqs. (3.21) and (3.22) are used.

2: return r =

[p`(T )

p`(T ) + κ2(y` (T )−w`

)] .End Function

4 Discrete Empirical Interpolation Method(DEIM)

In Algorithm 2, a POD method for solving the problem (1.1) is presented. In Line8 of this algorithm, for solving EQF(·, ·) = 0, a nonlinear equation solver is utilizedwhich generally uses an iterative method. In each iteration of the solver, functionEQF is evaluated at least one time, so the complexity of the evaluation of thisfunction is crucial in our method.

In EQF, the IVP (3.13) is solved by an appropriate time stepping scheme andin the nonlinear cases, in each time step, we need to compute N`

1(t) and N`2(t).

For obtaining the complexity of the evaluation of N`1(t), let the complexity for

evaluating the nonlinear function N1 with q components be O(%(q)), where % issome function of q. Thus, in each time step, the complexity for computing N`

1(t)by (3.21) is O(%(m) + 4m`). This means that the complexity of evaluation N`

1(t)depends on m. Moreover, the complexity of evaluation of N`

2(t) depends on m too.

By noting that, m is the number of the finite element basis functions and gener-ally is a large number, thus in the presence of the nonlinear term N1, the evaluationof function EQF is very time-consuming. As a result, the proposed Algorithm 2 isless efficient in the presence of nonlinearity. To overcome this drawback, we uti-lize DEIM, which is an efficient technique, with low computational complexity, forapproximating the nonlinear functions [3,4].

An Indirect Shooting Method using Proper Orthogonal Decomposition 13

In the follows, we use DEIM to present efficient formulas for approximatingN`

1(t) and N`2(t). For this purpose, we use the POD technique to construct a new

basis for approximating the nonlinear functions N1 and N2. We define:

n1(t) = N1(Qy`(t)). (4.1)

For approximation n1(t), at first we construct the POD basis φ11, ..,φ

1r1 ⊂ Rm

corresponding to the following snapshot space:

V1 = spanN1(y(t1)), . . . ,N1(y(tn)) ⊂ Rm, (4.2)

where r1 m denotes the size of the reduced representation of n1(t). For con-structing the POD basis φ1

1, ..,φ1r1, similar to the subsection 3.1, we consider the

following optimization problem:min

φ11,...,φ

1r1

n∑j=1

αj

∥∥∥N1(y(tj))−r1∑i=1

〈N1(y(tj)),φ1i 〉Mφ1

i

∥∥∥2M

s.t. 〈φ1i ,φ

1j 〉M = δij 1 ≤ i, j ≤ r1

(4.3)

in which the notations 〈·, ·〉M and ‖ · ‖M are the weighted inner product and thecorresponding weighted norm respectively, which are defined as:

〈u,v〉M = uTMv ‖u‖M = (〈u,u〉M)1/2 for u,v ∈ Rm. (4.4)

The optimization problem (4.3) can be reformulated as the following eigenvalueproblem [26,39]:

E1DET1 Mφ1i = λiφ

1i , (4.5)

where

E1 = [N1(y(t1)), . . . ,N1(y(tn))] ,

D = diag(α1, . . . , αn),

(4.6a)

(4.6b)

and M is defined in (2.8a). By using the constructed POD basis φ11, ..,φ

1r1, we

can approximate n1(t) by the following expansion:

n1(t) = Φ1c1(t). (4.7)

where Φ1 =[φ11, ..,φ

1r1

]∈ Rm×r1 and c1(t) =

[c11(t), . . . , c1r1(t)

]T ∈ Rr1 is theunknown corresponding coefficient vector. To evaluate the approximation (4.7),we need to obtain c1(t). DEIM algorithm selects r1 rows from the overdeterminedsystem (4.7) to specify c1(t). The row indices ρ11, . . . , ρ1r1, which are used fordetermining the coefficient vector c1(t), are selected inductively from the basisφ11, . . . ,φ

1r1 by the DEIM algorithm. For use of this algorithm we refer to [26] and

assume that ρ11, . . . , ρ1r1 are the obtained row indices. We consider correspondingmatrix P1 as:

P1 = [eρ11 , . . . , eρ1r1] ∈ Rm×r1 , (4.8)

14 Z. Sabeh et al.

where eρ1iis the ρ1i th column of the identity matrix Im ∈ Rm×m, i = 1, . . . , r1. By

multiplying (4.7) with PT1 , r1 rows of it are selected and with assumption PT1 Φ1

is invertible we have :

c1(t) = (PT1 Φ1)−1PT1 n1(t) = (PT1 Φ1)−1P1TN1(Qy`(t)). (4.9)

We note that the nonlinear functions N1 and N2 are evaluated componentwise attheir input vectors, thus the matrix PT1 can be moved into the nonlinear function:

c1(t) = (PT1 Φ1)−1N1(PT1 Qy`(t)). (4.10)

Now, we have everything to approximate N`1(t). By (3.21) and (4.7) we have

N`1(t) = QTMΦ1c1(t), (4.11)

and from (4.10), we get

N`1(t) ≈ Nr1

1 (t) = A1N1

(B1y`(t)

), (4.12)

where

A1 = QTMΦ1(PT1 Φ1)−1 ∈ R`×r1 and B1 = PT1 Q ∈ Rr1×`. (4.13)

We note that the matrices A1 and B1 can be precomputed and the nonlinearfunction N1 at each time step has to be computed at a r1-dimensional vector,that r1 m. More precisely, the corresponding computational complexity of thenonlinear term Nr1

1 (t) at each time step of IVP (3.13) is only O(%(r1) + 4r1`). Forapproximation N`

2(t) in (3.13), we follow the same approach as N`1(t) and finally,

we get:

N`2(t) ≈ Nr2

2 (t) = A2N2

(B2y`(t),B2p`(t)

), (4.14)

where

A2 = QTMΦ2(PT2 Φ2)−1 ∈ R`×r2 and B2 = PT2 Q ∈ Rr2×`. (4.15)

In summary, for utilizing the DEIM in the POD indirect shooting method, weneed to use the DEIM approximations (4.12) and (4.14) instead of approximations(3.21) and (3.22). Thus the shooting problem with POD/DEIM discretization canbe written as:

POD/DEIM-Shooting Problem (POD/DEIM-SP): Find η`,γ` ∈ R` such

that the solutions y`(t),p`(t) : [0, T ] → R` of the IVP (3.13), by considering

approximations (4.12) and (4.14), satisfy the terminal condition (3.15).

As a result, we have a POD/DEIM indirect shooting method that the compu-tational complexity of solving it is independent of the full dimension m. For thereader convenience, we summarize the POD/DEIM indirect shooting method inAlgorithm 3.

An Indirect Shooting Method using Proper Orthogonal Decomposition 15

Algorithm 3 : POD/DEIM-Indirect Shooting (POD/DEIM-IS) Algorithm

Input:m ∈ N as the number of finite element basis.n ∈ N as the number of snapshots.ηs,γs ∈ Rm as initial conditions for IVP (2.7) in order to generate snapshots.ε0 > 0 for estimating the appropriate number of POD basis.ε1 > 0 and ε2 > 0 for estimating the appropriate number of DEIM basis.

1: Begin2: Compute matrices M,K and vectors f(t), z(t),g,h,w as (2.8) and (2.10).3: Solve IVP (2.7) by the given initial conditions ηs, γs and call the solution as y(t) and

p(t).4: Construct the snapshot matrix Z and diagonal matrix D as (3.8).

5: Construct the snapshot matrix E1 and diagonal matrix D as (4.6).6: Construct the snapshot matrix E2 = [N2(y(t1),p(t1)), . . . ,N2(y(tn),p(tn))].7: Set d = rank(Z), solve the matrix eigenvalue problem (3.7) and obtain q1, . . . ,qd.8: Set d1 = rank(E1), solve the matrix eigenvalue problem (4.5) and obtain φ1

1, ..,φ1d1

.

9: Set d2 = rank(E2), solve the matrix eigenvalue problem E2DET2 Mφ2

i = λiφ2i and obtain

φ21, ..,φ

2d2

.

10: Use (3.9) to find the smallest integer numbers `, r1 and r2 such that ε(`) < 1− ε, ε(r1) <1− ε1 and ε(r2) < 1− ε2.

11: Set Q = [q1, . . . ,q`], Φ1 =[φ1

1, ..,φ1r1

]and Φ2 =

[φ2

1, ..,φ2r2

].

12: Determine row indices ρ11, . . . , ρ1r1 for φ1i r1i=1 and ρ21, . . . , ρ2r2 for φ2i

r2i=1 by DEIM

algorithm [26].13: Compute matrices A1,B1 by (4.13) and A2,B2 by (4.15).14: Use (3.17) to construct matrix K` and vectors f `(t), z`(t),g`,h`,w`.15: Solve the system of equations EQF(η`,γ`) = 0, by a solver and denote the solution by

(η∗`,γ∗`).16: Solve IVP (3.13) with (η`,γ`) = (η∗`,γ∗`) and using (4.12) and (4.14). Denote the solu-

tions by y∗`j (t) and p∗`

j (t).

17: Use y∗`j (t) and p∗`

j (t) in (3.23) and (3.24) to generate an approximation for the solution

of the problem (1.1).18: End Algorithm

Function r = EQFD(a,b)Global: A1,B1,A2,B2,K`, f `(t), z`(t),g`,h`,w`.1: Solve IVP (3.13) to obtain the approximate values of p`(T ), p`(T ) and y`(T ), where

I a and b are considered as initial values for p`(0) and p`(0),I For evaluation of N`

1(t) and N`2(t), Eqs. (4.12) and (4.14) are used.

2: return r =

[p`(T )

p`(T ) + κ2(y` (T )−w`

)] .End Function

5 Numerical results

This section is devoted to numerical experiments. We implemented the proposedAlgorithms 1, 2 and 3 with Matlab on a personal computer for three examples.In our implementations, we used the Matlab Pde Toolbox, for the finite elementspatial discretization with piecewise linear basis function [25]. Furthermore, theMatlab function ode45 was utilized for solving IVPs and for solving the systemof equations, the Matlab function fsolve was used.

The underlying algorithm of ode45 makes use of 4/5 Runge-Kutta integrationwith a variable time step. Furthermore, ode45 controls the error by two parametersRelTol and AbsTol. By these parameters, we can adjust the relative and absolute

16 Z. Sabeh et al.

error tolerances [33]. In our simulations, the error tolerances are set to RelTol =10−6 and AbsTol = 10−8. These considerations guarantee that the relative andabsolute errors in satisfying IVPs are less than 10−6 and 10−8, respectively.

As we said, we used fsolve to solve the nonlinear systems of equations. Usingthis solver, we can adjust the accuracy of the solution based on the two parame-ters TolFun and TolX, where the former specifies the termination tolerance on thefunction value and the later specifies the termination tolerance on the variables[32]. In our simulations, we continued the iteration of fsolve until TolFun and TolX

become less than 10−12.As numerical examples, we considered three examples, which the first two

examples are linear with exact solution [27] and the third example is nonlinearwithout exact solution [47]. In the cases that the exact solution is known, to assessthe accuracy of the methods, the following relative errors are reported

Em(y) =‖y∗ − ym‖L2(Q)

‖y∗‖L2(Q), Em(p) =

‖p∗ − pm‖L2(Q)

‖p∗‖L2(Q),

where y∗ and p∗ are the exact solutions and ym and pm are the approximatedsolutions corresponding to the discretization parameter m.

In addition, to further assess the quality of the obtained solutions, the RootMean-Square-Error (RMSE) and the Correlation coefficient (Corr) [45] are re-ported. For instance, let xi, i = 1, · · · ,m be the spatial nodes, tj be a time leveland

y∗j =[y∗(tj ,x1), · · · , y∗(tj ,xm)

], yj =

[y`(tj ,x1), · · · , y`(tj ,xm)

],

where y∗ is the exact solution and y` denotes the approximated solutions by thePOD-IS method. The RMSE and Corr between the exact state and the obtainedstate by the POD-IS method at time level j is defined as

RMSEy(y∗j , yj) =‖y∗j − yj‖2√

m,

Corry(y∗j , yj

)=

cov(y∗j , y∗j

)σy∗jσyj

,

where, “cov” denotes covariance and σ denotes standard deviation.In a similar manner, the RMSE and Corr between the obtained state or adjointfunctions by the POD-IS and POD/DEIM-IS at time level j can be defined.

5.1 Example 1

Let Ω = (0, 1), κ2 = 0, T = 1 and N1(y) = 0. Choose g(x) = sin(πx), h(x) = 0,f = −κ1sin(πx)(t−T )2 and z = 2sin(πx)+π2sin(πx)(t−T )2 +sin(πx)cos(πt). Theexact solution is y∗(t, x) = sin(πx)cos(πt) and p∗(t, x) = −κ1sin(πx)(t− T )2.

For solving this example, the IS and POD-IS methods are used. In the POD-IS method, the initial conditions ηs = γs = 0 are considered and 400 snapshots(n = 200) are generated from the obtained solution and POD basis is constructedby selecting ε = 0.001.

An Indirect Shooting Method using Proper Orthogonal Decomposition 17

In Table 1, we compare the relative errors and CPU time in the IS and POD-IS methods, for various values of κ1 and m. In this table, “POD setup” refers tothe computation time in the lines 2 to 7 of Algorithm 2 and “SUM” refers to theoverall time of the algorithm.

It is seen that the CPU time of the POD-IS method is much time smaller thanthe CPU time of the IS method, whereas, there is no significant difference betweenthe accuracy of two methods. Moreover, as m increases, the CPU time for solvingthe POD-SP algorithm is decreased. This is because that, by increasing m, theproblem is solved more accurately and the resulted snapshots can more accuratelycapture the solution space of the equation (2.2). Consequently, the more suitablePOD basis is obtained, which help to speed up the solution time of the POD-SPalgorithm.

We note that, for m = 400 the IS method, is not able to solve the problemwithin an acceptable computational time, whereas, the POD-IS method yields anaccurate solution in a reasonable CPU time.

Parameters Indirect Shooting Method POD Indirect Shooting Methodof Methods Error CPU Time Error CPU Timeκ1 m Em(y) Em(p) FSP Em(y) Em(p) POD setup POD-SP SUM

10 50 5.3e-4 2.5e-4 476 5.3e-4 2.5e-4 0.47 2.9 3.410 100 1.3e-4 6.4e-5 9724 1.3e-4 6.4e-5 1.9 2.7 4.610 200 3.3e-5 1.6e-5 102897 3.3e-5 1.6e-5 10.6 2.5 13.110 400 - - - 8.3e-6 4.1e-6 53 2.5 55.5

100 50 6.8e-4 2.9e-4 4055 6.8e-4 2.9e-4 0.36 3.1 3.5100 100 1.7e-4 7.3e-5 12342 1.7e-4 7.3e-5 1.3 3 4.3100 200 4.2e-5 1.8e-5 120112 4.2e-5 1.8e-5 6.2 2.8 9100 400 - - - 1.3e-5 7.1e-6 39 2.6 41.6

Table 1 (Example 1) Comparison of error and CPU time for IS method versus POD-ISmethod.

For further analyzing the obtained state and adjoint functions by the POD-ISmethod, in Figure 1, the RMSE and Corr between the exact and POD-IS solutionare plotted for various values of m. It can be seen that by increasing m, the RMSEbecomes sufficiently small and the correlation coefficient approaches to 1. Thismeans that, when m increases, the obtained POD-IS solutions are in more closeagreement with the exact solutions.

5.2 Example 2

Let Ω = (0, 1)2, κ2 = 0, T = 1, N1(y) = 0 and

g(x1, x2) = sin(πx1)sin(πx2), h(x1, x2) = sin(πx1)sin(πx2),

f = (1 + 2π2)etsin(πx1)sin(πx2)− κ1(t− T )2sin(πx1)sin(πx2),

z = (et + 2 + 2π2(t− T )2)sin(πx1)sin(πx2).

This problem has the following exact solution

y∗(t, x) = etsin(πx1)sin(πx2), and p∗(t, x) = −κ1(t− T )2sin(πx1)sin(πx2).

18 Z. Sabeh et al.

0 50 100 150 200

Time steps

10-8

10-6

10-4

10-2

RM

SE

y

m = 50

m = 100

m = 200

m = 400

0 50 100 150 200

Time steps

10-10

10-8

10-6

10-4

10-2

RM

SEp

m = 50

m = 100

m = 200

m = 400

(a) The RMSE of (left) state y and (right) adjoint p between the exact solution and POD-ISsolutions for various values of m (RMSE is plotted on logarithmic scale).

0 50 100 150 200

Time steps

0.99999999999985

0.99999999999990

0.99999999999995

1.00000000000000

1.00000000000005

C

orry

m = 50

m = 100

m = 200

m = 400

0 50 100 150 200

Time steps

0.99999999999980

0.99999999999985

0.99999999999990

0.99999999999990

1.00000000000000

1.00000000000005

Co

rr

p

m = 50

m = 100

m = 200

m = 400

(b) Correlation of coefficients of (left) state y and (right) adjoint p between the exact solutionand POD-IS solutions for various values of m.

Fig. 1 (Example 1 with κ1 = 10) The RMSE and Corr between the exact and POD-ISsolutions.

The results of applying the IS and POD-IS methods, with ηs = γs = 0, n = 200and ε = 0.001, are reported in Table 2. As we see in this table, for m = 177, thePOD technique significantly reduces the CPU time. Furthermore, for m ≥ 2577,the POD-IS method get the solution in a reasonable time but the IS method isnot able to solve the problem within an acceptable computational time.

Parameters Indirect Shooting Method POD Indirect Shooting Methodof Methods Error CPU Time Error CPU Timeκ1 m Em(y) Em(p) FSP Em(y) Em(p) POD setup POD-SP SUM

10 177 8.3e-3 5e-3 1893 8.3e-3 5e-3 3.7 2.5 6.210 2577 - - - 5.2e-4 3.1e-4 20.4 1.8 22.210 10145 - - - 1.3e-4 7.7e-5 63 1.7 64.710 40257 - - - 3.3e-5 1.9e-5 286 1.6 287.6

100 177 1e-2 4.5e-3 2751 1.1e-2 4.6e-3 3.7 2.1 5.8100 2577 - - - 6.8e-4 2.8e-4 20.7 2 22.7100 10145 - - - 1.7e-4 7.1e-5 64 1.8 65.8100 40257 - - - 4.2e-5 1.8e-5 283 1.7 284.7

Table 2 (Example 2) Comparison of error and CPU time for IS method versus POD-ISmethod.

Figure 2 shows the RMSE and Corr between the exact solution and the ob-tained POD-IS solutions for various values of m. It can be seen that improving

An Indirect Shooting Method using Proper Orthogonal Decomposition 19

0 50 100 150 200

Time steps

10-6

10-5

10-4

10-3

10-2

RM

SEy

m = 665

m = 2577

m = 10145

m = 40257

0 50 100 150 200

Time steps

10-10

10-8

10-6

10-4

10-2

RM

SEp

m = 665

m = 2577

m = 10145

m = 40257

(a) The RMSE of (left) state y and (right) adjoint p between the exact solution and POD-ISsolution by various values of m. (RMSE is plotted on logarithmic scale)

0 50 100 150 200

Time steps

0.9999970

0.9999975

0.9999980

0.9999985

0.9999990

0.9999995

1.0000000

1.0000005

Co

rr

y

m = 665

m = 2577

m = 10145

m = 40257

0 50 100 150 200

Time steps

0.999990

0.999992

0.999994

0.999996

0.999998

1.000000

1.000002

Co

rrp

m = 665

m = 2577

m = 10145

m = 40257

(b) Correlation of coefficients of (left) state y and (right) adjoint p between the exact solutionand POD-IS solution for various values of m.

Fig. 2 (Example 2 with κ1 = 10) RMSE and Corr between the exact solution and POD-ISsolution.

the quality of POD basis (by increasing m) leads to improvement in the accuracyof the POD-IS solutions.

5.3 Example 3

In the problem (1.1), let

Ω = (0, 1)2, T = 1, N1(y) = y3 − y, g(x1, x2) = sin(πx1)sin(πx2), h = 0,

z(t, x1, x2) = cos(πt)sin(πx1)sin(πx2), w(x1, x2) = cos(π)sin(πx1)sin(πx2).

This problem is nonlinear and we test the efficiency of the POD/DEIM-IS algo-rithm in comparison with the POD-IS algorithm. In all simulations, we select thefollowing input parameters

n = 200, ηs = γs = 0 and ε = ε1 = ε2 = 0.001.

By applying Algorithms 3 with m = 2577 on this example with κ1 = κ2 = 10,we get ` = 5 and r1 = r2 = 15. The obtained state and adjoint functions at timelevels t = 0.3s, t = 0.6s and t = 1.0s are plotted in Figure 3. Moreover, the first 40points selected by the DEIM for N1 and N2 are illustrated in Figure 4.

20 Z. Sabeh et al.

yℓ(0.3,x)

xy

0

1

0.05

0.1

1

0.15

0.2

0.5

0.25

0.5

0 0

yℓ(0.6,x)

xy

-1

1

-0.8

-0.6

0.81

-0.4

0.6

-0.2

0.4

0

0.5

0.2

0 0

yℓ(1.0,x)

x

y

-0.5

1

-0.4

-0.3

0.81

-0.2

0.6

-0.1

0.4

0

0.5

0.2

0 0

(a) The POD-DEIM-IS solution of state variable at time instances 0.3s, 0.6s and 1s.

pℓ(0.3,x)

xy

-0.3

1

-0.25

-0.2

0.81

-0.15

-0.1

0.6

-0.05

0.4

0

0.5

0.2

0 0

y

pℓ(0.6,x)

x

0

1

0.2

0.4

1

0.6

0.8

0.5

1

0.5

0 0

pℓ(1.0,x)

yx

-2

1

-1

0.8

0

1

×10-9

1

0.6

2

0.4 0.5

0.2

0 0

(b) The POD-DEIM-IS solution of adjoint variable at time instances 0.3s, 0.6s and 1s.

Fig. 3 (Example 3 with κ1 = κ2 = 10) The POD/DEIM-IS solution with m = 2577.

0 0.2 0.4 0.6 0.8 1

x1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

x2

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

1718

19

20

21

22

23

24

25

26

2728

29

30

31

32

33

34

35

36

37

38

39

40

0 0.2 0.4 0.6 0.8 1

x1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

x2

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19 20

21 22

23

24

25

26

27

28

2930

31

32

33

34

35

36

37

38

39

40

Fig. 4 (Example 3 with κ1 = κ2 = 10) The first 40 points selected by DEIM for the nonlinearfunctions (left) N1 and (right) N2, with m = 2577.

An Indirect Shooting Method using Proper Orthogonal Decomposition 21

0 50 100 150 200

Time steps

10-10

10-8

10-6

10-4

RM

SEy

r = 2

r = 15

r = 30

0 50 100 150 200

Time step

10-15

10-10

10-5

100

RM

SEp

r = 2

r = 15

r = 30

(a) The RMSE of (left) state y and (right) adjoint p between POD-IS solution and POD/DEIM-IS solution by various values of r1 = r2 = r. (RMSE is plotted in logarithmic scale)

0 50 100 150 200

Time steps

0.99996

0.99997

0.99998

0.99999

1.00000

Co

rr

y

r = 2

r = 15

r = 30

0 50 100 150 200

Time steps

0.9996

0.9997

0.9998

0.9999

1.0000

Co

rrp

r = 2

r = 15

r = 30

(b) Correlation of coefficients of (left) state y and (right) adjoint p between POD-IS solutionand POD/DEIM-IS solution by various values of r1 = r2 = r.

Fig. 5 (Example 3 with κ1 = κ2 = 10) RMSE and Corr between the POD-IS and POD-IS-DEIM solutions with m = 2577.

In this example, the exact solutions are unknown. So, for assessing the accuracyof POD/DEIM-IS algorithm, the RMSE and correlation coefficient between POD-IS solutions and POD/DEIM-IS solutions are reported in Figure 5. We see, byincreasing the number of basis for approximation of the nonlinear functions N1

and N2, the RMSE is decreased. This point is also highlighted by the correlationcoefficient.

In Table 3, for various values of m and κ, the obtained values of the cost func-tional and CPU times, with POD-IS and POD/DEIM-IS methods, are summarizedfor comparison. In this table, “ Setup Time” for POD-IS (POD-DEIM-IS) refersto computation time for the steps 1 to 7 of Algorithm 2 (steps 1 to 14 of Algorithm3.)

According to Table 3, we see that, the obtained cost values by POD-IS andPOD/DEIM-IS methods are very close to each other, moreover, in the two algo-rithms, the setup steps take almost the same CPU time. But the CPU time forPOD/DEIM-IS algorithm is smaller than POD-SP algorithm and for larger m thisdifference is more significant.

On the other hand, it can be observed that by increasing m, the CPU timeof POD-SP is increased. This is because that the computational complexity ofPOD-SP is dependent to m. In contrast, by increasing m, the computational timeof POD/DEIM-IS algorithm not only does not increase but actually decreases.The reason lies in the fact that, the computational complexity of POD/DEIM-IS

22 Z. Sabeh et al.

Parameters POD-IS POD-DEIM-ISof Methods J (y, u)

CPU Time J (y, u)CPU Time

κ m Setup Time POD-SP Setup Time POD/DEIM-SP

10 177 0.615689 7.5 5.05 0.615693 7.6 4.310 665 0.601846 27.4 5.3 0.601852 27.6 3.510 2577 0.583256 77.5 7.4 0.583256 77.9 2.710 10145 0.581629 362.3 13.5 0.581630 363.5 2.6310 40257 0.581222 1887 41.9 0.581222 1890 2.61

102 177 2.463484 7.5 6.7 2.463496 7.6 5.8102 665 2.373453 27.4 7.3 2.373487 27.6 5.3102 2577 2.350955 77.5 8.5 2.350957 77.9 3.7102 10145 2.345315 362.3 19.3 2.345315 363.5 3.2102 40257 2.343904 1887 61.2 2.343904 1890 3.2

103 177 3.995542 7.5 7 3.995553 7.6 6.1103 665 3.863917 27.4 8 3.863959 27.6 5.8103 2577 3.832058 77.5 11.7 3.832059 77.9 5.2103 10145 3.824125 362.3 23.5 3.824128 363.5 3.8103 40257 3.822150 1887 70.3 3.822151 1890 3.8

104 177 4.664583 7.5 9.3 4.664720 7.6 8.5104 665 4.449323 27.4 10 4.449358 27.6 8104 2577 4.408258 77.5 16.4 4.408278 77.9 7.4104 10145 4.398990 362.3 38.6 4.398991 363.5 6.3104 40257 4.396715 1887 116 4.396715 1890 5.7

Table 3 (Example 3) Comparison between the obtained value of cost functional and CPUtime, with POD-IS and POD/DEIM-IS methods, for various values of m and κ.

algorithm is independent of m. Moreover, as m increases, we can obtain more suit-able snapshots and thus more efficient POD and POD/DEIM basis are provided.Accordingly, the solution of POD/DEIM-IS can be obtained in less computationaltime.

6 Conclusion

In this paper, we utilized the POD technique in the indirect shooting method forsolving the optimal control of wave equation. We showed that the POD techniquegreatly improves the ability of the indirect shooting method to obtain solutionsquickly, where sacrifices no significant amount of accuracy. Moreover, in the pres-ence of the nonlinear term in the wave equation, to further speed up the solutiontime, a DEIM strategy is used for reducing the order of nonlinear calculations. Wefind that DEIM shows its impact on the problems, whose needs to a fine spatialdiscretization for obtaining reasonable accuracy. As a result, the reduced-ordermethods cause that the indirect shooting method becomes more applicable andattractive for the considered optimal control problems. We are currently consid-ering the extension of the POD method to the optimal control problems governedby PDE and inequality constraints.

An Indirect Shooting Method using Proper Orthogonal Decomposition 23

References

1. Banks, H.T., Keeling, S.L., Silcox, R.J.: Optimal control techniques for active noise sup-pression. Proceedings of the IEEE Conference on Decision and Control pp. 2006–2011(1988)

2. Betts, J.T.: Survey of numerical methods for trajectory optimization. Journal of Guidance,Control, and Dynamics 21(2), 193–207 (1998)

3. Chaturantabut, S., Sorensen, D.C.: Nonlinear model reduction via discrete empirical in-terpolation. SIAM Journal on Scientific Computing 32(5), 2737–2764 (2010)

4. Chaturantabut, S., Sorensen, D.C.: Application of POD and DEIM on dimension reductionof non-linear miscible viscous fingering in porous media. Mathematical and ComputerModelling of Dynamical Systems 17(4), 337–353 (2011)

5. Christie, I., Griffiths, D.F., Mitchell, A.R., Sanz-serna, J.M.: Product approximation fornon-linear problems in the finite element method. IMA Journal of Numerical Analysis1(3), 253–266 (1981)

6. Clason, C., Kaltenbacher, B., Veljovic, S.: Boundary optimal control of the westerveltand the kuznetsov equations. Journal of Mathematical Analysis and Applications 356(2),738–751 (2009)

7. Stefanescu, R., Navon, I.M.: POD/DEIM nonlinear model order reduction of an ADIimplicit shallow water equations model. Journal of Computational Physics 237, 95–114(2013)

8. Stefanescu, R., Sandu, A., Navon, I.M.: POD/DEIM reduced-order strategies for efficientfour dimensional variational data assimilation. Journal of Computational Physics 295,569–595 (2015). DOI 10.1016/j.jcp.2015.04.030

9. Du, J., Fang, F., Pain, C.C., Navon, I.M., Zhu, J., Ham, D.A.: POD reduced-order un-structured mesh modeling applied to 2D and 3D fluid flow. Computers and Mathematicswith Applications 65(3), 362–379 (2013)

10. Ervedoza, S., Zuazua, E.: Numerical Approximation of Exact Controls for Waves. Springer-Briefs in Mathematics. Springer New York (2013)

11. Fletcher, C.A.J.: The group finite element formulation. Computer Methods in AppliedMechanics and Engineering 37(2), 225–244 (1983)

12. Gerdts, M., Greif, G., Pesch, H.J.: Numerical optimal control of the wave equation: opti-mal boundary control of a string to rest in finite time. Mathematics and Computers inSimulation 79(4), 1020–1032 (2008)

13. Gubisch, M., Volkwein, S.: Proper orthogonal decomposition for linear-quadratic optimalcontrol. Tech. rep. (2013)

14. Gugat, M.: Penalty techniques for state constrained optimal control problems with thewave equation. SIAM Journal on Control and Optimization 48(5), 3026–3051 (2010)

15. Gugat, M., Keimer, A., Leugering, G.: Optimal distributed control of the wave equationsubject to state constraints. ZAMM Zeitschrift fur Angewandte Mathematik und Mechanik89(6), 420–444 (2009)

16. Heinkenschloss, M.: A time-domain decomposition iterative method for the solution ofdistributed linear quadratic optimal control problems. Journal of Computational andApplied Mathematics 173(1), 169–198 (2005)

17. Hesse, H.K.: Multiple shooting and mesh adaptation for PDE constrained optimizationproblems. Ph.D. thesis

18. Kammann, E., Troltzsch, F., Volkwein, S.: A posteriori error estimation for semilinearparabolic optimal control problems with application to model reduction by POD. Math-ematical Modelling and Numerical Analysis 47(2), 555–581 (2013)

19. Kroner, A.: Adaptive finite element methods for optimal control of second order hyperbolicequations. Computational Methods in Applied Mathematics 11(2), 214–240 (2011)

20. Kroner, A.: Numerical methods for control of second order hyperbolic equations. Ph.D.thesis, Fakultat fur Mathematik, Technische Universitat Munchen (2011)

21. Kroner, A., Kunisch, K., Vexler, B.: Semismooth newton methods for optimal control ofthe wave equation with control constraints. SIAM Journal on Control and Optimization49(2), 830–858 (2011)

22. Kunisch, K., Reiterer, S.H.: A gautschi time-stepping approach to optimal control of thewave equation. Applied Numerical Mathematics 90, 55–76 (2015)

23. Kunisch, K., Volkwein, S.: Galerkin proper orthogonal decomposition methods forparabolic problems. Numerische Mathematik 90(1), 117–148 (2001)

24 Z. Sabeh et al.

24. Lagnese, J.E., Leugering, G.: Time-domain decomposition of optimal control problems forthe wave equation. Systems and Control Letters 48(3-4), 229–242 (2003)

25. Langemyr, L.: Partial Differential Equation Toolbox User’s Guide: For Use with MATLAB.Math Works (1996)

26. Lass, O., Volkwein, S.: POD galerkin schemes for nonlinear elliptic-parabolic systems.SIAM Journal on Scientific Computing 35(3), A1271–A1298 (2013)

27. Li, B., Liu, J., Xiao, M.: A fast and stable preconditioned iterative method for optimalcontrol problem of wave equations. SIAM Journal on Scientific Computing 37(6), A2508–A2534 (2015)

28. Lions, J.L.: Optimal Control of Systems Governed by Partial Differential Equations.Springer-Verlag (1971)

29. Lions, J.L.: Control of distributed singular systems. Gauthier-Villars (1985)30. Lions, J.L.: Exact controllability, stabilization and perturbations for distributed systems.

SIAM Review 30(1), 1–68 (1988)31. Luo, X.B., Chen, Y.P., Huang, Y.Q.: A priori error estimates of finite volume element

method for hyperbolic optimal control problems. Science China Mathematics 56(5), 901–914 (2013)

32. Mehrpouya, M.A.: An efficient pseudospectral method for numerical solution of nonlin-ear singular initial and boundary value problems arising in astrophysics. MathematicalMethods in the Applied Sciences 39(12), 3204–3214 (2016)

33. Mehrpouya, M.A., Shamsi, M., Razzaghi, M.: A combined adaptive control parametriza-tion and homotopy continuation technique for the numerical solution of bang-bang optimalcontrol problems. ANZIAM Journal 56(1), 48–65 (2014)

34. Mordukhovich, B.S., Raymond, J.P.: Dirichlet boundary control of hyperbolic equations inthe presence of state constraints. Applied Mathematics and Optimization 49(2), 145–157(2004)

35. Mordukhovich, B.S., Raymond, J.P.: Neumann boundary control of hyperbolic equationswith pointwise state constraints. SIAM Journal on Control and Optimization 43(4), 1354–1372 (2005)

36. Mordukhovich, B.S., Raymond, J.P.: Optimal boundary control of hyperbolic equationswith pointwise state constraints. Nonlinear Analysis, Theory, Methods and Applications63(5-7), 823–830 (2005)

37. Nestler, P.: Optimales design einer zylinderschale - eine problemstellung der optimalensteuerung in der linearen elastizitatstheorie. Ph.D. thesis, Institut fur Angewandte Math-ematik, Universitat Greifswald (2010)

38. Sachs, E.W., Schu, M.: A priori error estimates for reduced order models in finance.ESAIM: Mathematical Modelling and Numerical Analysis 47(2), 449–469 (2013)

39. Studinger, A., Volkwein, S.: Numerical Analysis of POD: A-posteriori Error Estimationfor Optimal Control, pp. 137–158. Springer Basel (2013)

40. Troltzsch, F., Volkwein, S.: POD a-posteriori error estimates for linear-quadratic optimalcontrol problems. Computational Optimization and Applications 44(1), 83–115 (2009)

41. Volkwein, S.: Proper orthogonal decomposition: Theory and reduced-order modeling42. Wang, Y., Navon, I.M., Wang, X., Cheng, Y.: 2D burgers equation with large reynolds

number using POD/DEIM and calibration. International Journal for Numerical Methodsin Fluids 82(12), 909–931 (2016). DOI 10.1002/fld.4249

43. Wang, Z.: Nonlinear model reduction based on the finite element method with interpolatedcoefficients: Semilinear parabolic equations. Numerical Methods for Partial DifferentialEquations 31(6), 1713–1741 (2015)

44. Xiao, D., Fang, F., Buchan, A.G., Pain, C.C., Navon, I.M., Du, J., Hu, G.: Non-linearmodel reduction for the navier-stokes equations using residual DEIM method. Journal ofComputational Physics 263, 1–18 (2014)

45. Xiao, D., Fang, F., Buchan, A.G., Pain, C.C., Navon, I.M., Muggeridge, A.: Non-intrusivereduced order modelling of the navierstokes equations. Computer Methods in AppliedMechanics and Engineering 293, 522 – 541 (2015)

46. Xiao, D., Fang, F., Pain, C., Navon, I.M.: A parameterized non-intrusive reduced ordermodel and error analysis for general time-dependent nonlinear partial differential equationsand its applications. Computer Methods in Applied Mechanics and Engineering 317, 868– 889 (2017)

47. Yang, S.D.: Shooting methods for numerical solutions of control problems constrained bylinear and nonlinear hyperbolic partial differential equations. Ph.D. thesis, Departmentof Mathematics, Iowa State University (2004)

An Indirect Shooting Method using Proper Orthogonal Decomposition 25

48. Yang, S.D.: Shooting methods for numerical solutions of exact controllability problemsconstrained by linear and semilinear 2-D wave equations. International Journal of Numer-ical Analysis and Modeling 4(3-4), 625–647 (2007)

49. Zuazua, E.: Propagation, observation, and control of waves approximated by finite differ-ence methods. SIAM Review 47(2), 197–243 (2005)