Approximate solution of defect equation

7/26/2019 Approximate solution of defect equation

1/23

Approximate solution of defectequation

Ifum

is any approximation ofu and

rm =f Aum

is its residual, then the residual equation Aem = rm is

equivalent to the original equation: By solving for the

correction em, we obtain the solution u=um + em.

If we use, however, anapproximationAof A, such thatAem =rmcan be solved more easily, we obtain an iterative process

of the form

rm =f Aum, Aem =rm, um+1 =um +em (m= 0, 1, 2...)

This process is obviously equivalent to the general iterationwhere

Q= I (A)1A . Vice versa, ifQ is given, this yields an approximationAofA

according to A= A(I Q)1 .

Scient. Comput. and Sim./SlideNr. 1


2/23

Splitting, Preconditioning

An equivalent way of constructing Q is to start with a splitting

A=A R, Aum+1

=Ru

m

+ f .Here

Q= (A)1R=I (A)1A . A third, also equivalent, approach is based on the idea

of preconditioning. Here the original equation Au = f is

replaced by an equivalent equation

CAu= Cf

where C is an invertible matrix. C is called a (left)

preconditioner of A. The identification with the above

terminology is by

(A)1 =C.In other words, the inverse (

A)1 of any (invertible)

approximationAis a left preconditioner and vice versa. Richardsons iteration for the preconditioned system (with

= 1)

um+1 =um + C(f Aum) = (I CA)um + Cfis equivalent to the general iteration withQ= I CA.



3/23

Two ways of improving theconvergence

of basic iterative methods

For any approximation ui

h of the solution uh, we denote theerror byeih:=uh uih, and the defect (or residual) by

rih:=bh AhuihThe defect equation

Aheih=r

ih

is equivalent to the original equation, since

uh=uih+ e

ih.

This leads to the procedure

uih rih=bh Ahuih Aheih=rih uh=uih+ eih .This procedure is not a meaningful numerical process.

However, if Ah is approximated by a simpler operatorAh such that A1h exists, eih in Aheih = rih gives a newapproximation

ui+1h :=uih+eih.

The procedural formulation then looks like

uih rih=bh Ahuih Aheih=rih ui+1h =uih+eih .



4/23

Iteration matrix

The iteration operator of this method is given by

Qh=Ih (Ah)1Ah: G(h) G(h),whereIhdenotes the identity on G(h). We have

u

i+1

h =Qhu

i

h+sh with sh= (A)1

h bh (i= 0, 1, 2, . . .) .For the errors, it follows that

ei+1h =Qheih=Ih (Ah)1Aheih (i= 0, 1, 2, . . .)

This represents a general class of iterative schemes. For

example, with

Ah=Dh,the Jacobi scheme is regained; withAhthe lower triangularpart ofAhwe obtain Gauss-Seidel.



5/23

1. Coarse grid correction

The idea is to use an appropriate approximation AH of Ahon a coarser grid H, for instance the grid with mesh size

H= 2h. This means that the defect equation is replaced by

AHeiH=riH.AH :G(H) G(H), dimG(H) < dimG(h) and AH1exists.

AsriH andeiHare grid functions on the coarser grid H, weneed two (linear) transfer operators

IHh : G(h) G(H), IhH : G(H) G(h)

IHh is used to restrictrihtoH:

ri

H :=IH

h ri

h ,

andIhH is used to interpolate (or prolongate) the correctioneiHback toh: eih:=IhHeiH .



6/23

for Poissons equation

Smoothing iteration: lexicographic Gauss-Seidel method

Coarse grid discretization:

AHuH= 1

H2

0 1 0

1 4 10 1 0

H

.

The simplest example for a restriction operator is the

injection operator

rH

(P) =IHh

rh

(P) :=rh

(P) for P

H

h

,

A fine and a coarse grid with the injection operator are

presented:

h

h



7/23

Multigrid components

Prolongation operator:

y

2h

2h

x

Ih2he2h(x, y) =

e2h(x, y) for1

2

[e2h(x, y+ h) +e2h(x, y h)] for

12 [e2h(x+ h, y) +e2h(x h, y)] for14

[e2h(x+ h, y+ h) +e2h(x+ h, y h)+e2h(x h, y+ h) +e2h(x h, y h)] for



8/23

Altogether, one coarse grid correction step (calculating

ui+1h fromuih) proceeds as follows:

Coarse grid correctionuih ui+1h Compute the defect rih=fi Ahuih Restrict the defect (finetocoarse transfer) riH=I

Hh r

ih

Solve exactly onH AHeiH=riH Interpolate the correction eih=IhHeiH Compute a new approximation ui+1h =u

ih+eih

The associated iteration operator is given by

Ih BhAh with Bh=IhHAH1IHh .

Taken on its own, the coarse grid correction process is of nouse: It is not convergent! We have

Ih IhHAH1IHh Ah 1 .



9/23

high frequency components,

not visible on2h

low frequency components,

visible also on2h

High frequency components cannot be corrected on acoarse grid !

Coarse grid correction makes sense, if low frequencies are

dominating the error.

We can decompose the sum into partial sums:

p1

k,l=1 k,l

k,l

=highk,lk,l +lowk,lk,l

where lowk,lk,l =p/21k,l=1

k,lk,l

and

highk,l

k,l =p1

k,lp/2max(k,l)k,l

k,l .



10/23

Point-wise Gauss-Seidel:

ui,j =1

4[h2bi,j+ u

n+1i1,j+u

ni+1,j+ u

n+1i,j1+ u

ni,j+1]

The effect on the erroren =un uhis a local averaging effect:eni,j =

1

4[en+1i1,j+ e

ni+1,j+ e

n+1i,j1+ e

ni,j+1]

We have found already

||en+1|| (Q)||en||, (n ).

Analysis of the smoothing effect requires consideration ofeigenvalues/-vectors ofQ, which are closely related toA. Look

at the Fourier expansion of the error:

eh(x, y) =p1k,l=1

k,lsin kx sin ly=p1k,l=1

k,lk,l

The fact that this error becomes smooth means that the high

frequency components, i.e.,k,lsin kx sin ly withk orl large

become small after a few iterations, whereas the low frequency

components

k,lsin kx sin ly withkandl small

hardly change.



11/23

The two grid iteration

Correction scheme

It is necessary to combine the two processes of smoothingand of coarse grid correction.

Consider a linear problem Ahuh=bh on grid Gh(1) 1 smoothing steps

on the fine grid: uh =S1(u0h, bh)(2) computation of residuals

on the fine grid: rh:=bh Ahuh(3) restriction of residuals

from fine to coarse: rH :=IHh rh

(4) solution of the

coarse grid problem: AHeH=rH(5) prolongation of corrections

from coarse to fine: eh:=IhHeH

(6) adding the corrections to thecurrent fine grid approximation: uh =uh+ eh

(7) on the fine grid: u1h =S2(uh, bh) Steps (1) and (7) arepre and postsmoothing,

steps (2)...(6) form thecoarse grid correction cycle.



12/23

3. Multigrid

since 1973

Iterative methods likeJacobiandGauss-Seidelconvergeslowly on fine grids, however, theysmooththe erroruh u

05

1015

2025

3035 0

5

10

15

20

25

30

35

0

0.5

1

1.5

2

2.5

05

1015

2025

3035 0

5

10

15

20

25

30

35

0

0.2

0.4

0.6

0.8

1

1.2

1.4

Smootherrors can beapproximated well on

coarser grids (with

much less grid points)

05

1015

2025

3035 0

5

10

15

20

25

30

35

0

0.2

0.4

0.6

0.8

1

1.2

1.4

05

1015

2025

3035 0

5

10

15

20

25

30

35

0

0.2

0.4

0.6

0.8

1

1.2

1.4

Multigrid is a O(N)- method !



13/23

Multigrid components

Choice of coarse grid The choice of grid depends on the smoothness of the error.

Grid coarsening is particularly simple for structured grids.

For irregular finite volume/ finite element grids coarse grids

are chosen based on the connections in the matrix. In

this case, it is better to say that coarse matrices are

constructed.

It is possible to determine, based on matrix properties

(M-matrix, for example), where the error will be smooth

and accordingly how to coarsen algebraically (algebraic

multigrid, AMG).



14/23

2. Krylov subspace accleration

Basic Iterative solution Methods

We computed the iterates by the following recursion:ui+1 =ui + B1(b Aui) =ui + B1ri

Writing out the first steps of such a process we obtain:

u0,

u1 = u0 + (B1r0),

u

2

= u

1

+ (B

1

r

1

) =u

0

+B

1

r

0

+ B

1

(b Au0

AB1

r

0

),= u0 + 2B1r0 B1AB1r0,...

This implies that

ui u0 + span

B1r0, B1A(B1r0), . . . , (B1A)i1(B1r0)

.

The subspace Ki(A; r0) := span r0, Ar0, . . . , Ai1r0 is calledtheKrylov-spaceof dimension i corresponding to matrix A

and initial residualr0.

ui calculated by a basic iterative method is an element of

u0 + Ki(B1A; M1r0).



15/23

Recombination of Iterants

The acceleration of a basic iterative method by iterantrecombination starts from successive approximations

u1h, u2h, . . . , u

mh, from previous iterations.

In order to find an improved approximation uh,, we consider

a linear combination of them + 1 latest approximationsumih , i= 0, ,m,

uh,=umh + mi=1

i(umih umh) ,

(assumingm m) withi= 1. For linear equations, the corresponding residual, rh, = fh

Lhuh,, is given by

rh,=rmh +

m

i=1 i(rmih

rmh) ,

wherermih =fh Lhumih . To obtain an improved approximation uh,, parameters iare

determined such that residualrh, is minimized.



16/23

Minimizerh,, i.e.

||rmh +

m

i=1i(r

mih

rmh)

|| ,

with respect to theL2-norm | | | |2. This is a classical defect minimization problem. In principle,

the optimal coefficientsican be determined by a (Gram-

Schmidt) orthonormalization process.

Here, however, it is also possible to solve the system of linear

(normal) equations

H

12...

m

=

12...

m

,

where the matrixH= (hik)is defined by

hik = < rmih , r

mkh > < rmh, rmih >

< rmh, rmkh >+ < rmh, rmh > i= 1, . . . , m, k= 1, . . . ,m ,with the standard Euclidean inner product< ., . >and

i=< rmh, r

mh > < rmh, rmih > .

The work for solving the minimization problem is small.



17/23

The Chebyshev method

Supposeu

1

, . . . , u

k

have been obtained via a basic iterativemethod, and we wish to determine coefficients j(k),

j = 0, . . . , ksuch that

yk =k

j=0j(k)u

j

is animprovementofuk.

If u0 = . . .= uk = u, then it is reasonable to insist that yk = u.Hence we require

kj=0

j(k) = 1,

Consider how to choose the j(k)so that the error yk u is

minimized. Since error e(k+1) = Qke0 where ek = uk u. Thisimplies that

yk u= kj=0

j(k)(uj u) = k

j=0j(k)Q

je0.

Using the 2-norm we look for j(k) such thatyk u2 isminimal.



18/23


To simplify this minimization we use the following inequality:

yk u2 pk(Q)2u0 u2wherepk(z) =

kj=0

j(k)zj andpk(1) = 1.

Minimize

pk(Q)

2for all polynomials satisfyingpk(1) = 1.

Assumption that Q is symmetric with eigenvalues i that

satisfy n . . . 1


19/23


The solution of this problem is obtained by Chebyshev

polynomials. These polynomials cj(z) can be generated bythe following recursion

c0(z) = 1,

c1(z) =z,

cj(z) = 2zcj1(z) cj2(z).These polynomials satisfy

|cj(z)

| 1 on [

1, 1] but grow

rapidly in magnitude outside this interval. As a consequencethe polynomial

pk(z) =ck1 + 2 z

ck

1 + 2 1

satisfies pk(1) = 1, since1 + 2 1 = 1 + 2 1, and tends tobe small on[, ]. The last property can be explained by the

fact that1 1 + 2 z

1 for z [, ] so the

numerator is less than 1 in absolute value, whereas the

denominator is large in absolute value since 1 + 2 1 >1.



20/23

-method

This leads to

yk u2 pk(Q)2u0 u2 u u02|ck

1 + 2 1

|.

Calculation ofyk is costly, since all u0, . . . , uk should be kept

in memory. Furthermore, one needs to add k + 1 vectors,

which costs fork 5more than one matrix vector product. Using the recursion of the Chebyshev polynomials it is

possible to derive athree term recurrenceamong the yk.

Vectorsyk can be calculated as:

y0 =u0

solvez0 fromBz0 =b Ay0 theny1 is given byy1 =y0 + 22z

0

solvezk fromBzk =b Ayk theny(k+1) is given by

y(k+1) =4 2 2

ck

1 + 2 1

ck+1

1 + 2 1

yk y(k1) + 2

2 zk+y(k1)

The Chebyshev semi-iterative method associated withBy(k+1) = (B

A)yk + b.



21/23

The semi-iterative Chebyshevmethod

Theory

Note that only 4 vectors are needed in memory and the

extra work consists of the addition of 4 vectors.

Acceleration is effective with good lower and upper bounds

ofand. These parameters may be difficult to obtain.

Assumption in deriving the Chebyshev acceleration: the

iteration matrixB1(B A)is symmetric. Thus, analysis doesnot apply to the SOR iteration matrix B1

(B

A). To repair

this Symmetric SOR (SSOR) is proposed. In SSOR one SOR

step is followed by a backward SOR step. In this backward

step the unknowns are updated in reversed order.

Suppose that the matrix B1A is symmetric and positive

definite and that the eigenvaluesi are ordered as follows

0< 1 2 . . . n. It is then possible to prove the followingtheorem:

If the Chebyshev method is applied and B1Ais symmetric

positive definite then

yk u2 2

K2(B1A) 1K2(B1A) + 1

k

u0 u2.



22/23


Proof SinceB1A= B1(B (B A)) =IB1(B A) =IQwe see that the eigenvalues satisfy the following relation:

i= 1 i or i= 1 i.This leads to the inequality:

yk u2 u u02

|ck

1 + 2 (1(11))(11)(1n)

|.

So it remains to estimate the denominator. Note that

ck

1 + 2(1 (1 1))(1 1) (1 n)

= ckn+ 1

n 1= ck

1 + 1n

1 1n

.The Chebyshev polynomial can also be given by

ck(z) =1

2

z+

z2 1

k+

z

z2 1k

.

This expression can be used to show that

ck

1+1n1

1n

> 12

1+1n

11n

+

1+1n1

1n

2 1k

=

= 12

1+1n+21n1

1n

k = 121+1n1

1n

k .



23/23


The condition numberK2(B1A)is equal to n1 . This leads to

yk u2 2

K2(M1A) 1K2(M1A) + 1

k

u0 u2.

Chebyshev type methods which are applicable to a wider

range of matrices are given in the literature.


Documents

Approximate solution of defect equation