8 1’. V. Vasin

It is easily seen that the iterative process in fact stabilizes at the second iteration.

We used our method for solving systems of the 50-th order (the initial system was written

as two systems), of the 100~th order (the initial system was written as three systems), and of the

150~th order (the initial system was written as three systems), etc.

Computing experiments showed that our decomposition method for solving systems of

non-linear algebraic equations of large dimensionality, having the property described above,

provides the same convergence as the classical Newton’s method, i.e., the same as when the

original system as a whole is considered.

Translated b_v D. E. Brown

REFERENCES

1. ZHIDKOV, N. P., ILYSHEVA, N. P., and TIMOFEEV, D. V., Some numerical methods for electric network calculations, Zh. @chisl. Mar. mat. Fiz., 14, No. 5, 1317-1323, 1974.

2. KHACHATRYAN, V. S., Determination of steady-state conditions of large electric energy systems using the Newton-Raphson method, Zzv. Akad. lvaauk SSSR, Energetika transport, No. 4, 36-43, 1974.

3. KHACHATRYAN, V. S., Solution of the equations of the steady-state conditions of large electrical systems by the decomposition method, Elektrichestvo, No. 6, 12-19, 1976.

4. KHACHATRYAN. V. S.. A method for inverting matrices encountered in electrical engineering, IZv. Akad. llhuk SSSR, Energetika transport, No. 5, 105-108, 1969.

U.S.S.R. Comput. MathsMath. Ph,~s. 1’01. 19. p.S-19 0 Pergamon Press Ltd. 1980. Printed in Great Britain.

0041-5553/79/0201/0008S0: 50/O

DISCRETE CONVERGENCE AND FINITE-DIMENSIONAL APPROXIMATION OF REGULARIZING .4LGORITH.CIS*

V. V. VASIN

Sverdlovsk

(Recebed 20 Jurle 1977: relked 30 Augtst 1977)

FOR THE linear operator equation of the 1 st kind Au = f; the convergence of the general scheme

of finite-dimensional approximation (discretization) of the Tikhonov-regularized family of

approximate solutions is examined, along with the application of this scheme to projection and

quadrature methods.

Introduction

We consider the linear operator equations of the 1 st kind

Al.l=f (l-1)

*Zh. vychisl. Mat. mm. Fiz., 19, 1, 11-21, 1979.

Approximation of regularizing algorithms 9

in linear normed spaces, where, for given if and 7 E F, the element u E U is the unknown. We

assume that the initial problem is unstable (ill-posed), i.e. the operator either is not invertible,

or its inverse 2 - 1 is unbounded. When solving the well posed problem (]/A-’ jj < m) numerically,

we usually go over to its discrete analogue

A,11,=& (1.2)

and take the solution u,, of this “approximate” equation as the approximate solution of Eq. (1.1)

(see [l-4] ). If problem (1.1) is ill-posed, we cannot, in general, use this approach, since the

solution u, is unstable to perturbations of the operator and right-hand side. In this case, a stable

approximate solution can be obtained by the method of regularization [5].

The following two-stage scheme of solution is proposed in this paper:

(1) construction of the regularizing algorithm (regularization of the problem).

(2) finite-dimensional approximation of the regularizing algorithm (discretization of the

regularized problem).

In stage 1 the problem is regularized by Tikhonov’s method.

We will quote a familiar result on the convergence of the regularized family of approximate

solutions in this method.

Let {A.f} be approximate data of problem (1 .l), approximating the exact data {_z. ,7} :

JX--4 ji<lz. llf-,c G6, L!=(h. 6). (1.31

Let the space U have the Efimov-Stechkin property and be strictly convex, and let F be reflexive.

Denote by U(Q, A) the extremal element of the parametric variational problem

for given A > 0 and a > 0.

inf {IlAU-f~~2+CZ~~u~i2 : u=U} =d (1.4)

Theorent 1

Given any linear closed operator A and an element fC F, satisfying relations (1.3), problem

(1.4) is uniquely solvable, and the sequence of extremal elements u(a, A) converges to the unique

solutionii,withminimalnorm,ofEq.(1.1),i.e. liu(cr:A)-Ull+O, where inf {ijuJ/ :_$u=f}=

i/U/l, as A-+0, a(A) -0 and (6+h)‘la(3)+0.

At stage 2 problem (1.4) is approximated by the sequence of finite-dimensional problems

inf {llA.u,-f,l12~al~u,1~2: u,EU,} =d,. (I.51

where A,, fn, U,, are fmite-dimensional approximations of the initial quantities A, J C? In terms

of the discrete approximation (see Section 2) we can prove the convergence of the finite-dimensional

approximations u, (a, A), i.e. of the extremal elements of problem (1.5) to u((Y, A) (Section 3).

10 V. V. Vasin

In view of Theorem 1, (discrete) approximation of the solution of Eq. (1 .I) is thereby achieved.

From our fundamental Theorem 2 there follow, in particular, the well-known results on the

convergence of the projection method (Section 4) and the quadrature method (Section 6) (for

integral equations) of discrettiation of the regularized problem [5-131. In Section 5 we discuss

possible extensions of Theorem 2 to a wider class of spaces and operators.

Notice that our main result (Theorem 2) can be interpreted as a justification of the

convergence of discrete algorithms in the generalized method of least squares.

2. Discrete convergence of elements and operators

We shall give some necessary information about the discrete approximation, taking as our basis

the scheme described in [2-41.

Suppose we are given Banach spaces U, LJ,,, n = 1, 2, . . . , where U,, are not necessarily

subspaces of U, and given the family 9= {p,} of operators pn : U + U,,, satisfying the

conditions (see [2.3] )

Ilpn (au+cl’u’) -up,,u-n’p,,u’llr-,~O. 17-+= F’u. U’EC: Ii Yu. u'=Coll't.

(2.1)

(2.2)

Defiuitim 1. Operators pn, having the properties (2.1) and (2.2) are called connecting

operators (see [ ?] ).

Defirlition 2. The sequence of spaces {U,} forms a discrete approximation of space U

if a family of connecting operators B= {p,} exists (see [3] ).

Definitim 3. The sequence {u,}. where U, E U,, , is discretely convergent (or

9 -convergent) to u E U, if

lim JIun--pnuiicE =0 11-x

(see [2? 31). For discrete convergence we use the notation U, -+ U.

A direct consequence of Definition 3 is that the following basic properties of ordinary

convergence hold for discrete convergence:

uniqueness of the limit:

un-u, u,+u’=+u=ld

(here and henceforth, when passing to the limit, the subscript n runs over the natural series or

some infinite subset of it, i.e., r7 + 00):

(2.3)

convergence of a subsequence:

u,‘uJunk +u for {nR} c {n}; (2.4)

Approximation of regularizing algon’thms 11

linearity of passage to the limit:

u,-+u, u,‘~u’~au,+a’u~‘~au+a’u’,

where a, a’ = const;

(2.5)

(2.6)

where 8 is the zero element of space U (to simplify the writing, we shah henceforth use the single notation II . I( for the norm, instead of II . IIu, II . I(un, etc.);

matching of the norms:

~,-+~~liu~ll+ll71I/; (2.7)

Let the sequence of spaces {F,} form a discrete approximation of space F, and let the.family &= {q,} of operators qn : F + F,,, satisfy conditions (2.1) and (2.2).

Deftnition 4. The sequence {A,} of operators A, : U,, + F,, is discretely convergent ( or SQ -convergent) to A if, given any discretely convergent sequence {u,} we have the relation (see [2,3] ).

U,+U*A.U.+AU.

Definition 5. The sequence {g,} of linear functionals g, E V, * is discretely weakly

convergent (or weakly Y.-convergent) tog E U*, if (see [2,3])

we denote this as g, - + g.

Notice that discrete weak convergence satisfies (2.3)-(2.6); while the analogue of (2.7) is

g,+g* IIglIG lim inf llgllll. (2.9) n-m

In the case of Hilbert spaces, both discrete convergence, and discrete weak convergence, can be considered for sequences of elements, since we can identify U = CT* and U,, = U,,*. These concepts are connected by the relation (see [2], p. 28)

u,+u-u,--u, Il~nII+I:4/. (2.10)

Definition 6. The pair A, (A,) is called discretely weakly closed if we have the relation

(see 141)

US --+u, A,u,-*f=-UED (A), Au=f.

12 V. V. Vasin

3. Discrete approximation in Tikhonov regularization

We shall assume that the following conditions are satisfied.

Condition 1. U, U,, F, F,, are Hilbert spaces, U and F being separable.

Condifion 2. The sequence of spaces {U,} ({F,}) f orms a discrete approximation of

space U(F).

Condition 3. The sequence of linear bounded operators A,, : U, +fis discretely convergent

toA : C’+F.

Condirion 4. The pair A, (A,) is discretely weakly closed.

Condition 5. fn + f, i.e. is discretely convergent.

Tleorem 2

If conditions 1-5 hold, problem (1.5) has a unique solution u, (a, A) and U, (,a, A) -+

u (a. I).

Proof: Since A, is a linear bounded operator, and U, is Hilbert, the solvability of problem

(1.5) can be proved in the standard way. The uniqueness follows from the strict convexity of the

functional

The following inequalities, with o > 0. are obvious:

/ju,(a.A)li’G~ a

+n[p,~(a,A~l

G $ {[ liA,ll Ilp,u (a? A) ll+llfnll 12+allw (a, A) Ii’).

Since& --*f, then, by (2.7) II fn II + II f II, whence II fn II <cl; for the same reason

(see (2.&I), llp,,u (a, A) li~c,. By condition 3, the sequence A,, + A; hence II A, II < c3

(see [3], pp. 52. 53). Finally. Ilu,(a, A) IlGc, where c is a constant independent of n.

Let us show that

lim sup d,Gd. ?l+li?

Noting the discrete convergence p,u (a. A) +u (a, A) and condition 3, we have

A,p,u(a. A)*Au(a, A);

since fn + f, then, recalling (2 .S>,

(3.1)

By (2.7), we obtain

~,z-w(a, A) - f,-+Au(a,A)--f.

lim llAnpnu(a, A) -fnllZ=llA~(a, A) -fll’, *--

lim Ilp,,~(a, A> /12=Il~ia, A) II’. n-m

Approximution of regukwizing ~~gon’t~~$ 13

Relation (3.1) follows from the above relations. In fact,

lim sup da== lim sup (9, [an (a, A) ] n-io (I-cm

G lim sup Q),,~P~u(cL, A)]=d, I)-DD

Since the sequences (u, (a, A)}, {AJJ,, (a, A) } are bounded, there exist the discretely weakiy convergent subsequences (see 121, pp. 25-26)

uzzk (a, A)- -+k &n~na (a,A>- -i, {nJ={n}. (3.2)

Combining reIations (2.10) and (3.2), and conditions 4 and 5, we obtain i E D(A), A L=f, and

Ankunk (a, Ah-f,,- *A&f, (3.3)

whence, along with (3.1) and (2.9), we find that

+IIzznk (CI, A) 11% lim inf dnkd lim sup d,kG:d. (3.4)

k-cc k-r-

It follows from the above inequalities that i is the extremal element in problem (1.4), and hence u^ = u (cu, A).

We choose a subsequence of numbers {n,} E {nk} such that the following limits exist:

lim M,,u,,, (a, A) -fn n; li2=x, lim Iiu,, ,(a, A) 11*=h. 7n’p: ?n-m

In view of (3.4), we have the equation 5 + cy h = d. Let us show that x=jjdu (cr, !i)--fliz, ?.=

tlu(~~, A) II’. A ssume that, either ]lu(cz, A) II<?., or else llAu(t~, A) -j/12<ic (the reverse inequalities cannot hold by relations (3.2), (3.3), and (2.9)). We then arrive at the impossible inequality

Hence

moreover, it was shown above that

km (a, 1) ---W(cI, 3).

By relation (2.1(I), we finally obtain

~,,(,a, A) -+u(a, A). (3.5)

Since a subsequence with the property (3.5) can be extracted from any sequence (n,> E {n}

this implies the discrete convergence of the entire sequence.

14 V. V. Vasin

A’ote. If the connecting operators pn and qn are linear, bounded, and satisfy relation (2.1),

we can omit the condition that the spaces be separable in Theorem 2.

4. Projection methods

Suppose we are given in Hilbert space U(F),the chain of finite-dimensional subspaces

L-,= L- (F,cF) ) satisfying the condition

i U,=U (uF,=F). ?I=* n=,

Then. the linear operators of orthogonal projection P,, : U + U,, have the properties

P,u-+u ruf=u (Qnf-f f VW’).

llP,l!~ 1 (liQnllW.

(4.1)

(4.2)

(4.3)

1. As the connecting operators pn and qn we take the projection operators Pn and Q,,

respectively.

In (1.5) we put =l,=q,Ap,. .f%=q,,,i and we arrive at the projection scheme of

finite-dimensional approximation employed in [9. lo].

Let us check the conditions of Theorem 2. Conditions 1,2, and 5 are obviously satisfied

by virtue of (4.2). By (4.2) and (4.3), for the linear bounded operator A we have

By Theorem 1 of [2], p. 18, the discrete convergence An + A follows from (4.4) i.e. condition

3 is satisfied.

To prove that the pair A, (An), is discretely weakly closed, we first need two lemmas for

the case u,cc and Ilp,u--ull+O (‘t 1 IS not assumed that P,, is linear and bounded).

Lemma 1

We have u.+zI-+//u,,-u//+0.

Proof Let u, + u, i.e. 11 u,--p,u II + 0; then, recalling (4.2),

.4 pproximation of regulariring algon’thms 15

conversely, let ~~zz.---u~~+i). then,

Denote by U, + u (weakly), weak convergence in space c!

Lemma 2

We have the relation u,--+u-++u,+u (weakly).

Proof: Let u,, - + U; this means (see Definition 5) that, given any discretely convergent sequence u’, + u’, we have the relation

(u,‘. u=) + (u’, 12). (4.5)

Let us show that (z, u,) - (z. U) for all z E l_J. Put z, = P,,z; then, we have

I (--. un) - (2, u) 1 G 1 (2, %a) - (=n. Un) I+ 1 (z,, II,) - (2, u) I.

The second term tends to zero, since Z, + z, and (4.5) holds. The first term also tends to zero, since

where the sequence {II u, I~} is bounded (see [3], p. 57), while licn-:ll =i’p,:-:I;-0

We now consider the question of the pair A, (A,) being closed. Let U, - + U, i.e. discrete weak convergence. Let us show that the pair A, (An), has a stronger property than that of being closed, namely,

u,;- +u*A,,u,-*Au.

In fact, by the criterion of weak discrete convergence (see [2], p. 21), to prove this we simpl> have to show that

(-&un. qn!) + (Au. f) Yfd?

Notice that, by Lemma 2, u, + u (weaklyj, and hence Au,, + Au (weakly) (see [ 14]? p. 2 17 ),

and moreover, II qn f- f II + 0. Since

I (Anun. qnf)-- (Au, I) I= 1 (wbnun. qnf)- (Au: f) 1

= 1 (Au,. qnf) - (AK f) I.

the last expression tends to zero, by virtue of the continuity of the scalar product in this type

of convergence (see [14], p. 221).

2. Take a particular case of the method of finite-dimensional approximation (see Para. l), by putting q,, = I (identity operator) for all n. Then, A, = Ap,, and we obtain the Ritz projection

method. This method was discussed in [Il. 131 in the context of the regularized problem (1.4).

3.Letpn=q, = I (identity operator). In this case, the discrete convergence of linear bounded

operators A, implies point-wise convergence to the operator A, i.e. convergence on every element U:

16 V. V. Vasin

If we additionally require that U = F, ZJ, = Fn, and that the operators A, An be self-adjoint,

then condition 4 is also satisfied, since, if U, - + U, then v,, + u (weakly) and

I (A&n, r) - (Au, I) I = I (un, Ad) - (4 Af) I-4.

Hence all the conditions (of Theorem 2) for convergence of the extremal elements U,

(Q, A) of problem (1.5) to the extremal element u((Y, A) of problem (1.4), are satisfied. In the

general case, point-wise convergence is not sufficient for convergence of finite-dimensional

approximations (see an example in [6] ).

5. Some extensions of the fundamental theorem

1. Let the spaces U, (F,) be subspaces of the space U(F). Then the assertion of Theorem 2

holds if, instead of condition 1, we have the weaker condition:

Condition 6. The space F is reflexive, U is strictly convex and has the Efimov-Stechkin

property, i.e. in addition to its being reflexive, we have

zz,-+u (weakly), IIu~Il--t/l~II~II~,--ull’o.

Here, the fact of the pair A, (A,) in condition 4 being closed is understood with respect

to weak convergence in spaces U, F.

The proof of this extension of the theorem requires only slight modification of the original

proof.

2. Let the sequence of spaces U, (Fn) form a discrete approximation U(F,) and let an

isometric isomorphism 9 (G) exist, under which spaces U,, (Fn) are mapped into the

(finite-dimensional) subspaces U,‘cU (F,‘cF). Then, properties (2.1) (2.2)‘hold for the

operators pn ‘=(rpn (St’= qyn) and hence spaces {U,‘} ( {FR’} ) form a discrete

approximation of space U(F),

In this case, problem (1.5) is equivalent to the problem

inf {l~\1.An(r~i~~‘-f~‘j12+allu,1112: U,‘EU,‘}_. (5.1)

their solutions being connected by the equation ~,(a, 4) =q~-‘u,‘(a~ 4).

Hence the extension of Theorem 2 of Para. 1 can be applied for the operators B,=$A,(r-’

Example. Let U - L, [0, l] , We define some quadrature process, e.g. the rectangle process

where m(u) + 0 for any continuous function u(t) is continuous, we define the operator

pn: u(t)-+ (u(S:n’), . . . ,u(s,(n’ ) ) . As the connecting operators pn we take the continuation

of operators F,, into the entire space L, [0, l] . This continuation exists and in a certain sense, is

natural (see [2], p. 14). Let U, be an n-dimensional space with the norm

.4pproxirnariw of regularizing algon’tlrms 17

The mapping i-“. which associates the element II,= ( II,,, . . . . u,,) with the piecewise

constant function 21(t) =[I,, for tE (sl’-“, . s,‘“’ 3. is an isometric isomorphism of space

U into the subspace U,,’ E I:.

We can obtain in a similar way a discrete approximation in the multi-dimensional case

L,[Dl, DER, (see [3]. p. 88).

3. Theorem 2, on the convergence of finite-dimensional approximations, was stated under

the assumption that the operators A, (A,) are bounded, whereas Theorem 1, on the convergence

of regularized solutions, holds provided only that the operator A is closed. Let us describe a

device whereby the general case of a linear unbounded operator A can be reduced to the case

considered in Section 3.

Let T be a linear? in general unbounded, operator. with domain of definition D(T) C_ U and

range of values R(T) 5 F: these being linear manifolds. We norm the set D = D(T) by introducing

the T-norm:

Then, the linear operator A : D + F, defined by the relation .4rr = Tu, z/ E D = D(T), is bounded.

and ]I A 11 < 1. Similarly, we define the T,,-norm in D( T,, 1. and hence obtain bounded operators

A,,: d,,u=T,,u. UED (T.) (see [3] for details).

Note. Our results on the discrete approximation of a regularized family of approximate

solutions can be extended to the case when, instead of Tikhonov regularization, other variational

methods, such as e.g. the method of (generalized) discrepancy. or the method of quasi-solutions.

is employed.

6. Quadrature methods

Let us examine quadrature (finite-difference) methods from the general stand-point of

discrete approximation. For this, we consider the Fredholm integral equation of the 1 st kind

(6.1)

where K(r, s) is a continuous function with respect to both its variables, while assuming that the

operator A acts from U = L2 (0, I ] into F = L2 [0, I] , (space of Lebesgue square summable

functions).

Let Cp be a subset of continuous functions. Then, every function u(t) E Cp is Riemann

integrable and has the closure, with respect to the norm

I s*ls 19 / H

18 V. V. Vasin

We specify a convergent quadrature process, e.g. of rectangles:

Let @, be n-dimensional Euclidean space with the norm

j-1

We define the connecting operators for @ and @,, :

Since conditions (2.1), (2.2) are obviously satisfied for operators P,, , then we have discrete

approximation throughout the dense set @ in space L2 [0, I]. According to the result obtained

in [3], pp. 78,79, there then exists a unique continuation of the discrete approximation into the

entire space. If we define

then problem (1 S) becomes in this case

(6.2)

Determination of the extremal element of problem (6.2) reduces to solving the

corresponding system of linear algebraic equations.

We define the mapping q = $, which maps the element u,= (u,~. . . . , u,,) into the

piecewise constant function u (t) =u,, for 1~ (s,‘lll , sltn)] 1 sA”‘=O (see Para. 2 of Section 5).

Then ‘p realizes an isometric isomorphism of space U, into the n-dimensional subspace U,,’ C Ii.

The operator =I,‘=ll_A,T-l in problem (5.1) is the same, in subspace U,,‘, as the operator

1

A’“‘u= K‘“‘(t,s)u(s)ds, 5 0

where K’“‘(t.s)=K(s,‘“’ ,s,‘“‘) for s::: ad, . s::; <S& .

It can easily be seen that ][A--A’“‘//+0 as n+m. Hence, recalling Lemmas 1 and

2, we see from the inequalities

Approximation of regulariring algorithms 19

jlA’“‘U,-AUII~jjA’“‘U,-AU,li+IIAU,-AUII, 1 (A’%, -Au, f) I< 1 (A’“)u, ---Au,, !) I+ 1 (Au,--Au, f) 1

that the convergence A,’ -+ A is discrete, and the pair A, (A,‘) is discretely weakly closed.

By Theorem 2, this ensures the solvability of problem (5.1), and hence also the solvability

of problem (6.2), and the convergence u,‘(a. I) +u, i.e. u, (a. 3) +u.

These assertions regarding the convergence of quadrature (finite-difference) methods of

approximating regularizing algorithms, generalize and refine the well-known results obtained in

[5,8] for integral equations of type (6.1).

Translated by D. E. Brown

REFERENCES

1. KANTOROVICH, L. V. and AKILOV, C. P., Functional analysis in normed spaces (Funktsional’nyi analiz v normirovannykh prostranstvakh), Fizmatgiz, Moscowt 1959.

2. VAINIKKO, G., Analysis of discreti:ation methods (Analiz diskretizatsionnykh rnetodov), Izd-vo Tartuskogo un-ta, Tartu, 1976.

3. STUMMEL, F., Diskrete Konvergenz linearer operatoren, 1, Math. Ann., 190,45-92, 1970.

4. GRIGORIEFF, R. D., Theorie linearer approximations-regullrer Operatoren, I? Math. Nachr., 55, 233-249, 1973; II, 251-263.

5. TIKHONOV, A. N. and ARSENIN, V. Ya., Methods of solving ill-posed problems (Metody resheniya nekorrektnykh zadach), Nauka, Moscow, 1974.

6. DANILIN, A. R. and TANANA, V. P., On the convergence of projection methods for solving linear ill-posed problems, Matem. zap. Ural’sk. un-ta, 9, No. 4, 3-13, 1975.

7. TIKHONOV, A. N., On the solution of I&posed problems and the method of regularization, Dokl. Akad. Nauk SSSR, 153, No. 3,501-504,1963.

8. GONCHARSKII, A. V., LEONOV, A. S. and YACOLA, A. G., Finite-difference approximation of linear Ill-posed problems, Zh. @his/. Mat. mat. Fiz., 14, No. 1, 15-23, 1974.

9. VASIN. V. V., On a projection method for the approximate solution of ill-posed problems, Izv. vuzor’, Matematika, No. 11, 28-32, 1971.

10. VASIN, V. V., On fi-eonvergence of a projection method for non-linear operator equations, Zh. l$?hisl. Mat. mat. Fiz., 12, No. 2,492-497, 1972.

11. MOROZOV, V. A., Application of the method of regularization to the solution of an i&posed problem, Vestn. MGU, Matem., Mekhan., No. 4, 13-21, 1965.

12. IVANOV, V. V. and KUDRINSKII, V. Yu., Approximate solution of linear operator equations in Hilbert space by the method of least squares, 1, Zh. vychisl. Mat. mut. Fiz., 6, No. 5, 831-841, 1966; II, 7, No. 3,475-496, 1967.

13. VASIN, V. V. and TANANA, V. P., Approximate solution of operator equations of the 1st kind, Matem. zap. Ural’sk. un-fu, 6, No. 2, 27-37, 1968.

14. LYUSTERNIK, L. A., and SOBOLEV, V. I., Elements of functional analysis (Elementy funktsional’nogo am&a), Nauka, Moscow, 1965.