Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
OPTIMAL USE OF INFORMATION IN
CERTAIN ITERATIVE PROCESSES
Robert MEERSMAN
Vrije Universiteit Brussel
I. Introduction
To approximate numerically a zero _ of a real ana-
lytic function f,
f(e) = 0,
iteration is most widely used. We will discuss so-
called k-point stationary iterative processes
without memory, _, defined as follows• Let there
exist an interval around a such that for all x in1
this interval, the following functions
_i' i=2,...,k+l are well-defined :
z I = x 1
z2 = _2(Zl'N(z l;f))
z 3 = _3(Zl,Z2,N(Zl,Z2;f))
x2:=_(Xl)=Zk+l - _k+l (Zl'''''Zk'N(Zl'''''Zk ;f)) '
where
a) x 2 is called the new approximation to e ; we
assume that x 2 lies within the same interval a
and that the process is converging, i.e. put-
ting Xl:= x 2 and repeating the iterative step
produces a sequence which converges to _. Also
we assume f' (_) _ 0.
b) N(zl,...,zi;f) is called the information set ati iiii iJ • u ii lINK
This research was supported in part by the National ScienceFoundation under Grant GJ321]] and the Office of Naval Re-
search under Contract N00014-67-A-0314-00]0, NR 044-422.
the points Zl,...,z i and consists of the set of
all evaluations of f and/or its derivatives
used at those points when computing zi+ I. It is
not necessary to give all derivatives up to a
certain degree at any particular point z i.
Examples of specific N abound in the rest of
the paper. Other kinds of information sets can
be considered, but are not the topic of this
paper ; see for instance Kacewicz [75]. The read-
er should also be more or less familiar with the work off .
Woznlakowski [74] [75a].
2. Some known results and conjectures
Definition 2. I.
Let N(z l,...,zk;f) be given. We say that£%_ £_'%
= f mod N when for all f_JJ (z i) 6 N,
f(J) _zi) = _(J)(zi)
/As in Woznlakows_i [T5a] we adopt the following definitions"
Definition 2.2.
The order of iteration p(_) is the largest number
such that for all f with f(e) = 0,
for all _ - f mod N and having a zero e near e,
we have
[¢(xl;N) -lim sup
Xl._ IXl - elP (¢) <
It can be shown this definition coincides with the
usual definition in the literature (e.g. Traub[64])
under weak assumptions on the asymptotical constant
Definition 2.3.• . i i i
The order of information p(N) is the largest num-
ber such that for all f with f(a) = 0,
for all _ - f mod N and having a zero a near a,
we have
lim sup la - al _) <Ix -Xl 1
Remark that choosing a special f or _ - f mod N
will give upper bounds on these orders, by defini-
tion. See also Wo_niakowski [755]. Extensive use will
be made of the following two theorems.
Theorem 2.1. (Maximal Order Theorem)
The order p(¢) is bounded above by p(N) for all
using information N.
Theorem 2.2.
The maximal order is reached for the generalized
interpolating methods I N : p(I N) = p(N) .
For definitions and proofs, see Wo{niakoWski [75a].
Two kinds of problems arise now.
Problem i. Given N, compute p(N) . (In other words,
what is the maximal order achievable with informa-
tion N).
Problem 2. Given n = @N, the number of elements in| _ i __
N, determine P_ = max max p(_), where ¢ uses in-n @N=n
formation N.
Problem 2 is much harder than problem i. Kung and
Traub conjectured (Kung and Traub [74a]) that
(2.3) P_ = 2n-In
and exhibited two families of methods which rea-
lize this bound for each n. In a later paper, Kung
and Traub [74b] , they proved (2.3) for n = 1 and 2.
Remark that in view of Theorems 2.1 and 2.2, (2.3)
is equivalent to
2n-iQn =
where Qn = max p(N).N=n
The conjecture has been settled by Wo_niakowski in
one very important case.
Definition 2.4.
N is hermitean iff for all i, 1 _ i _ k, we have
f(k) (zi)6N_f(k-l) (zi)eN for all k > 0.
Theorem 2.3. (Wo_niakowski [755])
The conjecture of (2.3) is true if the maximum is taken over
hermitean /V. We will show later that a partial converse is
not true, i.e. that p(A/) = 2_v-] does not imply that A is
hermitean.
3. A solution to problem 1 for n = 3
We will prove in this section that P_3 = 4, showing
the correctness of the conjecture in this case.
Our proof uses special cases of some general re-
sults on certain n-evaluations iterations one of
which is to be treated in a later paper, namely
Theorem 3.1.
If N = {f(zl) ,fl(z2) ,...,f(n) (Zn+l) } then
p (N) _ 2n
Note : N is the so-called Abel-Con_arov informa-
tion.
In the proof of the following lemma and the rest
of the paper we assume that at each z i used in _,
some new information is computed. This is not a
restriction since otherwise we can substitute the
expressions for these z i in the other _j, obtai-
ning an equivalent iteration (with less points).
Lemma 3.1., ii
Let # be an iteration using two pieces of informa-
tion, i.e.
z = x1 1
z 2 = _2(Zl,N(Zl;f))
x2:=_(Xl)=Z3 = _3(Zl,Z2,N(Zl,Z2;f)) with @N = 2
(By the above convention, _ is a I- or 2-point
iteration) .
Then, if there exists a (known) constant C, C # 0
and C _ i, such that for all f
- z2 = C(_-z I) + 0(_-Zl) 2, (3.1)
then _ cannot be of second order.
Proof : If C _ I, then from (3.1) we could solve
for _ : r
z 2 - C z I 2= 1 - C + 0 (_-z I)
Z - C Z2 1
And z 2 = 1 - C would therefore produce a
second order approximation to _. Since P1 = I,
both pieces of information must be used then at z i'
but then _ is a one-point iteration, i.e. z 3 = z 2
by the convention, and from (3.1) and C _ 0 it fol-
lows that _ is only of first order.
Theorem 3.2.
P3 = 4
Proof : We prove that p_(N) _ 4 for all N with
_N = 3, where
p (N)=max {p lfor all _=f+G, G-0 mod N, _(_)=0,
and G monic polynomial of degree ( 3,
lim sup I_-_1 < _,}%
x Ix1- IPthereby restricting the class of _ such that
- f rood N.
Step 1
we need one evaluation of f at z to assure con-1
vergence so N is of the form
N = {f(zl) , f(i) (z2) , f(j) (z3) }
(Kung and Traub [74a]).
We will suppose z 2 and z 3 not necessarily diffe-
rent, unless of course i or j equals 0 or i = j.
It is clear this does not affect the bounds on the
optimal order.
Since now G is a monic polynomial of degree _ 3,
we can take i and j _ 2. Indeed, if i or j > 3,
G H 0 mod N is automatically satisfied ; if
i < j = 3, we can interpolate the zero function
for this information at z I and z 2 _ith a monicpolynomial of degree • 2 - from P2 = 2 it follows
that the optimal order is 2 < 4, and similarly if
j < i = 3. If i = j = 3 we can even take
G(z) = z - z I in which case the order of informa-
tion evidently is equal to 1 < 4.
Remark
The above argument can of course be generalized to
any n : it is closely related to the P61ya condi-
tions on the set N, see Wo_niakowski [75b], Sharma
[72].
The different cases for the information N.
With an obvious notation, in the following cases
the answer is already known :
Case 1 : {fl,f2,f3} : Hermitean N, order • 23-1=4
' f_} .- "Brant information with m=0"Case 2 : {fl,f2,
applying the results of Sac.
4, we find
if Zl_Z2:P(N)•m+2(k-l)-l=0+4-1=3<4 ;
if Zl=Z2:P(N)=m+2(k-l)+l=l+2+l=4
Case 3 : {fl,f_,f_} : "Abel-Goncarov" N, by Theo-rem 3.1. : order • 2.2 = 4
Case 4 • {fl' '• f_ f_} : Take again G(z)=z-z I as in
Step 1 ; order • 1 < 4 (here
the P61ya conditions are not
satisfied).
Step 3
Exhaustive checking of the remaining cases. Let us
set
G(z) = (z-z I) (z2+az+b)
Case 5 : {fl'f2'f3 }
Now G(z) = (z-zI)(z-z2) (z-c).
The condition _' (z 3) = 0 gives
(2z3-zl-z2) (z3-c)+(z3-zl) (z3-z 2) = 0
So in general, c is a function of z I, z 2 and z 3
which itself is also a function of z I and z 2.
Since P_ = 2, e-c cannot be of higher order than
(e-Zl) 22. Therefore
- _ = 0(G(_)) = 0(_(_)) = 0(_-z I) (e-z 2) (s-c)
4cannot be of higher order than (e-z I) since e-z 2
1 Thusis at most of order (_-z I) because P l = "
p (N) _ 4 for this N.
Case 6 : {fl'f2'f_ }
Completely analogous to case 5, the condition at
z 3 now is
(z3-c) + (2z3-zl-z 2) = 0.
Case 7 : {fl'f_'f3 }
Now G(z) = (z-zI) (z-z3)(z-c).
The condition at z2 : c = 3z 2 - z I - z 3, again
gives that (e-c) is at most of second order in
(_-zI).
Now (e-z 3) is at most of first order in (e-z I)
since the function
_(z) = z- z1
interpolates the zero function at the points z1
and z 2 for the given information. Thus with an ob-
vious notation,
_( ) - _ = 0(e-Zl) , and by theorem 2.1 and 2.2,
(_-z3) = 0(_-z I) at most.
Consequently,
e(G)-e = 0(_(e))=0(e-Zl) (e-z3) (e-c)
4is again at most of order (e-z I) .
Case 8 : {fl,f_,f_}
This is a permutation of the Abel-Gon_arov infor-
mation. It is an easy consequence of the proof of
Theorem 3.1 that also here we have,
p(N) _ 2 - 2 = 4.
A direct proof for this case can also be found,
and is left to the reader.
, f }Case 9 : {fl'f2' 3
G(z) = (z-zI) (z-z3) (z-c).
Again, (s-c) being of 3rd order or more in (e-Zl)
_ = 2 so if (e-z 3) is of orderwould contradict P2
1 in (e-z I) we are done. However (e-z 3) and (e-c)
cannot be both of second order, since the condi-
tion G' (z2) = 0 reads
(2z2-zl-z3) (z2-c)+(z2-z I) (z2-z 3) = 0
which is easily seen to be equivalent to
e 3(s-c)=e 2 (2el-3e 2)-(el-2e 2) (e3+(s-c))
i0
• = a - z i ; i=1,2,3.where e 1
Now e 3(a-c) cannot be of order 4 while by lemma
2 -3e ) is at most of Order 23.1 with C = 7' e2 (2el 2
and (el-2e2) (e3+(a-c)) is at least of third order.
Note that if 2z 2 - z I - z 3 = 0, this case becomes
non-poised (Sharma [72]) and the function
G(z) = (Z-Zl) (z-z 3) interpolates zero with respect
to this N, giving a maximal order of 3.
Theorem 3.3.
Hermitean information is not uniquely optimal.
Proof : We exhibit two examples for n = 3
a) Consider again case (6) of Theorem 3.9.
We have G(a) = (a-z I) (a-z 2) (a-c)
where c = 3z 3 - z I z 2.
To obtain order 4, this suggests we must have
2- c = 0(a-z I) , or
Zl+Z2 1 2(3.1) z3 - 3 + _y with y-a=8 (a-z I)
To find such a y, take
z I = x 1
z 2 = z I + f(z I)
and y the root of the interpolating (Ist degree)
polynomial at these two points, i.e.
z 2 - z 1
y = zI - f(zI) - f(zi)f(zI)
Then construct z 3 by (3.1) and
x2:= ¢(x I) := z 4
as the root of the (2nd degree) polynomial inter-
ii
polating f with respect to the information at Zl,
z2 and z 3. This _ethod is easily seen to be offourth order.
b) Case 3 with z I = z 2. Now G(z) = (Z-Zl)2(z-c)
and G" (z 3) = 0 gives 3z 3 = 2z I + c. By taking2 1
z3 = _Zl + _ Y where again _ - y = 0(_-Zl ) 2, for
example by a Newton-step, it is easy to show that
the following method has fourth order :
z I = x I (= z 2)
2 + 1 f (Zl)
z 3 : _ z I _ (z I - f,(zi))
X2:=#(x I) := z4 = zero of the second degree polyno-
mial interpolating f with respect to the informa-
tion at z I and z 3.
Remarks
The previous arguments permit the determination of
all arrangements of the information (_N = 3) which
can give optimal order. They are denoted by their
incidence matrices (Sharma [72]) as follows :
IOo0i]°IE) I 0 0 F) i 0 G) i 0 0]
100 10 01 0]001 01 00
The cases A) and C) have optimal generalizations
for all n > 3, it will be shown in a later paper
that also case F) can be generalized to a non-her-
mitean optimal case for all n.
12
4. A solution to a "problem 2" for N = Brent infor-mation.
Definition 4.1.
- i) i)Nm,£,k_{f(z ,f, (Zl) ,...,f(m) (z ;
f(£) (z2) ,...,f(£) (Zk) }
is called Brent-information, where the zi are dis-
tinct, m >. i, k >. 2 and _ _ i.
Brent has shown the following
Theorem 4.1. (Brent [ 74])
Assume £ _ m + I.
There exist methods using Nm Z k of order m+2k-l.f f
We will now prove that the Brent methods make op-
timal use of the information N Z k (with respectm, ,to order).
Theorem 4.2.
Let N _ k be as in definition 4.1. Then if Z_m+l,mt ,
p(N) = m + 2k - 1
If Z > m + i, p(N) = m + I.
Proof : A technique is used similar to theorem 3.2.
If _ > m + i, a function G(z) - 0 rood N is given
bym+ 1
G(z) = (z-z I)
And consequently I_- = •O{I = 0(_(e)) 0(e-Zl )m+l
By theorem 2.1 the maximal order is not more than
m + i. Methods realizing this order exist and are
trivial to find. Thus p(N) = m + i.
If £ ( m + i, we construct G(z) as follows.
13
To satisfy the conditions at z2,...,z k we must have
G (£) (z) = (Z-Zl)m-£+l (z-z2) (z-z3) ... (Z-Zk) H(z)
where H(z) is any (sufficiently regular) function.
Integrating £ times,
G(z) = (t-z) £-I (t-zl)m-£+l (t-z2) ... (t-Zk) H(t)dt.z 1
According to the remark at (2.2), we obtain an
upper bound by choosing a special H.
Take H(t) = (t-z 2) ... (t-zk) , making
G(z) = (t-z) £-i (t_zl)m-£+l [ (t_z2) ... (t_zk) I 2 dt.z1
Consider now G(_) = _(e). Transform G(e) to the
interval [-i,+I] ; after some easy calculations we
obtain
(_) = K(e-z I) £-I (__zl)m-£+l (__Zl) 2(k-l) (__Zl) .I(e)
where K M 0 does not depend on e or any of the z.1and where
+ 1 k z -z.I (e) = (l-T) £-l(l+T)m-£+l H (T+-I -i)2 _dT.
-I i=2 _-Zl
As is well known, I(_) is minimized when
k z -z.
H (T + 1 l) is equal to the (k-1) st monici=2 e-Zl
Jacobi polynomial corresponding to the weight
function (l-T) £-i (l+T)m-£+l.
Then I (e) >, c where c is independent of the zi.
(See for example G. Natanson : "Konstruktive Funk-
tionentheorie ") .
so = o = o p
14
with p = (£-i) + (m-Z+l) + 2(k-l) + 1
= m+ 2k- I.
Thus p(N) _ m + 2k - I, but by Brent's theorem and
theorem 2.1, we have equality.
Note : The previous theorem was independently dis-
covered by Wo{niakowski. A generalization of this
theorem is possible•
Theorem 4.3.
Let now N = {f(z I) ,f' (z I) ,... ,f(ml) (z I) ;
f(£) (z2) ,...,f(£+m2) (z2) ;
f(£) (z3) ,...,f(£+m3) (z3) ;
f(_) (Zk),...,f (£+mk) (zk) } ml>.l,
£_i.
Then if £ > m I + i, p(N) = m I + I, and if £_ml+l ,
(4•1) p(N) _ 1 + m I + 2. I ii=l
Proof : The proof runs analogously, with H replaced
byk e.
H(t) = H (t_zl) x with E i = f0 if m i odd
i=2 _i if m i even
(The Jacobi polynomial is now of degree 7 i ! )i=2
Remark 4.1.
Let m i = 1 for i = 2,...,k. Then p(N) _ m + 2k - I,
so we gain nothing compared to the Brent informa-
tion case ! In general the order cannot be raised
if all m i are even and we add the pieces of infor-
15
mation f (Z+mi+l) (zi) for i = 2,...,k.
Remark 4.2.
Contrary to Theorem 4.2 the inequality (4.1) is not
yet known to be an equality in general. If all m i
are equal, methods can be constructed by means of
so-called "s-polynomials" realizing the bound. For
a definition of these, see Ghizetti and Ossicini,
"Quadrature Formulae". Again, for details we refer
to a forthcoming paper on this subject.
Remark 4.3.
Kung and Traub's conjecture states that the optimal
order for a given number of pieces of information
will double by adding one extra piece of informa-
tion. That it is however possible to increase the
order more than twofold when it is not optimal, is
shown by the following example :
Let N = {f(zl) ,f'(zl) ,f"(z I) ; f'(z2) ,f"(z2) }.
By theorem 4.3 and remark 4.1 any method using this
information must have order at most 5. Adding the
element {f(z2)} to N, we get however an information
at which allows us to obtain order 12, as is easi-
ly shown. Although of course not a counterexample
to the conjecture - we believe it is true - it will
complicate any possible proof by induction.
Finally, we state without proof the following re-
sult, used in the proof of Theorem 3.1 :
16
Theorem 4.4.
If in Theorem 4.3 m I = 0 (and consequently, to
avoid trivial cases, 1 = i) the order of informa-
tion is bounded by
p(N) _ i + m 2 + I .i=3
References
Brent [74] Brent. R.. "Efficient Methods for Finding Zeroesof Functions whose Derivatives are Easy to Evalu-
ate," Department of Computer Science Report,
Carnegie-Mellon University, December, ]974.
Kacewicz [75] Kacewicz, B., "An Integral-interpolatory Meth-of for the Solution of Non-linear Scalar Equa-
tions," Department of Computer Science Report,Carnegie-Mellon University, January, ]975.
Kung and Traub [74a] Kung, H. T. and Traub, J. F., "OptimalOrder of One-point and Multipoint Iteration,"JACM 2], 643-65].
Kung and Traub [74b] Kung, H. T. and Traub, J. F., "OptimalOrder and Efficiency for Iterations with Two
Evaluations," to appear in SIAM J. Numer. Anal.
Sharma [72] Sharma, A., "Some Poised and Non-poised Problems
of Interpolation," SIAM Review ]4, ]972.
Traub [64] Traub, J. F., Iterative Methods for the Solution
of Equations, Prentice-Hall, ]964.
Woz'niakowski [74] Woz'niakowski, H., "Maximal StationaryIterative Methods for the Solution of Operator
Equations," SlAM J. Numer. Anal. ]], ]974, 934-949.
Wozniakowskl [75a] WozSniakowski, H., "Generalized Informa-tion and Maximal Order of Iteration for Operator
Equations," SlAM J. Numer. Anal. ]2, ]975, ]2]-]35.
17
/ i "Maximal Order of MultiWozniakowski [75b] Wozniakowski, H.,I!
point Iterations Using N Evaluations, to appear
in Analytic Computational Complexity, edited byJ. F. Traub, Academic Press, 1975.
UNCLASSIFIED
SECURITY CLASSIFICATION OF THIS PAGE (When Oila Entered)
REPORT DOCUMENTATION PAGE READINSTRUCTIONSBEFORE COMPLETING FORMt. RE_ORT
12 GOVT ACCESSION NO. 3. RECIPIENT'S CATALOG NUMBERNUMBER
ii
4. TITLE (and Subtitle) 5. TYPE OF REPOR _ & _ERIOO COVERED
OPTIMAL USE OF INFORMATION IN Interim
CERTAIN ITERATIVE PROCESSES 6 PERFORMING ORG DEDOR_ NU.REP
7. AUTHOR(m) 8. CONTRACT OR GRAN _ NUMBER_s_
Robert Meer sman NO 014-67 -A-0314-00 I0
NR 044-422.9 PERFORMING ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT. PROJECT TASK
AREA & WORK UNIT NUMBERS
Carnegie-Mellon University
Computer Science Dept.Pittsburgh, PA 15213
1 CONTROLLING ")_ICE NAME AND ADDRESS 12. REPORT DATE
Office of Naval Research ,, July 1975Arlington, VA 22207 ,_ NUMBER O_ PAGES
1814 MONITORING 4GENCv NAME & ADDRESSft/different from Contrnllini_ Office) 15. SECURITY CLASS (of this reDort)
UNCLAS S IFIED
15a. D_CLASSIFICATION DOWNGRADINGSCHEDULE
16 DISTRIBU_'_ON STATEMENT (of thts Report)
Approved for public release; distribution unlimited.
17 DIS'rRIBUTION STATEMENT (o! the abstract entered In Block 20, l! dl!!erent !zorn Report)
18. SUPDt EMENTARY NOTES
19. KEY WO_DS !Continue on reverse side tf necessary and Identity by block number)
20. A BS ? _ a __" ((_ontinuQ on reverse side If necese_P/ attd Idontifv b v block number)
none submitted
_OPM
_uu ,.,,,, ,_ 1473 EO,TION OF ' NOV 65 IS OBSOLETE UNCLASSIFIEDSECURITY CLASSIFICATION OF THIS PAGE (When ]ol, t_ Ent,_r_d)