On the Method of Least mTH Powers For a Set of Simultaneous Equations

Annals of Mathematics

On the Method of Least mTH Powers For a Set of Simultaneous EquationsAuthor(s): Dunham JacksonSource: Annals of Mathematics, Second Series, Vol. 25, No. 3 (Mar., 1924), pp. 185-192Published by: Annals of MathematicsStable URL: http://www.jstor.org/stable/1967908 .

Accessed: 12/11/2014 00:09

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

Annals of Mathematics is collaborating with JSTOR to digitize, preserve and extend access to Annals ofMathematics.

http://www.jstor.org

This content downloaded from 198.50.231.0 on Wed, 12 Nov 2014 00:09:05 AMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=annals

http://www.jstor.org/stable/1967908?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


ON THE METHOD OF LEAST mTH POWERS FOR A SET OF SIMULTANEOUS EQUATIONS.*

BY DUNHAM JACKSON.

1. Introduction. Let aii, a12, *. aip,

(1)* * * * * an,, ,2, .., ap

be a set of real quantities, the rank of whose matrix is n. That is, p > n, and the a's form n linearly independent sets of p quantities each. Let

bly N by. .. bp

be another set of p real quantities. The p simultaneous linear equations in Xie Xn:

ai Xi + + anl xn by

(2) a12 X1 + + an2 Xn - 2,

aip xi + + anp xn =bp

will have a solution if p n, but will generally have no solution if p > n. In the latter case, the question may be raised of determining x,, ***, x", so that the sum

? ai i+ * +ani Xn bia

for a given exponent m, shall be a minimum. For m = 2, this is the classical problem of least squares. For a general m,

it is intimately related to a problem which has been considered by the writer in a recent series of paperst on the approximate representation of

* Presented to the American Mathematical Society, December 30, 1920. t A. On functions of closest approximation, Trans. Amer. Math. Soc., vol. 22 (1921),

pp. 117-128; B. Note on a class of polynomials of approximation, ibid., vol. 22 (1921), pp. 320-326; C. On the convergence of certain trigonometric and polynomial approxima- tions, ibid., vol. 22 (1921), pp. 158-166; D. Note on an ambiguous case of approximation, ibid., vol. 25 (1923), pp. 333-337.

- 185



186 D. JACKSON.

a given continuous function. The function bi of the subscript i takes the place of the function f(x) of a continuous variable x, for which an approximate representation is sought, and the sets of numbers aki, k = 1, 2, .. ., ny regarded as constituting n linearly independent functions of i, correspond to the n linearly independent functions of x in terms of which the approximation is to be expressed. While the two cases, continuous and discrete, can be handled to a large extent along the same general lines, there does not seem to be so close an identity in detail as to render a separate treat- ment of the algebraic problem superfluous; and it is to a study of the latter problem that the present paper is devoted.

2. Existence of a solution of the problem of closest approximation,* m > 0. Let (3) 9)i aiixi +*** + anixns i 1~ 22 ... .^ py

and let H be the greatest of the p numbers I gi 1, for any given set of x's. We begin by establishing the lemma:

There exists a number P, completely determined by the matrix of the np numbers aki, such that

I xk I ? PH, k-1, 2, ., ny

however the numbers xk may be chosen. The order of the columns in the matrix (1) is immaterial, provided, of

course, that any permutation of them is accompanied by a corresponding permutation of the b's; let it be supposed for definiteness that the determinant d of the first n columns is different from zero. Let dki be the cofactor of ak, in the determinant o; then, from the first n of the equations (3),

Sk = U (4k 9)l + S02 + +dkn9)n)1 k = , 2,. . .,n.

Let 6' be the greatest of, the quantities I 6ki l; then, since | i ? < Hy

ixkl ? n~ld'/6$, and it suffices to take

P =n d'!.

* It is readily seen that the problem becomes trivial for m _ 0. The ground of this section is covered to some extent by F. Fliesz; Les systbmes d'6quations lintaires h une infnitO d'inconnues, Paris, 1913, Chapter m, see especially pp. 51, 54; but the purposes and methods of Riesz's discussion are altogether different, and he does not formulate the conclusions presented here.



THE METHOD OF LEAST mTH POWERS. 187

This proof is essentially the same as that given by Sibirani* for the continuous case. It would be possible, though not so simple, to give a demonstration corresponding more closely to that of Lemma I in the paper A.

For any exponent m> 0 and any given set of x's, let

Gmn

Since each term of the sum is positive or zero, no term can be greater than the whole sum, and

I 'mP l < Gms 9 gas l < Gj'nn X- 1 2, . -. 1 pr that is, H < KG,.".

Consequently, as a result of the preceding lemma, it is seen that: However the numbers xk may be chosen,

Ixk? < PGjtm k 1, 2,.,

where P has the same meaning as before. Now let a set of numbers b1, ..., bp, be given, or, as we shall say for

brevity, a vector (b1, ..., bp) (b). Let

gm = _ g i|

It is to be shown that the coefficients xk in the 9t's can be determined so that the value of gm shall reach its lower limit, the exponent m and the numbers aki and bi being held fast.

If the vector (b) is linearly dependent on the vectors (ak) = (akl, ..k., k= 1,2, . o ., n, it is possible to choose the x's so that as -= bi, i = 1, 2, . . .,p, and gm is reduced to its lower limit, zero. This means that the set of equations (2) has an exact solution.

If (b) is not linearly dependent on the vectors (ak), the a's and the b's together define a set of n +1 linearly independent vectors. The numbers

* Sulla rappresentazione approssimata delle funzioni, Annali di mat., ser. 3, vol. 16 (1909), pp. 203-221; p. 208; see also Tonelli, I polinomi d'approssimazione di Tchebychef, ibid., voL 15 (1908), pp. 47-119; pp. 61-62.



188 D. JACKSON.

(9'1 - bl, ..., o9p bp) result from a linear combination of these, with the coefficients xi, ... x., -1. If Q is, for the augmented system of vectors, the coefficient corresponding to the number P already associated with the original system, it will follow that

IXk I < Qgllm 1, 2, ...

the number Q, then, depends only on the a's and the b's. The last relation implies that if gm is near its lower limit, the point in

n-dimensional space which has the numbers xk for its coordinates belongs to a restricted region, which may be regarded as closed, and when gm is considered as a function of the x's, its lower limit is a minimum which can be attained by a suitable choice of the variables. Consequently, whether (b) is linearly dependent on the vectors (ak) or not,

There will exist at least one determination of x1, . .., xn, for wVhicZ the sum of the mth powers of the absolute values of the errors in the equations (2) is a minimum.

3. Uniqueness of the solution, m > 1. It remains to inquire whether the solution, just proved to exist, is unique. It is readily seen that when m < 1 this is not necessarily the case. For example, let the given quantities be taken as follows:

p - 2, n = 1, all = al 1, b 1, b2 -1.

The expression to be minimized is

gm I X. -m + I X1 + 1 Im.

As gm is an even function of xi, the determination of xl for the minimum can not be unique, unless it is the value xl = 0; but gm = 2 for xl = 0, while gm = 2"" for xi -4 1, so that the former value is not the minimum. The solution is in fact given by x] = 4-1.

If m = 1, the preceding example can still be used to show that the solution need not be unique.* The minimum in this case, g1 = 2, is reached for any value of xi in the interval 1 ? x1 < 1.

For m> 1, on the other hand, the solution of the problem of closest approximation is uniquely defined. Let the smallest value of gm be denoted by yTm. Let (x1, ..., x) and (xi', ..., xn) be two different sets of x's, if possible, which make gm = rm. Let A4 and cot' be the corresponding values

* This is an essential difference between the discrete case and the most important, at auy rate, of the continuous cases; cf. the paper B.




of spi. Because of the linear independence of the a's, 9sp and 9P' can not be equal for all values of i. Let h be a subscript such that 9h'+ t '. Then

|1(T (+ Tif?)-bh | 1(T' bh) + b ( h ) Im

< q ( | g41-bh I + I bh I'M)

This is an immediate consequence of the fact that the chord connecting any two points of the curve .q = I I"m lies above the curve, the mth power of the absolute value of the average of two numbers and the average of the mth powers of the absolute values of the same numbers being represented respectively by a point on the curve and a point with the same abscissa on a chord. (It is here that the demonstration fails for m < 1.) For any other subscript i, the same relations hold, except that the inequality becomes an equality whenever 9t = St'. At any rate, since the inequality holds at least once,

the quantity obtained on the right, on summation from i = 1 to i-= reducing to the value indicated. But the left-hand member in the last inequality is a value of gi, namely that given by Xk = I (xk + X), k = 1, 2, ..., n, and the hypothesis with regard to Tm is thereby contradicted.

4. Approximate solution in the sense of Tchebychef. A problem closely related to the one treated above, and suggested by the Tchebychef theory of approximation by means of polniomials,* is that of determining the x's so that the greatest of the errors I 9i -bi - in the equations (2) shall be as small as possible. It follows almost immediately from the first lemma of ? 2 that the problem has at least one solution; for the x's corresponding to restricted values of the errors will themselves belong to a restricted region, and the lower limit involved will be a minimum that can be attained.

Under the hypotheses made hitherto, the solution of the new problem is not necessarily unique. For example, let the given quantities be as in the illustration at the beginning of ? 3, except that a12 - 0 instead of 1. The problem is to determine xl so that the greater of the quantities

19'-bil = Ix1i-l, Iso2-.b.a bsf l, * Cf., e. g., Sibirani, Tonelli, loce. citt.; Borel, Lesons sur les fonctions de variables

rdeeles et les ddveloppements en sdries de polynomes, Paris, 1905, pp. 82-92; de la Valke Poussin, Legons sur l'approximation des fonctions d'une variable reelle, Paris, 1919, pp. 74-92.



190 D. JACKSON.

shall be as small as possible, and the requirement is satisfied by any value of xi in the interval 0 ? xi ? 2.

The solution uill be completely determinate, if it is required that every n-rowed determinant of the matrix (1) be differedt from zero*. The reasoning corresponds closely to that which is well known in the theory of Tchebychef polynomials.

In the first place, if x1, ..., xz form a solution of the problem, if T1, ..., 9pp have the values corresponding to these x's, and if I is the greatest of the errors I pi-bi 1, it must be that

Ipi-=bi I

for at least n + 1 different values of the subscript i. Suppose the maximum deviation were reached only r times, r _ n. Since the original order of subscripts is immaterial, there is no loss of generality in supposing this to occur for i = 1, 2, .., r. That is,

S - bi 1, i = ,...,r,

Sni -bil < 1,i r+l.,p

As every n-rowed determinant of the matrix (1), and, in particular, the determinant of the first n coluans, is assumed now to be different from zero, there will exist a set of numbers yl, ..., yn, satisfying the equations

aiiy, + * - * + aniYn = i -bi i = 1,2,. .., n.

The y's having been determined, let

=i - aliyl + *+ aniyn

for i = 1, 2, .. ., p. Let E be a positive number less than 1, and at the same time subject to the inequalities

EkbiI<l - i-biI, i = n+1,...,p. If the quantities

Xk Eyk, k =- 1, 2, ...,n

are substituted for the unknowns in the equations (2), the resulting errors

I j-Etpi-bi I

will be less than I in every case, and the hypothesis will be contradicted. * Cf. Sibirani, Su la rappresentazione approssimata di una funzione continua, etc, Rd.

Circ. Mat. Palermo, vol. .34 (1912), pp. 132-157.




Suppose now that there were two sets of numbers, (xI, .., x*) and (Xj',..., 4), each giving a solution of the minimum problem; and let 94 and , i = 1, 2, .. ., p, be the corresponding results of substitution in the left-hand members of the equations (2). For any particular subscript i, unless 4D = so' = bj?l, the quantity (94+94t'), corresponding to the substitution of s (4 + d'), k = 1, 2, . . ., n, will differ from bi by less than 1. As the difference can not exceed I in any case, the new values for the unknowns will also constitute a solution of the problem; by the preceding paragraph, it must be that I 1 (So + got') - bi =I for at least n + 1 values of i; and hence it must be that a, = t' for at least n + 1 values of i. But the equality of 94 and 9S', even for n values of i, implies that xk = xk4, k = 1, 2, ..., n. So the proof of uniqueness is complete.

5. Limiting form of the problem of least mth powers for m =co. Let xml, . .. xM. be the x's giving the solution of the problem of least mth powers, for an arbitrary value of m> 1; let SPn i = 1, 2, . . ., p, be the corresponding value of the left-hand member of the ith of the equations (2); and let 14 be the greatest of the differences ! gmi bi . Let lo be the limiting error in the problem of the preceding section, regardless, for the moment, whether the solution of that problem is unique or not, the hypothesis with regard to the matrix (1) being merely that it is of rank n; and let xol, ..., xo and 9Pol, ..., (pop be the corresponding set, or a corresponding set, of x's and SP's. It will be shown that

lim Im - to. In= 00

Let E be an arbitrarily small positive quantity, let Ym, as before, stand for the sum 7 l sj - bi I", and suppose that

to> lo+E

Then at least one term in the expression for ym is as great as (4+ e), and as the other terms are positive or zero,

rm > (? +)m.

On the other hand, because of the minimum property of ym, its value can not exceed that of the corresponding sum obtained by the substitution of any other set of x's, in particular xoi,..., xon. But



192 D. JACKSON.

and consequently

Spi - bil < pl1,an1

r in < P lo",, (lo + e)' < pl 4".

As the last relation ceases to be true for large values of m, it must be that

Im < lo + E

for all values of m from a certain point on, that is, limm= O m lo= Now let it be assumed that the determination of the numbers xo, .. ., Xon is

unique. From the preceding section, this will certainly be the case if the n-rowed determinants of the matrix (1) are all different from zero. By the proof just completed, im is bounded as m becomes infinite. That is, the quantities ! sei- bi I are bounded, and hence the same thing is true of the quantities !Pmi 1. From this it follows, by the first lemma of ? 2, that the points in n-dimensional space represented by the coordinates (xm,, .. *., Xm:nn) belong to a restricted region as m becomes infinite, and so have at least one limit-point* for nt = . On the other hand, if I represents the greatest of the p numbers I Ad bi I for an arbitrary set of x's, this I is a continuous function of the x's, and the value of I corresponding to any limit-point of a sequence of points (xmi, . . ., xmn) for mn = must be the limiting value of Im, namely lo. But, under the present hypothesis, the value 10 can correspond only to the single point (X0o, ..., XOn). So this is the only possible limit-point for m = x, and it must be that

idm Xnk =socks k =1, 2, .,n m =o

* That is, there will be a sequence of values of m, increasing without limit, such that the corresponding points (xCL..x..i) approach the point in question as a limit.

THE UNIVERSITY OF MINNESOTA,

MINN-EAPOLIS, MINN.



Documents

On the Method of Least mTH Powers For a Set of Simultaneous Equations