THE BASIC THEOREMS OF REAL LINEAR EQUATIONS, INEQUALITIES, LINEAR PROGRAMMING, AND GAME THEORY

David Gale* Brown University

1. PREFACE The purpose of these notes is to present short and direct proofs of five important

theorems of real linear algebra, three existence theorems concerning solutions of linear equations and inequalities, and the basic theorems of linear programming and game theory. Needless to say, none of these theorems is new, and in the case of the existence theorems it would probably be impossible to determine when they first became known. In the last decade, however, they have assumed a new importance because of their application to a wide class of problems in fields connected with mathematical economics, engineering, logistics, etc. Since the same theorems occur over and over again in these applications, it seems desirable to record them in an organized manner for reference purposes.

It should be emphasized that we have made no effort to give "best possible" proofs from a pedagogical point of view, because a number of expositions of this sort already exist. Thus, our treatment will not contain any numerical examples, plausibility arguments, dia- grams, bibliographical references, or convex cones. This presentation is intended rather for the reader who is willing to take his algebra straight.

On the other hand, an effort has been made to make the presentation as direct as pos- sible. There are no "prerequisites" except familiarity with the algebraic properties of the real numbers. There are no preliminary lemmas, and no concepts are introduced but those directly involved in the theorems.

A s to the choice of theorems, the inclusion of the results on games and programming requires no justification. Concerning the existence theorems, a word should be said.

Let u s consider the problem of determining whether or not a set of simultaneous equations has a solution. In case the system does have a solution, the proof of existence can be carried out "constructivelytt by the simple means of exhibiting a set of numbers and verify- ing that they satisfy the equations. On the other hand, if the system has no solution, there is no constructive way directly from the definition to prove nonexistence, for it is clearly impos- sible to examine in turn each of the infinite number of candidates for a solution and show that it fails to satisfy the equations. Now, the existence theorems in the next section a re intended to remedy this situation by giving constructive criteria for ascertaining when a system of equations or inequalities has no solution. The results actually take the form of nonexistence

*Manuecript received 23 September 1956 193

194 D. GALE

theorems, which say that a given system has no solution if , and only if, a certain lldualll system has a solution.

A s imilar situation obtains in the case of extremum problems. In order to show from the definition alone that a given function attains a certain value p as its maximum, one would have to examine all other values of the function and verify that they were no greater than p . This impossible task is circumvented in l inear programming by the duality theorem which asserts that p is the maximum of a certain function if, and only if, it is the minimum of the "dual" function. Now the problem can be solved constructively by simply exhibiting the argu- ments for which the values of the two functions are equal. (Note, on the other hand, that the problem of showing that a given value of a function is a maximum can be solved construc- tively by simply exhibiting a la rger value of the function.)

In the next section we list the familiar vector and matrix notations to be used in the statements and proofs of the theorems. While this may seem unnecessary, there is still so much diversity of notations, even in such a standard subject as linear equations, that once again for the sake of completeness and convenient reference we include our own glossary. The reader is assumed to be familiar with these matters and is encouraged to skip the next section and get on to the theorems, turning back if necessary to the section on definitions if some s tep in a proof is not clear.

its predecessor, except for Theorem 2, which is not needed for the resul ts that follow but is included because of its importance in many other applications. We have given each theorem a title in order to stress the logical sequence of ideas.

Finally, we remark that the only property of the real numbers used in our proofs is the fact that they form an ordered field. Thus all our theorems are also valid, with the real num- bers replaced, for instance, by the rational numbers. As an application of this fact it follows, f o r example, that a game with a rational payoff matrix has a rational value.

The theorems of the last three sections are listed in logical order, each depending on

2. DEFINITIONS AND NOTATION

is called the ith coordinate of x. We shall use the symbol 0 to denote the n-vector all of whose coordinates are zero. The symbol ei, the ith unit vector, is the n-vector (El, . . . , En) such that E i = 1, E . = 0 for j # i. Note that a 1-vector is simply a real number. We shall consistently denote vectors by Latin letters and real numbers by Greek letters.

operations on n-vectors.

An n-vector x is an ordered n-tuple of real numbers (El, . . . , En). The number E i

3

The set of all n-vectors is called real n-space, denoted by Rn. We define three

(1) Addition. If x = (El, . . . , E n) and y = (ql, . . . ,TI J, then

(2) Scalar Multiplication. If x = (El, . . . , E n) and X is a real number, then

Xx=(XE1, ... , A t n ) .

(3) Scalar Product. If x = (El, . . . , En) and y = (ql, . . . , q n) are vectors, then

BASIC THEOREMS ON EQUATIONS AND INEQUALITIES 195

We shall take for granted all the properties of these operations which follow imme- diately from the definitions and will u s e them freely in the proofs of the next sections without giving explicit justification. Examples of such properties are:

( A + u ) x = A x c b x ,

A(x+y)= A x + A y ,

(x, x) 20, and (x, x) = 0 if, and only if, x = 0 .

We also define a relation between n-vectors.

E . < n . for i = 1, ... , n . (4) Partial Order. If x = (El , . . . , E,) and y = (nl, . . . , nn) , then XI y and y > x if

One immediately verifies among others the following properties: If x 2 y and y 2 z , then X ~ Z ,

1- 1

If X 1 2 Y 1 and X2,Yp then + X 2 2 Y l + Y q t

If x k y , then AxkAy for A 2 0

Xx<Xy for XSO, If x > y - and a 2 0 , then (a, x)>(a, y ) .

We shall fly in the face of convention and call the vector x positive if x 20. The usual term for such vectors is 'Inon-negative." We have chosen the unusual terminology to avoid the frequent use of this awkward double negation in what follows. Note that in our terminology the vector as well as the number zero is positive (and also negative).

An mxn-matrix is an "array" of numbers CY.. i = 1, . . . , m, j = I, . . .., n. The ith 11'

row vector ai of A is the n-vector (ail, . . . , aln). The jth column vector a] of A is the

9'

m-vector (a lj, . . . , The number a i j is called the ijth coordinate of A. of the matrix A is an nxm-matrix whose' ijth coordinate is a!.

where a!. = aji. 4

(5) Matrix Multiplication. If A is an mxn-matrix with column vectors a , . . . , a , and x = (El, . . . , En) is an 1 n

n-vector, then

Immediate properties: If ai is the ith row vector of A, then the ith coordinate of Ax is (ai, x).

A (X + y) = A X + A y , A(Ax) = A A X ,

and especially important is the following:

196 D. GALE

If x is an m-vector, y an n-vector, then

(Ax, Y) = (x, A' Y) .

3. EXISTENCE THEOREMS

tively, ai and aj; x is an m-vector; and y an n-vector. In this section, A will be an mxn-matrix whose ith row and jth column are, respec-

THEOREM 1: (Positive Solutions of Linear Equations) The equation

A y = a

has no positive solution if, and only if, there exists x such that

A ' x L O and ( a , x ) > O .

PROOF: If y is a positive solution of (1) and A' x SO, then

(a, x) = (Ay, x) = (Y, A' XIS (Y, 0) = 0 ,

so (2) has no solution. The converse is proved by induction on n. For n = 1, (1) and (2) take the form

1 a n = a

(a', x) 5 0 and (a, x) > 0 . W1

(2)1

1 Assuming (1)1 has no positive solutions, if a' = 0 then x = a satisfies (2)1. If a # 0,

1 1 1 1 1 let x1 = (a , a ) a - (a , a) a and observe that (a , xl) = 0. If (a, xi) # 0, then one easily

verifies that x = (a, xl) x1 satisfies (2)1. Finally, if (a, xl) = 0, then (xa, xl) = (a , a )(a,xl) - (a', a ) ( a , xl) = 0 so x1 = 0, and hence a = ((a , a)/(a , a )) a . Since we have assumed

has no positive solution it follows that (a', a ) < 0, hence again x = a satisfies (2)1.

1 1

1 1 1 1 1

Now assume the theorem t rue for a matrix of n - 1 columns. For convenience we n- 1 0 write a = a . By the hypothesis of the theorem the equation C TI. ai = ao has no positive j =1 J

solution, hence by the induction hypothesis there exists x1 such that (aj, xl) 5 0 for j < n - 1

and (a , xl) 3 0 . If also (an, xl) SO, then x = x1 satisfies (2). If (an, xl) >0, define 0

- aj= (an, xl) aj - (aj, xl) an for j = 0, . . .,n - 1 .

BASIC THEOREMS ON EQUATIONS AND INEQUALITIES 197

Now, if

has a positive solution, then

is a positive solution of (l), contrary to hypothesis. Therefore, applying the induction hypothesis to (l'), there exists x2 such that

(d, x2)<O and (Z 0 , x2) >O for j = 1, ... , n - 1 . (2' 1

Let x = (an, x1)x2 - (an, x2)x1 and verify

(3)

(4)

(5)

(a', x ) = 61, x2) for j = 0, . . . , n - 1 ,

(an, x) = o .

Then (2'), (3), and (4) imply (2), completing the proof.

THEOREM 2: (Solutions of Linear Inequalities) The inequality

has no solution if, and only if, there exists a positive x such that

(6) A' x = 0 and (a, x) < O .

PROOF: If y is a solution of (5) and x is positive and satisfies A'x = 0, then

(a, x) >(fly, x) = (Y, A'X)= (Y, 0) = O ,

so (6) has no solution.

equations Conversely, suppose (6) has no positive solution. That is, letting a = (al, . . , , am), the

m C e ia i= 0 i= 1

m

i= 1 c eiai= - 1

have no positive solution, hence by Theorem 1 there exists a vector yl and number q such that

D. GALE

(ai, yl) + nai 5 0 , i = 1, . . . , m and - n >O .

Let y - l/n yl, and verify from (6') that y satisfies (5).

THEOREM 3: (Positive Solutions of Linear Inequalities) The inequality

A Y l a

has no positive.solution if, and only if, there exists a positive x such that

A'x 2 0 and (a, x) >O

PROOF: If y is a solution of (7) and x is positive and satisfies A' x = 0, then

(a, x) >(Ay, x) = (Y, A'x) >(Y, 0) = 0

so (8) has no solution. Conversely, i f (7) has no solution, then the equation

(7')

has no solution TI. >0, h i 20 where ei are the unit vectors in Rm. By Theorem 1 there exist x1 such that

3 -

(aj, xl) 5 0 , (ei, xl) 2 0 and (a, xl) >O .

Let x = - xl, and verify from (8') that x satisfies (8).

4. DUALITY THEOREM OF LINEAR PROGRAMMING Let A be an mxn-matrix, a an m-vector, b an n-vector. Let Y be the set of all positive solutions of

and let P = m a t (b, Y) . Y

Let X be the set of all positive solutions of

and let v = min (a, x) . X

BASIC THEOREMS ON EQUATIONS AND INEQUALITIES 199

THEOREM: A necessary and sufficient condition for the existence of p o r v is that both X and Y be non-empty, in which case p = v.

PROOF: We first show the necessity of the condition. If X is empty, then clearly v does not exist, so suppose x1 E X and Y is empty. Then (9) has no positive solution, so by

-em 3 there exists x2 2 0 such that A' x2 >O and (a, x2) < 0. Hence, for any positive A, x . ' v- satisfies (10) and (a, x) can be made arbitrari ly small by choosing X suffi- ciently large, ._ I

- * does not exist. A symmetric arb

Assuming now that X and Y are non-empty we prove the existence and equality of p :nt gives the corresponding result for p .

and v. First observe that

(11) (b ,y)<(a ,x) f o r a l l X E X , Y E Y ,

so it remains only to show that for some x E X and y E Y, (a, x) 5 (b, y), which is equivalent to the statement that the inequalities

have a positive solution. If they do not, then by Theorem 3, there exists a positive m-vector u and n-vector v and a number tl 2 0 such that,

(13) A' u - tlb 2 0, - A v + tl a 2 0 and (a, u) - (b, v) < 0 . First, suppose tl = 0. Then, from (131, A' u 2 0 and - Av 2 0 ; but since (9) and (10)

have solutions, it follows from Theorem 3 that (a, u) 2 0 and (-b, v) 20, contradicting the third inequality of (13).

On the other hand, if tl > 0 and li = l/nu, 7 = lltlv, then from (13)

A'E> b, AT La and (a, E) < (b, 7) , -

but then ii EX and 7 E Y and this contradicts (11). Thus (13) has no positive solutions, and the proof is complete.

m

i= 1

5. FUNDAMENTAL THEOREM OF GAME THEORY Let X be the set of all positive m-vectors x = (if,, . . . , 4,) such that

n Let Y be the set of all positive n-vectors y = (ql, . . . , qn) such that c

ti = 1 .

r\. = 1 . j= 1 I

Let A be an mxn-matrix.

THEOREM: There exists xo EX, yo E Y and a number w such that

(xo, ~ y ) > w > ( ~ ' x , yo) for all X E X , ye^.

200 D. GALE

PROOF: We f i r s t consider the case A >O. Let e be the m-vector, f the n-vector, all of whose coordinates are 1. Now the set Y1 of all positive solutions of A y < e is clearly non-empty (let y = 0) and bounded, for since A > 0, no coordinate of y can exceed l /min (a ).

(i,j) ij Therefore, the numbers (f, y), y E Y1 are bounded above, and hence by the duality theorem there exists y1 E Y1 and x1 positive such that A'xl 2f and (e, xl) = p = (f, yl) >O. Let

w = l/p, xo = wxl, yo = wyl. Then x,,EX, yo EY, AyoL we, A'xo >wf, and hence if X E X ,

yeY. Then

If A has some negative coordinates, choose v < min a! and let be the matrix with - ij

coordinates F.. = Q

w >@'X, yo) for all x EX, y EY. But x y = A y - vf and a' x = A x - ye , so that letting

w = G + u one again obtains (14).

- v . Then x > 0, so there exists xo EX, yo E Y, and W such that (xo, A y) 2 11 il -

* * *