Interior Point Methods for Linear Programming: Just Call Newton, Lagrange, and Fiacco andMcCormick!Author(s): Roy Marsten, Radhika Subramanian, Matthew Saltzman, Irvin Lustig, David ShannoSource: Interfaces, Vol. 20, No. 4, The Practice of Mathematical Programming (Jul. - Aug.,1990), pp. 105-116Published by: INFORMSStable URL: http://www.jstor.org/stable/25061374 .Accessed: 15/05/2011 04:03

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unlessyou have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and youmay use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at .http://www.jstor.org/action/showPublisher?publisherCode=informs. .

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printedpage of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

INFORMS is collaborating with JSTOR to digitize, preserve and extend access to Interfaces.

http://www.jstor.org

Interior Point Methods for Linear Programming:

Just Call Newton, Lagrange, and Fiacco

and McCormick! Roy Marsten

Radhika Subramanian

Matthew Saltzman

Irvin Lustig

David Shanno

School of Industrial and Systems Engineering

Georgia Institute of Technology Atlanta, GA 30332

School of Industrial and Systems Engineering

Georgia Institute of Technology

Department of Mathematical Sciences

Clemson University, Clemson, SC 29631

Deptartment of Civil Engineering and Operations Research

Princeton University, Princeton, NJ 08544

RUTCOR, Rutgers University New Brunswick, NJ 08903

Interior point methods are the right way to solve large linear

programs. They are also much easier to derive, motivate, and

understand than they at first appeared. Lagrange told us how to convert a minimization with equality constraints into an un

constrained minimization. Fiacco and McCormick told us how

to convert a minimization with inequality constraints into a se

quence of unconstrained minimizations. Newton told us how to solve unconstrained minimizations. Linear programs are

minimizations with equations and inequalities. Voila!

This Interfaces issue on the current

state of the art in linear programming could not be complete without a status re

port on the new interior point methods.

This report will be very up to date (as of

late March 1990) but brief and to the

point. We have been heavily involved in

implementing these methods, but the rapid

progress and constant excitement have

made it difficult to pick a time to pause

and explain things to a wider audience. In

terfaces, as an informal forum, is ideal for

what is really a dispatch from the battle

field rather than a scholarly paper. In 1984, Narendra Karmarkar [1984] be

gan the "new era of mathematical pro

gramming" with the publication of his

landmark paper. Shortly thereafter his em

ployer, AT&T, invited the professional mathematical programming community to

roll over and die. Speaking as representa tives of this community, we took this as

rather a challenge. Some of us proceeded to make dramatic improvements to the

Copyright ? 1990, The Institute of Management Sciences

0091-2102/90/2004/0105$01.25 This paper was refereed.

PROGRAMMING?LINEAR TUTORIAL

INTERFACES 20: 4 July-August 1990 (pp. 105-116)

MARSTEN, SUBRAMANIAN, SALTZMAN

simplex method; for example, Robert Bixby with CPlex, John Forrest with OSL, and

Michael Saunders with the new MINOS.

Many others worked on bringing Karmarkar's method, which at first ap

peared to be coming completely out of left

field, into our classical framework for opti mization. The first and most important re

sult in this effort was Gill et al. [1986], which demonstrated an equivalence be

tween Karmarkar's method and projected Newton barrier methods.

A great deal of theoretical and computa tional work has been done since the excel

lent Hooker [1986] Interfaces article. We

will focus on the algorithms, their imple

mentation, and their performance, rather

than on complexity results. This is, in fact,

a good time because the whole subject has

just recently gotten much clearer and sim

pler. The theoretical foundation for interior

point methods consists of three crucial

building blocks. First, Newton's [1687] method for solving nonlinear equations and hence for unconstrained optimization.

Second, Lagrange's [1788] method for op timization with equality constraints. Third,

Fiacco and McCormick's [1968] barrier

method for optimization with inequality constraints. Using these three pieces, we

will show how to construct the primal-dual interior point method. The primal-dual is

the most elegant theoretically of the many

variants and also the most successful com

putationally. See, for example, Marsten et

al. [1989] and Lustig, Marsten, and

Shanno [1989]. The primal-dual method

has been developed by Megiddo [1986],

Kojima, Mizuno, and Yoshise [1988], Monteiro and Adler [1989], McShane,

Monma, and Shanno [1989], Choi,

Monma, and Shanno [1988], and Lustig,

Marsten, and Shanno [1989]. Newton

Recall Newton's method for finding a

zero of a function of a single variable:

f(x) = 0. Given an initial estimate x? we

compute a sequence of trial solutions

xk+1 = xk-f(xk)/f'(xk)

for k = 0, 1, 2, ... , stopping when \f(xk)\

< e where e is some stopping tolerance; for

example; e =

10"8.

Suppose we have n equations in n

variables: f(x) =

0, where

r1 and x =

The Jacobian at the point x, J(x), is de

fined as the matrix with (/, j) component

dXj (X)'

and Newton's method looks like

xk+i =

xk __

[/(x*)]"1/^*).

Or, if we let

dx =

xk+1 -

xk

denote the displacement vector and move

the Jacobian matrix to the other side

J(x")dx =

-f(xk). (1)

Newton's method can be applied to the

unconstrained minimization problem as

well: to minimize g(x) take/(x) =

g'(x) and use Newton's method to search for a

zero of f(x). Each step of the method can

be interpreted as constructing a quadratic

approximation of g(x) and stepping di

INTERFACES 20:4 106

INTERIOR POINT METHODS

rectly to the minimum of this

approximation. If g is a real valued function of n vari

ables, a local minimum of g will satisfy the

following system of n equations in n

variables:

f:w=o

In this case the Newton iteration ( 1 ) looks

like

V2g(xk)dx =

-Vg(xk),

where Vg is the gradient of g and V2g is

the Hessian of g; that is, the matrix with

(/,;) component

??(x). dXjdXj

If g is convex, then any local minimizer is

also a global minimizer. If x* is a local

minimizer of g(x); that is, Vg(x*) =

0, and

if V2g(x*) has full rank, then Newton's

method will converge to x* if started

sufficiently close to x*.

Lagrange

Lagrange discovered how to transform a

constrained optimization problem, with

equality constraints, into an unconstrained

problem. To solve the problem minimize

f(x) subject to gi(x) = 0

for i =

1, . . ., m

form a Lagrangian function

m

L(x,y) =

f{x)- 2m<(*)

and then minimize the unconstrained

function L(x, y) by solving the system of

(n + m) equations in (n + m) variables:

?k = ?Lix)_ZvM(x)

= o dXj dxj() ?1y'dxj(

'

for j =

1, . . ., n and

for i = 1, . . ., m.

These equations can be solved by Newton's method.

Fiacco and McCormick

General inequality constraints can be

converted to equations by adding non

negative slack (or surplus) variables. So

the only essential inequalities are the non

negativity conditions: x > 0. The idea of

the barrier function approach is to start

from a point in the strict interior of the in

equalities (x? > 0 for all /) and construct a

barrier that prevents any variable from

reaching the boundary (xf =

0). For exam

ple, adding "?log (x7)" to the objective function will cause the objective to in

crease without bound as x? approaches 0.

Of course if the constrained optimum is on

the boundary (that is, some xj

= 0, which

is always true for linear programming), then the barrier will prevent us from

reaching it. The solution is to use a barrier

parameter that balances the contribution of

the true objective function against that of

the barrier function.

A minimization problem with non

negativity conditions can be converted into

a sequence of unconstrained minimization

problems in the following way. The

problem

July- August 1990 107

MARSTEN, SUBRAMANIAN, SALTZMAN

minimize

f(x) subject to x > 0

is replaced by the family of unconstrained

problems minimize n

B(x\ix)=f(x)-? 2 log(zy) 7 = 1

which is parameterized on the positive barrier parameter ?x. Let x(/x) be the mini

mizer of B(x|??). Fiacco and McCormick

[1968] show that x(p) -

x*, the con

strained minimizer, as \x - 0. The set of

minimizers x(??) is called the "central

trajectory."

As a simple example, consider the

problem minimize

(Xl + l)2 + (x2 + l)2

subject to

XX > 0, x2 > 0.

The unconstrained minimum would be at

( ?

1, ?1), but the constrained minimum is

at the origin (x*, x%) =

(0, 0). For any \i

>0:

?(x|M)-(x1 + l)2 + (x2+l)2-Mlog

(x1)-Mlog(x2)

dB a ? = 2(x1 +

l)-^ = 0

OXi X\

dB a - = 2(x2 +

l)-^ = 0

ox2 x2

Xi(M) =

X2(M) = -- + -Vl + 2M

which approaches (0, 0) as ?x -* 0.

In general we can't get a closed-form

solution for the central trajectory, but can

use the following algorithm: 1. Choose ju0 > 0, set k = 0.

2. Find xk{?xk), the minimizer of B(x\?xk). 3. If pk < e, stop.

Otherwise, choose /j,k+1 < ?k.

4. Set k = k + 1 and go to 2.

Then xk(nk) -> x* as ^

-> 0.

In step 2, we can find x{?i) by using Newton's method to solve the system of n

equations in n variables:

dg(;,U)_d/(*) M_Q dx; dx; x;

for; =

1, . . . , n.

In practice, we do not have to compute

x(ijl) very accurately before reducing ?i.

Fiacco and McCormick extended the ear

lier work on barrier methods of Frisch

[1954, 1955], Carroll [1961], and Huard

[1964]. The Primal-Dual Interior Point Method

Armed with Newton's method for un

constrained minimization and the Lagran

gian and barrier methods for converting constrained problems into unconstrained

ones, we are ready for a fresh look at

linear programming. Consider the primal-dual pair of linear

programs:

(P) minimize cTx subject to Ax = b

x> 0,

(D) maximize bTy subject to ATy

+ z = c

2> 0.

The equations can be handled by La

grange's method and the nonnegativity conditions by Fiacco and McCormick's

barrier method. The resulting uncon

strained functions can then be optimized

by Newton's method. Suppose that we

INTERFACES 20:4 108

INTERIOR POINT METHODS

first introduce the barriers and then form

two Lagrangians: n

Lp(x, y) = cTx

- ?i 2 log (Xj)

- yT(Ax

- b)

7 = 1

n

Ld(x, y, z) =

bTy + ?i 2 log (Zj) 7 = 1

- xT(ATy + z- c).

Computing all of the partial derivatives

and setting them to zero, we get just three

sets of equations (because of duplication). These are

Ax = b

ATy + z = c

XjZj =

/i for j =

1, . . . , n.

These are just primal feasibility, dual feasi

bility, and an approximation to comple

mentary slackness. That is, p =

0 corre

sponds to ordinary complementary slack

ness. But in the spirit of barrier methods,

we will start p at some positive value and

let it approach zero.

If we let diag { } denote a diagonal matrix with the listed elements on its

diagonal, and define

X = diag{x?-x2},

Z = diag {z\,

. . . ,z?n}, and

<?T =

(1,...,1);

then the ^-complementary slackness

conditions can be written as

XZe =

lie.

Now we can use Newton's method (1) to solve the equations

Ax - b =

0,

ATy + 2 - c = 0, and

XZe -

tie =

0.

(2)

Suppose that the starting point (x?, y?, z?) satisfies x? > 0 and z? > 0, and let

dP = b

- Ax?

dD =

ATy? + z?-c

denote the primal and dual residual vec

tors. Then Jacobian (1) applied to (2) gives

A 0 0 0 AT 1

Z 0 X

dx

dy dz

dp ~dD

pe -

XZe (3)

A little algebraic gymnastics shows that to

solve ( 3 ) we have to solve

(AZ'1XAT)dy = b- \xAZ~xe (4)

- AZ ~lXdD

for dy, and then compute

dz =

?dD -

ATdy

dx = Z~l[iie

- XZe - Xdz].

(5)

(6)

Having computed the Newton step (dx,

dy,dz),we use dx as a direction in x-space and (dy, dz) as a direction in (y, z)-space.

We then choose step sizes in the two

spaces that will preserve x > 0 and z > 0.

We use

av - 0.995*min \

aD =

0.995* min

dXi

1^1 dZj

dxj < 0

dZj < 0

where the 0.995 factor keeps us from ac

tually touching the boundary. The actual

step is then

x1 = x? + apdx

July-August 1990 109

MARSTEN, SUBRAMANIAN, SALTZMAN

y1 =

y0 + aDdy

z1 =

z? + aDdz.

This completes one iteration. Rather than

making several Newton steps to converge to the central trajectory for a fixed value of

ix, we will reduce p at every step. The

prime-dual method itself suggests how p

should be reduced from step to step. It is easy to show that the Newton step

reduces the duality gap. Assume that we

have primal feasibility (Ax? =

b) and dual

feasibility (ATy? + z? = c). (For the gen

eral case see Lustig, Marsten, and Shanno

[1989].) Let "gap (0)" denote the current

duality gap.

gap(0)= cTx?-bTy?

= (ATy? + z0)Tx0-(Ax0)Ty?

= (z?)V.

Let a < min {aP, aD}. Then as we move

gap (a) =

(z? + adz)T(x? + adx)

= (z0)Tx? + a[(z0)Tdx + (x?)Tdz]

+ a2dzTdx.

But primal and dual feasibility imply dz =

-ATdy and Adx = 0, so dzTdx = 0. Thus

gap (a) =

gap (0) + a[{Zdx)Te + (Xdz)Te]

= gap (0) + a[Zdx + Xdz]Te.

Now we can use the third row of equation

(3) to obtain

gap (a) =

gap (0) + a[pe -

XZe]Te

= gap(0) + a[n/*-gap(0)].

So we see that gap (a) < gap (0) as long

as

gap(O) n

In our computational experiments we have

used

cTxk -

bTyk ^ = e(n) (7)

for k =

0, 1, 2, . . .

2 if n < 5,000 where 6(n)

=

nV? n> 5,000

which produces a substantial reduction in

A* at each step. The equations (3) are

solved for each k, giving a sequence of

solutions (xk, yk, zk). The steps continue

until the duality gap is sufficiently small:

cTxk -

bTyk i + i&yi<e

where e = 10~8 gives eight digits of agree ment between the primal and dual

objective values. The primal and dual

objectives may not both improve at each

step, but they come closer together.

Implementation The most surprising and important char

acteristic of the interior point methods is

that the number of iterations required is

very insensitive to problem size. Almost all

problems solved to date have required be

tween 20 and 80 iterations. We shall pre sent some results below that indicate that

the number of iterations increases with the

log of the number of variables. Given this

behavior, the advantage of an interior

method as compared to the simplex method depends on how efficiently the in

dividual steps of the interior method can

be executed.

INTERFACES 20:4 110

INTERIOR POINT METHODS

In discussing implementation, we will

draw upon our experience in developing the OBI system, Marsten et al. [1989],

Lustig, Marsten, and Shanno [1989]. The work at each step of the interior

method is dominated by the requirement to solve the system of equations (4). This

consumes about 90 percent of the compu

tational time. There are two major compu

tational tasks. First, we must form the

AD2AT matrix, where D2 is a diagonal ma

trix that changes at each iteration (for the

primal-dual method, D2 = Z_1X). Second,

we must solve the system of equations

(which is a condensation of (4)):

(AD2AT)d = r (8)

where d is a direction vector, and r is some

modified right-hand-side vector. The

(AD2AT) matrix is symmetric and positive definite.

Variations on the interior method (pri mal or dual affine, primal or dual barrier,

primal or dual projective) all have system

(8) in common. Only the definition of D

and r is different.

Forming AD2AT can be done efficiently if

A can be accessed by both rows and col

umns. This typically consumes about five

percent of the total computation time.

Solving the system of equations (8) could be done in at least three different

ways: QR factorization of DAT, Cholesky factorization of AD2AT, or preconditioned

conjugate gradients. If DAT = QR where

the columns of Q are orthonormal and R is

upper triangular, then (8) becomes

(RTR)d = r

which can be solved by solving two trian

gular systems. See Duff, Erisman, and Reid

[1987]. The conjugate gradient method could be

used to solve (8) since AD2AT is symmetric and positive definite. See Luenberger

[1973]. The number of iterations required

by the CG method depends on the number

of distinct eigenvalues of AD2AT. This

number can be reduced by "precondition

ing." Suppose that we compute an

approximate factorization

AD2AT ? LLT (9)

where L is lower triangular. The CG

method will, in general, take fewer itera

tions to solve the following system, which

is equivalent to ( 8 ) :

(L-1AD2ATL~T)d = r

where d = LTd and r = L~lr. If the factori

zation (9) is exact, then L is the Cholesky factor of AD2AT and the preconditioning is

perfect. That is,

L-lAD2ATL-T = I

there is only one eigenvalue, and the CG

method takes only one step. What this

means is that all of the work goes into

computing L and none into the CG algo rithm. This suggests that partial or incom

plete Cholesky factorization should yield reasonable preconditioners. See Mehrotra

[1989]. Most computational work to date

has been based on complete Cholesky fac

torization, without conjugate gradients, and that is what we will discuss here.

The Cholesky factorization

AD2AT = LLT (10)

where L is lower triangular, reduces the

equations (8) to

(LLT)d = r

MARSTEN, SUBRAMANIAN, SALTZMAN

which can be solved by solving

Lg = r (11)

for g and then solving

LTd = g (12)

for d. Typically, the factorization (10)

takes about 75 percent of the total compu tational time, while solving systems of the

form (11) and (12) takes about 10 percent of the time. Almost all of the factorization

time is spent at the innermost level in a

two-line DO-loop that includes an indirect

array reference. This innermost loop can

be vectorized if the computer has gather scatter hardware, yielding a very dramatic

improvement in performance. The factori

zation used in OBI is based on the mate

rial in Chapter 10 of Duff, Erisman, and

Reid[1987]. The reason that barrier methods could

not compete with the simplex method in

the 1960s is that the numerical linear alge bra for symmetric positive-definite matrices

had not yet been worked out. This was all

done in the '70s and '80s for the purpose of solving large finite element models. See

George and Liu [1981]. The Cholesky factorization approach to

solving ( 8 ) is based on exploiting two key facts. First, the pattern of nonzeros in the

Cholesky factor, L, does not depend on D

and is therefore the same at every iteration

of the interior method. Second, the num

ber of nonzeros in L depends on the order

ing of the rows of A. The work that has to

be done during the factorization and the

subsequent triangular solves is directly

proportional to the number of nonzeros in

L. Therefore it is crucial to order the rows

of A so as to obtain a sparse factor, L.

There are a number of heuristic algorithms for this ordering problem, the most popu

lar being minimum-degree ordering. (Find

ing the ordering that minimizes the non

zeros in L is a NP-complete problem.) For

a complete development, see George and

Liu [1981]. OBI uses a more recent ver

sion, multiple minimum-degree, from Liu

[1985]. To illustrate the overriding impor tance of row ordering, consider the

NETLIB [Gay 1985] problem, SHIP04S. With the rows of A in the given order, L

has 39,674 nonzeros (61 percent dense).

After a minimum-degree

row reordering,

the alternate L has 3,252 nonzeros (five

percent dense). The time required for the

multiple minimum-degree algorithm is

usually modest, although it can become

significant for very large problems, particu

larly since the algorithm consists entirely of graph manipulations that do not

vectorize.

The main weakness of the interior point methods involves the issue of dense col

umns. This surfaced right at the beginning, with the notorious NETLIB problem, ISRAEL. A single column in A with a non

zero in every row will cause AA T to be

completely dense. This is regardless of the

row ordering, and of course means that L

is completely dense. Thus columns of A

with many nonzeros will lead to relatively dense L factors. Fortunately, LP Models

usually have fewer than 10 nonzeros per

column, independent of the number of

rows. There are, however, models that

have some very long columns. There are

two approaches to dealing with this diffi

culty, and they both involve treating the

dense columns separately. Adler et al.

[1986] compute a Cholesky factor using

INTERFACES 20:4 112

INTERIOR POINT METHODS

just the sparse part of A, and then use it to

precondition a conjugate gradient method.

Gill, Murray, and Saunders [1988] use a

Schur complement mechanism (see Duff,

Erisman, and Reid [1987]) that requires a

sparse factorization of just the sparse part

of A (just as in Adler) and a dense factori

zation of a matrix whose size is equal to

the number of dense columns removed

from A. We have tried both approaches, and OBI currently includes the Schur com

plement mechanism. Our conclusion is

that a small number of dense columns (say

10) can be handled successfully, but that a

large number presents a very real stum

bling block to the use of interior point methods. Of course the meaning of dense

is relative and depends on the size and

structure of the model as well as the actual

amount of fill-in during the factorization.

Sometimes removing dense columns can

cause more fill-in, since the min-degree or

dering is just a heuristic! See Lustig,

Marsten, and Shanno [1989]. For an

example of reformulating a model so as to

eliminate dense columns, see Lustig,

Mulvey, and Carpenter [1989]. Performance

The recent paper by Lustig, Marsten,

and Shanno [1989] contains extensive

computational results comparing the OBI

primal-dual interior point code to the new

MINOS 5.3 simplex code on the NETLIB

test set of small to medium-sized models

[Gay 1985]. These results show the inte

rior method starting to dominate the sim

plex method when m + n (number of

equations + number of variables) is greater than 2,500. Another recent paper by

Lustig, Mulvey, and Carpenter [1989] con

tains comparisons between OBI and

MINOS 5.3 on a set of stochastic general ized network problems. These results show

OBI running about 10 times faster than

MINOS 5.3 on a set of 10 test problems, which have m + n ranging from 7,391 to

29,424.

We recently solved a set of multi-period oil refinery models with OBI and the XMP

simplex code. The results are shown in Ta

ble 1. Solving the 16-period model with

MPSX, Version 2, took over 5,000 cpu

seconds on an IBM 3090-600S.

Shootouts between OBI and CPlex are

being held regularly on a Cray Y-MP su

percomputer, and the results change daily as both codes undergo constant refine

ment. Shootouts between OBI and the

new IBM product OSL are being arranged on an IBM 3090 with vector facility. The

Number of Time Periods in the Model

1 2 3 4 8 16

Equations 442 883 1,255 1,659 3,275 6,507

Variables 1,032 1,945 2,644 3,506 6,874 13,610 Nonzeros 7,419 14,280 19,494 25,727 50,659 100,523

XMP* 40 123 418 683 3,207 16,331

OBI* 62 139 217 306 656 1,441 *

These are cpu seconds on a DECstation-3100.

Table 1: OBI (interior point) versus XMP (simplex) on a set of multi-period oil refinery models.

July-August 1990 113

MARSTEN, SUBRAMANIAN, SALTZMAN

Simplex method is still very much alive,

but the advantage definitely shifts to the

interior method as problem size grows. AT&T has marketed their implementa

tion of Karmarkar's method under the

name "KORBX." KORBX is a hardware

software combination, with the hardware

being an Alliant FX/8 computer. Recently Carolan et al. [1990] released a paper that

documents the performance of the KORBX

system when it was benchmarked by the

Military Airlift Command. Sixteen of the

large models used in this benchmark have

been made publicly available by Jeff

Kennington. These models have been

solved using OBI on a Cray Y-MP system

(single processor). The results are con

tained in Table 2, along with the results

from Carolan et al. [1990]. Carolan ran all four methods available

in KORBX on all of the problems. The re

suits in the "best KORBX" column are the

best time for each problem, although the

best method varied from problem to

problem. Of course the Cray Y-MP supercomputer

is a much faster machine than the Alliant

FX/8 (which has eight vector processors) that KORBX runs on. However, the OBI

results can be obtained on a $5 million

Cray Y-MP system, while the price tag on

KORBX (as of this writing) is $8.9 million.

Finally, we have some very interesting results on a family of airline crew schedul

ing problems (set partitioning problems). All of these models have 837 rows. The

matrix generator that produces the A ma

trix generates a significant number of du

plicate columns, which we remove during

preprocessing. The results on number of

variables versus number of primal-dual it

erations are shown in Table 3. Each larger

Time in Seconds

OBI Best KORBX

Model Rows Columns Cray Y-MP Alliant FX/8

cre-a 3,516 4,067 8 121

cre-b 9,648 72,447 103 1,134 cre-c 3,068 3,678 7 102

cre-d 8,926 69,980 96 876

ken 07 2,426 3,602 2 36

ken 11 14,694 21,349 23 396

ken 13 28,632 42,659 67 1,236

ken 18 105,127 154,699 680 15,840 osa 007 1,118 23,949 14 162

osa 014 2,337 52,460 30 348

osa 030 4,350 100,024 57 732

osa 060 10,280 232,966 184 2,586

pds-02 2,943 7,535 11 105

pds-06 9,881 28,665 295 1,692

pds-10 16,558 48,763 1,521 11,880

pds-20 33,874 105,728 15,972 63,720

Table 2: OBI (interior point) versus KORBX (interior point) on a set of Military Airlift

Command models.

INTERFACES 20:4 114

INTERIOR POINT METHODS

instance was obtained by adding addi

tional columns to the previously solved

problem, but the problem was always re

started from the default starting point,

making no use of prior knowledge from

previously solved subproblems. If we fit a

regression curve y = a + fr*log (n) where y

is the number of iterations and n is the

number of unique columns, we get

y = -218.07+ 51.507*log(n)

with R2 = 0.979.

The Future

We now have a robust, reliable, and effi

cient implementation of the primal-dual interior point method for linear programs.

The immediate future holds the challenge of carrying this new methodology into the

areas of nonlinear and integer program

ming, and also to very large specially structured linear programs. A major re

search question that comes up in all of

these areas is how to do "hot starts." That

is, how to take advantage of a "good"

starting solution. This is not yet well un

derstood, but this is still a very new ball

game.

The simplex method is still the method

Number of Nonduplicate Variables Variables Iterations

50,000 35,331 21

100,000 68,428 26

200,000 134,556 42

300,000 197,764 60

500,000 320,228 64

1,000,000 604,371 76

2,000,000 1,112,872 99

4,000,000 1,999,199 105

Table 3: Number of variables versus number

of iterations for OBI (interior point) on a set of airline crew scheduling models.

of choice for "surface phenomena," for ex

ample, sensitivity and parametric analysis. For the beginnings of a theory of sensitiv

ity analysis without bases see Adler and

Monteiro[1989].

Acknowledgments We thank John Gregory of Cray Re

search, Inc. for making the computer runs

shown in Tables 2 and 3. Cray and Cray Y-MP are federally registered trademarks

of Cray Research, Inc. MPSX and OSL are

federally registered trademarks of IBM.

KORBX is a federally registered trademark

of AT&T.

References

Adler, I.; Karmarkar, N; Resende, M.; and

Veiga, G. 1986, "An implementation of

Karmarkar's algorithm for linear program

ming," Technical Report 86-8, Operations Research Center, University of California,

Berkeley, California.

Adler, I.; Karmarkar, N; Resende, M.; and

Veiga, G. 1989, "Data structures and pro

gramming techniques for the implementation of Karmarkar's algorithm," ORSA Journal on

Computing, Vol. 1, No. 2, pp. 84-106.

Adler, I. and Monteiro, R. D. C. 1989, "A geo

metric view of parametric linear program

ming," Department of Industrial Engineering and Operations Research, University of

California, Berkeley, California.

Carolan, W. J.; Hill, J. E.; Kennington, J. L.;

Niemi, S.; and Wichmann, S. J. 1990, "An

empirical evaluation of the KORBX algo rithms for military airlift applications," Oper ations Research, Vol. 38, No. 2, pp. 240-248.

Carroll, C. W. 1961, "The created response sur

face technique for optimizing nonlinear re

strained systems," Operations Research, Vol.

9, No. 2, pp. 169-184.

Choi, I. C; Monma, C. L.; and Shanno, D. F.

1988, "Further development of a primal-dual

interior point method," RRR#60-88, RUT

COR, Rutgers University, New Brunswick,

New Jersey.

Duff, I. S.; Erisman, A. M.; and Reid, J. K. 1987,

Direct Methods for Sparse Matrices, Oxford

University Press, New York.

July-August 1990 115

MARSTEN, SUBRAMANIAN, SALTZMAN

Fiacco, A. V. and McCormick, G. P. 1968, Non

linear Programming: Sequential Unconstrained Minimization Techniques, John Wiley and

Sons, New York.

Frisch, K. R. 1954, "Principles of linear pro

gramming?With particular reference to the

double gradient form of the logarithmic po tential method/

' Memorandum of October

18, 1954, University Institute of Economics,

Oslo, Norway.

Frisch, K. R. 1955, "The logarithmic potential method of convex programming/' Memoran

dum of May 13, 1955, University Institute of

Economics, Oslo, Norway.

Gay, D. M. 1985, "Electronic mail distribution

of linear programming test problems," Math

ematical Programming Society COAL News

letter, December.

George, A. and Liu, J. W.-H. 1981, Computer Solution of Large Sparse Positive Definite

Systems, Prentice-Hall Inc., Englewood Cliffs,

New Jersey.

Gill, P. E.; Murray, W.; Saunders, M. A.;

Tomlin, J. A.; and Wright, M. A. 1986, "On

projected Newton barrier methods for linear

programming and an equivalence to

Karmarkar's projective method," Mathematical

Programming, Vol. 36, No. 2, pp. 183-209.

Gill, P. E.; Murray W.; and Saunders, M. A.

1988, "A single-phase dual barrier method

for linear programming," Report SOL 88-10,

Systems Optimization Laboratory, Stanford

University, Stanford, California.

Hooker, J. N. 1986, "Karmarkar's linear pro

gramming algorithm," Interfaces, Vol. 16, No.

4 (July-August), pp. 75-90.

Huard, P. 1964, "Resolution des P.M. a con

straintes non-lineaires par la m?thode de

centres," Note E.D.F. HR 5.690, May 6, 1964.

Karmarkar, N. 1984, "A new polynomial-time

algorithm for linear programming,"

Combinatorica, Vol. 4, No. 4, pp. 373-395.

Kojima, M.; Mizuno, S.; and Yoshise, A. 1988,

"A primal-dual interior point algorithm for

linear programming," in Progress in Mathe

matical Programming, ed. N. Megiddo,

Springer-Verlag, New York, pp. 29-48.

Lagrange, J. L. 1788, M?canique Analytique,

Paris, France.

Liu, J. W.-H. 1985, "Modification of the

minimum-degree algorithm by multiple elimi

nation," ACM Transactions on Mathematical

Software, Vol. 11, No. 2, pp. 141-153.

Luenberger, D. G. 1973, Introduction to Linear

and Nonlinear Programming, Addison-Wesley, New York.

Lustig, I. J.; Marsten, R. E.; and Shanno, D. F.

1989, "Computational experience with a

prime-dual interior point method for linear

programming/' Technical Report SOR 89-17,

Department of Civil Engineering and Opera tions Research, Princeton University,

Princeton, New Jersey.

Lustig, I. J.; Mulvey, J. M.; and Carpenter, T. J.

1989, "Formulating stochastic programs for

interior point methods," Technical Report SOR 89-16, Department of Civil Engineering and Operations Research, Princeton

University, Princeton, New Jersey.

McShane, K. A.; Monma, C. L.; and Shanno,

D. F. 1989, "An implementation of a primal

dual interior point method for linear pro

gramming," ORSA Journal on Computing, Vol.

1, No. 2, pp. 70-83.

Marsten, R. E.; Saltzman, M. J.; Shanno, D. F.;

Pierce, G. S.; and Ballintijn, J. F. 1989, "Im

plementation of a dual affine interior point al

gorithm for linear programming," ORSA Jour

nal on Computing, Vol. 1, No. 4, pp. 287-297.

Megiddo, N. 1986, "Pathways to the optimal set in linear programming," in Progress in

Mathematical Programming, ed. N. Megiddo,

Springer-Verlag, New York, pp. 131-158.

Mehrotra, S. 1989, "Implementations of affine

scaling methods: approximate solutions of

systems of linear equations using precondi tioned conjugate gradient methods," Techni

cal Report 89-04, Department of Industrial

Engineering and Management Sciences,

Northwestern University, Evanston, Illinois.

Monteiro, R. C. and Adler, I. 1989, "Interior

path following primal-dual algorithms, Part I:

Linear programming," Mathematical Program

ming, Vol. 44, No. 1, pp. 27-42.

Newton, I. 1687, Principia, London, England.

INTERFACES 20:4 116