Poly Aff Algms 4 Lin Prog -- Clovis C. Gonzaga

8/8/2019 Poly Aff Algms 4 Lin Prog -- Clovis C. Gonzaga

1/15

Ma thema t ical Program ming 49 (1990) 7-21 7N o r t h - H o l l a n d

Polynomial affine algorithms for linearprogrammingClovis C. GonzagaDepar tment of Systems Engineering and Com puter Sciences, COPP E-Federal University of Rio de Janeiro,21941 Rio de Janeiro, R J, Bra silReceived 28 March 1988Revised manuscr ipt received 23 January 1989

The method of s teepest descent wi th scal ing (af f ine scal ing) appl ied to the potent ia l funct ion q log c'x -~i ' - i log xi solves the l inear pro gram min g p roblem in poly nomia l t im e for q~> n. I f q = n , then thea lgor i thm t e rmina tes in no more than O(n2L) i teratio ns; if q ~> n +x /n with q = O( n) the n i t takes nomore than O(nL) i terat ions. A modif ied a lgor i thm using rank-1 updates for matr ix inversions achievesrespect ively O(n4L) and O(n3"SL) a r i thmet ic comp ut ions.Key words: Linear programming, af f ine a lgor i thms, Karmarkar 's a lgor i thm, in ter ior methods.

1. Introduction

This paper is the result of a reflection on what mechanisms make Karmarkar'salgorithm [4] polynomial. The ingredients in the algorithm are: (i) the potentialfunction, (ii) scaling, (iii) steepest descent directions, (iv) the project ive transforma-tion int roduced in [4] or conical projections according to [3], and (v) the knowledgeof the cost of an optimal solution.

The conclusion seems to be very interesting: there is no need for the projectivetransformat ion to obtain polynomial algorithms whose analyses rest only on potentialfunctions. We shall study the method of steepest descent with scaling applied tothe potential function f ( x ) = q log c'x-~7_ 1 og xi, with the following results: ifq = n (and f ( . ) is Karmarkar' s potential function), then the algorithm detects anoptimal solution in no more than O(n2L) iterations; if q~> n+ x/ n with q=O(n)then it takes no more than O(nL) iterations. A modified algorithm using rank-1updates for matrix inversions achieves respectively O(n4L) and O(n35L) arithmeticcomputations.

Several presently known algorithms solve linear programming problems in poly-nomial time without projective transformat ions, achieving a complexity of O(x/~L)iterations by following the central trajectory defined by Bayer and Lagarias [1].These algorithms use Newton-Raphson iterates in a method of centers approach(Renegar [9] and later Vaidya [12]), in a penalty function approach (Gonzaga [2]),or in a primal-dual approach (Kojima, Mizuno and Yoshise [5], Monteiro andAdler [8]). Todd and Ye [11] studied several primal and primal-dual algorithmsand also arrived at interesting properties associated with the parameter q = n + x/~


2/15

8 C.C. Gonzaga / Polynomial affine algorithms for LPi n t h e p o t e n t i a l fu n c t i o n . A l t h o u g h m o s t p a t h - f o l l o w i n g m e t h o d s c a n b e i n t e r p r et e da s s c a l ing a nd s t e e pe s t de sc e n t a lgo r i t hms , a l l r e ly on some pa r a me te r t ha t i s( p r e se n t l y ) u p d a t e d b y s h o r t s t e ps : t h e c o m p l e x i t y a n a l y s es a r e b a s e d o n t h e s p e e dw i th w h ic h the se pa r a me te r s c a n be d r ive n to z e r o .

T he no ta t ion a nd a s sumpt ions a r e i n t r oduc e d in t h i s s e c t ion . S e c t ion 2 de sc r ibe ss o m e t o o l s n e c e s s a r y f o r p r o v i n g t h e c o m p l e x i t y b o u n d s i n S e c t i o n 3 . S e c t i o n 4m o d i f i e s t h e a l g o r i t h m t o u s e r a n k -1 u p d a t e s i n th e c o m p u t a t i o n o f in v e rs e m a t r i ce sa n d t h u s r e d u c e t he n u m b e r o f a ri t h m e ti c a l c o m p u t a t i o n s d e m a n d e d b y th ea lgo r i thms .The problemC o n s i d e r t h e f o l l ow i n g l i n ea r p r o g r a m m i n g p r o b l e m :

m i n i m i z e g ' xsubje c t to Ax = b , (1)

X~0,wh ere ~ c R" , b c ~m , /~ i s an m x n fu l l - r an k ma tr ix , 0 < m < n .

A ss um e tha t t he va lue ~ o f a n op t im a l so lu t ion i s z e r o , tha t t he f e a s ib l e s e tS= {x eR ' l x >~O, X,x= b}

i s c om pa c t , a nd tha t a n in i t i al n on - o p t im a l f e a s ib l e so lu t ion x i n t he r e l a ti ve i n t e r io ro f S i s know n . W e a l so a s s um e tha t t h i s i n i t i al so lu t ion i s suc h tha t f ( x ) < qL,w he r e f ( . ) i s t he po te n t i a l f unc t ion to be de f ine d in ( 2 ) .

T h e a b o v e h y p o t h e s e s m a y b e a s s u m e d w i t h o u t l o ss o f g e n e ra l it y , a s w a s s h o w nby K a r m a r k a r [ 4 ] . T he s t r onge s t hypo th e s i s i s c e r t a in ly the a s su mp t ion tha t ~ = 0 ,a nd i t c a n be c i r c umve n te d in tw o w a ys : f i r s t , by bu i ld ing a h ighe r d ime ns ionp r i m a l - d u a l p r o b l e m , f o r w h i ch t h e o p t i m a l c o s t (d u a l i ty g a p ) i s z e ro ; s e c o n d , b yu s i n g l o w e r b o u n d s t o a p p r o x i m a t e ~ w i t h s o m e u p d a t i n g s tr a te g y . T h i s l a st a p p r o a c hw a s d e v e l o p e d b y T o d d a n d B u r re l l [1 0 ] f o r K a r m a r k a r ' s a l g o r i t h m , a n d w i ll b ec o m m e n t e d o n i n S e c t i o n 5 .

T o t h i s p r o b l e m a logarithmic poten tial function i s a ssoc ia ted :x > O~-~f(x) -- q log Ux - ~ log xi, (2 )

i 1

w he r e q i s a pos i t i ve r e a l numbe r . T he po te n t i a l f unc t ion i s d i f f e r e n t i a b l e f o r a nyx > 0 and

Vf(x)=gq-~x8-[x[l], V f ( e ) = t / c - e ' g ' e (3)wh ere [z i ] den otes the vec t or wi th co m po ne nt s z i , i = 1 , . . . , n , and e = [1 1 1 ] '.

I t i s easy to see tha t i f a seque nce {x k} in S i s such th a t f ( x k) ~ - e c , t h e n a n ya c c u mu la t io n po in t ~ o f { x k} i s a n op t ima l so lu t ion f o r ( 1 ), s i nc e ne c e s sa r i l y U x k -> 0 .


3/15

C.C. Gonzaga / Polynomial afiine algorithms for LP 9The a lgo r i thm s to be s tud ie d ge ne r a t e s e que nc e s suc h tha t f o r a l l k , f ( x k ) < ~f ( x k - l) -- a , w h e r e a d o e s n o t d e p e n d o n k .T he a l gor it hm sThe a lgo r i thm s a r e s im p ly s t e e pe s t de sc e n t m e thods w i th sc a l ing ( a f f ine sc a l ing )a pp l i e d to the po te n t i a l f unc t ion . D i f f e r e n t c om ple x i ty r e su l t s w i l l be ob ta ine d f o rd i f f e re n t va lue s o f t he pa r a m e te r q a nd o f t he sc a l ing m a t r ix u se d in the a lgo r i thmm ode l be low . The sc a l ing ope r a t ion i s e xp la ine d in S e c t ion 2 .

T h e a l g o r i t h m u s e s a l o o s e l y s t a t e d a p p r o x i m a t e l i n e s e a r c h . O u r e x a c t i n t e n t i o ni s e xp la ine d a f t e r t he a lgo r i thm .Algorithm 1.1. G i v e n x > 0, x e S , an d e > O.k : = 0 .

R e p e a tS c al in g : A : = A D , c : = D6 , y : = D l xk , w h e r e D > 0 i s a d i a g o n a l m a t r i x , f ( y ) =

nq log c ' y - Y ~ i = l log Yi.P r o je c ti o n : h : = - P V f ( y ) , w h e re P = I - A ' ( A A ' ) ~ A i s t he p r o je c t ion m a t r ixo n t o N u l l ( A ) .L i n e s e a rc h : F i n d a n a p p r o x i m a t e s o l u t io n f o r

:= a rg m in {f (y + hh ) ]h > 0 , y + hh > 0}.Resu lt : y* := y + hh.B a c k to the o r ig ina l spa c e : x k~ := D y * .k : = k + l .

Un t i l Y x k 0 ~--> og a c E .


4/15

10 C.C. Gonzaga / Polynomial affine algorithms fo r L PC o n s i d e r a p o i n t a > 0 a n d a n i n c r e m e n t 6 > - a . Us i n g T a y l o r ' s t h e o r e m ,

6l o g ( a + 6 ) = l o g a + - + o ( a , 6 ) ,a

w h e r e1o(a , 6 ) - - - 6 2 O b e t w e e n O a n d 6.2(a + 0) 2

I t f o l l o ws t h a t6 6 2lo g( a + 6)/> log a -~ (4)a 2 ( a - l a l ) z "

In pa r t i cu la r , fo r a - ]a ] > 0.5,6log (a + 6 ) ~> log a + - - 26 2 . (5 )a

N o w c o n s i d e r t h e f u n c t i o ny ~ " , y > O ~ f z ( y ) = ~ logy~.

i - -1

Co ns id er a lso a po int y c R", y > 0 , a di rec t ion h ~" and a s teple ngt h A > 0 suchth at y + Ah > 0. Th ennf2(y+Ah) = Z log(y,+Ah,) ,

i - - 1

f 2 ( y + A h ) = ~ logyi+Ah'[yT1]+ ~ oi (Yi , Ahi).i = 1 i ~ l

I n p a r t i c u l a r , f o r Yi -a lh , I I> 0 .5, i = 1 , . . . , n ,2f 2 (y + A h ) ~ f2 ( y )+ A V f 2 ( y ) ' h - 2 A hi,

i = 1

orf z ( y + A h ) ~ f 2 ( y ) + A V f 2 ( y ) ' h - 2A2 [[h [[2. (6)

C o n s i d e r f i na l ly t h e p o t e n t i a l f u n c t i o n f ( ) . T h e f ir st term, f l (y ) = q log Uy, is strict lyc o n c a v e a n d c o n s e q u e n t l y o v e r e s t i m a t e d b y t h e l i n e ar a p p r o x i m a t i o n . I t f o l lo w s t h at

fl( Y + Ah) ~


5/15

C.C. Gonzaga / Polynomial affine algorithms fo r LP 11The scaling operationCon s ide r p rob le m (1) a nd a f ea s ib le po in t x k > O. A sca l ing op era t ion i s a coord ina tet r a n s f o r m a t i o n d e s c r i b e d b y x = D y, where D i s a s t r ic t ly pos i t ive d iagona l mat r ix .T h e s c a l e d p r o b l e m b e c o m e s

min imiz e c' ysubjec t to Ay = b , (9)

y ~ O ,w h e r e c = D ~ , A = A D , a n d y k = D l X k is fe a s ib l e . In pa r t ic u l a r , i f D = X k =d i a g ( x k , . . . , x k) , t h en D - i x k = e.

I t is e a sy to s e e t ha t a s c a li ng op e ra t i on a f fe c ts t he po t e n t i a l fun c t ion by a d d inga co ns tan t to i t, tha t is , for x = D y , f ( x ) = f ( Y ) - ~ i " = ~ log Di . I t fo l lows th a t thevariat ions of t he po t e n t i a l func t ion a re no t a f fe c t e d by sc al ing .Polynomial boundsPolynomia l bounds wi l l be e s t a b l i she d by p rov ing tha t t he po t e n t i a l func t ionde c re a se s a t e a c h i t e ra t i on a t l e a s t a n a moun t t ha t doe s no t de pe nd on the i t e ra t i on

f ( x k ) < ~ f ( x k ' ) - a , k = l , 2 , . . . . ( 1 0 )

Le m ma 2 . 1. I f condit ion (10) above is satisfied, then problem (1) is solved in no morethan O ( ( q / a ) L ) i terations, where L is the total length of the input data.Proof . We sha l l c ons ide r t he p rob le m to be so lve d whe n a po in t X k i s found suc htha t g 'xk


6/15

12 c . c . Gonzaga / Polynomial affine algorithms fo r LPT h e c o n d i t i o n l o g g' x k


7/15

C.C. Gonzaga / Polynomial affine algorithms or LP 13sin ce c ')3 = 0, e ' e = n. Bu t f i ~ 0 an d so e 'f i= Ilfi l l l~ 11)311.I t f o l l o w s t h a t

h'(33 - e ) /> q - n + [[)311a n d h e n c e

Ilhl111)3- el l ~ q - n + 11)311.N o w n o t e t h a t 11 )3-el l ~ 11)311 Ilell--11)311+ , / L a n d i t f o l l o ws t h a t

>_ q - n + ]]fil]Ilh ~ ~n ~- i l ~ . (12)W e c a n n o w r e a c h t h e fi na l c o n c l u s i o n s f o r b o t h c a s es :

(i) Im m e d ia te ly , if q - n ~> x/ n th en I[ h 11 ~ 1.( i i ) I f q - n = 0 t h e n

1I l h l l ~ 1 + 4 ~ / 1 l ; 1 1 "B u t )3 m u s t h a v e a c o m p o n e n t y j su c h t h a t y j ~ 1 , o t h e r w i se d = e - ) 3 i s a n o n - n e g a t i v ef e a s i b l e d i r e c t i o n : t h is c o n t r a d i c t s t h e c o m p a c t n e s s o f S , s i n c e t h e n e + / z d ~ S f o rany ~ ~ 0 . I t fo l lows tha t I[;11> / 1 , a n d f r o m ( 1 2 ) ,

1 1Ilh ll ~ > - - ~ > - - for n >~2,l + , / - d e , # dc o m p l e t i n g t h e p r o o f . [ ]

W e a r e n o w r e a d y t o p r o v e t h e m a i n r e s u l t :

L e m m a 3 . 2 . Consider the scaled problm (9 ) a s a b o v e a n d h = - P V f ( e ) .(i) I f q>~ n +x /- ff th en f ( y * ) - f ( e ) > ~ - O . l .(ii) I f q = n t h en f ( y * ) - f ( e ) O,f ( y * ) < - f( e + A h)

In par t ic ula r , fo r h e [0 , 0 .5/ I I h I I] we ha ve A [[ h I1~ 1. C h o o s i n g A = 0 . 3 / [ [ h l l ,f ( y * ) - f ( e )


8/15

14 C.C. Gonzaga / Polynomial affine algorithms for LP(ii) If q = n t h en ex am in e tw o cases : If I] h II ~> l th en ch o os e A = 0.3/11 h I] as a bo ve ,

to ob ta inf ( y * ) - f ( e)


9/15

C.C. Gonzaga / Polynomial affine algorithms fo r L P 15O ne de f in i t i on a nd one l e m m a a r e ne e de d f r om [ 2 ]. G ive n tw o ve c to r s x , d c

Nn, d > 0 , we de f ine the fo l low ing no rm for x :I l x l l , , = II[x,/ d,]l l. (is )

T h i s c o r r e s p o n d s t o IID-'xl[ f o r D = d i a g ( d l , . . . , d , ) , th a t i s, t h e n o r m o f t h e v e c t o rr e su l t i ng f r om a s c a l ing ope r a t ion a bou t d . T he in f in i ty no r m Ilxl17 i s de f ineds imi la r ly .L e m m a 4 .1 . L et x, d c R", x, d > O, be such tha t [[x - d II~


10/15

16 C.C. Gonzaga / Polynomial affine algorithms fo r LPB o u n d s o n t h e n u m b e r o f u p da t e sNow the ana lys i s i n Sec t ion (5 ) o f [ 2 ] app l i e s to the sequence gene ra t ed by thea lgor ithm . Fo r each k = 0 , . . . , K , whe re K i s t he la s t i t e ra t ion , de f ine A x k = Xk+l -- X k.Lemma 4.3 . Cons ider the sequenc e (x k ) gene ra te d by the a lgor i thm and l e t T be thet o ta l num b e r o f rank - 1 upda t e s c om pu t e d un t i l it era ti on K . T he n

KT ~ 12~,/n ~ IIAxkllxk. (18)k=0

Proof . Th i s expre ss ion i s t he l a s t inequa l i ty in the p r oo f o f Th eore m 5 .2 in [ 2 ],be for e subs t i tu t ing a va lue fo r Ax k. The cons tan t 12 was ob ta ined by subs t i tu t ingnum er ica l va lues in to tha t i nequa l ity . [ ]

Let K be the inde x of the las t ite ra t ion of the a lgor i thm, wi th the l inesearchabove . We a r e r eady to e s t ab l i sh bound s on the num ber o f upda te s in te rm s of K .Not i ce tha t t he ove ra l l bou nds dep end on the lim i t a tions on K to be p roved be low.Theorem 4 .4 . L e t T be the t o t al num be r o f upda t e s c om pu t e d by t he a lgor it hm .

(i) I f q >~ n + x/ n the n T


11/15

C.C. Gonzaga / Polynomial affine algorithms fo r L PLemma 4.5. Cons ider the scaled probl em (9 ) a n d a s s u m ei = 1 , . . . , n , whe re fl c (0, 1).

(i ) I f q > ~ n + x / n t he n I lhl l~>/3.(ii) l f q = n then Ilhll ~>/~/( 24-~ ).Proof . We fo l low st ep by s tep t he pro of o f Le mm a 3.1 .

V f( y ) = c-~qyc-y, wh ere )7 = [y-~ ],h ' (~ - y ) = -V f (y ) ' ( f i - y ) = q - )7'y + )7'~.

On the r i gh t -hand s ide ,)7'y -- n, )7'5 ~> /3e '~ ~>/3 I1~11

On the le f t -hand s ide ,Ilyll < ~ l l e l l - - ~ - ,

a nd he nc eh'(~-y) q - n ~ II~ll ,

SO

Ilhl[~ ~ q-n[311Yll - ~ ~ l l f il l "

( i) i s now imm edia te . Le t us prove ( ii ): I f q - n = O, thenIlhl l~/3 1 ~/-~/( /3 I1~11)

As be fore, for som e co m po ne nt j , )3j > yj >t fl, a nd t hus1 f13 f13IIh11~1+,/~/~ ~+4~ 2,/-~'

com ple t ing the proof . [ ]

17t ha t ~ < ~ y i ~ l / f l f o r

Lem ma 4.6. Consider the sca led problem (9 ) and y* de termined by the a lgor i thm forapproximate scal ings. Then

(i ) I f q >~ n + ~ then f ( y * ) - f (y )


12/15

18 c. c. Gonzaga / Polynomial affine algorithms for LPProof . We fol low c lose ly the pro of of Lem ma 3.2. Note ini t ia l ly tha t by const ruc t ionin Algor i thm 4 .2 , fo r i = 1 , . . . , n ,

- -0.1 < (Xki -- di) / d i < 0. 1o r

0.9 < x ~ / d i < 1.1 < 1/0 .9,and f ina l ly

/3 0.9 -0 .3 ~> 0.5,and (7) can be appl ied . W e now fo l low the pro of o f Lem ma 3 .2 , ob t a in ing

f ( Y * ) - f ( Y ) n +x/-ff th en fro m (1 6) ,/z =0.3/ l lhl l , an d

f ( y * ) - f ( y ) /3 = 0.9, an d th u s/ z 3. It follo ws tha t

f ( y * ) - f ( y ) /33 /(2v/- n)-

Merging these two express ions and subst i tut ing/3 = 0.9 ,0.1 2x/-m < 14-n /33 3 ,

and hencef ( y * ) - f ( y )


13/15

C.C. Gonzaga / Polynomial affine algorithms fo r LP 19Proof . S imply fo l l ow e xa c t ly t he sa me pa th a s i n the p r oo f o f The o re m 3 .3 , u s ingthe re su lt s o f Le m ma 4 .6 [ ]

Boun ds on the number of arithmetic computationsTheorem 4.8. Consider the m odified algorithm applied to problem (1) with q = O(n) ,and let C be the total number of arithmetical computations.

(i) I f q>~n+ x/n then C n +x/-n. Fr om (12) , we ca nn ot p rov e tha t I lh ll inc reases m uc h wi th q ,un l e s s we f i nd some bound fo r I t , l l . Sinc e t he c o mple x i ty t ha t we c a n p ro ve isO ( ( q / a ) L ) i t e ra t i ons , i nc re a s ing t he o rde r o f q wi l l i nc re a se t he c omple x i ty .


14/15

20 C.C. G onzaga / Polynomial affine algorithms for LPI t is a m u s i n g t o n o t i c e t h a t i f we c o u l d g u a r a n t e e ]])~]]~ 0 . 5 , t h e n i t i s e a sy t o p r o v e t h a t a n i t e r a t i o n w i t h t h i s p o t e n t i a lf u n c t i o n r e s u lt s i n a g o o d r e d u c t i o n f o r f ( . ). O t h e r w i s e , v c a n b e i n c r e a s e d u n t il

0.5 ~< IIPVN ( e)II < 1.T h e r e s u l t i n g v a l u e m u s t b e a l o w e r b o u n d , s i n ce o t h e r w i s e IIPVL(e)II >1 1.

T h i s c o m p l e t e s t h e u p d a t i n g s c h e m e , w h i c h i s v e r y s i m i l a r t o t h e o n e p r o p o s e db y T o d d a n d B u r r e ll [ 10 ] f o r K a r m a r k a r ' s a l g o r i th m .

References

[ 1 D. Bayer and J.C. Lagarias, "Th e non-linear geometry of linear programming, I. Affine and projectivescaling trajectories, II. Legendre transform coordinates, III. Central trajectories," Preprints, AT&TBell Laboratories (Murray Hill, NJ, 1986).


15/15

C.C. Gonzaga / Polynomial affine algorithms for LP 21[2] C. Gonzaga, "A n algori thm for solving l inear progra mm ing problems in O(n3L) o perat io ns ," in :

N. Megiddo, ed., Progress in Mathem atical Programming: Interior-Point and Rel ated M etho ds(Springer, New York, 1988) pp. 1-28.

[3] C. Gonzag a, "C onical project ion a lgorithms for l inear progra mm ing," Mathem atical Programming43 (1989) 151-173.[4 ] N. Karmarkar , "A new po lynomia l t ime a lgor ithm fo r l inea r p rogramm ing ," Combinatorica 4 (1984)

373-395.[5] M. Kojima, S . Mizuno and A. Yoshise , "A prima l-du al in ter ior point method for l inear program -

ming," in : N. Megiddo, ed . , Progress in M athem atica l Programming: Interior-Point and Relate dMethods (Springer, New York, 1988), pp. 29-48.

[6] N. Megiddo , "O n the complexity of l inear progr amm ing," in : T. Bewley, ed ., Advances in EconomicTheory (Cambridge University Press, Cambridge, 1987) pp. 225-268.

[7 ] N. Meg iddo and M. Shub , "Boundary behav iour o f in te r ior po in t algo ri thms in l inea r p rogramm ing ,"Research Re port RJ 5319, IBM Thom as J. Watson Rese arch Center (Y orktown Heights , NY, 1986).

[8] R.C. Monteiro and I . Adler , "An O (n3L) pr ima l-du al in ter ior point a lgori thm for l inear program-ming ," Manu scrip t , Dep artm ent of Industr ia l Engineering and Operat ions Research, Univers i ty ofCalifornia (Berkeley, CA, 1987).

[9 ] J . Renegar, " A po lynomia l - t ime a lgor i thm based on Newton ' s me thod fo r l inea r p rogramm ing ,"Mathem atical Programming 40 (1988) 59-94.

[10] M. Todd and B. Burrel l , "An extension of Karmarkar 's a lgori thm for l inear programming usingdual var iables ," AIgorithmica 1 (1986) 409-424.

[11] M.J . Tod d and Y. Ye, "A centered projective a lgori thm for l inear progra mm ing," Technical Repo rt763, School of Operat ions Research and Industr ia l Engineering, Cornell Univers i ty (I thaca, NY,1987).

[12] Pravin M. Vaidya, "A n algo ri thm for l inear program ming which requires O ( ( ( m + n ) n 2 +( m + n ) lS n ) L ) ari thmetic opera t ions ," Preprin t , AT&T Bell Laborator ies (M urray Hil l , N J , 1987).

Documents

Poly Aff Algms 4 Lin Prog -- Clovis C. Gonzaga