A new technique for the Simplex basis LU factorization updateaurelio/artigos/arquivo2.pdf · Techniques of the Simplex basis LU factorization update are developed to improve the solution

A new technique for the Simplex basis LUfactorization update

Daniela R. Cantane1

School of Electrical and Computer Engineering(FEEC), University of Campinas(UNICAMP), Sao Paulo, Brazil

Aurelio R. L. Oliveira2

Institute of Mathematics, Statistics and Scientific Computing (IMECC),University of Campinas (UNICAMP), Sao Paulo, Brazil

Christiano Lyra1

School of Electrical and Computer Engineering (FEEC), University of Campinas(UNICAMP), Sao Paulo, Brazil

Abstract

The objective of this work is to develop more efficient alternatives for Simplexmethod implementation. Techniques of the Simplex basis LU factorization updateare developed to improve the solution of the Simplex method linear systems toachieve a column static reordering of the matrix. A simulation of the Simplexmethod is implemented, with the change of basis obtained from MINOS. Onlythe factored columns actually modified by the change of the basis are carriedthrough to achieve an efficient LU factorization update. Thematrix columns arereordered according to three strategies: minimum degree, block triangular formand the Bjorck strategy. Sparse factorizations are obtained for any basis with onlya small computational effort to obtain the order of columns,since the reorderingof the matrix is static and basis columns follow this ordering. Computationalresults for Netlib problems show the robustness of this approach and good com-putational performance, since there is no need for the periodical factorizationsused in traditional updating methods. The proposed method achieves a reductionin the nonzero entries of the basis with respect to MINOS.

Keywords: linear optimization, sparse matrix, factorization update, Simplexmethod.

1Av. Albert Einsten, 400, CEP:13083-852, Campinas, SP, Brazil.E-mails: [email protected], [email protected]

2Sergio Buarque de Holanda, 651, CEP:13081-970, Campinas,SP, Brazil.E-mail: [email protected]

1

1. Introduction

This paper presents innovations to improve the solution of the large linear sys-tems that form the Simplex method computational core — proposed by Dantzigin 1947 [5], Simplex is the first efficient method for solving linear optimizationproblems.

The solution of large-scale linear systems can be approached through genericmethods, such asLU factorization of the basis and its update, or through prob-lem specific structure exploitation, as in the network flow problem [8]. The mainfocus of this study was to develop a static reordering of the basis columns ofthe Simplex method, together with an efficient update of the entering and leavingcolumns of the basis, for general linear optimization problems. TheLU factor-ization is performed only in the columns actually modified bythe entering and/orleaving columns in the basis.

The new proposed update presents a completely different approach when com-pared with the traditional approaches [6, 10]. In theLU factorization update, theoperations caused by a column leaving the basis are undone; in other words, op-erations are performed in the reverse order of the factorization. The operationsfor the factor basic matrix, obtained with the entering column of the basis, arethen performed. Finally, it is necessary to perform the operations on the columnslocated after the entering column in the basis, always considering sparsity.

The next section reviews approaches to performLU factorization updates.Subsequently, the proposedLU factorization update methods are discussed.

2. LU Factorization Update Methods

TheLU factorization update technique with partial pivoting was proposed byBartels [1]. Two variants of this algorithm proposed in Reid[11] aim to balancethe sparsity and numeric stability in the factorization. The latter variant is an im-provement over the former.

Several methods have been proposed to update the triangularfactors of a mod-ified matrix. Forrest and Tomlin [7] performed a study of three methods with theperspective of solving large scale optimization problems;the method proposedby Brayton et al. [4] appeared to be the most convenient for practical imple-mentations. Computational experiments with this method inlinear optimizationproblems of medium size were promising.

TheLU factorization update proposed by Markowitz [9] and its application tothe basis update in the Simplex method is described in Maros [10]. This book alsopresents the ideas of an efficient implementation proposed by Suhl and Suhl [13],which is a good combination of the symbolic and the numericalphase of pivoting,and represents a compromise between sparsity and numericalstability.

The column ordering method proposed by Bjorck [3] for theQR factorizationwas adopted in this work to deal with theLU factorizations. Santos-Palomo and

2

Guerrero-Garcıa[12] described an upper trapezoidal sparse orthogonal factoriza-tion update, where the sparseQR factorization is used; these ideas can also beused withLU factorizations. Other update techniques are described in Duff, Eris-man and Reid [6].

The next section presents the update method proposed in thisstudy.

3. The Proposed LU Factorization Update Method

The LU factorization update method proposed in this work performsoper-ations only in the columns actually modified by the change of basis, using thestructure of the sparse matrix as best as possible .

Changes may not occur in columns located after the entering column (e) at thebasis, and/or the leaving column (l), due to the sparse structure of the columnsinvolved. For instance, if a matrix entry is in thek row and in the original enteringcolumne it is equal to zero (B(k, e) = 0). Also if the matrix entry is ink row andin the leaving columnl of the existing basis is equal to zero(B(k, l) = 0), thenthe columnk does not change by basis update.

In order to improve efficiency and to reduce fill ins, the columns of the basisare ordered according to three strategies:

• Minimum degree(R1). The first step is to find the nonzero entries numberof the matrix, column by column. The column that has the smallest num-ber of nonzero entries is selected and permuted to the first position of thematrix. This column is marked as unavailable. The procedureis repeatedby searching a new column and exchanging it likewise. Then, the columnsof the matrix are reordered, according to the number of nonzero entries, inincreasing order.

• Block triangular form(R2). The nonzero entries number of the matrix isfound, column by column. The column that has the smallest number ofnonzero entries is selected. It is necessary to determine which position inthis column will be pivoted; the row that has the smallest number of nonzeroentries will be chosen. After selecting the pivot position in the given col-umn, this column is marked as unavailable and one degree is decreased foreach column that has a nonzero entry in the chosen row, from the pivot po-sition until the end. Subsequently, the actual position moves to the left topposition. The matrix will be in an upper block triangular form.

• Bjorck’s strategy(R3): This strategy was applied to column reordering, in-stead of row reordering, as described in Bjorck [3]. Letfi(A) andli(A) bethe row index for the first and last nonzero entries in the i-thcolumn ofA,respectively. First, sort the columns after an increasingfi(A), such that, ifi ≤ k thenfi(A) ≤ fk(A). Next, for each set of columns withfi(A) = k,k = 1, . . . , maxifi(A), sort all the columns after increasingli(A).

In the proposedLU factorization update, the operations caused by the leavingcolumnl are undone, in reverse order of the factorization, from the last column to

3

thes + 1 column, considering the sparsity.

The next step is to verify which columns from the entering one(e+1 column)to the last remain unchanged by entering columne at the basis. The operation willbe performed only in the other columns, after the column thatenters the basise isupdated.

4. Implementation Issues

The objective of the implementation developed here is to simulate Simplexiterations using the static reordering and theLU factorization update for generallinear optimization problems. The basis sequence obtainedfrom MINOS is usedin the simulation. A procedure was implemented for finding the initial basis, the“Crash” basis (described in Maros [10]) and the leaving and entering columns ofthe basis from MINOS’ output.

The three static reordering approaches described in Section 3 (R1, R2 andR3) are used, leading to sparse factorizations with a small computational effortto obtain the order of columns. Two column update approacheswere used in theimplementation and the number of nonzero entries for each one was compared.When the entering columne in the basis follows the static ordering the approachis calledU1. In the second approach, calledU2, the entering columne entriesthe basis in the position of the leaving columnl. In both approaches, aLU fac-torization update of the basis considering its sparsity is performed. The notationUMINOS is used for MINOS update results.

The introduced error in the factorization was estimated with the objective ofverifying the robustness of theLU factorization update method. Each factoriza-tion is completely undone in every iteration of the Simplex method. Thus, the op-erations are performed in the reverse order of theLU factorization and the matrixthat will simulate the basis factorization update is obtained from the one factoredfrom the very first column. Therefore, the original basis is obtained with someamount of numerical error and the accumulated errors are a worst case estimationfor each iteration.

5. Numerical Experiments

A subset of the Netlib problems was used in the numerical experiments [2].Initially, some numerical experiments were performed in order to verify the effi-ciency of the basis column update approaches presented in the previous section.Figure1 shows the number of nonzero entries for three static reordering strategies.

5.1. Computational Experiments with MatlabFirst of all, computational experiments with Matlab were performed for a few

small Netlib problems (computational times are too long when large problems areused). Figures2 and3 show the number of nonzero entries of the basis factor-ization, for the same linear optimization problems, using and not using the Crashbasis.

4

Even thoughR1 andR3 are a little more sparse thanR2, R2 was chosen for theremaining experiments, because it works better for the largest problems that weretested. TheU1 basis update approach reduces the number of nonzero entriesupto 52% for the problem scfxm1 and78% for the problem bandm, in comparisonwith the U2 approach in the Figures2 and3, respectively. Thus,U1 is used fortheLU factorization update proposed in this work. TheU1 update approach is alittle more dense that theUMINOS update, but as will be seen, it does not need torefactorize the basis. Thus, it will probably be faster thanother approaches.

Figure 4 shows the flop number (floating point operation count) for each fac-torization presented in the previous section; the MINOS’ Crash option was turnedoff in these experiments. TheF1 factorization proposed in this work reduces theflop number with respect to theFMINOS factorization.

In Figure5, the error estimate introduced by floating point operationswas ve-rified, by computing the norm of the difference between the basis obtained bycompletely undoing the factorization and the original basis obtained directly fromthe constraint matrix.

It can be concluded that the proposed method for factorization updating is ro-bust. The maximum accumulated error in the worst case is of the order of10−12;in other words, in this approach, due to its robustness and sparsity characteristics,it is not necessary at all to refactorize the basis, as is usually done in the traditionalmethods.

5.2. Computational Experiments with C programming language

Computational experiments for the large scale problems were achieved withsimulation of the Simplex method using the proposedLU factorization update,implemented in C programming language. Figures 6, 7, 8 and 9 present the num-ber of nonzero entries using the three static reordering approaches presented inSection 3, without the Crash basis.

In only 20.45% of the cases theR3 reordering obtained a more sparse basis;R1 andR2 obtained a more sparse basis in most of the cases. However, for thelarger problems tested, the reorderingR2 is more efficient. Thus, the reorderingR2 is used in the computational tests below.

The number of nonzero entries in theLU factorization of basis using the threeupdating columns, as described in the previous section, areshown in Figures 8, 9,10 and 11. In theU1 andU2 update approaches, theF1 factorization update pro-posed in this work is used in order to compare with theUMINOS update approach(obtained from MINOS), which adopts the Bartels-Golub factorization update.

Turning off the Crash basis, Figures 8 and 9 show that theU1 update proposedin this paper obtains a more sparse basis for85% of the cases, while the updateadopted by MINOS provides a more sparse basis in only12.5% of cases. TheU2 update approach obtained better sparsity for only one case.The U1 updateapproach proposed in this paper obtained a reduction of48.11% in the number

5

of nonzero entries for the scsd8 problem with respect to theU2 approach. Forthe scagr7 problem, a reduction of43.89% in the number of nonzero entries wasachieved, with respect to theUMINOS.

In Figures 10 and 11, theU1 update approach proposed herein provided a moresparse basis in87.5% of the cases, while the update obtained by MINOS obtaineda more sparse basis for only10% of the cases. TheU2 update approach obtainedbetter sparsity of the basis for only one case, the ship08s problem. TheU1 updateapproach proposed in this paper obtained a reduction of46.43% in the number ofnonzero entries for the brandy problem, in comparison withU2. For the sctap1problem, a reduction of43.67% in the number of nonzero entries was achieved incomparison withUMINOS.

6. Conclusions

This study proposed a static reordering of matrix columns for linear program-ming problems, leading to a Simplex basis with sparseLU factorizations and in-expensive factorization updates. The reordering has no initialization or updatingcosts since there is no need to reorder the columns in the factorization.

Three reordering approaches to reduce fill-ins were considered: minimum de-gree, block triangular form and Bjorck’s strategy. The method that obtains a re-sulting matrix in the block triangular form achieved betterresults, in comparisonwith the minimum degree and the Bjorck strategy. Thus, thismethod was used intheLU factorization update proposed in this work.

For the computational experiments with small problems using Matlab, the col-umn updating approach that follows the reordering reduced the number of nonzeroentries with respect to the updating approach, where the entering column uses thesame positions of the column that leave the basis. In spite ofthe fact that thisupdating approach obtains a basis somewhat more dense than the MINOS updat-ing approach in a few cases, it never requires a refactor. Therefore, this createsthe possibility of a much faster approach, compared with other updating strategies.

The number of floating print operations of the factorizationupdate approachproposed in this work is smaller than the equivalent number of operations used byMINOS updating strategy. Error estimation presented in Figure 5 demonstratesthe robustness of the proposed update approaches. Therefore, it is safe to concludethat no periodical factorization is needed for these problems; periodical factoriza-tions are usually necessary with other update schemes [6, 10]. As a further result,if the starting basis is the identity matrix, it is not necessary to compute anyLU

factorization at all.

For the computational experiments with large scale problems using the C im-plementations, the column update approach that follows thereordering, togetherwith the factorization update proposed in this work, achieved a more sparse basisin most cases.

6

kb2 adlittle sc205 israel scsd1 0

50010001500200025003000

Problems

Num

ber o

f non

zero

ent

ries


100200300400500600

bandm scfxm1 beaconfd scsd6 0

1000

2000

3000

4000

5000


1500

3000

4500

6000

Ave

rage

Min

imum

R1 R2 R3

Max

imum


1500

3000

4500

6000

7500


400

800

1200

1600

2000

Figure 1: Nonzero entries number with Crash basis in Matlab.


1000

2000

3000

4000

5000

6000

Max

imum


1500

3000

4500

6000

7500

9000

U1 U2 UMINOS


1000

2000

3000

4000

Ave

rage

Problems

Num

ber o

f non

zero

ent

ries


1000

2000

3000

4000

5000

6000

Figure 2: Nonzero entries number with Crash basis in Matlab.

7


1000

2000

3000

4000

5000

6000


5000

10000

15000

20000

Max

imum

Num

ber o

f non

zero

ent

ries

Problems


5001000150020002500300035004000


5000

10000

15000

20000

U1 U2 UMINOS

Ave

rage

Figure 3: Nonzero entries number without Crash basis in Matlab.


50001000015000200002500030000


20000

40000

60000

80000


2000400060008000

1000012000

Ave

rage

Max

imum



400000800000

1200000160000020000002400000

Up UMINOS

Tota

l

Flop

s nu

mbe

r

Problems


2000000

4000000

6000000

8000000

Figure 4: Flops number ofLU factorization in Matlab.

8

kb2 adlittle scs205 israel scsd1 bandm scfxm1 beaconfd scsd6 -1,0-0,50,00,51,01,52,0

50

kb2 adlittle scs205 israel scsd1 bandm scfxm1 beaconfd scsd6 -4,5-3,0-1,50,01,55060

kb2 adlittle scs205 israel scsd1 bandm scfxm1 beaconfd scsd6

-0,2-0,10,00,12,02,5

Aver

age

Min

imum

e-13

Max

imum

e-1

5e-1

3

Problems

Erro

r Est

imat

ion

Figure 5: Error estimate of updated basis without Crash basis in Matlab.

sc50b sc50a sc105 kb2 adlittle blend0

100200300400500600

scagr7 sc205 share2b lotfi share1b scorpion0

500

1000

1500

28003000


40

80

120

160

200

Ave

rage

Min

imum

Max

imum

Problems

Num

ber o

f non

zero

ent

ries


400

800

1200

24002600


75

150

225

300

375


400

800

1200

160026002800

R1 R2 R3

Figure 6: Nonzero entries number of the reordering in C without Crash basis.

9

scagr25 sctap1 brandy scsd1 israel bandm0

75015002250300037504500

Max

imum

Min

imum

Ave

rage

Num

ber o

f non

zero

ent

ries

Problems

R1 R2 R3

agg grow7 scfxm1 e226 beaconfd scrs80500100015002000250030003500


750

1500

2250

3000



750

1500

2250

3000

3750


500

1000

1500

2000


scsd6 degen2 agg2 agg3 seba forplan0

100020003000400050006000

scfxm2 grow15 ship04s bnl1 fffff800 ganges01250250037505000625075008750


75015002250300037504500



1000

2000

3000

40005000


Max

imum

Min

imum

Ave

rage

Problems

Num

ber o

f non

zero

ent

ries

R1 R2 R3


10

scfxm3 sctap2 grow22 ship04l ship08s sierra0

2000

4000

6000

8000

10000

ship12s scsd80

2000

4000

6000

8000


1000

2000

3000

4000

5000

ship12s scsd8050010001500200025003000


1000

2000

3000

4000

5000

ship12s scsd80

750

1500

2250

3000

3750

Max

imum

Min

imum

Ave

rage

Problems

Num

ber o

f non

zero

ent

ries

R1 R2 R3


sc50b sc50a sc105 kb2 adlittle scagr7 sc2050

200400600800

100012001400

U1 U2 UMINOS

share2b lotfi share1bscorpionscagr25 sctap1 brandy0500100015002000250030003500

Max

imum


100200300400500600700800

Num

ber o

f non

zero

ent

ries


500

1000

1500

2000

2500


100200300400500600700800


Problems

Ave

rage

Min

imum

Figure 10: Nonzero entries number of the columns update in C and of the MINOS update withoutCrash.

11

scsd1 israel bandm agg grow7 sxfxm1 e2260

100020003000400050006000

beaconfd scrs8 scsd6 agg2 agg3 seba0500100015002000250030003500

Max

imum


5001000150020002500300035004000

Num

ber o

f non

zero

ent

ries

Problems

U1 U2 UMINOS


Min

imum


1000

2000

3000

4000

5000


Aver

age


forplanscfxm2grow15ship04s bnl1 gangesscfxm30

10002000300040005000600070008000

sctap2 ship04l ship08s sierra ship12s scsd801000200030004000500060007000


1000200030004000500060007000


Max

imum


10002000300040005000600070008000


Min

imum

Aver

age

Problems

Num

ber o

f non

zero

ent

ries

U1 U2 UMINOS


12


200400600800

10001200140016001800

share2b lotfi share1bscorpionscagr25sctap1 brandy05001000150020002500300035004000


100200300400500600700800


U1 U2 UMINOS


100200300400500600700800900

1000


Aver

age

Min

imum

Max

imum

Problems

Num

ber o

f non

zero

ent

ries

Figure 13: Nonzero entries number of the columns update in C and of the MINOS update withCrash.


50010001500200025003000350040004500


Max

imum


500

1000

1500

2000

2500



50010001500200025003000


Min

imum

Aver

age

Num

ber o

f non

zero

ent

ries

Problems

U1 U2 UMINOS


13


10002000300040005000600070008000



1000200030004000500060007000



10002000300040005000600070008000


Num

ber o

f non

zero

ent

ries

ProblemsAv

erag

eM

inim

umM

axim

um

U1 U2 UMINOS


Acknowledgements

This research was sponsored by the Foundation for the Support of Researchof the State of Sao Paulo (FAPESP) and the Brazilian National Research Council(CNPq).

References

[1] R. BARTELS AND G. GOLUB, The Simplex method of linear programmingusing the LU decomposition, Communications of the Association for Com-puting Machinery, 12 (1969), pp. 266–268.

[2] R. BIXBY , Implementing the Simplex method: the inicial basis, ORSA Jour-nal on Computing, 4 (1992), pp. 267–284.

[3] A. BJORCK, Numerical Methods for Least Squares Problems, SIAM,Philadelphia, PA, 1996.

[4] R. BRAYTOM , F. GUSTAVSON, AND R. WILLOUGHBY , Some Results onSparse Matrices, IBM Research Center, Yorktown Heights, N.Y., 1969.

[5] G. DANTZIG, Linear Programming and Extensions, Princeton UniversityPress, 1963.

[6] I. DUFF, A. ERISMAN, AND J. REID, Direct Methods for Sparse Matrices,Clarendon Press, Oxford, 1986.

14

[7] J. FORREST AND J. TOMLIN , Updating triangular factors of the basis tomaintain sparsity in the product form Simplex method, Mathematical Pro-gramming, 2 (1972), pp. 263–278.

[8] J. L. KENNINGTON AND R. V. HELGASON, Algorithms for Network Pro-gramming, Wiley, New York, 1980.

[9] H. M ARKOWITZ, The elimination form of the inverse and its applications tolinear programming, Management Science, 3 (1957), pp. 255–269.

[10] I. M AROS, Computational Techniques of the Simplex Method, Kluwer Aca-demic Publishers, 2003.

[11] J. REID, A sparsity-exploiting variant of the Bartels-Golub decomposi-tion for linear programming bases, Mathematical Programming, 24 (1982),pp. 55–69.

[12] A. SANTOS-PALOMO AND P. GUERRERO-GARCIA , Updating and down-dating an upper trapezoidal sparse orthogonal factorization, IMA Jounal ofNumerical Analisys, 26 (2006), pp. 1–10.

[13] U. SUHL AND L. SUHL, A fast LU update for linear programming, Annalsof Operations Research, 43 (1993), pp. 33–47.

15

Documents

A new technique for the Simplex basis LU factorization updateaurelio/artigos/arquivo2.pdf · Techniques of the Simplex basis LU factorization update are developed to improve the solution