New pivot based methods inlinear optimization, and an appli ationin the petroleum industryZsolt CsizmadiaPhD ThesisSupervisor: Tibor IllésAsso iate Professor, PhDEötvös Loránd University of S ien es, Institute of Mathemati s,Do toral S hoolDire tor of do toral s hool:Professor Miklós La zkovi hmember of the Hungarian A ademy of S ien esApplied mathemati s do toral programDire tor of program:Professor András Prékopamember of the Hungarian A ademy of S ien es

This thesis was written at the Department of Operations Resear h at theEötvös Loránd University of S ien es.BudapestMay, 2007

Contents1 Introdu tion 61.1 S ope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.2 Stru ture of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.3 Notation and problem de�nitions . . . . . . . . . . . . . . . . . . . . . . . . 101.4 A short history of linear optimization . . . . . . . . . . . . . . . . . . . . . . 122 Linear feasibility problems 202.1 Linear algebra: an elementary and short introdu tion . . . . . . . . . . . . . 202.2 The alternate theorem of linear feasibility problems . . . . . . . . . . . . . . 242.3 The monotone build-up simplex method for feasibility problems . . . . . . . 252.3.1 Ba kground . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262.3.2 A weaker degenera y on ept . . . . . . . . . . . . . . . . . . . . . . 282.3.3 The algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.3.4 Complexity bound under the weak degenera y assumption . . . . . . 362.4 Finiteness with s-monotone pivot rules . . . . . . . . . . . . . . . . . . . . . 392.4.1 The on ept of s-monotone pivot rules . . . . . . . . . . . . . . . . . 392.4.2 An anti-degenera y pro edure based on index sele tion rules . . . . . 402.5 s-monotone pivot rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452.6 A re ursive algorithm to handle degenera y . . . . . . . . . . . . . . . . . . . 472.7 An appli ation in the petroleum industry - blending . . . . . . . . . . . . . . 562.7.1 Properties of the blends . . . . . . . . . . . . . . . . . . . . . . . . . 572.7.2 General onstraints of the blending model . . . . . . . . . . . . . . . 592.7.3 Example of a simpli�ed blending . . . . . . . . . . . . . . . . . . . . 592.7.4 Numeri al aspe ts of the blending example . . . . . . . . . . . . . . . 612.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 653 Linear programming 663.1 Duality theorem of linear programming . . . . . . . . . . . . . . . . . . . . . 672

3.1.1 The equivalen e of the linear feasibility problem with the linear programmingproblem pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693.2 The simplex method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693.2.1 Finiteness of the simplex algorithm . . . . . . . . . . . . . . . . . . . 713.2.2 Finding a feasible basis for the simplex algorithm . . . . . . . . . . . 733.2.3 Complexity bound under the weak degenera y assumption . . . . . . 753.3 The monotoni build-up simplex method for linear programming problems . 763.3.1 Finiteness of the MBU simplex algorithm . . . . . . . . . . . . . . . . 793.4 The blending example for linear programming . . . . . . . . . . . . . . . . . 833.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 854 Linear omplementarity problems 874.1 Ba kground . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 884.1.1 Su� ient matri es . . . . . . . . . . . . . . . . . . . . . . . . . . . . 894.1.2 The alternative theorem of the linear omplementarity problem . . . 914.2 The Criss-Cross method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 934.2.1 Almost terminal tableaux of the riss- ross method . . . . . . . . . . 954.2.2 Finiteness of the riss- ross method . . . . . . . . . . . . . . . . . . . 1024.3 EP theorems and the linear omplementarity problem . . . . . . . . . . . . . 1034.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1095 The appli ation of linear optimization in the petroleum industry 1115.1 Introdu tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1115.2 Bilinear programming in pra ti e . . . . . . . . . . . . . . . . . . . . . . . . 1135.3 Bilinear optimization in the petroleum industry . . . . . . . . . . . . . . . . 1165.3.1 Distributive re ursion . . . . . . . . . . . . . . . . . . . . . . . . . . . 1175.3.2 Some further properties of the model . . . . . . . . . . . . . . . . . . 1185.3.3 Solution in pra ti e (using PIMS) . . . . . . . . . . . . . . . . . . . . 1205.4 Improving the solution te hnique . . . . . . . . . . . . . . . . . . . . . . . . 1245.4.1 Analysis of the model . . . . . . . . . . . . . . . . . . . . . . . . . . . 1255.4.2 Analysis of the optimization phase . . . . . . . . . . . . . . . . . . . 1265.5 Future tenden ies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1325.5.1 Convex envelopes used in re�nery models . . . . . . . . . . . . . . . . 1345.6 Con lusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135Summary \ Összefoglaló 136Bibliography 1383

A knowledgementsNowadays, our studies never end. However, if we have to de�ne where an end ould be,it is probably this point. I have been given the han e to rea h this point with undiminishedvigor. Now, I wish to say thanks for this.I met those tea hers to whom I am mu h obliged for my professional development at theuniversity1. My interest turned to optimization and operations resear h due to the in�uen eof my supervisor Tibor Illés and Béla Vizvári. Without the work and support of professorsIllés and Vizvári, I ouldn't have a hieved so mu h.Several grant organizations like OTKA (T 049789) or TÉT (SLO-4/2005), national andinternational onferen es of operational resear h organizations, visits at foreign universitieslike Rutgers in the USA, M Master in Canada, the Te hnis he Universität of Berlin inGermany or the University of Coimbra in Portugal, moreover, ompanies like Fran e Tele omand MOL �the Hungarian Oil Company� provided the opportunity to widen my professionalexperien e and to meet real life work hallenges.I hereby thank the resear h fellowship of MOL Pl . This fellowship not only supportedme during my studies, but even more importantly, it opened the door to an irrepla eablework experien e.My results are only partially mine. The prime merit is due to the ir umstan es, thatput me on the path and kept me there. Naturally, my family is of primary importan e.Albeit the help and guidan e of my Mother, Father and my Grandparents was the mostdetermining, I have always had more than one family supporting me ba all means.Last but not least, I'm grateful to my friends who have a ompanied me on this path.I gave thanks to the path that it has led me here.

1I'm also grateful to the opposers of my thesis, whose areful work and suggestions signi� antly improvedits quality. 4

KöszönetnyilvánításMai világunkban tanulmányainkat sosem fejezzük be. Ha mégis meg kell mondani, hogyhol a vége, talán ez az. Nekem megadatott, hogy idáig töretlenül jussak el. Ezért mondokmost köszönetet.Egyetemi tanulmányaim során ismerkedtem meg azon tanáraimmal, kiknek szakmai fej-l®désemet köszönhetem1. Témavezet®m Illés Tibor, és Vizvári Béla hatására fordult érdek-l®désem az optimalizálás, és az operá iókutatás felé. Illés és Vizvári Tanár Úr munkája éstámogatása nélkül ma nem tartanék itt.A tanulmányok minél szélesebb körben való folytatásához, a kell® szakmai tapasztalatmegszerzéséhez a lehet®séget számos pályázat � mint amilyen az OTKA (T 049789), a TÉT(SLO-4/2005)� különböz® operá iókutatási szervezetek rendezvényei, külföldi egyetemekentett látogatások � mint az amerikai Rutgers egyetem, a kanadai M Master egyetem, a BerliniM¶szaki Egyetem vagy a portugál Coimbrai Egyetem � illetve számos vállalat � mint a Fran eTele om és a MOL �biztosította.Ezúton szeretnék külön köszönetet mondani a MOL-nak egyrészt az ösztöndíjért mellyeltanulmányaim támogatták, másrészt ami talán ennél is fontosabb, a szakmai tapasztalatértmelyhez így hozzájuthattam.Eddigi eredményeim sak töredék részben az enyémek. A f® érdem a körülményeké,melyek engem az útra tereltek, és ott meg is tartottak. A legfontosabb természetesen a saládom. Els®sorban Édesanyám, Édesapám és a nagyszüleim iránymutatása és segítségevolt a meghatározó, azonban sosem fogom feledni, hogy szinte onnantól, hogy valamire isemlékszem, végig több saládom volt melyek egymástól függetlenül mindenben támogattak.Végül, de nem utolsósorban hálás vagyok barátaimnak, kik az úton végigkísértek.Köszönöm az útnak, hogy ide vezetett.

1Hálával tartozom az opponenseknek is, kiknek gondos munkája és hasznos észrevételei jelent®sen növeltéka dolgozat min®ségét. 5

Chapter 1Introdu tionLinear optimization is one of the most widely applied optimization te hniques today. Itspopularity is based on its broad spe trum of appli ability in real life situations, as well onthe existen e of the ever improving algorithms and state of the art softwares. Not only manye onomi al, logisti al and industrial problems may be formulated as linear programmingproblems, but often the approximations of more omplex systems yield linear stru tures.Moreover, several optimization methods require to solve linear programs as part of a so-phisti ated algorithm, like in the ase of mixed integer linear programming (MILP) or thesu essive linear programming (SLP) method.The pra ti al importan e of linear optimization is out of question, making it an evergreenarea of resear h. The �eld of algorithmi resear h in linear optimization onsists of severaldomains. There are the highly appli ation driven areas, like the algorithms based on the lassi al pivot methods, or the relatively new, but already highly e� ient interior pointmethods. Complementing these, there are su h methods that are often ine� ient in pra ti e,but feature great theoreti al importan e, like for example the ellipsoid method.This thesis on entrates on pivot methods, and on the appli ation of linear optimiza-tion in the petroleum industry. While pivot methods are often onsidered to be a maturesubje t, there still exists a quest to �nd a strongly polynomial algorithm for linear pro-gramming. For linear feasibility problems and for linear programming problems, we onsidermonotone s heme pivot methods, with �exible index sele tion rules to in rease their pra ti alappli ability and e� ien y. A new omplexity analysis appli able to most monotone s hemepivot methods [11℄ and a new re ursive method to handle degenera y [10℄ (that �ts into themonotone frame better, than the usual anti-degenera y rules) are presented. Finiteness isalso shown with lassi al pivot rules as well. To do so, a general framework is introdu ed,that makes it possible to prove onvergen e of several, so alled s-monotone pivot rules [11℄simultaneously. Several s-monotone pivot rules feature favorable �exibility in hoosing the6

pivot position. This �exibility may help to improve numeri al stability of the methods.In the study of algorithms for linear omplementarity problems (LCP), matrix lassesplay a fundamental role, determining the appli ability and e� ien y of the algorithms. MostLCP solvers require a priori information about the input matrix. One of the widest matrix lasses, for whi h both the riss- ross method [79℄ and several interior point algorithms [51℄are �nite, is the lass of su� ient matri es [18℄. Su� ien y of a matrix is hard to he k (nopolynomial time method is known1). We present a new riss- ross method [19℄, that applies�exible pivot rules, and requires no information on the properties of the input matrix inadvan e. The algorithm terminates with one of the following ases in a �nite number of steps:it solves the LCP problem, solves its dual problem (proves that the problem is infeasible),or it gives a erti� ate that the input matrix is not su� ient, thus y ling may o ur. Thisalgorithm is an extension of the algorithm presented in [3℄ for su� ient matri es, with asimpli�ed proof of �niteness at the same time.Several examples help to demonstrate the presented pivot algorithms.Petroleum industry was among the �rsts to use linear optimization models to improvee� ien y in luding oil extra tion, re�ning, blending and distribution [21, 85℄. While theoriginal models were linear, later mixed integer linear models, and �nally the lassi al bi-linear (sometimes multilinear) models of today have been evolved. These models are solvedwith sequential linear programming approa h in the majority of ases. These models arenot only large s ale models, but often numeri ally very badly onditioned due to the stri texpe tations on quality parameters given by standards of the produ ts and e onomi al om-petition driven goals (i.e. pro�t maximization). Although the omputational apa ities andsolution methods steadily improve, the omplexity of the model in reases even faster, mak-ing modelling questions and �ne-tuning of solution methods more and more important and ompli ated. To keep the large, ever hanging models up to date and stable, along with theo� the shelf softwares and highly skilled petroleum engineering experts tending the model,the solvers and models have to be onstantly �ne-tuned onsidering numeri al and operationresear h on epts. This thesis also summarizes spe i� aspe ts of model onsisten y he ksand solution methods, that proved to be highly e� ient for petroleum industry models [20℄.1.1 S opeThis thesis was written as part of the requirements for the degree of do tor of philosophyin applied mathemati s. The sele ted �eld is linear programming, and in a wider sense,1When this thesis was written. 7

operations resear h. As a thesis written in the �eld of applied mathemati s, it tries to ombine the theoreti al aspe ts of linear programming with pra ti al issues.From the theoreti al part, several new results for pivot methods in linear optimizationare presented. All these results are related to the pivot algorithms of linear systems, and areeither new algorithms, extensions of lassi al algorithms or provide new analysis to existingalgorithms. The results are based on [10, 11, 19℄. These results are joint work with mysupervisor, Tibor Illés.As a demonstrative appli ation, the produ tion planning problem of an oil re�nery ispresented. The arising mathemati al programming problems are numeri ally hallenging.Several methods, mainly based on the stru tural analysis of the models are presented, whi hproved to improve the reliability and e� ien y of the solution methods greatly. These resultsare joint work with my supervisor, Tibor Illés, and my fellow PhD student, Marianna Nagy.The ombination of theoreti al and pra ti al aspe ts naturally lead to di�eren es inpresentation. While the theoreti al part is hopefully of a higher standard mathemati ally,the des riptions of the appli ations are more detailed, and when appropriate, even withredundan y.A detailed des ription of the new results and the stru ture of the thesis is given in thenext se tion.1.2 Stru ture of the thesisIn this introdu tory se tion, after presenting the notations used throughout the thesis, wegive a short summary on the history of linear optimization.In Chapter 2 linear feasibility problems are onsidered. After de�ning the linear algebrai ba kground, we present the Farkas lemma [17, 29, 39℄, the famous alternative theorem oflinear feasibility problems, as well as the �exible pivot rules used to resolve the phenomenaof degenera y. The main results of the hapter on ern monotone s heme pivot algorithms,su h as the monotoni build-up simplex method for feasibility problems (or the �rst phaseof the simplex method). For su h algorithms, a nontrivial iteration bound on the numberof pivot steps required is presented under a weak degenera y assumption. Although thebound is not polynomial in general, it yields a new, non ombinatorial way to analyze theadvan ement made by the algorithm in ea h pivot step. For strongly degenerate problems,two fundamentally di�erent ways are provided to handle degenera y. The �rst relays onwell-known �exible pivot rules, but the �niteness proofs are presented in a new, uni�ed way,based on the on ept of s-monotone pivot rules [11℄. The se ond approa h is based on a8

re ursive pro edure, that takes advantage of the spe ial properties of the monotoni build-upsimplex algorithm. This hapter relies on the results presented in [11, 10℄.After some numeri al examples, as a demonstrative appli ation, the feasibility problem ina simpli�ed blending problem from the petroleum industry is presented, and a small instan eis solved in a Matlab implementation of the presented MBU algorithm. The presentation ofthe blending problem also serves as an introdu tion to the last hapter.In Chapter 3 the lassi al simplex method and the original version of the monotoni build-up simplex algorithm [4℄ for linear programming problems are presented. The �niteness ofboth algorithms is shown to be assurable by the new on ept of s-monotone pivot rules. Itis also shown, that the new omplexity bound an also be generalizable to the these lassi alalgorithms.As a numeri al demonstration, as well as some small examples, the blending problempresented in Chapter 2 is extended with an obje tive fun tion, and solved with both thesimplex and the MBU simplex algorithm.Chapter 4 deals with the linear omplementary problem. After an introdu tion to thefundamental results of LCPs, we onsider the so alled su� ient matri es. This matrix lassis one of the most studied matrix lasses in the �eld of linear omplementarity. For su hmatri es, we present a new, generalized riss- ross method using s-monotone pivot rules,that also omplies with the requirement of EP theorems. These results are based on [19℄.Throughout the thesis, a gasoline blending appli ation from petroleum industry is usedto demonstrate the pra ti al importan e of linear optimization. At the end of ea h hapter,those aspe ts of this appli ation are dis ussed, that are most relevant to the hapter. Thisway, several theoreti ally and pra ti ally important properties of the blending model aredis ussed. Chapter 5 summarizes the model, and pla es it into the framework of supply hain management problems of petroleum industry, where these models are used in every dayoptimization problems. These models are extremely omplex, and numeri ally hallenging.As an attempt to meet these hallenges, several model onsisten y and stru tural he kingmethods are presented that turned out to be very bene� ial in pra ti e. Some methods arealso elaborated to in rease omputational e� ien y. The results presented in this hapterare based on [20℄.All studies onne ted to the petroleum industry have been elaborated in a resear h o-operation with MOL Pl . [68℄. The data for the numeri al blending example has also beenmade available by MOL.The thesis is losed by a summary. 9

1.3 Notation and problem de�nitionsWe use the following notations throughout the thesis. S alars and indi es are denoted bysmall Latin letters, ve tors by small boldfa e Latin letters, matri es by apital Latin lettersand �nally, index sets by apital alligraphi (s ript) letters.The linear feasibility problem is de�ned byAx = b, x ≥ 0, (1.1)where A ∈ Rm×n,b ∈ Rm (n ≥ m).The linear programming problem onsists of a linear feasibility problem in the presen eof an obje tive fun tion.

min cTx

Ax = b, x ≥ 0, (1.2)where c ∈ Rn, A ∈ Rm×n,b ∈ Rm (n ≥ m).The linear omplementarity problem repla es the obje tive fun tion with the so alled omplementarity ondition.−Mu+ v = qu v = 0 (1.3)u, v ≥ 0,where M ∈ Rn×n, q ∈ Rn and uv = (u1v1, . . . , unvn).The bilinear programming problem is a nonlinear, generally non- onvex optimizationproblem of the form

max cTx + xT Dy + dTy

aTi x + xT Fiy + bT

i y ≤ bi i = 1, . . . ,m (1.4)Nx + My ≤ d,where c, ai ∈ Rn, D, Fi ∈ Rn×k,d ∈ Rk, bi ∈ R, i = 1, . . . ,m, N ∈ Rl×n,M ∈ Rl×k.In problems (1.1)-(1.2), without loss of generality, we may assume that the matrix ofthe linear onstraints (A) is of full rank, for the feasibility he k of the linear system andthe removal of the redundant equations an be done simultaneously with the Gauss-Jordanelimination. In ase of problem (1.3), the matrix [−M, I] is always of full rank.For an index set G ⊆ I := {1, 2, . . . , n} and a ve tor v ∈ Rn, we denote the subve torof v indexed by G by vG, in other words (vG)i := vi, i ∈ G. For any matrix T ∈ Rm×n and10

index sets F ⊆ J := {1, . . . ,m} and G ⊆ I, we denote the submatrix of T , whose rows and olumns are indexed by F and G, respe tively, by TFG. For a matrix A ∈ Rm×n and indexs, the ve tor as ∈ Rm denotes the olumn of A indexed by s.Let B ∈ Rm×m be a nonsingular square submatrix of A ∈ Rm×n and N be the matrix onsisting of the olumns of A not in B. The nonsingular matrix B is alled a basis of theproblem while the index sets IB ⊂ I and IN ⊂ I denote the index sets of the basi andnonbasi variables, respe tively. The well-known orresponding basi solution is given bythe formula b := xIB

:= B−1b, and T = B−1N ∈ Rm×(n−m) de�nes the short pivot tableau orresponding to basis B. The ve tor of nonbasi variables is xIN:= 0.The notations used in the thesis are summarized in Table 1.1.

x, xi bold hara ters for ve tors, normal for s alars.vu the oordinate produ t (Hadamard) of v and u ve tors.A the matrix of the feasibility or linear programming problem, A ∈ Rm×n.M the matrix of the LCP problem, M ∈ Rn×n.B basis, an m × m (n × n) nonsingular submatrix of [−M, I] ∈ Rn×2n (A ∈ Rm×n).T short pivot tableau for a given basis.TB short pivot tableau for a given basis, if the notationmakes it ne essary to emphasize that it belongs to basis B.I alligraphi letters denote sets.IB indi es belonging to basis B.IN := IB indi es outside of basis B.⊕, ⊖, +, − nonnegative, nonpositive, positive, negative element.〈. . . 〉 spanned ve tor spa e.b the transformed right-hand side in the pivot tableau.Spe ial notations used for linear programming problems:t(c) the fundamental ve tor for the obje tive fun tion rowin the pivot tableau.c the transformed obje tive fun tion in the pivot tableau.Spe ial notations used for linear omplementarity problems:i = n + i if i ∈ {1, . . . , n} and = i − n if i ∈ {n + 1, . . . , 2n}.tq the fundamental ve tor for the right-hand sidein the pivot tableau.q the transformed right-hand side in the pivot tableau.Table 1.1: Notations used.11

1.4 A short history of linear optimizationA very large sele tion of books are available on every aspe ts of linear programming. Thebooks range from overing simplex type methods like [21℄, interior point type methods like[73℄, or theoreti al aspe ts of linear programming like [75℄. Several well-organized summariesand studies deal with the development and history of linear optimization. A very detailedand interesting des ription of the early development of linear programming is found in [21℄presented by George Dantzig. András Prékopa gives an ex ellent and detailed des riptionof the history of the development of optimization and the Farkas lemma [70, 69℄, but evenre ent papers like [97℄ enri h the existing literature.The solution of linear equation systems with three variables by elimination method al-ready appeared in The Nine Chapters on the Mathemati al Art by Chinese mathemati iansapproximately in 150 B.C. The Gauss-elimination method, whi h is the base of one of themain methods in linear optimization was published by Carl Friedri h Gauss (1777 − 1855)and was later generalized as Gauss-Jordan elimination method, published in the Handbookof Geodesy (1873) by W. Jordan (1842-1899). The �rst alternate theorem, hara terizingwhen a linear equality system has a solution was formulated by A. Capelli, Leopold Kro-ne ker (1823 − 1891) and Eugene Rou hé (1832 − 1910) before 1892.The �rst well-known model in the literature to onsider linear inequalities is the Fourierme hani al prin iple, whi h also provided a starting point for the works of Gyula Farkas(1847 − 1930). Solvability of linear inequality systems was investigated by Gyula Farkas[27, 28℄ in the late 19th entury. His paper �written in German, summarizing his earlier resultsregarding linear inequalities [29℄� published in 1901 be ame one of the most frequently itedpaper in the literature of optimization. His famous alternate theorem, the Farkas lemma hara terizes the solvability of linear inequality systems. The �rst known omputationalmethod for spe ial linear inequality systems was formulated by Jean Baptiste Joseph Fourier(1768 − 1830, famous for the Fourier series as well) in 1823. This method was generalizedto arbitrary linear systems of inequalities by Theodore Samuel Motzkin (1908 − 1970), andsin e then this method has been known as the Fourier-Motzkin elimination [69℄.Well before the development of really e� ient methods for solving linear inequality sys-tems, the famous alternative theorem of Farkas was published, �rst in 1894 in Hungarian[27℄. The original form of his theorem states that a homogenous linear inequality gTx ≤ 0is a onsequen e of a set of other homogenous linear inequalities, if and only if a set of signrestri ted s alars exists, with whi h the weighted algebrai sum of the inequalities yield theinequality gTx ≤ 0. As Prékopa notes in [69℄, the original proof was not omplete, but ouldbe orre ted. In 1918, Alfréd Haar (1885 − 1933) generalized Farkas's theorem [38, 39℄.12

His results may be interpreted in the urrent terminol-ogy, as the following: if a linear programming prob-lem has an optimal solution, then a so alled dual sys-tem also has. Similar results have been a hieved byGeorgy Fedoseevi h Voronoy (1868 − 1908) [91℄, Her-mann Minkowski (1864 − 1909) [63℄ and later by Johnvon Neumann (1903− 1957). Several other isolated at-tempts were made to handle linear systems, in luding apaper by de la Vallée Poussin (1866− 1962) in 1911, orlater by Leonid Vitalyevi h Kantorovi h (1912 − 1986)in 1939. The development and the appli ation of op-timization methods be ome possible with the develop-ment of omputers. The �rst tra es of e� ient omput-ers began to emerge during World War II, thanks to thepioneer work of su h great s ientists like Neumann orKantorovi h. Gyula Farkas.Sin e then omputational apabilities and algorithmi te hnologies improved dramati- ally, making it possible to solve huge problems in a very short time.The early development of stru tures and models used in linear programming was in�u-en ed by the work of the Nobel prize winner (1976) e onomist, Wassily Leontief (1906−1999)who in his work Interindustry Input-Output Model of the Ameri an E onomy (1932) wasamong the �rsts to work out a omplete e onomi model based on a simple, but large matrixstru ture. His model's importan e lies in the fa t that after formulating the model, he wasable to olle t the ne essary input data, and later onvin e the poli y makers to apply theresults. An equally important result in�uen ing the period is due to von Neumann. Hisresults in game theory (1928) and on steady e onomi growth (1937) both paved the wayfor the way of thinking required in linear programming [21℄.One of the undisputed founding fathers of linear optimization is George Dantzig (1914−

2005). In his famous book [21℄, he summarizes many interesting fa ts on how the simplexmethod and linear programming evolved during its early de ades.The simplex algorithm to solve linear programming problems was developed by Dantzigin 1947. Before this date, most results on erned only systems of linear equalities and in-equalities ( alled the linear feasibility problem today), without the presen e of an obje tivefun tion. Originally, the �rst version of Dantzig's model, developed with military appli a-tions in mind, had no obje tive fun tion either. The military appli ation meant that su h amodel was required whi h was large s ale, dynami , and ould e� iently be omputed.13

The �rst model, alled a tivity analysis model byDantzig, was developed for the Ameri an Air For e andsubstituted the obje tive fun tion with ertain rules toguide the strategy sele tion. The main role of theserules was to de rease the number of feasible solutions.Dantzig with his resear h group soon realized that e�- ient omputational apabilities provided only by om-puters were essential for solving the new models, andsu essfully persuaded the Pentagon to fund the de-velopment of omputers. By the se ond half of 1947,Dantzig formulated the �nal form of his model, whi hsatisfa torily represented the te hnologi al features andrelations in pra ti al problems. George Dantzig.With Dantzig's own words, we present the three requirements he de�ned:1. the total amount of ea h type of item produ ed or onsumed by the system as a wholeis the algebrai sum of the amounts inputted or outputted by the individual a tivitiesof the system;2. the amounts of these items onsumed or produ ed by an a tivity are proportional tothe level of a tivity, and3. these levels are nonnegative.The resulting mathemati al model onsisted in the minimization of a linear form subje tto a system of linear equalities and inequalities, and thus linear programming in its urrentform was born.The simplex method itself was proposed by Dantzig shortly afterwards, during the sum-mer of 1947.The usage of the term �programming� is strongly related to this environment, in whi hthe model was proposed. The term �program� itself in the military vo abulary refers tovarious plans regarding s hedules, logisti s, deployment, et . Dantzig's �rst paper on linearprogramming, written while he worked for the Ameri an Air For e, was thus alled Program-ming in a Linear Stru ture. This way, the term �program� in the optimization environmentwas established long before it be ame known as a word for omputer programs, whi h wereat that time simply referred to as � odes�. Later, in early 1949, Robert Dorfman was the�rst to use the phrase �mathemati al programming�, sin e he found the expression �linear�to be too restri tive. 14

The name of the simplex method is due to Motzkin, who realized that in the olumnspa e of the problem, the pivot operations performed by the simplex methods may be viewedgeometri ally as moving from a simplex to an adja ent one.The term dual was already used in mathemati al ontext before linear programmingbe ame widespread. However, it's a bit surprising that the term primal was new, proposedby Dantzig's father, Tobias Dantzig (1884−1956) around 1954 to repla e the phrase �originalproblem whose dual is...�.Only two years after the dis overy of linear programming, the �rst onferen e on math-emati al programming, sometimes referred to as the Zero Symposium, was held in Chi ago.Soon after that the simplex method be ame available, the �rst appli ations began toemerge. Among the �rst su essful industrial appli ations of linear programming were thepetroleum industry (in luding oil extra tion, re�ning, blending and distribution), the foodpro essing se tor, the meat pa king bran h, iron and steel industry, metalworks, paper millsand �nan ial management. Nowadays, literally every aspe t of our every day life is somehowrelated to linear optimization.Large s ale methods based on the simplex algorithm began with the development of theso alled Dantzig-Wolfe de omposition in 1955 by Neumann and Philip Wolfe, [22℄, and itsdual form developed by J. F. Benders in 1962 [9℄.Soon after the development of the simplex method, the �rst alternative pivot methodsbegan to emerge. Still, Dantzig's simplex algorithm (or its heavily re�ned variants) seem tobe the most e� ient pivot algorithm for the great majority of problems. A great variety ofindex sele tion rules and pivot methods are available today. For a summary, see T. Terlakyand S. Zhang [82℄.The �rst pivot methods for the linear omplementarity problems were developed in 1962−

63 by Ri hard Cottle. In 1965, Lemke invented his famous method [55, 56℄, whi h presenteda histori breakthrough into the non onvex domain.A general linear omplementarity problem di�ers from the linear feasibility problem andfrom the linear programming fundamentally, sin e it is NP- omplete, be ause the feasibilityproblem of linear equations with binary variables an be des ribed as an LCP problem,too. The LCP problem remains NP- omplete, even if we restri t the matrix M to the lassof negative semide�nite matri es [16℄, and so it does on the lass of P0 matri es [51℄, too.Linear omplementarity problems an be obtained from the Karush-Kuhn-Tu ker optimality onditions for quadrati programming. For onvex quadrati obje tive fun tions, the matrixM of the LCP problem is bisymmetri , whi h is part of the lass of su� ient matri es, thewidest lass of matri es for whi h both several pivot methods and interior point methods15

are proved to be �nite.Possibly the most important feature of LCPs is that most pivot and interior point meth-ods developed for linear programming an somehow be adapted to handle LCPs at leastwith PSD matri es, thus providing e� ient algorithms to solve linearly onstrained onvexquadrati problems.The minimal index riss- ross method, a pure ombinatorial style pivot method was �rstinitiated by Zionts [100℄ in 1969. He was unable to prove the orre tness of his method, thusthe result was not widely published, and it remained virtually unknown. Independently,Chang [15℄, Terlaky [80, 79, 78℄ and Wang [92℄ have published riss- ross methods, the lattertwo results being presented for oriented matroids. The development of the riss- ross methodanswered a long lasting open question, namely, whether the solution of a linear program maybe obtained in only one phase or not.Although the simplex algorithm was qui k to prove it-self in pra ti e, its theoreti al omplexity remained anopen question until 1970, when Vi tor Klee and GeorgeMinty reated an example, on whi h the simplex al-gorithm with the usual minimal index rule makes anexponential number of pivot operations started from aproper hosen vertex before �nding the optimal vertex.This example, the famous Klee-Minty ube, is a slightlyperturbed hyper ube, whi h proved to be a very usefultheoreti al tool ever sin e. Although several alterna-tive index sele tion rules were invented, for most asesan exponential example exists [6, 50, 65, 64, 66, 72℄, orat least, onje tured to exist. The Klee-Minty ube in 3 dimension.Thus, one of the most hallenging open questions in linear optimization remains, whethera polynomial time pivot method exists. This question is in lose onne tion with the famousHirsh onje ture [49℄. This onje ture states that in any polyhedra of dimension d, thereexists a path of length no more than d + 1 between any two verti es, using adja ent verti esonly. The strongest result related to the the Hirsh onje ture is due to Fukuda and Terlaky[35℄. They proved in 1999 that travelling on the ut system of the polyhedra (i.e. not onlyon feasible verti es), with using only so alled admissible pivots, there exists su h a path oflength no more than d + 1. Unfortunately, they were able to prove only the existen e andthe ra e to �nd an algorithm using su h a path, or prove that none exists ontinues.Although theoreti ally not polynomial, a ording to numeri al experien es, the simplexmethod still a ts like a polynomial algorithm in pra ti e for the majority of ases, moreover,16

the average running time is linear (or at most quadrati ). This behavior was justi�ed by awide range of papers on the expe ted number of pivot steps required by the simplex method[1, 2, 13, 83℄, although all these results assume stri t probabilisti onditions on the inputdata of the problem.The �rst polynomial algorithm for linear programming � developed in 1978 � was aspe ialization of a nonlinear programming te hnique, answering the long standing question,whether linear programs belonged to the �easy� polynomial lass of problems or not. Thefoundations of Leonid G. Kha hian's (1953−2005) ellipsoid method were developed by otherSoviet mathemati ians, Shor, Yudan and Nemrivskii. Despite its theoreti ally favorable omplexity and its great impa t on the theories using linear programming (espe ially in theapproximation theory of ombinatorial optimization) unfortunately, the method proved tobe very ine� ient in pra ti e, sin e the number of iterations needed by the implementationstends to losely approximate the enormous theoreti al bounds.The �rst polynomial time algorithm for linear program-ming that proved to be e� ient in pra ti e as well waspublished by Narenda Karmarkar [46℄ in 1984. Simi-larly to the ellipsoid method, the origins of this newpromising favorite had their roots in nonlinear program-ming. A year after the publi ation, the formal equiv-alen e of Karmarkars methods with the lassi al loga-rithmi barrier methods originally developed for non-linear programming was shown in [36℄. Soon, the rela-tionship with lassi al methods like the A�ne S alingMethod of Dikin [26℄, the Logarithmi Barrier Methodof Fris h (the Nobel prize winner e onomist) [31℄ andthe Method of Centers of Huard [42℄ was shown, manyof whi h ould also be proven to be polynomial [73℄. Narenda Karmarkar.The su ess of the new interior point algorithms in pra ti e started the revolution ofinterior point methods. This new lass of algorithms also brought the on e separate �elds oflinear and nonlinear optimization loser together. After the onne tion between Karmarkarsmethod and the logarithmi barrier method be ame widely known, it also be ame apparentthat these new methods are promising andidates to e� iently solve quadrati and linear omplementarity problems, too.It is interesting to note that regarding the omplexity of the simplex and the interior pointalgorithms, the same phenomena may be observed: the theoreti al omplexity bound is mu hhigher than the pra ti ally required number of iterations. An interesting breakthrough was17

a hieved for the interior point methods re ently by Terlaky et al. [24℄ who showed that by arefully adding an exponential number of redundant inequalities to the Klee-Minty ube,the entral path, and thus the interior point methods may be for ed to visit an arbitrarysmall surrounding of ea h vertex of the ube. By slight perturbations of the redundantinequalities, even today's state of the art solvers were unable to dete t the redundan y inthe presolve pro ess, thus were for ed to make the predi ted number of interior point steps.In a subsequent paper, Terlaky et al. showed that these results may be used to tighten thelower bounds on the steps required by most interior point methods [25℄.Although it seemed that the appearan e of the interior point methods may ause thepivot based methods to de line, it turned out that these two methods perfe tly omplementea h other, both having advantages and disadvantages over the other. The pros and ons ofthe two methods are summarized in [43℄. The ompetition of these two methods resulted in aboost for both algorithms, making the implementations �nely tuned and in reased e� ien yby at least two magnitudes.The in redible su ess of linear programming would have been impossible without theexisten e of e� ient softwares of the highest quality. The �rst ommer ial grade software wasdeveloped in 1954, when already several modern implementation issues like sparsity, numeri- al stability or the ompa t form of bases inverses were addressed in the ode. Unfortunately,several modern implementation issues remained business se rets. This was espe ially true forthe simplex based methods. Re ently in 2003, István Maros published his book on the om-putation methods of the simplex algorithm [57℄, making several aspe ts of the omputationaland implementation issues widely available.In the 1950's, when the simplex algorithm was developed, problems with a few hundredvariables and onstraints were onsidered to be large s ale problems. Today, state of the artsolvers (both ommer ial and free odes) are able to handle problems often with millions ofvariables.Still, ill onditioning may ause numeri al problems even for today's sophisti ated solverseven in the ase of relatively small problems. Several real life appli ations, like those arisingin petroleum industry, yield numeri ally very hallenging linear problems, on whi h falseinfeasibility reporting, or in some ases even y ling, may be observed for simplex based odes. Often, the numeri ally more robust interior point methods are inadequate, sin e theappli ation may require some spe ial feature of the simplex like algorithms (like e� ientwarm start). Moreover, in appli ations requiring a basi solution on several numeri allyhard problems, the usual basi identi� ation pro edures (espe ially after infeasible interiorpoint methods) su�er from similar problems like the simplex based methods.As the demand for more and more sophisti ated and omplex models in industry in-18

reases, often even faster than the evolvement of omputers, modelling and modell onsis-ten y questions be ome more and more important.

19

Chapter 2Linear feasibility problemsIn this hapter, we onsider the linear feasibility problemAx = b, x ≥ 0, (2.1)where A ∈ Rm×n,b ∈ Rm (n ≥ m). Without the loss of generality, we may assume thatrank(A) = m. Pra ti ally, the Gauss-Jordan elimination method is applied for the feasibility he k of the linear system and the removal of redundant equations.2.1 Linear algebra: an elementary and short introdu tionIn this se tion, we summarize some basi results in linear algebra whi h are used in thisthesis. For a omplete introdu tion, see [77℄.De�nition 2.1.1 Let a �nite set of ve tors {a1, . . . , ak} ⊂ Rn and s alars {λ1, . . . , λk} ⊂ Rbe given. Any ve tor b ∈ Rn of the form

b = λ1a1 + λ2a2 + . . . + λkakis a• linear ombination of ve tors {a1, . . . , ak},• a�ne ombination of ve tors {a1, . . . , ak} if the s alars satisfy λ1 + λ2 + . . . + λk = 1,• onvex ombination of ve tors {a1, . . . , ak} if the s alars are nonnegative and satisfy

λ1 + λ2 + · · · + λk = 1.De�nition 2.1.2 A set K ⊂ Rn is said to be onvex, if for any �nite set {a1, . . . , ak} ⊂ Kany onvex ombination of {a1, . . . , ak} also belongs to set K.20

De�nition 2.1.3 A ve tor b ∈ Rn is said to be linearly independent from ve tors{a1, . . . , ak} ⊂ Rn if there exist no su h s alars λi ∈ R, i = 1, . . . , k that

b = λ1a1 + · · · + λkak,that is b may not be written as a linear ombination of ve tors {a1, . . . , ak}. Otherwise, bis said to be linearly dependent from ve tors {a1, . . . , ak}.De�nition 2.1.4 The ve tors of a set {a1, . . . , ak} ⊂ Rn is said to be independent fromea h other, if neither of them is dependent on the others.For any set of ve tors, a maximal ardinality independent subset of ve tors has the same ardinality. This number is alled the rank of the set of the ve tors.De�nition 2.1.5 A subset V ∈ Rn is alled to be a linear subspa e,1. if x,y ∈ V then x + y ∈ V ,2. if x ∈ V and λ ∈ R then λx ∈ V .It is easy to observe, that by the se ond axiom of linear subspa es 0 ∈ V for any linearsubspa e V .De�nition 2.1.6 A subset L ∈ Rn is alled to be an a�ne subspa e, if there exists a linearsubspa e V ∈ Rn and a ve tor x0 ∈ Rn su h that L = x0 + V = {x0 + x | x ∈ V }.De�nition 2.1.7 A subset C ∈ Rn is alled to be a onvex one, if it is onvex and for anyx ∈ C and λ ∈ R it holds that λx ∈ C.The following proposition establishes the onne tion between a�ne subspa es and linearequality systems.Proposition 2.1.1 A subset L is an a�ne subspa e of Rn, if and only if there existsA ∈ Rm×n and b ∈ Rm that L = {x | Ax = b}.It is often interesting to onsider the smallest linear subspa e or onvex one that ontainsa given set of ve tors.De�nition 2.1.8 Let a set of ve tors {a1, . . . ak} ⊂ Rn be given. Then the linear subspa espanned by is de�ned as

〈{a1, . . . ak}〉 = {x | x = λ1a1 + · · · + λkak, λi ∈ R}.21

De�nition 2.1.9 Let a set of ve tors {a1, . . . , ak} ⊂ Rn be given. Then the onvex onespanned by is de�ned asCone ({a1, . . . ak}) = {x | x = λ1a1 + · · · + λkak, λi ∈ R⊕}.For a ompa t des ription of a linear subspa e, it is usual to use generator systems.De�nition 2.1.10 A set of ve tors {a1, . . . , ak} from a linear subspa e V ⊂ Rn is said tobe a generator system of V , if 〈{a1, . . . , ak}〉 = V .De�nition 2.1.11 A set of ve tors {a1, . . . , ak} of a linear subspa e V ⊂ Rn is said to be abasis of V , if it's linearly independent and is a generator system of V .It an be proved, that any basis of a linear subspa e has the same number of ve tors.This number is alled the dimension of the subspa e.It is natural to onsider the solvability of any linear system Ax = b as asking whetherve tor b is in the spanned subspa e of the olumns of matrix A or not. Similarly, thesolvability of Ax = b,x ≥ 0 is equivalent with the question whether x is in the spanned onvex one of olumns of matrix A or not.Consider the linear equality system Ax = b, and let a basis B of the olumns of A begiven, and suppose that b is in the subspa e spanned by the olumns of A. Then for any hoi e of xIN= s ∈ Rm−n, by setting the basis variables to xIB

= B−1b − B−1Ns we mayget every possible solution of Ax = b.The algorithms of this theses manipulate the so alled short pivot tableaus. Given a setof ve tors and a basis, the short tableau des ribes how the ve tors outside the basis may bede omposed to a linear ombination of the ve tors in the basis.De�nition 2.1.12 Let S ⊂ Rn be a given set of ve tors, its omponents indexed by J .Furthermore, let JB ⊆ J be su h that the ve tors indexed by them de�ne a basis of S. LetJN = J \JB be the indi es of the ve tors in S outside the basis B. Then the short pivottableau belonging to JB is the matrix TB ∈ R|JB |×|JN |, where tij is the oe� ient of theve tor belonging to j ∈ JN in its representation in the basis for i ∈ JB.The following theorem states that if tij 6= 0 in the previous de�nition, then the basisve tor orresponding to row i may be repla ed with the nonbasis ve tor orresponding to olumn j. The transformation of the short pivot tableau alled a pivot operation may bedone in O(m × (n − m)) arithmeti operations, where m is the number of ve tors in basisand n is the total number of ve tors. 22

Theorem 2.1.1 Let a �nite set of ve tors {ai | i ∈ J }, the indi es of a basis JB ⊂ J andthe orresponding short pivot tableau T be given whose olumns are indexed by JN = J \JB.If trs 6= 0 then ar, r ∈ JB an be ex hanged with as, s ∈ JN in the following way:(1) t′ij = tij −trjtistrs

i ∈ J ′B, i 6= s, j ∈ J ′

N , j 6= r,(2) t′sj =trj

trsj ∈ J ′

N , j 6= r,(3) t′ir = − tistrs

i ∈ J ′B, i 6= s,(4) t′sr = 1

trs.where J ′

B = (JB \ {r}) ∪ {s} is the index set of the new basis and t′ij are the s alars in theupdated short pivot tableau, while J ′N = J \ J ′

B.It is important to note that for the linear optimization problems (1.1),(1.2) and (1.3) ifa (an optimal) solution exists, than there exists a (an optimal) basis solution as well.De�nition 2.1.13 A ve tor x that satis�es the linear equality onstraints of problems (1.1),(1.2)or (1.3) is alled feasible, if x ≥ 0 also holds.De�nition 2.1.14 Problems (1.1),(1.2),(1.3) or (1.4) are alled feasible, if they yield afeasible solution. Otherwise the problem is alled infeasible.Most of our �niteness proofs are based on the well-known orthogonality theorem [48℄ thatdes ribes an orthogonal property between di�erent pivot tableaux of the set of ve tors. Letus de�ne the ve tors t(i), i ∈ JB and tj, j ∈ JN ∪{b} from the short pivot tableau as follows:(t(i))

k=

tik, if k ∈ JN ∪ {b}

1, if k = i

0, otherwiseand(tj)k

=

tkj, if k ∈ JB

−1, if k = j

0, otherwise.Observe that in the de�nition we onsidered the right hand side ve tor b as a olumn ofthe short pivot tableau, and indexed it with b.De�nition 2.1.15 We all the above de�ned ve tor t(i) a fundamental ir uit, and theve tor tj a fundamental o ir uit [71, 32℄. 23

We an now state the orthogonality theorem, [48℄:Theorem 2.1.2 For any matrix A ∈ IRm×n and its arbitrary bases B′ and B′′, the ve torst′(i) and t′′j belonging to TB′ and TB′′ respe tively, are orthogonal.For most pivot based methods, the analysis of the algorithm greatly depends on whetherthe intermediate solutions generated by the method lie in a smaller subspa e generated byve tors of the a tual basis, or are only spanned by the whole set of ve tors in the basis.De�nition 2.1.16 Let us onsider the linear feasibility problem (2.1). A basis B or solution

x = B−1b is alled degenerate, if there exits an i ∈ I su h that xi = 0. Otherwise, it's allednondegenerate.It is important to in lude the geometri al interpretation of basi solutions. Before doingthis, we need to de�ne some geometri al obje ts.De�nition 2.1.17 The interse tion of a �nite number of half spa es is alled a polyhedron.In other words, a polyhedron is the solution set of a system of inequalities {x | Ax ≤ b},where A ∈ Rm×n and b ∈ Rm.De�nition 2.1.18 A polytope is de�ned as the onvex hull of a �nite number of points inRn.De�nition 2.1.19 Let a onvex set C ⊂ Rn be given. A point x0 ∈ C is alled a vertex ofC, if x0 ∈ [a,b] where [a,b] ∈ C implies a = b = x0.Here [a,b] = {x | x = λa + (1 − λb), λ ∈ [0, 1]} denotes the se tion de�ned by a and b.Theorem 2.1.3 Let the feasibility problem of form F = {Ax = x, x ≥ 0} be given. Thenx0 is a vertex of F if and only if it's a feasible basi solution of {Ax = x, x ≥ 0}.Theorem 2.1.4 Any �nite polyhedron P is equal with the polytope de�ned by the vertexpoints of P .2.2 The alternate theorem of linear feasibility problemsThe famous alternate theorem of linear feasibility problems hara terizes the feasibility ofa system of linear inequalities. For an infeasible problem, the Farkas lemma provides a erti� ate of infeasibility, namely, a solution of an alternate system.24

Theorem 2.2.1 (Farkas) Let a matrix A ∈ Rm×n and b ∈ Rm be given. Then the linearfeasibility problem of formAx = b, x ≥ 0has a solution if an only if its alternative system of

yT A ≤ 0,

yTb = 1has no solution.Proof. We follow the lassi al way to prove Farkas's theorem. It's easy to see that bothsystems annot have a solution simultaneously. Let us suppose the ontrary. For any feasiblepair x and y

0 ≥ yT Ax = yTb = 1,holds, whi h is a ontradi tion.To show that at least one of the alternative systems always has a solution, we de�ne analgorithm that only terminates if it su essfully solved one of the problems, and prove thatthe algorithm is �nite, thus the proof will be omplete with the results in Subse tion 2.4 or2.6. �Intuitively, Farkas's theorem states that a linear feasibility problem either has a solutionor a ontradi tion may be dedu ed from it.2.3 The monotone build-up simplex method for feasibilityproblemsBased on the pivot sele tion rule of Anstrei her and Terlaky [4℄ we de�ne a monotoni build-up (MBU) simplex algorithm for linear feasibility problems [10℄. An mK upper bound forthe iteration bound of our algorithm is given under a weak non-degenera y assumption,where K is determined by the input data of the problem and m is the number of onstraints.Number K annot be bounded in general by a polynomial of the bit length of the input data[11℄.The �niteness of the problem for strongly degenerate problems may be ensured usings-monotone pivot rules. As well as elaborating this approa h, noti ing an interesting lo alproperty of degenera y, a re ursive pro edure is presented to handle degenerate problems.25

The essen e of this pro edure is as follows. If a degenerate pivot tableau is obtained, wede�ne a subproblem, restri ting the pivot position to a smaller part of the tableau. On thissubproblem, we follow the same prin iples as before to hoose the pivot position. The smallerproblem is either solved ompletely or a new, smaller degenerate subproblem is identi�ed. Ifthe subproblem is solved then we return to the starting larger problem, where either we anmake a non-degenerate pivot or dete t that the problem is infeasible. It is easy to see thatthe maximum depth of the re ursively embedded subproblems is smaller than 2m [10℄. Thegeneral idea of the re ursion is somewhat similar to Wolfe's ad ho method [96℄, howeverthe subproblems of the re ursion are de reasing in size not only by rows, but by olumns aswell, and no perturbation of any kind is applied.Analogous omplexity bounds for the linear programming versions of MBU and the �rstphase of the simplex method an be proved under our weak non-degenera y assumption.2.3.1 Ba kgroundThe most interesting open question of feasibility problems is whether there exists a poly-nomial pivot algorithm for solving (2.1). We provide an answer to an easier question: weshow that the monotoni build-up simplex algorithm using similar pivot sele tion rule likeAnstrei her and Terlaky's algorithm [4℄ an be formalized for the problem (2.1), whi h hasan iteration bound of mK, where K is a onstant determined by the problem data; more-over, the omplexity analysis of the algorithm requires a spe ial non-degenera y assumption.A ording to our best knowledge, sin e the development of Dantzig's simplex algorithm [21℄,there have been no pivot algorithms known to have an iteration bound other than the �nite-ness of the pivot sele tion rule. Most pivot algorithms start from a given (often feasible)basis and make an exponential number of iterations on the Klee-Minty ube [50℄, or on someother problems with very similar stru ture, before obtaining the optimal solution. On theother hand, in pra ti e, linear programming problems an usually be solved in O(m) pivotoperations (or at most in O(m2)) [57℄, where m is the number of onstraints of the prob-lem. The pra ti al experien e has been supported by many probability based analyses. Ithas been shown, under reasonable probabilisti assumptions, that the expe ted number ofiterations of the simplex algorithm is polynomial; see e.g. Borgwardt [13℄ and Todd [83℄.After Karmarkar's famous paper [46℄ on interior point methods (IPM), new algorithmsand their analyses have be ome the most popular resear h area in linear programming,pushing the theoreti al study and the elaboration of new pivot methods into the ba kground.A rare ex eption is the monotoni build-up simplex algorithm of Anstrei her and Terlaky [4℄that represents a major innovation over the former simplex type algorithms. This algorithm26

has the following main properties: (i) feasibility of the dual variables are a hieved in amonotone way, (ii) it does not ne essarily keep primal feasibility during the iterations, (iii)it ompares two kinds of pivot positions, (iv) in ase of degenera y it uses some kind ofanti-degenera y rule like the minimal index rule, (v) if the feasibility of a primal variable isviolated, it is restored after a pivot is made on the so- alled driving variable.We de�ne an algorithm for solving linear feasibility problems following the main idea ofthe monotoni build-up simplex method of Anstrei her and Terlaky [4℄, namely, the ompar-ison of the two kinds of pivot positions. We develop both the primal and the dual versionsof our algorithm [10℄. Both of them have the property that on e a variable be omes feasible,it remains so during the algorithm.This pivot sele tion rule di�ers from the usual ones in the literature for feasibility (lin-ear programming) problems. We show that our algorithm � under a lo al non-degenera yassumption � makes at most mK iterations before solving the linear feasibility problem [10℄.Sin e the ful�llment of the non-degenera y assumption annot be known a priori, weintrodu e versions of our algorithm that an solve general degenerate problems as well. Aswell as showing �niteness with s-monotone pivot rules introdu ed in Se tion 2.4.1, we de�nea primal-dual re ursive version of the algorithm that is apable of solving general degenerateproblems without applying any earlier known anti- y ling sele tion rule (e.g. minimal index,lexi ographi ordering of variables, et ).Re ursive methods to repla e index sele tion rules have already been su essfully appliedto handle degenera y. These methods in lude Bland's lassi al re ursive rule and Jensen'sgeneral re ursion [82℄. Among the re ursive methods, Jensen's re ursion is the most similarto ours, where both the number of variables and the number of rows in the subproblems hange through the re ursions, just as in our algorithm. The major di�eren e betweenthe two methods is that while Jensens's re ursion on entrates on the feasible part of the alling subproblem's pivot tableau, the subproblems de�ned by our re ursion in lude onlydegenerate rows and one spe ially sele ted infeasible row of the pivot tableau (or olumn fordual side subproblems).A geometri interpretation of our algorithm is the following. Like the riss- ross methods[79℄, the algorithm generates neighboring points de�ned by the interse tions of the onstraints� not ne essarily feasible basis � of the problem. If in any iteration the generated point lieson the feasible side of any de�ning hyperplane, then any further point generated will lie onthe same side of the hyperplane.In Se tion 2.3.3, we de�ne the MBU type simplex method for the linear feasibility prob-lem, and prove some of its important properties. In ase of not strongly degenerate pivot27

sequen es, we al ulate an upper bound for the omplexity of the algorithm. In Se tions 2.4and 2.6, the �niteness of the algorithm is proved without any non-degenera y assumption.2.3.2 A weaker degenera y on eptOur MBU algorithm for feasibility problems work under a weaker degenera y assumptionthan usual.For a given basis B, we use the following partition:IB = I+

B ∪ I0B ∪ I−

B ,whereI+

B = {i ∈ IB | xi > 0} , I0B = {i ∈ IB | xi = 0} and I−

B = {i ∈ IB | xi < 0} .If we want to emphasize the feasibility of a variable xi, we will use the notationI⊕

B = I+B ∪ I0

B and say that i ∈ I⊕B .A sample basi tableau a ording to the above partition is shown in Figure 2.1.

i −−...−

}I−

B

j0...0

}I0

B

k ++...+

}I+

BFigure 2.1: Partition of a basis.In most pivot algorithms pivots are made only on positive elements in primal feasiblerows (like the simplex algorithm), and only on negative elements in primal infeasible rows(like the dual simplex algorithm), or the two strategies are ombined (like the riss- rossmethod). Re ognizing this, Fukuda and Terlaky [34, 35℄ have de�ned the on ept of the so alled admissible pivot operations. Our algorithm arries out more general pivot operationsas well. A pivot on a positive value in the row of a stri tly feasible (positive) variable an beviewed as the primal feasibility equivalent of the dual side admissible pivots. While handlingdegenera y, our algorithm makes pivots on both negative and positive values in degeneraterows.De�nition 2.3.1 For a given basis B and pivot tableau T , a pivot element tij is alled ageneralized admissible pivot if 28

1. i ∈ I−B and tij < 0, or2. i ∈ I+B and tij > 0, or3. i ∈ I0B and tij 6= 0.Our algorithm uses generalized admissible pivots. Before formulating the algorithm, weneed to introdu e some de�nitions and also re�ne the on ept of degenera y. For s ∈ IN letus introdu e the set

Ks ={i ∈ I0

B | tis > 0}

.De�nition 2.3.2 A basis B is alled degenerate if the orresponding basi solutionxIB

= B−1b has at least one zero omponent, and it is alled non-degenerate otherwise.The degenera y on ept de�ned above is often referred to as primal degenera y andanalogously, primal non-degenera y. The phenomenon of degenera y is a lo al propertydepending on the a tual basis. A basis is degenerate if and only if the right-hand sideve tor is the linear ombination of less olumns of the basis matrix than its dimension. Wedistinguish between two kinds of degenera ies.De�nition 2.3.3 A degenerate basis B is alled weakly degenerate with respe t to indexs ∈ IN if Ks = ∅, and strongly degenerate with respe t to index s if Ks 6= ∅.Let us assume that for a given basis B we have hosen the index s as a olumn of ageneralized admissible pivot in a non-degenerate row. Observe that su h a pivot does notmake any degenerate variable infeasible if and only if Ks = ∅, i.e., if and only if basis B isweakly degenerate with respe t to index s. Su h a weakly degenerate tableau is shown inFigure 2.2.

s−...−

}I−

B

⊖...⊖

0...0

}I0

B

+...+

}I+

BFigure 2.2: A weakly degenerate pivot tableau.29

De�nition 2.3.4 Let a basis B and an index r ∈ I−B be given. We all a pivot operation on

tij an xr in reasing pivot if1. the pivot made on tij is a generalized admissible pivot,2. I⊕B ⊆ I⊕

B′, and3. xr > xr holds,where B′ = B ∪ {j} \ {i} denotes the new basis, and x denotes the new basi solution.Our aim is to formulate an algorithm based on xr in reasing pivots. Up to our bestknowledge, the only su h pivot algorithm for feasibility (or general linear programming)problems known from the literature is the dual version of the algorithm of Anstrei her andTerlaky [4℄ (whi h starts from a dual feasible basi solution).Unfortunately, there are basi tableaux where no xr in reasing pivot exists, as shown bythe example of Figure 2.3. This problem has a feasible solution of x1 = x2 = 0, x3 = x4 = 1,but has no in reasing pivot for the only infeasible row.x3 x4

x1 −1 0 −1

x2 1 −1 0Figure 2.3: A strongly degenerate tableau with no xr in reasing pivot, where r = 1.The example shows that one may expe t no su h algorithm that performs only xr in- reasing pivots, sin e the only nondegenerate pivot would possibly make the variable x2infeasible.2.3.3 The algorithmIn this se tion we formulate our MBU type simplex algorithm for solving linear feasibilityproblems of form (2.1). The proposed algorithm possesses similar properties to Anstrei herand Terlaky's MBU simplex method for linear programming problems [4℄. The main ideaof Antrei her and Terlaky was that for a given dual feasible solution, the algorithm buildsup the feasibility of primal variables monotoni ally, while in intermediate pivot steps dualfeasibility is allowed to be violated, but is restored when the hosen primal driving variablea hieves feasibility. Our algorithm is related to this algorithm in the sense that in our asethe number of primal feasible variables in rease monotoni ally and the pivot sele tion ruleis similar. 30

In order to a hieve monotoni ity, our algorithm follows an intuitive path. We hoosean infeasible variable in the given basi solution whi h we want to make feasible in thenext iteration(s). As long as this variable remains infeasible, it is referred to as the drivingvariable. We redu e the infeasibility of the driving variable xr by means of xr in reasingpivots. We show that this an always be done if the pivot tableau is non-degenerate orweakly degenerate with respe t to the pivot olumn.The pseudo ode of our algorithm is given in Figure 2.4, while the �ow hart of thealgorithm is presented in Figure 2.5. In ase of strongly degenerate tableaux, the algorithmis for ed to use some anti-degenera y methods. As well as the use of s-monotone indexsele tion rules, a re ursive method DegPro is presented in Se tion 2.6. This anti-degenera ymethod is di�erent from the lassi al, well-known methods from the literature and ensure�niteness too.In order to �nd an xr in reasing pivot, two ratio tests must be performed in the sele ted olumn. The relationship between these two ratio tests determines whether the drivingvariable an be made feasible in one pivot step, or (perhaps several) xr in reasing pivotsteps are ne essary.Let s denote the index of a sele ted pivot olumn. The value of the two ratio tests aredenoted byθ1 :=

br

trs

, and θ2 := min

{bk

tks

| k ∈ I⊕B , tks > 0

}.From their de�nition, it is easy to see that θ1 > 0 and θ2 ≥ 0. Furthermore, if the basis isnon-degenerate or weakly degenerate, then θ2 > 0 holds. We use the onvention that theminimum taken over the empty set is in�nity. The aim of the inner y le of the algorithm isto make the driving variable feasible.

31

MBU type simplex algorithm for feasibility problemsInput data: A ∈ Rm×n,b ∈ Rm, B.Output: x feasible solution or a message that no feasible solution exists.BeginT := B−1A, b := B−1b, I−

B := {i ∈ J | bi < 0}.While I−B 6= ∅ doLet r ∈ I−

B be arbitrary (driving variabe), rDone := false.While rDone = false doJ −

r := {j ∈ IN | trj < 0}.If J −r = ∅ thenNo feasible solution exists, ReturnEndifLet s ∈ J −

r be arbitrary, Ks := {i ∈ I0B | tis > 0}.If Ks 6= ∅ then

(T, l) = DegProc(T, I0B, r).If l ∈ IN then

s := l.elseNo feasible soution exists ReturnEndifEndifθ1 := br

trs, θ2 := min

{bk

tks| k ∈ I⊕

B , tks > 0}

.If θ1 ≤ θ2 thenPivot on trs, rDone := true.elseq := arg min

{bk

tks| k ∈ I+

B , tks > 0}

, pivot on tqs.EndifEndwhileI−

B := {i | bi < 0}.EndwhileReturn: x is a feasible solution.EndFigure 2.4: The MBU type simplex algorithm for feasibility problems.32

feasibleThe solution isSTOP STOP

r ∈ I−B

rDone = false

J −r := {j ∈ IN | trj < 0}

yes

No feasible. . .

no+

yesJ −

r = ∅yes

Ks := {i ∈ I0B | tis > 0}

⊕

(T, l) = DegProc(T, I0B, r)

yesno

q := arg min{

bk

tks| k ∈ I+

B , tks > 0}

rDone = true+

−− −

loop:false

s ∈ J −r

solution⊕ r

Ks 6= ∅

l ∈ IN

0

−

⊖

⊖ 0

...r

−

yesθ1 ≤ θ2

rDone =

noθ2 := min

{bk

tks| k ∈ I⊕

B , tks > 0}

θ1 := br

trs...−

yesno⊕

STARTI−

B := {i ∈ J | bi < 0}

loop...⊕

I−B 6= ∅

q

r

s s

no no −

Figure 2.5: Flow hart of the MBU type simplex algorithm for feasibility problems.We prove that if the given basis B is non-degenerate or weakly degenerate, then thealgorithm makes only xr in reasing pivots. First we investigate the ase when the drivingvariable leaves the basis.Proposition 2.3.1 For a given basis B, let r ∈ I−B be the index of the driving variable and

q ∈ I⊕B , tqs > 0. Suppose that

bq

tqs

= θ2 ≥ θ1 =br

trs

> 0,where br < 0, trs < 0, bq ≥ 0 and tqs > 0. In this ase a pivot is arried out on trs. Let usdenote the new basis by B′, thenI⊕

B ∪ {s} ⊆ I⊕B′33

holds, thus ∣∣I⊕B′

∣∣ >∣∣I⊕

B

∣∣ .Note, that θ2 > 0 also implies q ∈ I+B .Proof. When trs is the pivot position, the variable xr leaves the basis, while variable xsenters it. Let us denote the new basi solution (thus the new right-hand side of the pivottableau) by b′. We distinguish between the following ases.a) For index s the right-hand side be omes b′s = br

trs= θ1 > 0, making the driving variable

x′r = 0 feasible.b) For i ∈ I+

B , i 6= s we have b′i = bi −tisbr

trs. If tis ≤ 0 then by using bi ≥ 0, br < 0 and

trs < 0 we have b′i > 0, be ause we add a nonnegative number to an already positivebi. Otherwise if tis > 0 then by b′i = tis

(bi

tis− br

trs

) using the ondition bi

tis≥ θ2 ≥ θ1 =

br

trs> 0 we get that b′i ≥ 0. ) For i ∈ I0

B we have bi = 0, so b′i = − tisbr

trs, and by θ2 > 0 we have tis ≤ 0; thus

b′i = − tisbr

trs≥ 0.As we have seen, no feasible basi variable turns infeasible, while the infeasible drivingvariable xr leaves the basis (thus be oming feasible), and the entering variable xs enters ata feasible level, as well. It follows that ∣∣I⊕

B′

∣∣ >∣∣I⊕

B

∣∣. �In the next proposition we investigate the ase when a pivot is made outside the row ofthe driving variable. If the basis is non-degenerate or weakly degenerate for the enteringnonbasi variable, the pivot made by the algorithm is still xr in reasing.Proposition 2.3.2 In a given iteration of the algorithm, for the a tual basis B, let r ∈ I−Bbe the index of the driving variable and q ∈ I⊕

B , tqs > 0. Suppose that0 ≤

bq

tqs

= θ2 < θ1 =br

trs

,where br < 0, trs < 0, bq ≥ 0 and tqs > 0. In this ase a pivot is arried out on tqs. Letus denote the new basis by B′. Then I⊕B\{q} ⊆ I⊕

B′\{s} and 0 > b′r ≥ br. Furthermore, ifθ2 > 0, then 0 > b′r > br holds.Proof. Using the notations introdu ed in the proof of Proposition 2.3.1. one an see thatdue to the ratio test, for any index i ∈ I⊕

B we have i ∈ I⊕B′ if i 6= q, as proved in Proposition2.3.1. 34

Furthermore, b′s = bq

tqs≥ 0, so s ∈ I⊕

B′ thusI⊕

B\{q} ⊆ I⊕B′\{s},proving that the already feasible variables remain feasible. For the index of the drivingvariable b′r = br −

trsbq

tqs, where − trsbq

tqs≥ 0, using that trs < 0, tqs > 0 and bq ≥ 0. By the ondition θ2 < θ1 we have 0 ≥ trsbq

tqs> br thus

0 > b′r = br −trsbq

tqs

≥ br.If the basis is non-degenerate or weakly degenerate, then θ2 > 0 holds by de�nition, sobq

tqs> 0, thus −trs

bq

tqs> 0, and

0 > b′r = br − trs

bq

tqs

> br, ompleting the proof. �Geometri ally, Proposition 2.3.2 tells us that the new solution is loser to the nonnega-tivity onstraint of the driving variable.Summarizing the results of Propositions 2.3.1 and 2.3.2 we obtain the following result.Corollary 2.3.1 If the MBU type simplex algorithm performs only non-degenerate or weaklydegenerate pivots, then the algorithm makes only xr in reasing pivots, and thus it is �nite.Proof. The number of di�erent basis is �nite; therefore, it su� es to prove that the algo-rithm is not y ling, or in other words, that a basis may not o ur twi e. Sin e we assumedthat the algorithm does not visit strongly degenerate basis, it follows from Propositions 2.3.1and 2.3.2 that at ea h iteration, the algorithm makes xr in reasing pivots. In ea h step anew variable be omes feasible or the value of the driving variable in reases, thus the samebasis may not return. �Propositions 2.3.1 and 2.3.2 present important results for our MBU type simplex algo-rithm. Anstrei her and Terlaky in their paper [4℄ have proved similar results for their primalalgorithm for linear programming problems.In the next se tion we give a lower bound on the in rement of the value of the drivingvariable, and onsequently, we provide an upper bound on the iteration number of thealgorithm. Most lassi al and primal MBU simplex algorithms an be analyzed in a similarway like the one presented in the next se tion.To demonstrate the algorithm, onsider the following small example, given with a shortpivot tableau. 35

Example 2.3.1 Consider the following simple problem:x1 − x4 = −2

x2 − x4 + x5 = −1

x3 − x5 = −3

x1, x2, x3, x4, x5 ≥ 0We solve this example with the MBU algorithm with MIR index sele tion rule. In this exam-ple, the driving variable is always in bold, as is the pivot element of the short pivot tableau.The row of the driving variable is further emphasized by drawing other variables in grey. Ob-serve that variables x1, x2, x3 form a anoni al basis. The orresponding short pivot tableauis shown in Tableau 1.x4 x5

x1 −1 0 −2

x2 −1 1 −1

x3 0 −1 −3Tableau 1.x1 x5

x4 −1 0 2

x2 −1 1 1

x3 0 −1 −3Tableau 2.x1 x2

x4 −1 0 2

x5 −1 1 1

x3 −1 1 −2Tableau 3.x3 x2

x4 −1 −1 4

x5 −1 0 3

x1 −1 −1 2Tableau 4.A ording to the MIR index sele tion rule, the �rst driving variable is sele ted to be x1. Theonly negative element in the row of x1 de�nes an in reasing pivot, thus x1 is made feasiblein one iteration. The only remaining infeasible variable a ording to the se ond tableau isx3, the new driving variable. This variable annot be made feasible in one iteration, thusthe algorithm performs an in reasing pivot, and the value of x3 is in reased by 1. In thelast iteration, the driving variable x3 is made feasible, sin e the pivot in the row of x3 is anin reasing one. The found basi solution of the problem is x = (2, 0, 0, 4, 3).2.3.4 Complexity bound under the weak degenera y assumptionIn this subse tion we assume that our algorithm visits only non-degenerate or weakly degen-erate tableaux. Degenera y is handled in Se tions 2.4 and 2.6.By the de�nition of the pivot tableau and the basis, for the olumn indexed by s ∈ INof the tableau we have ts = B−1as and b = B−1b; thus ve tors ts and b an onsidered to36

be the unique solutions of the linear equations Bu = as and Bv = b. For any index i ∈ IB,Cramer's rule yieldstis =

det(Bis)

det(B)and bi =

det(Bi)

det(B),where matrix Bis ∈ Rm×m is the modi� ation of the regular basi matrix B su h that its ith olumn is repla ed by ve tor as, and similarly matrix Bi is obtained from B by repla ing its

ith olumn by ve tor b. For an xr in reasing pivot that does not make the driving variablefeasible, we haveb′r = br −

trsbq

tqs

=det(Br)

det(B)−

det(Brs)det(B)

det(Bq)

det(B)

det(Bqs)

det(B)

=det(Br)

det(B)−

det(Brs) det(Bq)

det(Bqs) det(B),where

−det(Brs)

det(Bqs)

det(Bq)

det(B)> 0holds by the fa t that the basis is not strongly degenerate, as seen in Proposition 2.3.2. Let

∆A := min

{−

det(Brs) det(Bq)

det(Bqs) det(B)

B is a regular submatrix of A, anddet(Brs)det(B)

< 0, det(Bq)

det(B)> 0, det(Bqs)

det(B)> 0

}be the minimal in rease of the driving variable's value. Assuming that in all pivot transfor-mations of the tableau θ2 > 0 holds, we have that ∆A > 0 is a �nite number andb′r = br −

trsbq

tqs

=det(Br)

det(B)−

det(Brs) det(Bq)

det(Bqs) det(B)≥

det(Br)

det(B)+ ∆A,thus an xr in reasing pivot either makes the driving variable feasible or in reases its valueby at least ∆A. We now bound the maximum absolute value that an infeasible variable antake during the algorithm. Let

∆max := max

{−

det(Br)

det(B)

sgn(det(Br)) = −sgn(det(B)),

B ∈ Rm×m is a regular submatrix of A

}be the maximal possible RHS value determined by the help of Cramer's rule. If there is anybasis for whi h there is a negative right-hand side value, then the number ∆max is positiveand �nite. Let K ∈ Z be su h that K =⌈

∆max

∆A

⌉, thus K ∈ N.We are now ready to bound the number of pivots ne essary to make the driving variablefeasible.Proposition 2.3.3 Assume that the algorithm visits only non-degenerate or weakly degen-erate pivot tableaux. Let r ∈ I−B be the index of the driving variable. There an be at most

K pivot operations before the driving variable be omes feasible.37

Proof. By the de�nition, the value of the driving variable annot be smaller than −∆max.The value of the driving variable in reases by at least ∆A in every iteration; thus, there annot be more than K iterations before the next xr in reasing pivot makes the drivingvariable feasible. �We are now ready to prove the bound on the omplexity of the algorithm.Theorem 2.3.1 Consider the feasibility problem (2.1). Assume that the MBU type pivotalgorithm visits only non-degenerate or weakly degenerate pivot tableaux in solving (2.1).Then the algorithm is �nite, and there an be at most mK pivots.Proof. By Proposition 2.3.3, there an be at most K pivot operations before the algorithmrea hes feasibility in the row of the driving variable or proves infeasibility. The numberof driving variables during the algorithm is bounded by the number of rows, be ause byPropositions 2.3.1 and 2.3.2 the number of infeasible variables de reases monotoni ally; thus,the algorithm may not y le, and there an be at most mK pivots before solving the problemor proving that it is infeasible. �We have proved under the non-degenera y assumption that the algorithm is �nite, andwe an bound the required number of pivot operations. This upper bound is generally nottight, and may even be larger than the obvious upper bound O(2n). However, it would beinteresting to �nd su h problem lasses where the upper bound an easily be determined;that is, the values of ∆A and ∆max are easily omputable. A naturally arising problem lass would be one, for whi h the matrix A is totally unimodular. Most problems omingfrom ombinatorial optimization are highly degenerate; thus, the omplexity estimate wouldonly bound the number of non-degenerate and weakly degenerate pivot steps. In ase A istotally unimodular and b ∈ Zm, we get K ≤ ‖b‖1. This means that in this spe i� ase Kdepends pseudo-polynomially on the (binary) input size. It would also be interesting to �ndproper perturbations for given problem lasses to handle strong degenera y. Note that theperturbation te hniques known from the literature (like the ǫ-perturbation te hnique [61℄)do not give an appropriate answer to the problem, sin e they usually drasti ally a�e t thevalue of ∆A, as shown in the next simple example.Example 2.3.2 Let A ∈ Zm×n be any matrix, and onsider the following linear feasibilityproblem: A

1 . . .1

x =

−1

. . .

−1

.38

After hoosing a properly small ǫ, the perturbed problem be ome A

1 . . .1

x =

−1 + ǫ

. . .

−1 + ǫm

.It is lear that (with properly sele ting some oe� ients in A) for this problem ∆A < ǫn,while ∆max > 1 + ǫ, thus K >

(1ǫ

)n.This example learly demonstrates that applying the ǫ-perturbation te hnique may resultin a drasti in rease in the iteration bound K.2.4 Finiteness with s-monotone pivot rulesIn this se tion, we de�ne a possible pro edure to handle degenera y (DegPro ), using so alled s-monotone index rules. Algorithmi ally, degenera y is often handled by perturbationor index sele tion rules. As we will see, this is also possible in the ase of the MBU typesimplex algorithm.2.4.1 The on ept of s-monotone pivot rulesIn most ases, our proofs of �niteness will use so alled s monotone pivot rules. This lassof pivot rules ontain the lassi al minimal index rule as well as several �exible pivot ruleslike LIFO (Last-In-First-Out) and MOSV (Most-Often-Sele ted). The advantage of thesesele tion rules is that they o�er reasonable freedom in variable sele tion (mostly at thebeginning of the algorithm1), thus providing possibility to avoid numeri ally instable pivots.Let a basis sequen e B0, B1, . . . , Bk be generated by a pivot algorithm. For ea h basisof the sequen e, we de�ne a ve tor s ∈ Nn⊕. These ve tors depend not only on the basisitself, but on the whole set of bases through whi h the basis was generated by the algorithm.Sin e many proofs of �niteness with index sele tion rules are very similar, we aim to give a ommon framework for our proofs.Let a pivot algorithm and a sequen e of ve tors s ∈ Nn

⊕ be given, the oordinates ofwhi h orrespond to the variables of the problem.De�nition 2.4.1 The sequen e of ve tors s is alled a preferen e ve tor, if their valuesde�ne how a given index sele tion rule would sele t a variable from any set of variables.1The LIFO rule if �exible only while it has to sele t among variables that have not yet moved at all.39

We onsider su h pivot rules that feature a someway monotone preferen e ve tor. Theaim of index sele tion rules is to break any ties in the pivot algorithm (i.e. when the stru tureof the pivot tableau does not uniquely de�ne the pivot position).De�nition 2.4.2 Consider a pivot rule with preferen e ve tor s. The sequen e of ve tors sshould satisfy the following riteria:1. In any iteration (pivot operation), the entries of ve tor s may not de rease and mayonly be hanged for entries orresponding to variables moving in or out of the basis. Ifthe pivot rule would allow for several variables to enter (or leave) the basis, one witha highest s value is to be sele ted.2. The s ve tor provides that one of the following ases must o ur:(a) After basis Bk, the algorithm terminates in a �nite number of steps.(b) If there exist su h variables that move in�nitely many times after basis Bk, thenlet the index set of su h variables be denoted by I∗. After basis Bk, in a �nitenumber of iterations, a basis B∗ is generated, for whi h the variable outside thebasis with a minimal s value is unique with respe t to I∗. Let this variable bedenoted by xl.3. When after basis B∗, variable xl enters the basis, then until it leaves it again thefollowing holds: in any iteration, the s value of those variables that entered the basisafter variable xl has a higher s value than variable xl.Those pivot rules for whi h a preferen e ve tor s satis�es 1− 3 are alled s-monotone pivotrules. Observe, that basis B∗ should simultaneously satisfy riteria 2 and 3.First, we demonstrate how this de�nition may be exploited in the �niteness proofs. InSe tion 2.5 we show that several well-known pivot rules satisfy the 3 riteria.2.4.2 An anti-degenera y pro edure based on index sele tion rulesThe key issue of the analysis presented in the previous se tion was that the tableaux visitedby the algorithm were all non-degenerate or weakly degenerate. The pro edures for handlingdegenera y presented in this and the following se tion sele t pivot positions based on therow of the driving variable and the degenerate submatrix ( onsisting of all degenerate rows).These pivots do not hange the urrent basi solution, but transform the basi tableau insu h a way that it either be omes primal infeasible, or there will be at least one olumn thathas a negative entry in the row of the driving variable and is weakly degenerate.40

The �rst version of pro edure DegPro is presented in Figure 2.6, where by alling tableauwe mean the pivot tableau of the original problem. The �ow hart of the algorithm ispresented in Figure 2.7. We note that it would be possible (and su� ient) to use the sameve tor s throughout the whole MBU algorithm, however, it is su� ient to introdu e oneonly when strong degenera y o urs. For the sake of simpli ity, we hoose this strategy. Theproper initialization of ve tor s depends on the sele ted pivot rule. Some of the possible ases are expli itly given the next subse tion.Anti degenera y pro edure with s-monotone index rulesPro edure DegPro (s-monotone version)Input: (T, I0B, r).Output: (T modi�ed pivot tableau, l status �ag).BeginInitialize ve tor s (a ording to the sele ted pivot rule).While ((J −N := {j ∈ IN : trj < 0}) 6= ∅) doLet k ∈ J −

N be arbitrary with maximal value respe t to s.If ((I+

k := {i ∈ I0B | tik > 0}) = ∅

) thenThe alling tableau is weakly degenerate Return(T, k).EndifLet l ∈ I+k be arbitrary with maximal value respe t to s.Pivot on tlk, update ve tor s.EndwhileThe alling tableau is infeasible, Return(T,−1).EndFigure 2.6: A possible pro edure to handle degenera y using s-monotone index sele tionrules.The terminal tableaux of pro edure DegPro (presented in Figure 2.6) for the degeneraterows and the row of the driving variable is presented in Figure 2.8.Note that both terminal tableaux ensure �niteness of the MBU algorithm, sin e in the�rst ase an in reasing pivot is possible, guaranteeing that the same basis will never return,while in the other ase the problem is proved to be infeasible, and thus the MBU algorithmterminates. So it is su� ient to show that the anti-degenera y pro edure is �nite.41

J −N 6= ∅no

I+k = ∅no

yesyes Return(T,k) 00⊖

⊖

START

Pivot on tlkl ∈ I+

kupdate ve tor s.... ...-00⊕. . .

...⊕ - -Return(T,-1)

k ∈ J −N

STOPFigure 2.7: Flow hart of a possible pro edure to handle degenera y using s-monotone indexsele tion rules.⊖ 0... ...⊖ 0

− −

0...0

⊕ . . . ⊕ −Weakly degenerate InfeasibleFigure 2.8: Terminal tableaux of the anti-degenera y pro edure with s-monotone indexsele tion rules, with respe t to the degenerate rows and the row of the driving variable.We prove that while solving the degenerate subproblem, the a tual basi solution of thealgorithm does not hange.Proposition 2.4.1 Let a degenerate pivot tableau T ∈ Rm×n be given, and denote the indexset of degenerate rows by D = I0B. Then any pivot made on the elements of submatrix TDINdoes not hange the urrent basi solution.Proof. Let the hosen generalized admissible pivot element be tij ∈ TDI . We show that theright-hand side of the tableau does not hange after the pivot, namely b′i = bi

tij= 0 = bi and

b′k = bk + tkjbi

tij= bk + 0 = bk, where k 6= i. �We are ready to show the �niteness of the pro edure. From this follows the �niteness ofthe MBU algorithm as well.Theorem 2.4.1 If the anti-degenera y pro edure DegPro uses an s-monotone index sele -tion rule, then it's �nite. 42

Proof. In ase of pro edure DegPro , the whole problem onsists of degenerate rows andthe single nondegenerate row of the driving variable.Suppose the ontrary, namely that the algorithm is not �nite. Sin e the number ofdi�erent basis is �nite, this is only possible if the algorithm y les2. Let us onsider aminimal y ling example. In su h an example, with the ex eption of the driving variable,every variable moves in�nitely many times.Sin e the index sele tion rule is s-monotone, there exists a basis B′ and variable xl outsidethe basis, satisfying the se ond riterion of s-monotoni ity. Thus, when this variable entersthe basis, its value in s among the other variables outside the basis is minimal (also usingthe �rst riterion). Sin e the algorithms sele t the one with the largest s value from thepossible variables, this is only possible if for the ve tor t′(r) it holds that trl < 0 and tri ≥ 0for any i ∈ I \ {l} orresponding to basis B′.Let us now onsider basis B′′, when the variable xl next leaves the basis. Let the variableentering the basis be xk. A ording to the rule of the anti-degenera y pro edure, among thepositive entries of t′′k, variable xl has the highest value in s′′, otherwise the pro edure wouldhave hosen an other variable. The variables with lower s values, by the third riterion of

s-monotoni ity have not moved sin e basis B′. Let the set of these variables be K, thusK = {j ∈ IB′′ | t′′jk > 0} ∩ IB′ . Furthermore, let L = {j ∈ IB′′ | t′′jk ≤ 0}. Then l 6∈ L ∪ K.The pivot tableaux orresponding to basis B′ and B′′ are shown in Figure 2.9.

B′ : B′′ :

0...0

xr ⊕ · · ·⊕ − ⊕ · · ·⊕ −

xl

0

xl +...0

xr − −

xk

}K

}L

Figure 2.9: Basis B′ and B′′.Sin e t′ri = 0 if i ∈ K and t′′ik = 0 if i ∈ IN ′′ , it follows thatt′(r)

Tt′′k =

∑

i∈IN′′

t′rit′′ik +

∑

i∈K

t′rit′′ik +

∑

i∈L

t′rit′′ik + t′rlt

′′lk + t′rrt

′′rk ≤ t′rlt

′′lk + t′rrt

′′rk

= t′rlt′′lk + t′rk < 0,2In ase of the �exible index sele tion rules, by y ling we mean that the algorithm returns to a givenbasis in�nitely many times. 43

sin e t′rj ≥ 0 if j ∈ L and t′rit′′ik = 0 if i ∈ K ∪ {r}, sin e these variables have not movedbetween the two bases. Using that t′rl < 0 and t′′lk > 0, as well as t′rr = 1 and t′′rk < 0 itfollows that t′rlt

′′lk < 0, ontradi ting the orthogonality theorem. �We formally state, that the MBU algorithm using an anti-degenera y pro edure with

s-monotone index sele tion rule is �nite.Theorem 2.4.2 The MBU algorithm, in the ase of applying an s-monotone index sele tionrule, is �nite for any feasibility problem.Proof. For not strongly degenerate pivot tableaux, the in reasing pivot operations ensurethat a basis may not o ur twi e. By Theorem 2.4.1, for strongly degenerate pivot tableaux,the anti-degenera y pro edure provides a weakly degenerate tableau in �nite steps, or provethat the problem is infeasible, by Lemma 2.4.1, without hanging the right-hand side of theproblem. �Remark 2.4.1 We have also ompleted the proof of Theorem 2.2.1.Remark 2.4.2 The presented DegProc pro edure may also be viewed as a spe ial Criss-Criss method with s-monotone pivot sele tion rules.Example 2.4.1 To demonstrate how the pro edure against degenera y works, onsider thefollowing problem:x1 − x4 + x6 = −1

x2 + x4 − x5 − 2x6 = 0

x3 − x5 + x6 = 0

x1, x2, x3, x4, x5, x6 ≥ 0We solve this example with the MBU algorithm using the anti-degenera y pro edure withthe MIR index sele tion rule. As was for Example 2.3.1, the row of the driving variable isemphasized by drawing the other variables in grey. Observe, that variables x1, x2, x3 form a anoni al basis. The orresponding short pivot tableau is shown in Tableau 1.x4 x5 x6

x1 -1 0 1 -1x2 1 -1 -2 0x3 0 -1 1 0Tableau 1.

x4 x5 x6

x1 -1 0 1 -1x2 1 -1 -2 0x3 0 -1 1 0Tableau 2.44

The only infeasible variable of the �rst tableau is x1, whi h thus be omes the drivingvariable. Sin e the olumn of the only negative pivot position is strongly degenerate, theanti-degenera y pro edure is alled. A ording to MIR, a pivot is made in the row of x2 andthe olumn of x4. The new subtableau is dual infeasible, meaning that the original problemhas a weakly degenerate olumn. After making the orresponding pivot, the next tableau isfeasible.x2 x5 x6

x1 1 -1 -1 -1x4 1 -1 -2 0x3 0 -1 1 0Tableau 3.

x2 x5 x6

x1 1 -1 -1 -1x4 1 -1 -2 0x3 0 -1 1 0Tableau 4.

x2 x1 x6

x5 -1 -1 1 1x4 0 -1 -1 1x3 -1 -1 2 1Tableau 5.The basi solution found is x = (0, 0, 1, 1, 1, 0).2.5 s-monotone pivot rulesWe now show, that several well-known pivot rules satisfy the requirements of s-monotonepivot rules. Throughout the proofs, we prove that if the (a) version of the se ond riteriadoes not hold, then (b) does.The lassi al index sele tion rule of Bland hooses the variable with the smallest possibleindex. The Bland rule is often referred to as the minimal index sele tion rule (MIR).Lemma 2.5.1 The minimal index sele tion rule of Bland is s-monotone.Proof. Let us represent the minimal index rule of Bland, using the ve tor s that is de�nedas

si = n − i, i ∈ I.It is easy to see, that the maximal value in any subve tor of ve tor s orresponds to thevariable with the lowest index, thus orre tly representing the minimal index rule.The �rst and se ond riteria of s-monotoni ity easily follow from s being onstant.For basis Bk+1 the index arg min sii∈INk+1

∩I∗

= u is unique, sin e every value of s is di�erent(su = n− u), thus u is the highest index in the set INk+1∩ I∗. If v = arg min si

i∈IBk+1∩I∗

is su h thatv ≥ u (then v > u also holds), then let B∗ = Bk+1 and l = u, otherwise let B∗ be the �rstbasis when variable xv moves outside the basis, and let l = v.45

The third riterion of the rule holds, be ause the order of the variables with respe t to sremains the same throughout the algorithm. �The LIFO index sele tion rule hooses the variable that has moved most re ently. The on ept behind LIFO is that if a variable was sele ted to enter the basis, and it soon turnedout that it be ame infeasible, than that variable may have not been a good hoi e, and thealgorithm should try an other.Lemma 2.5.2 The index sele tion rule LIFO is s-monotone.Proof. Let us initiate the ve tor s to be onstantly zero. In a pivot when xl leaves and xkenters the basis in the pth iteration, the values of s are modi�ed to favor these variables:s′i =

{p if i ∈ {l, k},si otherwise. }The �rst riterion of the rule holds by de�nition.Let M = INk

∩ I∗. We follow the moves of these variables, and when variable xi entersthe basis, we delete the orresponding index from M, thus M := M\ {i}. Sin e M ⊂ I∗,it follows that after a �nite number of iterations | M |= 1 holds. Let the last element of Mbe denoted by l. For the basis orresponding to this state, l = arg min sii∈IN∩I∗

holds, thus thisbasis is a proper hoi e of B∗ with variable xl.The third riterion follows immediately from the de�nition of LIFO. �The MOSV rule prefers those variables that moved most often before. The idea may beinterpreted in the following way. Most index sele tion rules ensuring �niteness are re ursivetype rules, meaning that there is an upper bound on how many times a variable may moveduring the algorithm. For example, in the ase of MIR, it is easy to see that while theoreti- ally the variable x1 with the lowest index may move up to 2n−1 times, the variable xn withthe highest index may move only on e. This way, MOSV may be interpreted as trying to �xthose variables that already moved the most, by exhausting the possible freedom to move.Lemma 2.5.3 The index sele tion rule MOSV is s-monotone.Proof. Let the ve tor s be initialized as onstant zero. In a pivot when xl leaves and xkenters the basis in the mth iteration, the values of s are modi�ed to in rease the favor ofthese variables:s′i =

{si + 1 if i ∈ {l, k},

si otherwise. }46

The �rst riterion follows immediately from the de�nition of the index sele tion rule.Let MN = INk∩ I∗ and MB = I∗ \MN . We de�ne the numbers ti as follows:

ti =

{si, if i ∈ MN

si + 1, if i ∈ MB

.Let P = {i ∈ I∗ | i ∈ arg minj∈I∗

tj} and minj∈I∗

tj = p. We ontinue the iterations a ording tothe pivot rule. Sin e P ⊂ I∗, thus for any i ∈ P there exists su h a basis, when variablexi enters the basis for the �rst time after basis Bk. When this happens, we delete its indexfrom P , thus P := P \ {i}. After �nitely many iterations, su h a set P is obtained, forwhi h | P |=| {l} |= 1. After this happens, let the �rst basis when variable xl enters thebasis be B∗. We show that in B∗ the hoi e of xl is unique. Observe, that in this asesl = p, regardless whether xl was in the basis Bk or not. Be ause of the pivot rule, p < siif i ∈ I∗ \ P and sin e every variable with index i ∈ P \ {l} has at least on e entered thebasis after Bk and now is outside the basis, their values in s must be at least p + 2. On theother hand, if it was a basis variable in Bk than its s value is at least p+1. Thus the se ond riterion also holds.Sin e the variable xl enters the basis in basis B∗, and every other variable with index inI∗ entering the basis after xl already had a higher s value than xl in basis B∗, a ording tothe MOSV rule, the third riterion also holds. �In the ase of LIFO and MOSV, when several variables have the same value in s, when a hoi e has to be made, the algorithm may hoose taking other preferen es into onsideration,like numeri al stability or a preferred basis stru ture.Remark 2.5.1 The de�nition of s-monotoni ity ensures that if an algorithm using s-monotoneindex sele tion rule y les, than there exists a size minimal y ling example. Observe how-ever, that in ontrast with the usual methodology, it is not ne essarily true that in anyminimal size y ling example every variable moves in�nitely many times. A simple exampleis the MBU algorithm for feasibility problems.2.6 A re ursive algorithm to handle degenera yIn this se tion, we dis uss a new, re ursive DegPro of the MBU type algorithm presentedin Figure 2.4.Similarly as in the ase of the s-monotone index sele tion rules, the re ursive algorithmmakes pivots only in degenerate rows and terminates with the same subtableaux as presentedin Figure 2.8. In this ase, for solving the subproblems we use the dual (primal) version of47

our MBU type simplex algorithm, thus the solution pro ess of the subproblems arries thesame already shown basi properties over the iterations as the MBU type simplex algorithm.Primal Dual ⊕ Primal⊖Ax = b yT A = cT Ax = b

x ≥ 0 y ≥ 0 x ≤ 0strongly 0 0degenerate +... + − +

...0 0

− − − 0 . . . 0 − +

⊕ ⊖feasible ⊕ ⊖... ...⊕ ⊕ ⊕ . . . ⊕ ⊖

ւր րցւտ

⊖infeasible ⊕ . . . ⊕ −... ⊕ . . . ⊕ +

⊖

−

ւր րցւտweakly ⊖ 0 ⊖ 0degenerate ... ... ⊕ . . . ⊕ +... ...

⊖ 0 ⊖ 0

− − 0 . . . 0 − − +Figure 2.10: Key tableaux of the algorithm.The anti-degenera y method des ribed in the sequel uses a re ursive stru ture. We use thetableaux illustrated in Figure 2.10. Suppose that a strongly degenerate tableau of the primalproblem is obtained. In this ase, the algorithm de�nes a so alled dual side subproblem.The tableau of this subproblem onsists of all the degenerate rows without the primal right-hand side and the row of the driving variable, as dual right-hand side. Obviously, the size ofthe subproblem is smaller than the size of the original primal problem. Now suppose thatthe dual subproblem is solved. If the dual subproblem is feasible, its stru ture is the same48

as the stru ture of an infeasible tableau of the original primal problem orresponding tothe degenerate rows and the driving variable; while if the dual subproblem is infeasible, itsstru ture is the same as the stru ture of a weakly degenerate tableau of the original primalproblem, restri ted again to the degenerate rows and the row of the driving variable.A similar, interrelated onne tion may be observed between problems Dual⊕ and Primal⊖,as illustrated in Figure 2.10. The problem Primal⊖ is an analogous version of the origi-nal problem, where all the variables are required to be non-positive, instead of being non-negative. The de�nition for the analogous algorithm will be given in Figure 2.12, while the onne tions used by the re ursions will be illustrated in Figure 2.15.It may be observed, that the form of the subproblem is mainly based on the parity ofthe level of re ursion, as shown in Figure 2.11.level of re ursion original n = 2k + 1 n = 2kproblem k = 0, . . . k = 1, . . .form of Ax = b yT A = cT Ax = bsubproblem x ≥ 0 y ≥ 0 x ≤ 0Figure 2.11: Form of subproblem depending on the level of re ursion.We need to note that the dual subproblem ould naturally be solved by arbitrary pivotmethods, but we prefer the dual version of our MBU type simplex algorithm.The nature of the pro edure handling degenera y makes it onvenient to formulate itas a re ursive method. The pseudo- odes of the subpro edures are summarized in Figures2.12. and 2.13. In these pro edures, the term alling tableau refers to the pivot tableau orresponding to those pro edures, that alled the subpro edure re ursively. The pivotsmade in the subpro edures are arried out on the whole tableau, but onsider only thesubtableau de�ned by the index sets F and G. The olumn indexed by b and the row indexedby c play the roles of primal and dual right-hand sides respe tively, of the subproblems takeninto onsideration.The primal subproblems alled by DegPro are instan es of type Primal⊖ where the right-hand side values are required to be non-positive. In what follows we all a primal variablefeasible if its sign is adequate for the orresponding feasibility problem or zero, and otherwiseinfeasible. For the sake of larity, in the dis ussion of the subproblems, we talk only of valuesinstead of variables when referring to the right-hand side values; be ause only for the originalproblem do primal variables a tually orrespond to the right-hand side values.The analysis of the pro edure for solving the dual subproblems is ompletely analogous to49

The dual side MBU algorithm for solving degenerate subproblemsPre edure DualSubMBUInput: (T, c,F ,G).Output: (T modi�ed pivot tableau, l status �ag).BeginWhile (J −B := {j ∈ G | tcj < 0}) 6= ∅ doLet r ∈ J −

B be arbitrary (driving variable), rDone := false.While rDone = false doIf (I+r := {i ∈ F | tir > 0}) = ∅ thenThe alling pivot tableau is weakly degenerate, Return(T, r).EndifLet s ∈ I+

r , Ks = {k ∈ G | tck = 0, tsk < 0} .If Ks 6= ∅ then (when subtableau TFG is strongly degenerate)(T, s) := PrimalSubMBU(T, r,F , {j ∈ G | tcj = 0}).If s = −1 thenThe alling tableau is weakly degenerate, Return(T, r).EndifEndif

θ1 := tcr

tsr, θ2 := max

{tck

tsr| k ∈ G, tck > 0, tsk < 0

}.If θ1 ≤ θ2 thenpivot on tsr, rDone := true.elseq := arg max

{tck

tsr| k ∈ G, tck > 0, tsk < 0

}.pivot on tsq.EndifEndwhileEndwhileThe alling tableau is infeasible, Return(T,−1).EndFigure 2.12: The dual subpro edure starts with the solution of a subproblem of the formDual⊕.

50

the primal version. Using Figure 2.10, the on ept of dual side weak and strong degenera y,as well as the on ept of the xr in reasing pivot an be de�ned analogously as in Propositions2.3.2, 2.3.3 and 2.3.4.We show that the primal and dual subpro edures have similar properties as the primalalgorithm working on non-degenerate problems.Proposition 2.6.1 Let us assume that for a given subproblem either the PrimalSubMBU orthe DualSubMBU pro edure is used. If the orresponding tableau is non-degenerate or weaklydegenerate, then the pivot steps arried out are in reasing pivots for the olumn indexed byb or for the row indexed by c, respe tively.Proof. The �rst part of the proposition follows immediately from Proposition 2.3.1, whilethe part orresponding to the dual subalgorithm an be proved similarly. �Although we have already stated the main idea behind the re ursive algorithm, we nowformalize it for both subpro edures.Proposition 2.6.2 Suppose that the MBU type simplex algorithm is started from problemP0. Let us assume that repeated strongly degenerate tableaux o urred and denote them byP1 := DualSubMBU(T1, k1,F1,G1), P2 := PrimalSubMBU(T2, k2,F2,G2), . . . ,Pl := PrimalSubMBU(Tl, kl,Fl,Gl) (or Pl := DualSubMBU(Tl, kl,Fl,Gl)). Then the piv-ots arried out while solving subproblem Pl do not hange any right-hand side of problemsPi, where (i = 1, . . . , l − 1) and ki /∈ Fl ∪ Gl.Proof. The re ursive steps involve only degenerate rows and olumns, thus by Proposition2.4.1 our statement holds. �Observe that sin e Fl ⊂ Fl−1 ⊂ . . .F2 ⊂ F1 and Gl ⊂ Gl−1 ⊂ . . .G2 ⊂ G1 hold, duringthe solution of a degenerate subproblem, we either make in reasing pivots or swit h to asubproblem of stri tly smaller size.The pro edure starting the re ursion and handling degenera y an easily be formalizedas shown in Figure 2.14.The relationship of the di�erent subproblems is shown in Figure 2.15. The �gure shows apossible primal-dual-primal alling sequen e. Phrases PrimalSubMBU and DualSubMBUrefer to the type of the subproblem. The stru ture of the basi tableaux orresponding tostrongly degenerate basis has already been presented in Figure 2.10.51

The primal side MBU algorithm for solving degenerate subproblemsPro edure PrimalSubMBUInput: (T, b,F ,G).Output: (T modi�ed pivot tableau, l status �ag).BeginWhile (I+B := {i ∈ F | tib > 0} 6= ∅) doLet r ∈ I+

B be arbitrary (driving variable), rDone := false.While rDone = false doIf (J −r := {j ∈ G | trj < 0}) = ∅ thenThe alling tableau is weakly degenerate, Return(T, r).EndifLet s ∈ J −

r , Ks := {k ∈ F : tkb = 0, tks > 0} .If Ks 6= ∅ then (when subtableau TFG is strongly degenerate)(T, s) := DualSubMBU(T, r, {i ∈ F | tib = 0},G).If s = −1 thenThe alling tableau is weakly degenerate, Return(T, r).EndifEndif

θ1 := |trb|trs

, θ2 := min{

|tkb|tks

| k ∈ F , tkb < 0, tks > 0}

.If θ1 ≤ θ2 thenpivot on trs, rDone := trueelseq := arg min

{|tkb|tks

| k ∈ F , tkb < 0, tks > 0}.pivot on tqs.EndifEndwhileEndwhileThe alling tableau is infeasible, Return(T,−1).End

Figure 2.13: The PrimalSubMBU starts with the solution of a subproblem of form Primal⊖.We now prove the �niteness of the MBU algorithm without any non-degenera y assump-tion. 52

The anti-degenera y pro edure starting the re ursionPro edure DegPro (re oursive version)Input: (T, I0B, r).Output: (T modi�ed pivot tableau, l status �ag).Begin

(T, l):=(PrimalSubMBU(T, I0B , {1, . . . , n}, n + 1)).Return(T, l).End

Figure 2.14: A possible anti-degenera y pro edure, DegPro .Theorem 2.6.1 The MBU type simplex algorithm using the re ursive version of Degpro is �nite for any feasibility problem.Proof. While the algorithm visits only non-degenerate or weakly degenerate problems, thealgorithm arries out xr in reasing pivots a ording to Proposition 2.6.1; thus, the samebasis may not return. Then the algorithm may not y le. In ase the orresponding pivottableau is strongly degenerate for a hoi e of a nonbasi variable, the algorithm alls thePrimalSubMBU or DualSubMBU subpro edures for stri tly smaller problems, thus the depthof re ursion is at most 2m ≤ n + m.Consider the ase when the re ursively alled DualSubMBU solves the subproblem.When it stops with an infeasible subtableau, then the orresponding alling (sub)problembe omes weakly degenerate to the proper nonbasi variable; thus, the pro edure ontinueswith an in reasing pivot. When the DualSubMBU stops with a feasible subtableau, thenthe primal (sub)problem above be omes infeasible. This means the infeasibility of the orig-inal problem when the alling pro edure was the DegProc pro edure. Otherwise, the dualsubproblem one step above in the re ursion be omes weakly degenerate; thus, it ontinueswith an in reasing pivot.Similar onne tions hold when the PrimalSubMBU solves the orresponding subprob-lem.Be ause the depth of the re ursion is bounded and the returning sub-pro edures providethe possibility of an in reasing pivot for the alling pro edure, no basis may o ur twi e.The number of di�erent basis is �nite, thus the algorithm is �nite. �53

+

⊕⊕ ⊕ ⊕

⊕⊕

⊕

⊕

⊕

⊕

⊕⊕ ⊕ ⊕

. . .

. . .. . .

. . .

. . .

��������

��������

��������

�����

�����

�����

− − −

− − − −

−

⊖

⊖

⊖

⊖

������

���

������

���

���

0

0

0

0

0 0

0

0

0 0

0

0

�� ��

�� ��

�� ��

��� ��

��� ��

��� ��

���� �����

���� �����

������ ������

������ ��

������ ��

��������� ��

Figure 2.15: A Primal-Dual-Primal sequen e of subproblems.54

Example 2.6.1 For demonstrating the re ursion, we solve the linear feasibility problem ofExample 2.4.1 with the presented re ursive s heme.The a tual subproblem being solved is emphasized in normal bla k olor, while the rest ofthe short pivot tableaux is presented in grey.x4 x5 x6

x1 -1 0 1 -1x2 1 -1 -2 0x3 0 -1 1 0Tableau 1.

x4 x5 x6

x1 -1 0 1 -1x2 1 -1 -2 0x3 0 -1 1 0Tableau 2.The only infeasible variable of the �rst tableau is x1, whi h thus be omes the drivingvariable. Sin e the olumn of the only negative pivot position is strongly degenerate, a re- ursive subproblem (of form Dual⊕)is de�ned. The only dual infeasible olumn of the �rstsubproblem shown by Tableau 2 is the olumn of x4, while the only nonzero pivot position forthis olumn is in the row of x2. The subproblem is also strongly degenerate, sin e any pivotin reasing the dual driving variable would make the olumn of x5 infeasible. Thus, an evensmaller subproblem (of form Primal⊖) is de�ned, as shown in Tableau 3.

x4 x5 x6

x1 -1 0 1 -1x2 1 -1 -2 0x3 0 -1 1 0Tableau 3.

x4 x5 x6

x1 -1 0 1 -1x2 -1 -1 2 0x3 -1 -1 3 0Tableau 4.

x4 x2 x6

x1 -1 0 1 -1x5 -1 -1 2 0x3 -1 -1 3 0Tableau 5.In this se ond-level of re ursion, the row of the driving variable is the row of x2, forwhi h there are two possible pivot positions. A pivot in the olumn of x5 would make thesubproblem feasible in one iteration, while the olumn of x6 is strongly degenerate, and aneven smaller subproblem would have to be de�ned. For simpli ity, assume that we make apivot in the olumn of x5. Sin e the se ond-level subproblem was found feasible, the �rstlevel subproblem is infeasible. The infeasible olumn of the �rst level dual subroblem de�nesa weakly degenerate olumn for the original primal problem, thus an in reasing pivot for thedriving variable x1 in the olumn of x4 is now possible.55

x4 x2 x6

x1 -1 0 1 -1x5 -1 -1 2 0x3 -1 -1 3 0Tableau 6.

x1 x2 x6

x4 -1 0 -1 1x5 -1 -1 1 1x3 -1 -1 2 1Tableau 7.This pivot makes the original problem feasible, thus the algorithm ends. The found basi solution is x = (0, 0, 1, 1, 1, 0).Observe that both the algorithm and its re ursive subalgorithms make in reasing pivotsto the orresponding (sub)problem and use re ursion. Thus, it is possible to generalize the omplexity bound of the non-degenerate and the weakly degenerate ase; however, be auseof the re ursion, the implied bound greatly depends on the number of degenerate subproblem alls.Although the analysis presented in Se tion 2.3.4. an be arried out for the �rst phaseof the simplex algorithm, the presented anti- y ling re ursion pro edure annot be naturallyapplied to the simplex algorithm. The main reason for this is, when a subproblem is feasible,we make use of the infeasibility of the driving variable to rea h the on lusion that the allingproblem is infeasible.As already stated, it would be possible to solve the degenerate subproblems with an ar-bitrary pivot method, similarly to the ase of the Hungarian method for linear programming[47℄, in whi h the riss- ross method was used [79℄.In pra ti e, the e� ien y of the algorithm ould be in reased by exploring the freedomon the hoi e of the variable to enter the basis. This freedom may help, espe ially in ase ofnumeri ally hallenging problems.2.7 An appli ation in the petroleum industry - blendingThe petroleum industry was among the �rst to employ linear optimization methods to iden-tify optimal produ tion plans and to maximize pro�t. This se tion deals with one of thesimplest versions of the feasibility problem behind the blending problems in petroleum pro-du tion, and on entrates on the feasibility problem itself. A more detailed introdu tion tothe optimization models and pra ti al hallenges fa ed in real life re�neries is presented inChapter 5.We use the so alled �ow model [5℄, where the main unknowns of the model are the �owvalues between the di�erent blends. 56

The wide range of possible saleable �nal produ ts of a petroleum re�nery is de�ned bystandards. These standards usually de�ne stri t bounds on the a eptable levels of several hemi al and physi al properties of the end produ ts.In an idealized environment, the blending problem is the following. There is a given setof input materials with known properties. The task is to blend these materials together inorder to produ e new materials that satisfy the standards. There may be several di�erentblends, often referred to as pools in the pro ess. The fundamental simpli� ation omparedto real life pra ti e is that now we suppose that we exa tly know the di�erent qualities ofthe inner pools (i.e. not the �nal produ ts), while usually these values are only guessed inadvan e, and an iterative method is used to �ne-tune them. This means that instead ofhaving both the �nal qualities and quantities as variables in the model, the qualities of theinner pools are now given in advan e. The general problem is dealt with in Chapter 5.2.7.1 Properties of the blendsWe start the des ription of the �ow model with the most spe i� aspe t of blending: thequalities of the blends.Suppose, that n input liquid materials Ii, i = 1, . . . , n are blended together into a single�nal produ t F . The l qualities of interest are Qj, j = 1, . . . , l. The amount of the inputmaterials blended into the �nal produ t is xi, i = 1, . . . , n, usually given in volume. Letthe volume of the �nal produ t be denoted by xF . The density (sometimes referred to asSPG = spe i� gravity) of the materials is a distinguished quality parameter, given by ρifor the input materials and ρF for the �nal produ t. Furthermore, suppose that the qualityrequirements refer to the relative amount of some spe i� materials in the blends, and thuslet the quality Qj of the input material Ii be given in density by qji. We suppose that ea hquality blends linearly.First, let us suppose that an inner produ t is blended, for whi h the quality parametersshould equal pj for quality Qj, j = 1, . . . , l. These pj values are usually not exa t values,and are guessed in advan e. The volumetri balan e for the blend is given byxF − (x1 + · · · + xn) = 0. (volumetri balan e)In a general model, ea h su h balan e equation may have two extra variables, one for theopening, and one for the losing inventory. The absolute amount of a spe i� materialrequired by quality Qj in the blend is given by pjxF . The sum of this spe i� materialtransferred into the blend by the di�erent input materials is qj1x1 + · · ·+ qjnxn. Sin e the pjvalues are usually not exa t, it is allowed for the blend to ontain an extra amount or la k57

of the spe i� material. These di�eren es are handled with the help of the so alled errorve tors. For ea h inner blend and quality, a separate error variable RjF is de�ned. If theinner blend is blended only from su h input materials that are not blends themselves (thushave no asso iated error ve tors), the orresponding equation is−qj1x1 − · · · − qjnxn + pjxF + RjF = I. (re ursed rows)In this des ription, we assume that I = 0. In a general model, this value equals the di�eren eof the opening and losing inventory multiplied with the quality guess. These equations willbe referred to as re ursed rows, and will be dis ussed in more detail in Chapter 5.Now suppose, that a �nal, saleable produ t is blended. There are both input materials

Ii, i = 1, . . . , n, and already blended inputs in the mixture. The quality (guess) of theseinner blends is given by pji for quality Qj and blend i, i = n + 1, . . . , s, where s is thesum of the number of input materials and inner blends. The orresponding error ve torsare Rji. Typi ally, the standards de�ne stri t upper and lower bounds on the quantity ofgiven hemi al omponents of the blends. Let the lower and upper bound given in densityfor quality Qj be denoted as StdNj and StdXj, a ording to the proper standard. Let theamount of the �nal blend again be denoted by xF , and the amount of input materials addedto the blend by xi, i = 1, . . . , s. Then the allowed relative amount of the hemi al omponentfor Qj falls between StdNjxF and StdXjxF .The role of the error ve tors needs some further explanation. The values in the errorve tor ( onsisting of several error variables for the di�erent qualities) of a blend are dis-tributed among the blends, into whi h the blend is further mixed. The ratio, in whi h theerror ve tor is distributed, is the same as the volumetri ratio in whi h the inner blend isblended into the further blends. Naturally, the values in these ratios, (i.e. the amount ofmaterials blended together) are unknowns of the model. These values are determined in aniterative pro ess. For now, let us suppose that these �ow values are known. The details ofthe iterative pro ess will be given in Chapter 5. Let the ratios in question be denoted byDi, i = n + 1, . . . , s. Thus, the orresponding lower and upper inequality for quality Qj ofthe �nal produ ts are given by

StdNjxF −n∑

i=1

qjixi −s∑

i=n+1

pjixi −s∑

i=n+1

DiRji ≤ 0, (lower bound)StdXjxF −

n∑

i=1

qjixi −s∑

i=n+1

pjixi −s∑

i=n+1

DiRji ≥ 0. (upper bound)58

2.7.2 General onstraints of the blending modelA typi al �ow-based blending model grabs several general modelling and te hni al aspe ts.We present some typi al onstraint types in luded in these models like: the use of availableraw materials, apa ity restri tions (of blending units) and minimal or maximal produ tionrequirements.Let the available (pur hased) amount of the input materials be given by Purci for materialIi, i = 1, . . . , k. Suppose that input material Ii is blended into blends Bj, j = 1, . . . ,m. Letthe amount of these materials (�ow values) be denoted by xij. Usually, the amount ofpur hases is de�ned in weight. Sin e the �ow variables are usually in volume, the variableshave to be s aled by their (known) spe i� gravity. The following inequalities state, that theusage should not ex eed the available amounts.

−Purci + ρ1xi1 + · · · + ρmxim ≤ 0. (weight balan e)Usually, the models also in lude opening and losing sto k values and requirements. These onstraints are formulated in the straightforward way.Capa ity onstraints are often formulated with range variables. Suppose that �ow xi,i = 1, . . . , n use a given blender with apa ity Cap. Then the apa ity onstraint may bemodelled as

x1 + · · · + xn − RgBCap = Cap, (blender apa ity)−Cap ≤ RgBCap ≤ 0. (range variable)Usually, the size of the error ve tors used in re ursed rows is bounded both from belowand upper. These bounds are usually tightened in the iterative pro ess presented in Chapter5. A typi al model also in ludes onstraints on the usage of given utilities like steam pres-sure, etanol additive, et .2.7.3 Example of a simpli�ed blendingWe are ready to formulate a small produ tion model of an imaginary re�nery. Although thesize and stru ture is greatly simpli�ed and arti� ial, it grabs the main aspe ts of a typi almodel. Also, the model is big enough to be numeri ally hallenging.The input data for this model was provided by MOL [68℄. The model and the orre-sponding linear programming problem was generated with Aspen XPIMS [67℄.In this example, eleven raw materials, abbreviated by IBU , NBU , IC5, I6C, C9A,

CCE, ALK, MTB, ETA, REF and ETB are blended together. The qualities of interest59

are abbreviated with SPG, SPM , RV I, RV 8, RON , MON , 070, 100, 150, 205, OX1,BIB, ET%, BUT , BNV , ARV , OLV . The explanation of the abbreviations is presentedin Table 2.1 Input materialsabbr. textIBU Iso ButaneNBU Normal ButaneIC5 Iso PentaneI6C Iso HexaneC9A AromatolCCE Etered f naphtaALK AlkylateMTB MTBE DFETA Ethanol DFREF ReformateETB ETBE DFR4A Blended ReformateFinal produ tsabbr. textUNH ESZ-95 (1020) DRUNB ESZ-95 (1020) DR BIO

Properties of interestabbr. textSPG Spe i� gravitySPM Sulfur, ppmRVI Reid Vapour Pressure IndexRV8 Reid Vap ethanol addRON Resear h ONMON Motor ON070 % O� at 070 elsius100 % O� at 100 elsius150 % O� at 150 elsius205 % O� at 205 elsiusOX1 Oxygen Content, wt%BIB Ether vol%-FAME in dieselET% Ethanol ontBUT Butane, vol%BNV Benzene, vol%ARV Aromati s, vol%OLV Ole�ns, vol%Table 2.1: Some explanation of the abbreviations.Two �nal produ ts, UNH and UNB are blended, with one inner blend R4A. The �owstru ture of the model is presented in Figure 2.16.The quality parameters of the eleven input materials are given in Table 2.2.Only a few aspe ts of the model is still missing. In order to provide a full des ription,we now provide the missing details. There is only one material sto kpiled, the blendedreformulate R4A, with an opening inventory of 5000 and a required losing inventory of2000. We mention that some properties have to be transformed in the proper onstraints,like for OX1 and RV I. For simpli ity, in this example, the quality guess of the inner blendblended reformulate R4A equals the known quality parameters of the reformulate REF .The quality requirements for the �nal produ ts UNH and UNB de�ned by the standards,and the blending apa ities are presented in Table 2.3. Where there are no values presented,60

Figure 2.16: The �ow stru ture of the example model.the standard de�nes no bounds. The GB1 in the blender table is the gasoline blender forthe �nal produ ts UNH and UNB.Now we are ready to formulate the orresponding linear feasibility problem. To ensureprodu tion at all in the feasibility problem (sin e no obje tive fun tion is introdu ed at thisstage), a lower bound is pla ed on some �ow variables.The size properties of the linear feasibility problem orresponding to this example arepresented in Table 2.4.The orresponding linear feasibility problem is relatively small. The density of the prob-lem is 9.04%, that ounts as rather high. However, for larger, real life instan es, the typ-i al sparsity is mu h smaller, for rolling plans for several planning periods, sparsity valuessmaller than 0.01% are not rare. The two neutral rows are the minimization and maximiza-tion versions of the obje tive fun tion (that is onstant zero for the feasibility version of theproblem).2.7.4 Numeri al aspe ts of the blending exampleWe brie�y demonstrate the MBU simplex algorithm with minimal index sele tion rule onthe blending problem presented above. 61

IBU NBU IC5 I6C C9A CCE ALK MTB ETA REF ETBSPG 0.56 0.58 0.62 0.67 0.88 0.75 0.70 0.75 0.79 0.86 0.74SPM 0 0 0.40 1.60 0 6.10 0.50 12 20 0.10 14RVI 1749.81 1019.50 472.99 153.19 5.66 123.97 188.12 149.78 0 64.41 177.49RV8 1788.56 1058.25 511.74 191.94 44.42 162.72 226.87 188.53 38.75 103.16 216.24RON 99 93.80 91 81 110 93.30 94.51 110 120 102.76 107MON 97.60 89.60 86 82 90 82.95 92.49 98 97 91.30 99070 100 100 100 100 0 23.03 6.33 100 207 1.90 80100 100 100 110 105 0 45.58 31.11 100 112 11 100150 100 100 100 100 50 82.75 91.13 100 100 71 100205 100 100 100 100 100 100 100 100 100 100 100OX1 0 0 0 0 0 0.75 0 18 34.78 0 15BIB 0 0 0 0 0 6.12 0 98 0 0 95ETBUT 100 99 0.20 0 0 0.27 0.04 0 0 0.90 1BNV 0 0 0.07 0 0 0.96 0 0 0 0.10 0ARV 0 0 0.07 0 99.07 32.94 0.07 0 0 84.46 0OLV 0 0 0 0 0 15.97 0 0 0 2.04 0Table 2.2: The quality parameters of the input materials.On the implementationThe ode was implemented in Matlab 7.0.1. Using a full short pivot tableau proved to benumeri ally far too instable, thus a revised simplex method based approa h was applied.In the revised simplex method, instead of the pivot tableau or the inverse of the basis, ade omposition of the basis matrix is al ulated, and updated a ording to the pivot sele -tion. Ea h time the transformed right-hand side, redu ed osts or a transformed row or olumn of the pivot tableau are required, a linear equation system is solved. The solutionof these systems is numeri ally stable and very e� ient using the de omposed basis. Theimplementation used QR de omposition and the qrupdate fun tion of Matlab. To in reasenumeri al stability, the basis was regularly reinverted (i.e. the QR de omposition reatedfrom s rat h). For a detailed des ription on the revised simplex method, see [57℄.It must be emphasized that this implementation was purely meant as a demonstrationof the algorithm. For example, in a state of the art implementation, the problem is typi allyleft in it original form, and the ode is generalized to handle any problem in any form.Meanwhile, in this example, the feasibility problem was transformed to the anoni al formof Ax = b, x ≥ 0. Although this transformation further in reases the ondition number ofthe matrix, for this form, the standard des ription of the algorithm may be used.62

UNH UNBlower upper lower upperSPG 0.7200 0.7800 0.7200 0.7800SPM 10 10RVI 145.54 229.95 145.54RV8 268.6978718RON 95.0 95.0MON 85.0 85.0070 20.0 48.00 20.0 48.00100 48.0 48.0150 75.0 75.0205 100.0 100.0OX1 2.7 2.7BIB 0.0 15.0 0.0 15.0ETBUT 5.000 5.000BNV 0.0 1.0 0.0 1.0ARV 30.0 30.0OLV 18.0 18.0

Blenderfor material Capa ityGB1 2628.0R4A 1681.9CCE 1681.9ALK 841.0MTB 473.0ETB 473.0I6C 788.4IC5 367.9IBU 131.4NBU 131.4C9A 131.4ETA 131.4Table 2.3: Quality requirements (standards) and blending apa ities.Number of rows: 96 Bounds:of equal type: 25 lower: 20less or equal: 45 upper: 17greater or eaual: 24 free: 1neutral: 2 �xed: 1Number of variables: 68 Sparsity: 9.04%Number of nonzeros: 584 RHS: 58.62% of entries are zeroTable 2.4: Size of the linear feasibility problem orresponding to the blending example,generated by XPIMS.Numeri al experien esThe matrix A of the anoni al form of the blending problem presented above had 116 rowsand 174 olumns. To he k the transformation and the results, both the original form ofthe problem and the transformed version were solved with Cplex and the linprog fun tion63

of Matlab. Both softwares solved the problem without any problem3.The ondition number of matrix A is 6.148599 ∗ 105, and has a full rank of 116. To get astarting basis, the rref fun tion of Matlab was used, whi h al ulates a redu ed row e helonform of the given matrix, and gives ba k the indi es of a possible basis as well. With thedefault parameters, this fun tion returned a basis for whi h the rank fun tion reported arank of 115, and its ondition number was 8.084732 ∗ 1018. To over ome this problem, wetried to in lude the logi al arti� ial variables introdu ed to the problem while transformingit to the anoni al form. To do this, we simply reversed the order of olumns in matrix A,thus the arti� ial olumns be ame the �rst ones. After doing this, the basis returned byrref had full rank with a ondition number of �only� 3.414651 ∗ 1012. Although still quitehigh, starting from this basis, the algorithm had no numeri al problems in al ulating theQR de ompositions of the o ured basis during the MBU simplex algorithm. Moreover, thisbasis still had many stru tural variables in it, thus served as a promising initial basis4.Simplex based algorithms apable of solving large s ale problems rely on several, oftendynami ally hosen ǫ values, used to numeri ally distinguish between zero and nonzero. Thisimplementation used 4 di�erent ǫ values: ǫrow and ǫcolumn for values in the pivot tableau,ǫRHS for the transformed right-hand side, and ǫratio used to al ulate when two ratio testare onsidered the same. The relationship between these values were ǫrow ≃ ǫcolumn, ǫRHS ≃

ǫratio, with ǫcolumn >> ǫRHS. In these experiments, too small ǫrow and ǫcolumn values oftenled to long running times, sometimes even to y ling5.The fundamental idea of the MBU simplex algorithm is to keep the feasible variablesfeasible. To do this, a primal ratio test was used. A ording to our experiments, it is hardto identify nonzero values using a traditional ǫ value in ea h iteration (too sensitive to the hoi e of ǫ). To stabilize the algorithm, a logi al array was used, and on e a variable be amebigger than a given −ǫ value, it was always onsidered as nonnegative, regardless its value ompared to the given ǫ value.The next driving variable was always sele ted to be the most infeasible one. This strategyis the analog of the lassi al pri ing rule of Dantzig for linear programming problems.3Although a bug was found in our version of Matlab, namely it ignored the lower bounds on the variablesgiven as a parameter, so the nonnegativity bounds were given instead as onstraints.4In sophisti ated systems, the initial basis generated by the so alled rash-heuristi s often tries to balan ebetween in luding as many stru tural variables in the starting basis as possible, while preserving numeri alstability, sin e numeri al experien e suggests that su h starting bases are usually more e� ient[57℄.5Despite the use of index sele tion rules. 64

The MBU simplex algorithm solved the problem in 165iterations. It must be noted, that when this value is ompared to the �rst phase iteration number of the las-si al simplex method and the MBU simplex method forlinear programming problems, the 116 pivot operationsmade by the rref fun tion should also be added (i.e.the sele tion of the �rst basis). 0 20 40 60 80 100 120 140 160 1800

5

10

15

20

25

30

35

40

Iteration number

Num

ber

of n

egat

ive

varia

bles

Number of infeasible variables.2.8 SummaryWe have presented a �nite MBU type simplex algorithm for linear feasibility problems,as well the possible use of s-monotone pivot rules [11℄. A new re ursive anti-degenera ypro edure [10℄ has been introdu ed and dis ussed. A weaker on ept of degenera y hasbeen de�ned. Based on the assumption that the problem is not strongly degenerate, wehave shown that the algorithm solves the feasibility problem in at most mK pivot steps,where K is a onstant determined by the input data of the problem and m is the number of onstraints [10℄. The onstant K annot be bounded in general by a polynomial of the bitlength of the input data. Similar omplexity analysis an be made for the �rst phase of theprimal simplex algorithm, and the original MBU simplex algorithm under the assumptionthat the tableaux visited are not strongly degenerate, as will be shown in Chapter 3. Thepresented new, re ursive anti-degenera y pro edure uses the spe ial sign stru ture of ourMBU type simplex algorithm; therefore, it is not dire tly appli able to the simplex method:in the primal simplex method, the right-hand side is always nonnegative, while the presentedanti-degenera y pro edure uses the negativity of the driving variable when returning from afeasible dual-side re ursion.The re ursive anti-degenera y rule di�ers from the lassi al, well-known methods in theliterature. The omplexity analysis of the algorithm works for weakly degenerate pivotsequen es. It would be interesting to �nd a perturbation that makes the problem nondegen-erate and �ts well into the presented omplexity analysis. We have shown that unfortunately,the lassi al perturbation methods like the ǫ-perturbation [61℄ riti ally worsen the omplex-ity bound presented.To demonstrate the algorithm, as well as to present some small numeri al examples, ablending problem of the petroleum industry has been onsidered. It is shown, that a simpleimplementation of the MBU simplex algorithm is apable of solving small instan es of theproblem. 65

Chapter 3Linear programmingIn this hapter, we extend our study to linear programming problems. The linear pro-gramming problem (LP) onsists in the minimization or maximization of a linear obje tivefun tion over the feasible set of a feasibility problem. The standard form of an LP ismin cTx

Ax = b (3.1)x ≥ 0where A ∈ Rm×n, c ∈ Rn and b ∈ Rm.The linear programming problem is the most widely studied and applied area of linearoptimization. The form given in (3.1) is the most appropriate for easy formulation of thealgorithms presented in the hapter. However, in pra ti e, LP an be presented in a moregeneral form, in luding lower and upper bounds both on the variables and on the onstraints.In spite of the several known pivot algorithms and their variants developed for solvingthe linear programming problem, the vast amount of numeri al experien e and �ne-tuningof the simplex methods make the simplex algorithm the most ompetitive pra ti al hoi eof pivot algorithms.In this hapter, we formulate the primal simplex method and the original monotoni build-up simplex algorithm of Anstrei her and Terlaky [4℄. The �niteness of both algorithmsis presented using the new on ept of s-monotone pivot rules.The obje tive fun tion is often regarded as a new variable and onstraint of form

z = cTxwith variable z, that is always in the basis. Due to this presentation of the LP problem, theorthogonality Theorem 2.1.2 may be generalized in a straightforward way to linear program-ming problems and to the row of the obje tive fun tion.66

Often, the values of the transformed obje tive fun tion (i.e. the z row of the pivottableau) are referred to as redu ed osts.3.1 Duality theorem of linear programmingEvery linear program of form (3.1) has its dual pair, de�ned asmaxyTb (3.2)ATy ≤ c.Let us introdu e the so alled primal feasible set and the dual feasible set as

P = {x | Ax = b, x ≥ 0},

D = {ATy ≤ c}.The set of optimal solutions is de�ned asP∗ = {x∗ ∈ P | cTx∗ ≤ cTx ∀ x ∈ P},

D∗ = {y∗ ∈ D | y∗Tb ≥ yTb ∀ y ∈ D}.A solution of the dual problem may serve as a erti� ate that the primal problem isbounded, while an optimal solution of the dual problem is a erti� ate that the primalproblem also has an optimal solution.Theorem 3.1.1 (Weak duality) For any feasible solution x ∈ P and dual feasible solutiony ∈ D

cTx ≥ yTbholds, and with equality if and only if the so alled optimality ondition of omplementarysla kness holds:x · s = 0, where s = c − ATy ≥ 0.Proof.

yTb = yT Ax ≤ cTx,sin e x ≥ 0 and ATy ≤ c. To prove the optimality ondition, observe thatcTx − yTb = cTx − xT ATy = cTx − yT Ax =

(cT − yT A

)x = sTx.

�Sometimes, a natural orollary of the weak duality theorem is stated as the weak equilib-rium theorem. 67

Theorem 3.1.2 (Weak equilibrium) If a feasible solution x ∈ P and a dual feasiblesolution y ∈ D is su h that cTx = yTb, then x ∈ P∗ and x ∈ D∗.The strong duality theorem providing the hara terization of optimal solutions of a lin-ear programming problem is a powerful tool often exploited in algorithms for large s aleproblems.Theorem 3.1.3 (Strong duality) For any primal-dual linear programming system, at leastone of the following holds.1. P = ∅.2. D = ∅.3. if P 6= ∅ and D 6= ∅ then P∗ 6= ∅ and D∗ 6= ∅, and for any pair x∗ ∈ P∗ and y∗ ∈ D∗

cTx∗ = y∗Tb.Problem (3.1) has an optimal solution if and only if problem (3.2) has. In su h ases,the optimal obje tive fun tion values are equal.Proof. The proof is based on the �niteness of the simplex algorithm, thus will be ompletedby Theorem 3.2.1. �The possible o urring ases are summarized in Table 3.1.Primal problemoptimal unbounded infeasibleoptimal X × ×Dual problem unbounded × × Xinfeasible × X XTable 3.1: The possible primal dual solution status ases, where X denotes the possible ases.Intuitively, the duality theorem states that the minimum of the obje tive fun tion equalsthe stri test dire t bound dedu ible from the restri ting onstraints. This may espe iallybe well illustrated by the following, equivalent form of the linear programming problempresented in Figure 3.1. 68

max cTx

Ax ≤ b

minyTb

yT A = c

y ≥ 0Figure 3.1: An intuitive primal-dual LP pair.3.1.1 The equivalen e of the linear feasibility problem with thelinear programming problem pairIt is worth mentioning that any linear programming problem may be transformed to anequivalent feasibility problem with the help of the duality theorem.It is easy to see that by the strong duality theorem, �nding an optimal solution for bothproblems (3.1) and (3.2) is equivalent with the solution of the following system.Ax = b, x ≥ 0,

ATy + s = c, s ≥ 0,

cTx − bTy = 0.This well-known result is often used to onvert linear programming problems to equiv-alent linear feasibility problems. It is important to note however, that this equivalen y istheoreti al and usually quite unpra ti al. In pra ti e, it is usually more e� ient to generalizethe algorithms in su h a way that it should be able to handle ea h problem exa tly in thesame form as it arises in the appli ations [57℄.3.2 The simplex methodIn this se tion, we onsider the famous simplex method of Dantzig [21℄.A spe ial property of the simplex algorithm is that it requires a feasible basis to startfrom. Thus, the algorithm is lassi ally divided into two phases. In the �rst phase, usingan embedding te hnique presented in Subse tion 3.2.2, a feasible basis is identi�ed. In these ond phase, while generating only feasible solutions, the obje tive fun tion is optimized.The simplex algorithm also has a monotone feature. If the basis sequen e generatedby the algorithm is primal nondegenerate (weak degenera y is su� ient), then in the �rstphase, in ea h iteration, the sum of infeasibilities is de reased, while in the se ond phase,the value of the obje tive fun tion is in reased. We must also note, that often the feasibility69

of the generated basi solution is regarded as a strength of the method, sin e in the �rstphase a measure of infeasibility, while in the se ond phase a feasible solution is available ifthe algorithm is terminated for any reason.The pseudo- ode of the algorithm is presented in Figure 3.2, while the �ow hart of thealgorithm is shown in Figure 3.3.The primal simplex algorithm with s-monotone pivot rulesInput: A ∈ Rm×n,b ∈ Rm, c ∈ Rn a feasible basis B, initialized s ve tor.Output: An optimal solution, or a erti� ate that the solution is unbounded.BeginI−

N := {i ∈ IN |ci < 0}.While I−N 6= ∅ doLet q ∈ {i ∈ I−

N |ci < 0} be arbitrary with maximal value respe t to s.If tq ≤ 0 thenStop: problem is unbounded, erti� ate is tq.ElseLet ϑ := min{

bi

tiq| i ∈ IB, tiq > 0

} be the value of the primal ratio test.Let p ∈ IB arbitrary, su h that bp

tpq= ϑ and with maximal value respe t to s.EndifPivot on (p, q).EndwhileThe solution is optimal.End.Figure 3.2: The primal simplex algorithm with s-monotone pivot rules.One of the most hara teristi features of the simplex algorithm is the so alled ratio testwhi h ensures that the next basis remains feasible.Proposition 3.2.1 Given a feasible starting basis, the primal simplex algorithm generatesa feasible basis sequen e.Proof. The proof is by indu tion. Let a feasible basis of the sequen e be given, and let thenext position of the pivot be given by (p, q). The proposition is straightforward for index p,thus let an index i 6= p be given. The new right-hand side b′ is determined by

b′i = bi − bp

tiqtpq

.70

Problem is unbounded,noϑ := min

{bi

tiq: i ∈ IB, tiq > 0

}

tq ≤ 0

q ∈ {i ∈ I−N |ci < 0}

erti� ate is tq.p ∈ IBPivot on (p, q).: bp

tpq= ϑ

loop:START

STOPnoI−

N 6= ∅

The solutionis optimal.loop

STOPyes yes

Figure 3.3: Flow hart of the primal simplex algorithm with s-monotone pivot rules.If tiq = 0 then b′i = bi ≥ 0, while otherwiseb′i = tiq

(bi

tiq−

bp

tpq

)≥ 0,be ause of the ratio test determining p. �Note that in the primal nondegenerate ase, �niteness immediately follows from the fa tthat any nondegenerate pivot stri tly improves the obje tive fun tion value.3.2.1 Finiteness of the simplex algorithmThe proof of �niteness of the algorithm is based on ontradi tion. Sin e the number of basesis �nite, the algorithm fails to terminate if and only if it visits some bases in�nitely manytimes. Let us onsider su h a minimal example. Be ause of minimality and the pivot ruleof the algorithm, in su h an example ea h variable moves in�nitely many times.First, we prove that in su h an example ea h variable is at a zero level in every possiblebasis.Lemma 3.2.1 A minimal y ling example has a onstant zero right-hand side.Proof. Observe that a pivot on a nondegenerate basi variable would in rease the obje tivefun tion. Sin e a pivot in a nondegenerate row doesn't make the row degenerate and a pivoton a degenerate row doesn't hange the right-hand side, the right-hand side ve tor mustequal onstant zero, or otherwise the obje tive fun tion would have to in rease in�nitelymany times, ontradi ting that the number of possible bases is �nite. �71

A onstant zero right-hand side ve tor greatly simpli�es the ratio test, sin e ea h ratiois the same. This way the hoi e of the leaving variable is left solely to the index sele tionrule.Consider now the situation des ribed in the se ond riterion of s-monotoni ity. Considerbasis B′ of the minimal y ling example, when the variable xl with the lowest s value entersthe basis. A ording to the olumn sele tion rule of the simplex algorithm, the obje tivefun tion row of the pivot tableau for basis B′ has a negative entry for the nonbasi variablexl and nonnegative entries for all other nonbasi variables.In a minimal y ling example, sin e every variable moves in�nitely many times, variablexl must leave the basis. Consider basis B′′ when xl leaves the basis for the �rst time afterB′. By Lemma 3.2.1, the hoi e of the leaving variable is sele ted from those basi variablesthat have a positive value in the olumn of the leaving variable and have a maximal valuea ording to ve tor s.The pivot tableaux (the so alled almost terminal pivot tableaux) for bases B′ and B′′ ispresented in Figure 3.4.

xl

+ 0

0...0

0...0

− ⊕ . . . ⊕

xl enters the basis in basis B′.

xk

xl + 0

K

+...+

0...0

L

⊖...⊖

0...0

−

xl leaves the basis B′′.Figure 3.4: Almost terminal pivot tableaux for the simplex algorithm.We are ready to prove the �niteness of the simplex algorithm with s-monotone indexsele tion rules.Theorem 3.2.1 The simplex algorithm with s-monotone index sele tion rule is �nite.Proof. Let us assume the ontrary, and onsider a minimal y ling example, with enteringvariable xl in basis B′ des ribed in the se ond riterion, and basis B′′ with entering variable72

xk, when variable xl leaves the basis for the �rst time after B′, as des ribed in the third riterion of s-monotone index sele tion rules.Consider ve tor t′(c) for basis B′ and ve tor t′′k for basis B′′. LetK = {i ∈ IB′′ | t′′ik > 0} \ {l}, and L = {j ∈ IB′′ | t′′jk ≤ 0}.Then

t(c)′T t′′k =∑

i∈K

t′cit′′ik +

∑

j∈L

t′cjt′′jk + t′clt

′′lk ≤ t′clt

′′lk,using that t′cj ≥ 0 and t′′jk ≤ 0 for all j ∈ L, and t′ci = 0 for all i ∈ K be ause by the �rst riterion, the values of s may only in rease, and those variables that have moved sin e basis

B′ have a greater value in s than variable xl. By the third riterion of s-monotone indexsele tion rules, these variables have not moved sin e basis B′, thus have a orresponding zerovalue in t′(c). Sin e t′cl < 0 and t′′lk > 0, we have t(c)′T t′′k < 0, ontradi ting the orthogonalitytheorem. �Remark 3.2.1 We have simultaneously proved �niteness for the index sele tion rules MIR,LIFO and MOSV.Remark 3.2.2 Sin e in an optimal tableau the obje tive fun tion row of the pivot tableaualso de�nes an optimal dual solution, we have also ompleted the proof of the duality Theorem3.1.3.3.2.2 Finding a feasible basis for the simplex algorithmThe identi� ation of a feasible starting basis is usually done with the use of an embeddingte hnique. Let the anoni al form (3.1) be given, and assume that b ≥ 0. The assumptionon b may be assured by simple multipli ation. Consider the following embedded problem,where 1 = (1, . . . , 1) is of proper dimension.max(1T A)x

Ax + u∗ = b (3.3)(x,u∗) ≥ 0.For this problem, B = I is a feasible basis, sin e b is nonnegative.Theorem 3.2.2 The linear programming problem (3.1) has a feasible solution if and onlyif the existing optimal value for the embedded linear programming problem (3.3) equals 1Tb.73

Proof. It is easy to see that problem (3.3) is bounded, sin e Ax + u∗ = b yields Ax ≤ busing the nonnegativity of u∗, and thus (1T A)x ≤ 1Tb. Therefore, problem (3.3) always hasan optimal solution. Furthermore, any feasible solution to (3.1) orresponds to a solution of(3.3) (and vi e versa) with u∗ = 0, and an obje tive fun tion value of 1Tb. �Note, that although it would be straightforward to minimize the sum of the so alledarti� ial variables u∗. A drawba k of this approa h would be that the orresponding tableauwouldn't trivially in lude an initial feasible basis, sin e the row of the obje tive fun tionwould has to be transformed �rst. After the transformation, one gets ba k the obje tivefun tion −1T A, whi h is the maximization equivalent of 1T A.Example 3.2.1 Let us onsider the following problem.min−x1 − 3x2 + x3

2x1 + 2x3 ≤ 110

−2x1 + 2x2 = 80

x2 + 2x3 ≥ 40

x1, x2, x3 ≥ 0Sin e the problem has no trivial feasible starting basis, we have to introdu e arti� ial sla kvariables, and the se ondary obje tive fun tion. There is a sla k and a surplus variable(u1, u2) and two arti� ial sla k variables (v∗1, v

∗2) introdu ed.

( min−x1 − 3x2 + x3 )

max−x1 − 2x2 − 2x3 + u2

2x1 + 2x3 + u1 = 110

−2x1 + 2x2 + v∗1 = 80

x2 + 2x3 − u2 + v∗2 = 40

x1, x2, x3, u1, u2, v∗1, v

∗2 ≥ 0In this transformed problem, the olumns of variables (u1, v

∗1, v

∗2) form a trivial feasible basis.The short pivot tableau of the problem is given in Tableau 1. We solve the problem withthe simplex method using the MIR index sele tion rule. The row of the original obje tivefun tion is denoted by z, while the row of the arti� ial obje tive fun tion with z∗. While thearti� ial obje tive fun tion is the a tive one, the original is drawn in grey. In the row of theoriginal obje tive fun tion, in the olumn of the right-hand side ve tor there is the negative ofthe value of the obje tive fun tion for the urrent basi solution. For the arti� ial obje tivevalue, this value is usually initialized as −1Tb=, where b= denotes the part of the right-handside ve tor that belongs to rows in whi h an arti� ial sla k variable was introdu ed. Thus,this value serves as an infeasibility measure for the urrent basi solution.74

x1 x2 x3 u2

u1 2 0 2 0 110v∗

1 -2 2 0 0 80v∗

2 1 0 2 -1 40z -1 -3 1 0 0

z∗ 1 -2 -2 1 -120Tableau 1.x1 v∗

1 x3 u2

u1 2 0 2 0 110x2 -1 0.5 0 0 40v∗

2 1 0 2 -1 40z -4 1.5 1 0 120

z∗ -1 1 -2 1 -40Tableau 2.By sele ting the pivot olumns a ording to the MIR index sele tion rule and the pivotrow by the ratio test (whi h is unique in this problem) in two pivots both arti� ial variablesleave the basis, and the pivot tableau is optimal in respe t to the arti� ial obje tive fun tion.Sin e the optimal arti� ial obje tive value equals the required 120, the algorithm has nowidenti�ed a feasible solution of the problem.v∗

2 v∗1 x3 u2

u1 -2 0 -2 2 30x2 1 0.5 2 -1 80x1 1 0 2 -1 40z 4 1.5 9 -4 280

z∗ 1 1 0 0 0Tableau 3.x3 u2

u1 -2 2 30x2 2 -1 80x1 2 -1 40z 9 -4 280Tableau 4.It is onvenient to delete the olumns of the arti� ial variables and the row of the arti� ialobje tive fun tion, sin e none of these has any further rule. The solution is ontinued fromTableau 4. with the same algorithm.

x3 u2

u1 -2 2 30x2 2 -1 80x1 2 -1 40z 9 -4 280Tableau 5.

x3 u1

u2 -1 0.5 15x2 1 0.5 95x1 1 0.5 55z 5 2 340Tableau 6.Tableau 6. is optimal. The found optimal solution is x = (55, 95, 0).3.2.3 Complexity bound under the weak degenera y assumptionWe show that the new omplexity bound presented in Chapter 2 may easily be generalizedfor the simplex method. 75

First observe that for weakly degenerate tableaux, the ratio test sele ts a nondegeneratepivot row. Thus, the nonzero hange in the obje tive fun tion value (i.e. the negative of theright-bottom orner of the pivot tableau) is given by|znew − zold| = cq

bp

tpg

, (3.4)where tpg > 0 is the pivot element, cq > 0 is the redu ed ost of the sele ted variable to enterthe basis and bp > 0 is the right-hand side in the row of the variable leaving the basis.The minimum of the in rease in the obje tive fun tion automati ally yields a boundfor the �rst phase of the algorithm, where the value of the obje tive fun tion is boundedboth from below (0) and above (1Tb=). For the se ond phase of the algorithm, an upperbound on the obje tive fun tion is ne essary. This may be de�ned analogously as was inthe ase of ∆max, after observing that the redu ed ost in the short pivot tableau de�ned byc = cN − cT

BB−1N whi h is the solution of Bx = c, whereB =

[B 0

cB 1

], with the inverse of [ B−1 0

−cTBB−1 1

].3.3 The monotoni build-up simplex method for linearprogramming problemsThe monotoni build-up simplex algorithm was introdu ed in [4℄. In Chapter 2, we havepresented its simpli� ation to feasibility problems and proved its �niteness with s-monotoneindex sele tion rules, as well as with a new, re ursive pro edure. In this se tion, we presentthe original algorithm for linear programming problems and prove its �niteness with s-monotone index sele tion rules.The monotoni build-up simplex algorithm starts from a feasible basis and sets the feasi-bility of the dual variables one by one, while maintaining the feasibility of the already feasibledual variables. Although primal feasibility may be violated in some bases generated by thealgorithm, primal feasibility is always restored when the sele ted driving variable be omesfeasible.This algorithm also requires a feasible basis to start from. To a hieve su h a basis, a �rstphase may be introdu ed similarly as was for the simplex method, or the feasibility versionof MBU presented in Chapter 2 may be used.The pseudo ode for the algorithm is presented in Figure 3.5, while the �ow hart isshown in Figure 3.6. 76

The MBU simplex algorithm with s-monotone pivot rulesInput: A ∈ Rm×n,b ∈ Rm, c ∈ Rn a feasible basis B.Output: An optimal solution, or a erti� ate that the solution is unbounded.BeginInitialize ve tor s.I−

N := {i ∈ IN |ci < 0}.While I−N 6= ∅ doLet the driving variable s ∈ {i ∈ IN |ci < 0} be arbitrary.While cs < 0 doLet Ks = {i ∈ IB|tis > 0}.If Ks = ∅ then Stop: problem is unbounded, erti� ate is ts.ElseLet ϑ := min

{bi

tis|i ∈ Ks

} the value of the primal ratio test.Let r ∈ Ks arbitrary su h that br

trs= ϑ and with maximal value respe t to s.Let θ1 := |cs|

trs, and let J = {i ∈ IN |ci ≥ 0, tri < 0}.If J = ∅ then

θ2 := ∞.Elseθ2 := min

{ci

|tri||i ∈ J

} the value of the dual ratio test.Let q ∈ J arbitrary su h that θ2 =cq

|trs|and with maximal value respe t to s.EndifIf θ1 ≤ θ2 thenPivot on (r, s), (driving pivot).ElsePivot on (r, q), (auxiliary pivot).EndifEndifEndwhileEndwhileThe solution is optimal.End Figure 3.5: The MBU simplex algorithm with s-monotone pivot rules.77

+s

nos

no

−

−

s

−

−

yes+

−

yes

Problem is unbounded.

−−

STOPyesnono -

s rPivot on (r, s)r + −-q

Pivot on (r, q)

s ∈ {i ∈ IN |ci < 0}

Ks = {i ∈ IB|tis > 0}

Ks = ∅

J = ∅ θ2 := ∞

θ2 := ci

|tri|

θ1 ≤ θ2

J = {i ∈ IN |ci ≥ 0, tri < 0}

...

loop:

cs < 0

...θ1 := |cs|

trsr ∈ Ks

⊖

...

q ∈ J ++

s yes

⊖

......

loopyesnoThe solution is optimal.

⊕

STARTI−

N 6= ∅

...STOP

⊕ ⊕. . .⊕

⊖ ⊕

⊕

Figure 3.6: Flow hart of the MBU simplex algorithm with s-monotone pivot rules.To establish the orre tness of the algorithm, we repeat the key theorems proved in [4℄.We will all a pivot in the olumn of the driving variable a driving pivot, while any otherpivot an auxiliary pivot.Theorem 3.3.1 ([4℄) Consider any pivot sequen e produ ed by the MBU algorithm orre-sponding to an initial feasible basis and the hoi e of a driving variable xs. Then followingan auxiliary pivot, the next basis produ ed by the algorithm has the following properties:1. cs < 0,2. if bi < 0, then tis < 0,3. max{

bi

tis| bi < 0

}≤ min

{bi

tis| tis > 0

}.78

Remark 3.3.1 The proof of Theorem 3.3.1 states that ondition 3 always holds for anysequen e of auxiliary pivots generated by the algorithm.Remark 3.3.2 Condition 2 of Theorem 3.3.1 assures that the problem is really unboundedwhenever the algorithm laims it, even if the urrent basis is infeasible.Based on the above stated theorem, we may state that the MBU algorithm is well-de�ned.Theorem 3.3.2 [4℄ Whenever the MBU algorithm performs a driving pivot, the next basisis feasible.Note that if the problem is both primal and dual nondegenerate, then �niteness is ensuredby the fa t that similarly to the simplex method, the obje tive fun tion stri tly in reases inea h iteration [4℄. However, weak primal nondegenera y is not su� ient for �niteness in this ase.3.3.1 Finiteness of the MBU simplex algorithmIn the original paper [4℄, lexi ography was used to ensure �niteness. In this se tion, we provethat the algorithm is �nite whenever s-monotone pivot rules are applied. First, we need toexamine some further properties of the algorithm.Lemma 3.3.1 Any pivot made in a dual nondegenerate olumn in reases the value of thedriving variable.Proof. A driving pivot makes the dual infeasible driving variable dual feasible, while anauxiliary pivot in reases the value of the driving variable without making it nonnegative.� The next lemma states a further monotone property of the MBU algorithm.Lemma 3.3.2 In any sequen e of auxiliary pivots generated by the algorithm for drivingvariable xr, the value max

{bi

tis| bi < 0

} never de reases.Proof. By Remark 3.3.1 the third ondition of Theorem 3.3.1 states thatmax

{bi

tis| bi < 0

}≤ min

{bi

tis| tis > 0

} (3.5)always holds. Observe, that by the primal ratio test arried out by the algorithm, for anauxiliary pivot made on position (r, q), the minimal ratio of min{

bi

tis| tis > 0

} is obtained.Sin e the auxiliary pivot is arried out on a negative pivot element, the new right-hand side79

value for index r be omes negative, while the ratio of the right-hand side and the pivotelement remain the same, thus in the next iteration this ratio o urs in the left-hand side of(3.5). �Before examining a possible y ling example, we need a te hni al-type lemma. Thislemma is new, and plays a fundamental role in the proof of �niteness with s-monotone pivotrules.Lemma 3.3.3 Let a, b, Θ ∈ R su h that b 6= 0 and ab

= Θ. Let c, d, λ ∈ R su h that d ·λ 6= 0and b + λd 6= 0. Then if a+λcb+λd

= Θ, then cd

= Θ.Proof.a + λc

b + λd= Θ

a + λc = Θ(b + λd)

Θ + λc

b= Θ

(1 + λ

d

b

)

c

b= Θ

d

bc

d= Θ.

�We are ready to prove that the MBU simplex algorithm with s-monotone pivot rules is�nite. Our proof is based on ontradi tion. Let us onsider a minimal example, for whi h thealgorithm is not �nite. As usual, sin e the number of possible bases is �nite, the algorithmmust visit the same basis in�nitely many times. It is lear, that be ause of minimality, insu h an example ea h variable moves in�nitely many times, with the possible ex eption ofone single variable, whi h may remain an infeasible driving variable throughout the whole y le.Lemma 3.3.4 For a minimal y ling example, the following properties hold:1. Any basis generated by the algorithm is dual degenerate for all variables ex ept onesingle variable. This variable remains the same throughout the algorithm and neverenters the basis.2. All variable moves in�nitely many times, ex ept one, whi h never enters the basis.3. No driving pivot is made. 80

4. The primal ratio test always yields the same value. Furthermore, in any basis generatedby the algorithm, for the olumn r of the driving variable, the ratio bi

tiris the same forall i, where tir 6= 0.Proof. By Lemma 3.3.1, a pivot made in a nondegenerate olumn stri tly in reases thevalue of the driving variable. Observe, that while a pivot in a nondegenerate olumn leavesthe olumn of the short pivot tableau nondegenerate, a degenerate pivot doesn't hange therow of the obje tive fun tion in the tableau.It is easy to see, that in a y ling example, there exists an infeasible driving variable

xr that never be omes feasible, thus on e this variable is sele ted for the role of a drivingvariable, only auxiliary pivots are made. Be ause of minimality, all other variables movein�nitely many times. However, by the observation made above, namely that ci = 0 for alli 6= r, so it follows that 1 holds.

2 and 3 follow immediately from 1.Sin e the driving variable never hanges, by Theorem 3.3.1 and Lemma 3.3.2, the valueof the ratio test be omes a onstant value after �nitely many iterations. By the te hni alLemma 3.3.3, it yields that the ratio must be the same for any basis generated by thealgorithm. �By Lemma 3.3.4, a minimal y ling example ontains a single infeasible dual variablesele ted as driving variable. Furthermore, both the primal and dual ratio tests are trivial,and the sele tion of indi es is solely based on the index sele tion rule. Let xr be the drivingvariable. Consider now variable xl with basis B′ as des ribed in the se ond riterion ofs-monotone index sele tion rules, and let B′′ be the basis when xl leaves the basis after B′for the �rst time. (Observe, that sin e the driving variable never enters the basis, l 6= r.)Using the observations stated in Lemma 3.3.4, the almost terminal pivot tableaux for basesB′ and B′′ have a sign stru ture as presented in Figure 3.7.We are ready to prove that the algorithm is �nite.Theorem 3.3.3 The MBU simplex algorithm with s-monotone index sele tion rule is �nite.Proof. Let us assume the ontrary, and onsider a minimal y ling example with enteringvariable xl and leaving variable xk in basis B′ des ribed in the se ond riterion of s-monotoneindex sele tion rules, and basis B′′ when variable xl leaves the basis for the �rst time afterB′. Consider ve tor t′(k) for basis B′ and ve tor t′′r for basis B′′. Let

K = {i ∈ IB′′ | t′′ir > 0} \ {l}, and L = {j ∈ IB′′ | t′′jr ≤ 0}.81

xr xl

∗...∗

xk + − ⊕ . . . ⊕ ∗

∗...∗

− 0 0 . . . 0

xl be omes basi in basis B′.

xr

xl + ∗

K

+...+

∗...∗

L

⊖...⊖

∗...∗

− 0 0

xl be omes nonbasi in basis B′′.Figure 3.7: Almost terminal pivot tableaux for the MBU simplex algorithm.Thent(k)′T t′′r =

∑

i∈K

t′kit′′ir +

∑

j∈L

t′kjt′′jr + t′krt

′′rr + t′klt

′′lr ≤ t′krt

′′rr + t′klt

′′lr,using that t′kj ≥ 0 and t′′jr ≤ 0 for all j ∈ L, and t′ki = 0 for all i ∈ K be ause by the �rst riterion, the values of s may only in rease, and those variables that have moved sin e B′have a greater value in s than variable xl. By the third riterion of s-monotoni ity, thesevariables have not moved sin e basis B′, thus have a orresponding zero value in t′(c). Sin e

t′kl < 0 and t′′lr > 0, furthermore t′kr > 0 and t′′rr = −1, we have t(k)′T t′′r < 0, ontradi tingthe orthogonality theorem. �Note that no index sele tion rule is ne essary for the sele tion of the driving variable.Example 3.3.1 We demonstrate the MBU simplex algorithm for linear programming prob-lems on the problem also solved in Example 3.2.1. To reate a feasible starting basis, thesame sla k and arti� ial sla k variables may be introdu ed. We start from the �rst tableauof Example 3.2.1. As before, we emphasize the driving variable in bold and show the othervariables in grey.x1 x2 x3 u2

u1 2 0 2 0 110v∗

1 -2 2 0 0 80v∗

2 1 0 2 -1 40z -1 -3 1 0 0

z∗ 1 -2 -2 1 -120Tableau 1.v∗

1 x2 x3 u2

u1 1 2 2 0 190x1 -0.5 -1 0 0 -40v∗

2 0.5 1 2 -1 80z -0.5 -4 1 0 -40

z∗ 0.5 -1 -2 1 -80Tableau 2.82

First the driving variable is x2. The primal ratio test is unique. A ording to the dualratio test, an auxiliary pivot is ne essary. However, the next pivot is a driving one, and afeasible basis is identi�ed.v∗

1 v∗2 x3 u2

u1 0 -2 -2 2 30x1 0 1 2 -1 40x2 0.5 1 2 -1 80z 1.5 4 9 -4 280

z∗ 1 1 0 0 0Tableau 3.x3 u2

u1 -2 2 30x1 2 -1 40x2 2 -1 80z 9 -4 280Tableau 4.

x3 u1

u2 -1 0.5 15x1 1 0.5 55x2 1 0.5 95z 5 2 340Tableau 5.In one more driving pivot, we obtain the same optimal solution x = (55, 95, 0), as we didfor the simplex method.The omplexity bound for the nondegenerate ase is analogous, as was in the ase ofthe simplex method after observing that an auxiliary pivot also in reases the value of theobje tive fun tion.3.4 The blending example for linear programmingTo demonstrate the algorithms, we present numeri al experiments with the simplex and theMBU simplex algorithms for the blending problem presented in Se tion 2.7.3, whi h is nowextended to an optimization problem. To do this, pri es are introdu ed on the raw materials,and pro�t values are de�ned for the two �nal produ ts. These osts are presented in Table3.2.Numeri al experiments on the simplex methodWe use the same framework as in Se tion 2.7.3. It must be emphasized, that the results areby no means a omparison of the algorithms, just a demonstration.For the simplex algorithm, using the embedding te h-nique presented in Subse tion 3.2.2, the �rst phase takes

314 iterations. We must note, that there is a small te h-ni al detail not yet dis ussed, namely that in the opti-mal solution of the se ondary obje tive fun tion it mayhappen that arti� ial sla k variables are still in the ba-sis on zero level. These variables are pivoted out on thebases on the numeri ally most stable positions (sin ethey are degenerate, no ratio test is required). 0 50 100 150 200 250 300 3500

0.5

1

1.5

2

2.5x 10

4

Iteration number

Sum

of i

nfea

sibi

lity

Sum of infeasibility.83

Raw material Raw materialIBU 596 ETA 886NBU 595 REF 521IC5 602 ETB 726I6C 603C9A 526CCE 544 Final produ tALK 532 UNH 575MTB 579 UNB 590Table 3.2: Cost of the raw materials and pro�t on the �nal produ ts.Note, that the starting arti� ial basis of the �rst phase has a ondition number of 1.After the �rst phase two arti� ial variables are still inbasis. Then with two degenerate pivots, these arti�- ial variables be ome nonbasi and are removed fromthe tableau. Now the se ond phase is applied. The ba-sis found in the �rst phase has a ondition number of6.394720 ∗ 106. The se ond phase only requires 29 it-erations to rea h optimality. The ǫ values used are thesame as were for the feasibility version of the problem. 0 5 10 15 20 25 30

0

1

2

3

4

5

6x 10

4

Iteration number

Obj

ectiv

e fu

nctio

n

The obje tive value.Numeri al experiments on the MBU simplex methodFor the feasibility version of the MBU simplex method, we used a logi al array to stabilizeour algorithm. This te hnique proved to be mostly unne essary for the original MBU simplexalgorithm for linear programming problems. However, as we will shortly see, the la k of su ha te hnique resulted in small �u tuations in the number of infeasible dual variables.Although it was su� ient to he k the feasibility of the dual variables with a simple omparison to a well-sele ted numeri al ǫ, still, it proved to be very important to do everydriving pivot step. In our experiments it happened, that due to numeri al round o� errors,driving variables be ame feasible (numeri ally onsidered to be feasible) after an auxiliarypivot. While in su h ases (in the feasibility version of the algorithm) we sele ted new drivingvariables, in the linear programming version it is vital to do the driving pivot (although itmay not even be an admissible pivot) to ensure that the basis be omes primal feasible again.For the MBU simplex algorithm for linear programming problems, the �rst phase required84

297 pivot iterations, out of whi h 54 was a driving pivot, while 242 was an auxiliary one.The feasible basis found by the �rst phase had a ondition number of 2.6338 ∗ 107.0 50 100 150 200 250 300

0

0.5

1

1.5

2

2.5x 10

4

Iteration number

Sum

of i

nfea

sibi

lity

0 50 100 150 200 250 3000

10

20

30

40

50

60

70

80

90

Iteration number

Num

ber

of in

feas

ible

dua

l var

iabl

es

Sum of infeasibility. Number of infeasible dual variables.Note, that the main �u tuation in the number of infeasible dual variables o urred whenthe sum of infeasibility was already very low, thus it shows some sensitivity to the sele tedǫ value.It is interesting that the se ond phase of the MBU simplex algorithm required only12 iterations. The number of infeasible dual variables in this ase arries no informationwhatsoever, sin e the problem had only one dual infeasible variable.

0 2 4 6 8 10 124

4.2

4.4

4.6

4.8

5

5.2

5.4x 10

4

Number of iterations

Obj

ectiv

e fu

nctio

n

0 2 4 6 8 10 120

1

2

3

4

5

6

7

8

Iteration number

Num

ber

of in

feas

ible

dua

l var

iabl

es

The obje tive fun tion. Number of infeasible dual variables.The fast onvergen e seems to be due to the lu ky situation that the �rst phase (a i-dentally) has found a very good basis.3.5 SummaryWe have proved that the lassi al simplex algorithm and the monotoni build-up simplex al-gorithm for solving linear programming problems are �nite, if the new on ept of s-monotoneindex sele tion pivot rules is applied. 85

We have also shown, that the omplexity bound presented in Chapter 2 for nondegenerateproblems may be generalized for the simplex and the monotoni build-up simplex algorithmfor linear programming problems as well.Based on the blending problem introdu ed in Chapter 2, we have presented some numer-i al experiments demonstrating the algorithms.

86

Chapter 4Linear omplementarity problemsThis hapter onsiders the linear omplementarity problem (LCP) in the standard form: �ndve tors u,v ∈ Rn, su h that−Mu+ v = q, u v = 0, u, v ≥ 0 (4.1)where M ∈ Rn×n, q ∈ Rn and uv = (u1v1, . . . , unvn) ∈ Rn.In this hapter, we present the well-known riss- ross method [81℄, using s-monotoneindex sele tion rules. Most LCP solvers require a priori information about the input matrix.One of the most general properties often required is su� ien y. However, su� ien y of amatrix is to he k (no polynomial time method is known) [87℄.We modify our basi algorithm in su h a way that it may start with an arbitrary matrix

M , without having any information about the properties of the matrix (su� ien y, bisym-metry, positive de�niteness, et ) in advan e. Even in this ase, our algorithm terminateswith one of the following ases in a �nite number of steps: it solves the LCP problem, solvesits dual problem or it gives a erti� ate that the input matrix is not su� ient, thus y lingmay o ur [19℄.It is worth mentioning that although the algorithm presented here is further general-ized than in [19℄ with the use of s-monotone pivot rules, the �niteness proof is not more ompli ated.Furthermore, the �niteness proof of the algorithm provides a onstru tive proof to Fukudaand Terlaky`s LCP duality theorem as well.The results of this hapter are based on the results presented in [10℄, however, the originalproofs were presented using the LIFO pivot rule. In this thesis, the results are furthergeneralized by using the so alled s-monotone pivot rules.87

4.1 Ba kgroundThe linear omplementarity problem is one of the most studied areas of mathemati al pro-gramming. A large number of pra ti al appli ations and the wide range of � unsolved boththeoreti ally and algorithmi ally � problems make it an attra tive �eld of resear h.There are several di�erent pivot algorithms to solve LCP problems with di�erent matri es.The riss- ross algorithm is one of those whi h were developed independently � for di�erentoptimization problems � by Chang [15℄, Terlaky [79℄ and Wang [92℄. Sin e then, the riss- ross method has be ome a lass of algorithms that di�ers in the index sele tion rule.Akkele³, Balogh and Illés [3℄ developed their riss- ross algorithm for LCP problemswith bilinear matri es. They used the LIFO (last-in-�rst-out) and the MOSV (most-often-sele ted-variable) pivot rules. It is an interesting question whi h is the widest lass of matri esfor whi h the riss- ross algorithm with the above mentioned index sele tion rules an beextended, preserving �niteness.The lass of su� ient matri es was introdu ed by Cottle, Pang and Venkateswaran [18℄.Su� ient matri es an be interpreted as generalizations of P and PSD matri es. Väliaho[86℄ showed that the lass of su� ient matri es is the same as the lass of P∗ matri es,introdu ed in [51℄ for interior point methods of LCP problems. It was proved by den Hertog,Roos and Terlaky [23℄, that the su� ient matri es are exa tly those matri es for whi h the riss- ross algorithm with the minimal index pivot sele tion rule solves the LCP problemwith any right-hand side ve tor.This hapter presents a generalization of the riss- ross algorithm for s-monotone pivotrules to LCPs with su� ient matri es.At present, there is no known e� ient algorithm to de ide whether a matrix is su� ientor not. (Väliaho [87℄ developed an indu tive method to he k su� ien y, but the algorithmis not polynomial). Most algorithms developed for LCP problems so far have the pra ti allyunattra tive property that they need the a priori information that the matrix is su� ient orpossesses some other good properties. Fukuda, Namiki and Tamura [32℄ gave the �rst su halgorithm, based on the LCP duality theorem of Fukuda and Terlaky [33℄ used in the form ofEP theorems, that did not require a priori information on the su� ien y of the matrix. If thealgorithm annot pro eed or would begin to y le, it provides a polynomial size erti� atethat the input matrix is not su� ient.Later in this hapter, the extension of the generalized algorithm is modi�ed in su h away that in ase of an arbitrary matrix M and right-hand side q, it either solves the LCPproblem, or provides a polynomial size erti� ate that the matrix M is not su� ient [19℄.This property improves the value of the algorithm signi� antly, be ause it makes it appli able88

to a wide range of LCP problems, without requiring a priori information on the propertiesof the matrix. Indeed, the improved freedom of the pivot position sele tion ompared tothe minimal index rule gives the possibility to avoid numeri ally instable pivots, furtherextending the pra ti al appli ability.The proof of this modi�ed algorithm provides a onstru tive proof for the LCP dualitytheorem in the stronger EP form.4.1.1 Su� ient matri esThe solvability of the linear omplementarity problems and the e� ien y of the solutionmethods depend on the properties of matrix M . We survey some properties of su� ientmatri es below.We will use the following te hni al-type de�nition.De�nition 4.1.1 For matrix M ∈ Rn×n and subset J = {α1, . . . , αk} ⊆ {1, . . . , n} ofindi es, the re tangular submatrix MJJ is alled a prin ipal submatrix.Now, we de�ne the su� ient matri es, [18℄.De�nition 4.1.2 The matrix M ∈ IRn×n is alled olumn su� ient if no x ∈ Rn ve torexists, for whi h{

xi (Mx)i ≤ 0 for all indi es i ∈ {1, . . . , n}

xj (Mx)j < 0 for at least one index j ∈ {1, . . . , n}(4.2)and we all it row su� ient if its transpose is olumn su� ient. A matrix M is alledsu� ient if it is both olumn and row su� ient at the same time.It an be shown that olumn su� ient matri es are exa tly those, for whi h the solutionset of linear omplementarity problems is onvex [18℄.The lass of su� ient matri es were introdu ed by Cottle, Pang and Venkateswaran [18℄.They have shown that these are generalizations of P and positive semide�nite matri es.They have also shown that su� ient matri es are spe ially stru tured P0 matri es.Later, den Hertog, Roos and Terlaky [23℄ proved that su� ient matri es are exa tlythose, for whi h the riss- ross algorithm with the minimal index pivot rule an solve linear omplementarity problems for any right-hand side ve tor q, in a �nite number of iterations.To show some important properties of su� ient matri es, we will need the de�nition ofstri tly sign reversing and stri tly sign preserving ve tors, [32℄:89

De�nition 4.1.3 We all a ve tor x ∈ R2n stri tly sign reversing ifxixi ≤ 0 for all indi es i = 1, . . . , n

xixi < 0 for at least one index i ∈ {1, . . . , n} .We all a ve tor x ∈ R2n stri tly sign preserving ifxixi ≥ 0 for all indi es i = 1, . . . , n

xixi > 0 for at least one index i ∈ {1, . . . , n} .Let us introdu e the subspa esV :=

{(u,v) ∈ R2n | [−M, I](u,v) = 0

}andV ⊥ :=

{(x,y) ∈ R2n | [I,MT ](x,y) = 0

},where u,v,x and y are all ve tors of length n. Obviously, V and V ⊥ are orthogonal om-plementary subspa es of R2n.Lemma 4.1.1 A matrix M ∈ Rn×n is su� ient if and only if no stri tly sign reversingve tor exists in V and no stri tly sign preserving ve tor exists in V ⊥, [32℄.A basis B of the linear system −Mu + v = q is alled omplementary, if for ea h index

i ∈ I exa tly one of the olumns orresponding to variables vi and ui is in the basis. Ashort pivot tableau is alled omplementary, if the orresponding basis is omplementary.The next lemma shows the sign stru ture of short omplementary pivot tableaux of LCPswith su� ient matri es. This stru ture is the only signi� ant property that we use in therest of the paper.Lemma 4.1.2 (Cottle, Pang and Venkateswaran [18℄) Let M be a su� ient matrix, B a omplementary basis and M = [mij | i ∈ JB, j ∈ JN ] the orresponding short pivot tableau.Then(a) mii ≥ 0 for all i ∈ JB; furthermore(b) for all i ∈ JB, if mii = 0 then mij = mji = 0 or mij · mji < 0 for all j ∈ JB, j 6= i.We have to mention that the proof of the previous lemma is onstru tive, so if the givenstru ture of the matrix is violated, we an easily obtain the erti� ate from tableau M , thatM is not su� ient. The oding size of this erti� ate is bounded by a polynomial of theinput length of matrix M .By the permutation of M ∈ Rn×n, we mean the matrix P T MP , where P is a permutationmatrix. 90

Lemma 4.1.3 [23℄ Let M ∈ Rn×n be a row ( olumn) su� ient matrix. Then1. any permutation of matrix M is row ( olumn) su� ient,2. the produ t DMD is row ( olumn) su� ient, where D ∈ Rn×n+ is a positive diagonalmatrix,3. every prin ipal submatrix of M is row ( olumn) su� ient.It is easy to see that matrix M is also su� ient after any number of arbitrary prin ipalpivots, if M is su� ient. The lass of su� ient matri es is also losed under prin ipalblo k pivot operations [87℄, namely, their properties are preserved during the riss- ross typealgorithms, too.For a matrix M ∈ Rn×n and J ⊆ {1, . . . , n}, if MJJ is nonsingular, we will denote theblo k pivot operation belonging to J by ηJ .Lemma 4.1.4 Let MJJ be a nonsingular submatrix of the row ( olumn) su� ient matrix

M . Then matrix M′

= ηJ (M) is row ( olumn) su� ient, [23℄.So the lass of su� ient matri es is losed also under the operation of blo k pivoting.The ex hange pivot operations arried out by the riss- ross algorithm de�ned later mayalso be viewed as a blo k pivot operation of size 2 × 2.4.1.2 The alternative theorem of the linear omplementarity problemThe de ision problem, whether an arbitrary linear omplementarity problem has a solutionor not, is in NP, and not always in co-NP. For some matrix lasses it belongs to co-NP, [16℄.The lass of su� ient matri es is su h a lass. Let us formulate (1.3) again:(u,v) ∈ V (M,q) :=

{(u,v)

−Mu + v = q

uv = 0, u, v ≥ 0

}(P − LCP )In the theory of optimization, a naturally arising question is whether there is anotherproblem de�ned by the same input data whi h has a solution, exa tly when problem (P −

LCP ) has no solution.Fukuda and Terlaky [33℄ answered this question in a very general form for omplemen-tarity problems on oriented matroids.Following the approa h of Fukuda and Terlaky, we de�ne the problem (D − LCP )

(x,y) ∈ V (M,q)⊥ :=

{(x,y)

x + MTy = 0, qTy = −1

xy = 0, x, y ≥ 0

}(D − LCP )91

The next theorem plays an important role in the analysis of LCP problems. In thisse tion, only the sket h of the proof is presented, the whole proof depends on the �nitenessof the riss- ross algorithm whi h is proved in Subse tion 4.2.2.To hara terize the feasibility of linear omplementarity problems, we an formulate thetheorem of Fukuda and Terlaky as follows:1Theorem 4.1.1 For a su� ient matrix M ∈ Rn×n and a ve tor q ∈ Rn, exa tly one of thefollowing statements holds:(1) the (P-LCP) problem has a feasible omplementary solution (u,v),(2) the (D-LCP) problem has a feasible omplementary solution (x,y).Proof. Let us suppose that both of them an be solved, and let (u,v) be the solution of(P − LCP ) and (x,y) be the solution of (D − LCP ). Then from the ondition

−Mu + v = q,after taking the left matrix produ t with yT , we get−yT Mu + yTv = yTq = −1.Using the �rst ondition of the (D −LCP ) problem and the nonnegativity of our variables,we have the relation

0 ≤ xTu + yTv = −1,whi h is a ontradi tion.We prove that one of the problems (P − LCP ) and (D − LCP ) is feasible. To do this,we use the riss- ross algorithm with s-monotone pivot rules where the �niteness of thealgorithm will omplete the proof of the alternative theorem. �This also means that it an be well- hara terized when problem (P−LCP ) has no feasible omplementary solution if the matrix M is su� ient and rational, be ause a polynomial size erti� ate an be given, namely the solution of the problem (D − LCP ).1Fukuda and Terlaky [33℄ all their result as a duality theorem, but we think that the term alternativetheorem represents its meaning more pre isely. 92

4.2 The Criss-Cross methodLet us denote with I := {1, 2, ..., n, 1, 2, ..., n} ∪ {q} the (ordered) set of the indi es and|I| = 2n + 1. To simplify the notation let α = α for all α ∈ I \ {q}, so the omplementarypair of α is α.The initial omplementary solution of the linear omplementarity problem (1.3) is u = 0and v = q. Let the matrix of the short pivot tableau be denoted by T as usual.Our aim is to generate a feasible omplementary solution from our initial solution, usingpivot operations. Lemmas 4.1.3. and 4.1.4. ensure that the su� ien y of the matrix ispreserved during the algorithm. Moreover, we only make su h pivot operations that preserve omplementarity, too. To do this, we need to re�ne the on ept of s-monotone pivot rulesfor LCP problems.De�nition 4.2.1 An s-monotone pivot rule applied to an LCP problem is symmetri , ifs(i) = s(i) always holds for any i ∈ I.In the riss- ross algorithm presented below, we apply symmetri s-monotone pivot rules.We explain the pivot operations (diagonal and ex hange pivot) and the role of the ounterve tor s when we analyze the algorithm. The omplementary variable pairs (ul, vl) willusually be referred to simply as variable pairs.Let us assume that vj is a basi variable and the value of variable vj is infeasible. Iftjj < 0, we may perform a diagonal pivot where variable uj enters the basis while variablevj leaves it.If tjj = 0, we have to pivot on su h an index k for whi h tjk < 0. The solution obtainedafter a pivot like this is not omplementary any more, so to restore the omplementarity ofthe solution, we have to pivot on the position (k, j) as well. Lemma 4.1.2. ensures that insu h a ase tkj > 0. These two pivots together are alled an ex hange pivot.During an ex hange pivot, variables uj and uk enter the basis, while vj and vk leave it(see the �gure of ex hange pivot). We say that uj and vj are hosen a tively, while uk andvk are hosen passively. Thus, the terms passively and a tively refer to the order in whi hthe indi es are sele ted. The two types of pivot operations are presented in Figure 4.1.In ase of an ex hange pivot, the ve tor s of the s-monotone pivot rule is updated to thepassively sele ted pair of variables �rst, and only then for the a tively sele ted pair, thusthe symmetry of ve tor s is also maintained. This way, from the s ve tor point of view,an ex hange pivot is onsidered as two pivots. This rule will play an important role in the�niteness proof.The pseudo- ode of the riss- ross type algorithm is presented in Figure 4.2.93

uj...vj . . . − . . . −...Diagonal pivot

uk uj...vk +...vj . . . − . . . 0 . . . −...Ex hange pivotFigure 4.1: Diagonal and ex hange pivot.Criss- ross type algorithm with symmetri s-monotone pivot ruleInput: problem (1.3), where M is su� ient, T := −M , q := q, r := 1. Initialize s.Begin

J := {i ∈ I : qi < 0} .While (J 6= ∅) doJmax := {j ∈ J | s(j) ≥ s(α), for all α ∈ J } , let k ∈ Jmax arbitrary index.If (tkk < 0) thenDiagonal pivot on tkk, update ve tor s for variable pair (uk, vk), let r := r + 1.Else

K := {i ∈ I : tki < 0}.If (K = ∅) then Stop: The LCP problem has no feasible solution.ElseKmax = {β ∈ K | s(β) ≥ s(α), for all α ∈ K} , let l ∈ Kmax arbitrary.Ex hange pivot on tkl and tlk.Update ve tor s for variable pair (ul, vl) as in iteration r + 1,then for variable pair (uk, vk) as in iteration r + 2 and let r := r + 2.EndifEndifEndwhileStop: A feasible omplementary solution has been omputed.End

Figure 4.2: The riss- ross type algorithm.94

The algorithm starts from the trivial omplementary solution and be ause it only makesdiagonal and ex hange pivots, it preserves omplementarity. The su� ien y of matrix Tensures that (Lemma 4.1.2) the sign of the hosen pivot elements will be as desired, if wehave to do an ex hange pivot. The algorithm terminates only if there is no solution or ifit has found the solution. Thus, we have to prove that it is �nite. Be ause the number ofpossible bases is �nite, we have to show that the riss- ross type algorithm with s-monotonepivot rule does not y le.

0 ... ...

?

feasiblesolution nono

l

k no solutionyes

yes no

ex hangepivot possiblek

k− −

−

− k

k

l

START

STOP yesfeasible?

diagonalpivot possible?

STOP

omplementarytableau⊕

⊕

⊕

⊕

⊖⊖

...... omplementary

Figure 4.3: Flow hart of the algorithm.4.2.1 Almost terminal tableaux of the riss- ross methodLet us assume that an example exists for whi h the algorithm is not �nite. The number ofbases is �nite, at most (2n

n

), so the algorithm makes an in�nite number of iterations only if95

ul

(a) ⊕

⊕......⊕

⊕

vl − −

uj ul

(b) ⊕

vj + ⊕......⊕

⊕

vl − 0 −Figure 4.4: Variable ul is a tively sele ted to enter the bases. y ling o urs. Let us onsider a minimal size y ling example. In su h an example, be auseof minimality, every variable moves during the y le.Let us onsider the situation des ribed by the se ond riterion of s-monotoni ity andwith variable ul outside the bases. Consider basis B′ where variable ul enters the bases forthe �rst time after this situation o urred. Sin e ul and vl have the smallest value a ordingto ve tor s and the symmetry of the rule, vl is the only infeasible variable in basis B′. Theshort pivot tableaux (presented in Figures 4.4 and 4.5) for this ase an be as follows:1. The algorithm hooses ul to enter the basis.The diagonal element tll < 0, so a diagonal pivot is possible [tableau (a)℄: ul entersthe basis, while vl leaves it. The ve tor s is modi�ed symmetri ally for variable pair(ul, vl).2. The algorithm hooses variable ul to enter the basis, but tll = 0, so an ex hange pivotis ne essary [tableau (b)℄. Variables ul and uj enter the basis, while variables vl andvj leave it. The ve tor s is modi�ed symmetri ally, �rst for variable pair (uj, vj) andthen for variable pair (ul, vl) as if in the next iteration.The olumn of q is the same as in tableau (a). In this ase, it is not important whether

uj or vj is in the basis. We onsider the ase when vj is in the basis.3. The algorithm hooses a variable uj to enter the basis, but tjj = 0, so an ex hangepivot is needed and the algorithm hooses the variable ul as well (passively), [tableau(c)℄.A ording to the situation when ul enters the basis, the row of vj only in the olumnsof ul and q may ontain negative elements, and be ause tjj = 0, using Lemma 4.1.2,96

ul uj

(c) ⊖...⊖

vl +⊖...⊖

vj ⊕ · · · ⊕ − ⊕ · · · ⊕ 0 −Figure 4.5: Variable ul is passively sele ted in a a se ond pivot position in an ex hange pivotto enter the basis.we an also �ll in the sign stru ture of the olumn of uj. In this ase, it is on e againnot important whether uj or vj is in the basis. We onsider the ase when vj is in thebasis. The ve tor s is modi�ed symmetri ally �rst for variable pair (ul, vl) and thenfor variable pair (uj, vj) as if in the next iteration.For later use, we state that in ases 1. and 2. we used only the pivot rule when we �lledin the sign stru ture, while we used the su� ien y of the matrix in the third ase for the olumn of uj.Now we onsider basis B′′, when ul leaves the basis for the �rst time after basis B′. Thepivot tableau for this iteration an have three di�erent stru tures (as presented in Figures4.6 and 4.7), a ording to the third riterion of s-monotoni ity.A. A ording to the pivot rule, we hoose variable ul to leave the basis, tll < 0, so adiagonal pivot takes pla e [tableau (A)℄.B. The pivot rule hooses variable ul to leave the basis, but tll = 0, so an ex hange pivotis needed: vk (or uk) enters the basis, while uk (or vk) leaves it [tableau (B)℄.C. The algorithm hooses variable uk (or vk), but tkk = 0, so an ex hange pivot takespla e and vl enters the basis, while uk leaves it [tableau (C)℄.Below, we show that none of the tableaux (a)−(c) an be followed by one of the tableaux(A)− (C) if matrix M is su� ient. It will be important to note where the su� ien y of thematrix is used. 97

vl(A) ⊕...⊕

−...−

ul − −

vk vl(B)uk +

ul − 0 −Figure 4.6: Variable ul is a tively sele ted to leave the bases.vl vk(C)

ul +

uk − 0 −Figure 4.7: Variable ul is passively sele ted in a se ond pivot position in an ex hange pivotto leave the basis.Auxiliary lemmasFirst, we onsider the ases that do not use the su� ien y of matrix M . We begin by showingthat tableau (c) annot be followed by tableaux (A) or (B).Lemma 4.2.1 Let us denote the tableau of ase (c) by TB′ and the tableau of A (or B) byTB′′. Consider the ve tors t′

(æ) and t′′q , read from the row of the basi variable vj in tableauTB′, and from the olumn of q in TB′′. Then

(t′(j))T t′′q > 0.Proof. Let K′′ := { i ∈ IB′′ | q′′i < 0 }. Using the third riterion of s-monotone pivot rules,variables referring to these indi es (with the possible ex eption of index j and l) have notmoved sin e B′, or else their s value would have to be bigger than of ul, thus the algorithmwould have hosen one from K′′ instead of ul, so K′′ ⊆ IB′ ∩IB′′ . This indi ates that t′ji

= 098

for all indi es i ∈ K′′ \ {j, l}, thus∑

i∈J ′′\{j,l}

t′jit′′iq = 0. (4.3)Sin e at the ex hange pivot made on B′, the s value was �rst updated for variable pair

(ul, vl) and only in the next pivot for (uj, vj), we also know from s-monotoni ity that t′′jq andt′′jq

are nonnegative (one is outside the basis, and sin e they have moved sin e B′, their svalue is bigger than of ul).Furthermore, it an be read from tableau (c) that t′jj

= 0, t′jj

= 1, t′j l

= 0, t′jl

< 0 andt′jq

< 0 sot′jj t′′jq + t′jjt

′′jq + t′j lt

′′lq + tjl t

′′lq + t′jq t′′qq ≥ t′jlt

′′lq − t′jq > 0, (4.4)be ause t′′qq = −1 by de�nition, and t′′lq < 0 a ording to the pivot rule of the algorithm(tableaux (A) and (B)).If h /∈ K′′ ∪

{j, j, l, l, q

}, we know again from the tableaux that t′jh

≥ 0 and by thede�nition of K′′ it holds that t′′hq ≥ 0, so∑

h 6∈K′′∪{j,j,l,l,q}

t′jh t′′hq ≥ 0. (4.5)The result follows as we sum up inequalities (4.3)-(4.5).�From tableau (c) we onsidered the stru ture of the row for variable vj, while fromtableaux (A) and (B) the stru ture of the olumn of q. In none of these ases did we usethe su� ien y of the matrix, and the proofs used only the ombinatorial nature of the s-monotone pivot rules. Thus tableaux (c) and (A) (or (B)) are ex lusive be ause of theorthogonality theorem and the lemma above, regardless of the su� ien y of the matrix.We now prove that tableaux (a) and (b) annot be followed by tableau (C).Lemma 4.2.2 Let us denote tableau (a) (or (b)) by TB′ and tableau (C) by TB′′. Considerthe ve tors t′q and t′′(k) belonging to the olumn of q in tableau MB′, and to the row of uk intableau MB′′. Then

(t′′(k))T t′q > 0.Proof. Like in the previous lemma, K′′k := {i ∈ IN ′′ | t′′ki < 0} ⊂ IN ′ holds be ause of thethird riterion of s-monotoni ity, so t′iq = 0 for every i ∈ K′′

k \ {l}, thus∑

i∈K′′k\{l}

t′′ki t′iq = 0. (4.6)99

Furthermore, for an index h /∈ K′′l ∪ {j, j, l, l, q}, t′′kh ≥ 0 and t′hq ≥ 0, therefore

∑

j 6∈K′′l∪{j,j,l,l,q}

t′′kjt′jq ≥ 0. (4.7)From tableaux TB′ and TB′′ it an be read that t′qq = −1, t′′kk = 1, t′′

kk= t′′kl = t′lq = 0 and

t′′kl

< 0, t′′kq < 0, t′lq

< 0, t′kq

≥ 0 and t′kq ≥ 0 sot′′kkt

′kq + t′′kk t′kq + t′′kl t

′lq + t′′klt

′lq + t′′kq t′qq = t′kq + t′′kl t

′lq − t′′kq ≥ t′′klt

′lq − t′′kq > 0.The result follows as we sum up inequalities (4.6) − (4.8).

�We an now onsider tableaux where the su� ien y of the matrix plays an importantrole.In the following, we show that tableaux (a) (or (b)) annot be followed by tableaux (A)or (B).Lemma 4.2.3 Let the omplementary solutions (u′,v′) and (u′′,v′′), belonging to tableaux(a) (or (b)) and (A) (or (B)) be given. Then

(u′ − u′′) M (u′ − u′′) < 0.Proof. We prove all four ases simultaneously.(u′ − u′′) M (u′ − u′′) = (u′ − u′′) (q + M u′ − q − M u′′) = (u′ − u′′) (v′ − v′′)

= u′ v′ − u′ v′′ − u′′ v′ + u′′ v′′ = −u′ v′′ − u′′ v′,where the last equation holds be ause of the omplementarity of the given solutions. LetK′′ := { i ∈ IB′′ | q′′i < 0 }. As before, a ording to s-monotoni ity, variables indexed by K′′have not moved sin e bases B′, or else the algorithm would have hosen one from them, thusfor all i ∈ K′′\ {l}, the value of u′

i (or v′′i ) and u′′

i (or v′i) is zero:

u′i v′′

i + u′′i v′

i = 0. (4.8)From tableau (a) (or (b)), and tableau (A) (or (B)), it an be read that u′l = 0, v′

l < 0 andu′′

l < 0, v′′l = 0 so,

u′l v′′

l + u′′l v′

l > 0. (4.9)Furthermore, for any h /∈ K′′ it holds that u′h, v

′h, u

′′h, v

′′h ≥ 0, thus

u′h v′′

h + u′′h v′

h ≥ 0. (4.10)100

To sum it up, the ve tor (u′ − u′′) is su h that (u′ − u′′) M (u′ − u′′) < 0.

�Note, that the proof is onstru tive be ause the ve tor u′ − u′′ proving the la k of su�- ien y of our matrix an easily be obtained from the bases B′ and B′′.In the last auxiliary lemma, we will investigate the ase when tableau (c) would befollowed by tableau (C).Lemma 4.2.4 Let us denote tableau (c) by TB′, and tableau (C) by TB′′. Consider theve tors t′j and t′′(k) belonging to the olumn of uj in tableau TB′ and to the row of uk intableau TB′′. Then(t′′(k))T t′j < 0.Proof. Let K′′

k = {i ∈ IN ′′ : t′′ki < 0}\ {j}. Using the third riterion of s-monotone pivotrules again, the variables of the indi es K′′k have not moved sin e B′, so (IN ′′ \ K′′

k) ⊂ IB′and K′′k ⊂ IN ′ , thus t′ij = 0 if i ∈ K′′

k. Based on these observations, we have∑

i∈K′′k∪{q}

t′′ki t′ij = 0. (4.11)Furthermore, if h /∈ K′′

k ∪{q, l, l, j, j, k, k

} then t′hj ≤ 0 a ording to tableau (c). By thede�nition of K′′k, t′′kh ≥ 0, so ∑

h 6∈K′′k∪{q,l,l,j,j,k,k}

t′′kh t′hj ≤ 0, (4.12)From tableaux TB′ and TB′′ , taking the de�nition of ve tor t into onsideration, we get thatt′lj = t′′kk = t′jj = t′qj = t′′kl = 0, t′′kk = 1, t′jj = −1 and t′kj ≤ 0, t′′kl < 0, t′lj > 0so

t′′kq t′qj + t′′kl t′lj + t′′kl t

′lj + t′′kj t′jj + t′′kj t′jj + t′′kkt

′kj + t′′kk t′kj < −t′′kj. (4.13)By the de�nition of the algorithm, at the ex hange pivot in tableau c, the �rst variablepair for whi h the update of the s ve tor is applied is (ul, vl), and only after for (uj, vj). Hen evariable (uj, vj) is already onsidered to be moved sin e B′, thus by the third riterion of

s-monotone pivot rules their s value is bigger than of index l. This is only possible if t′′kj ≥ 0either be ause out of bases, or be ause of its asso iated s value.The result follows as we sum up inequalities (4.11) − (4.13).�101

(a) (b) (c)

(A) ∗ ∗

(B) ∗ ∗

(C) ∗Figure 4.8: The ases when su� ien y of the pivot matrix is used.4.2.2 Finiteness of the riss- ross methodIn this se tion, we prove the �niteness of the riss- ross algorithm, making the proof ofTheorem 4.1.1 omplete.Theorem 4.2.1 The riss- ross type algorithm with s-monotone pivot rule is �nite for thelinear omplementarity problem with su� ient matri es.Proof. Let us assume the ontrary, that the algorithm is not �nite. Be ause a linear omplementarity problem has �nitely many di�erent bases, the algorithm an have an in�nitenumber of iterations only if it is y ling. Then we have a y ling example. Let us hoosefrom the y ling examples one with a minimal size. Then, every variable moves during the y le. Taking the auxiliary lemmas into onsideration, after variable ul enters the basis afterbasis B′, it annot leave it again:If it enters in ase (a) or (b) and leaves the basis in ase (A) or (B), Lemma 4.2.3. ontradi ts the su� ien y of matrix M .If it enters in ase (c) and leaves the basis in ase (A) or (B), Lemma 4.2.1. ontradi tsthe orthogonality theorem.If it enters in ase (c) and leaves the basis in ase (C), Lemma 4.2.4. ontradi ts theorthogonality theorem.If it enters in ase (a) or (b) and leaves the basis in ase (C), Lemma 4.2.2. ontradi tsthe orthogonality theorem.All possible ases lead to a ontradi tion, therefore the algorithm is �nite. �The following shows the ases in whi h we took the su� ien y of matrix T into onsid-eration during the proof of �niteness of the riss- ross type algorithm.Example 4.2.1 To demonstrate the riss- ross method, onsider the following matrix andthe orresponding short pivot tableau of the LCP problem, where the identity matrix orre-sponding to v serves as an initial omplementary basis.102

u1 u2 u3

v1 -2 1 1 -4v2 0 0 -1 -2v3 1 1 -1 1Tableau 1.

v1 u2 u3

u1 -0.5 -0.5 -0.5 2v2 0 0 -1 -2v3 0.5 1.5 -0.5 -1Tableau 2.We solve the LCP with the riss- ross method using the MIR rule. After a diagonal andan ex hange pivot, the tableau be omes feasible.

v1 u2 v2

u1 -0.5 -0.5 -0.5 3u3 0 0 -1 2v3 0.5 1.5 -0.5 0Tableau 3.

v1 v3 v2

u1 -1/3 1/3 -2/3 3u3 0 0 -1 2u2 1/3 2/3 -1/3 0Tableau 4.The found omplementary solution is (u,v) = ((0, 0, 2), (3, 0, 0)).However, this example demonstrates that the riss- ross algorithm may solve a LCP prob-lem even if the matrix is not su� ient. To show that M is not su� ient, onsider the ve tor

x = (1,−1,−1). For this ve torx · (Mx) = (−4,−1,−1),thus x is a stri tly sign preserving ve tor. This proves that M is not su� ient.4.3 EP theorems and the linear omplementarity problemNow we generalize the algorithm in the sense of EP theorems. An EP theorem is usuallya olle tion of several possible statements, but from whi h one always holds, and if anystatement holds, then a polynomial size (in the length of the input data) erti� ate mustexist. It may also be viewed as a general framework for making pra ti ally provable theories.Observe, that the Farkas lemma and the strong duality theorem in Chapter 3 were EP-type theorems, sin e ea h had at least one of its possible out omes true for every input, andan optimal or infeasible basis may serve as a erti� ate of polynomial size.We annot expe t that we will be able to solve every linear omplementarity problemwith an arbitrary matrix using the riss- ross type method. If our generalized riss- rosstype algorithm was not able to solve a linear omplementarity problem, it would give a erti� ate that the matrix of the problem is not su� ient.The modi� ation of our riss- ross type algorithm is based on the theory developed byCameron and Edmonds [14℄. They introdu ed the so alled EP�theorems.103

The general form of an EP (Existentially Polynomial time) theorem is as follows:[∀x : F1(x) or F2(x) or . . . or Fk(x)]where Fi(x) is a statement of the form

Fi(x) = [∃yi for whi h ‖yi‖ ≤ ‖x‖ni and fi(x,yi)] .We generalize the riss- ross type algorithm for general linear omplementarity problem.Here ni ∈ Z+, ‖z‖ denotes the oding size of z, while f(x,y) is a predi ate for whi h thereis a polynomial-time algorithm.Before formalizing the duality theorem of the linear omplementarity problem in EPform, we need to state some de�nitions and theorems.We all the support of ve tor x the set {i | xi 6= 0}. From a given set of ve tors, a ve torwith a minimal support is alled a ir uit.We will use the onform de omposition of a ve tor, [32℄.De�nition 4.3.1 Let V ⊆ Rn be an arbitrary linear subspa e and x,x1, . . . ,xk ve tors fromsubspa e V . We say that ve tor x is onformly de omposed into ve tors x1, . . . ,xk, ifx = x1 + · · · + xk andxi = 0 =⇒ x1

i = · · · = xki = 0,

xi > 0 =⇒ x1i , . . . , x

ki ≥ 0,

xi < 0 =⇒ x1i , . . . , x

ki ≤ 0 for all indi es i = 1, . . . , n.For a linear subspa e, the following holds [71℄.Lemma 4.3.1 Let V be a linear subspa e in Rn. In this ase, any x ∈ V an onformly bede omposed into ir uits c1, . . . , ck of V .With the lemma above, we an show that the erti� ate of non su� ien y an be givenusing a ir uit or the sum of two ir uits, [32℄.Lemma 4.3.2 If M is not olumn (row) su� ient, then a stri tly sign reversing (preserving) ir uit in subspa e V (V ⊥) exists or a stri tly sign reversing (preserving) ve tor x in subspa e

V (y in V ⊥) exists, that an be de omposed into the sum of two omplementary ir uits.Theorem 4.3.1 Let the matrix M ∈ IRn×n be not su� ient. In this ase, a erti� ate existsthat M is not su� ient, the oding size of whi h is polynomially bounded by the input lengthof matrix M [32℄. 104

We want to state the duality theorem of linear omplementarity problems in EP form.To do this, �rst we have to form it in su h a way, that the su� ien y of the matrix is nolonger a ondition of the theorem.Theorem 4.3.2 For any matrix M ∈ Qn×n and ve tor q ∈ Qn, at least one of the followingholds:(1) the problem (P-LCP) has a omplementary, feasible solution (u,v).(2) the problem (D-LCP) has a omplementary, feasible solution (x,y).(3) the matrix M is not su� ient.The theorem is not in an EP form yet. When (1) or (2) holds, the solution itself ispolynomial sized. In ase (3), we have to show that there is a polynomial size erti� ate,that matrix M is not su� ient.Now, we an state the LCP duality theorem in the EP form, [32℄.Theorem 4.3.3 For any matrix M ∈ Qn×n and ve tor q ∈ Qn, at least one of the followingstatements holds:(1) problem (P-LCP) has a omplementary, feasible solution (u,v) the en oding size ofwhi h is polynomially bounded by the input length of matrix M and ve tor q.(2) problem (D-LCP) has a omplementary, feasible solution (x,y) the en oding size ofwhi h is polynomially bounded by the input length of matrix M and ve tor q.(3) matrix M is not su� ient, and there is a erti� ate the en oding size of whi h ispolynomially bounded by the input length of matrix M .Note that ases (1) and (2) are ex lusive, while ase (3) an hold alone or together witheither ase (1) or (2). Moreover, it is a naturally arising ondition that the entries of thematrix should be rational numbers.To prove the theorem, we have to modify our algorithm and prove its �niteness.We modify our algorithm so that it either solves problem (P − LCP ) or its dual, orproves the la k of su� ien y of the input matrix2, giving a polynomial size erti� ate.Lemma 4.1.2 ensures that the pivot operations an always be done if our matrix issu� ient, and if it is not, it provides the required erti� ate that matrix M is not su� ient.2There is no known e� ient, polynomial algorithm to he k the su� ien y of a matrix.105

The modi�ed riss- ross type algorithmInput: T = −M, q = q, r = 1, Initialize Q and s.BeginWhile ((J := {i ∈ I | qi < 0}) 6= ∅) doJmax := {β ∈ J | s(β) ≥ s(α), for all α ∈ J }.Let k ∈ Jmax be arbitrary.Che k −u′ · v′′ − u′′ · v′′ with the help of Q(k).If (−u′ · v′′ − u′′ · v′′ � 0) thenStop: M is not su� ient, erti� ate: u′ − u′′.EndifIf (tkk < 0) thenDiagonal pivot on tkk, update s.

Q(k) = [JB, tq], r := r + 1.ElseIf (tkk > 0)Stop: M is not su� ient, reate erti� ate.Else /* tkk = 0 */K := {α ∈ I | tkα < 0}If (K = ∅) thenStop: DLCP solution.Else

Kmax = {β ∈ K | s(β) ≥ s(α), for all α ∈ K}.Let l ∈ Kmax be arbitrary.If ((tk, tk) or (tl,tl) sign stru ture is violated) thenStop: M is not su� ient, reate erti� ate.EndifEx hange pivot on tkl and tlk, update s �rst for (uk, vk),then for (ul, vl) as in a next iteration.Q(k) = [JB, tq], Q(l) = [∅,0], r := r + 2.EndifEndifEndWhileStop: we have a omplementary feasible solution.End

Figure 4.9: The modi�ed riss- ross type algorithm.106

We now have to deal with the ase of y ling.Avoiding y ling: Note, that minimality of the y ling example is not ne essary inthe proof of �niteness of the original algorithm. Every proof remains valid for any y lingexample, sin e those variables that do not move during a y le do not hange their basisstatus, thus have a zero value in the orthogonality theorem.Let us onsider an arbitrary y ling example. Let the index set of the variables involvedin the y ling be R, and onsider an iteration, when y ling has already begun. Let the basisB′ su h that is satis�es the se ond riterion of s-monotoni ity with variable xl.Then the stru ture of tableaux MB′ and MB′′ restri ted to indi es R and ve tor q isexa tly like in ase (a) − (c) and (A) − (C). Between these two tableaux, a variable whoseindex is not in R has not moved. Thus, in the produ t of the fundamental ir uits in Lemmas4.2.1, 4.2.2 and 4.2.4, for the indi es not in R and not q, exa tly one of the orrespondingvariables is in basis, so the produ t for these indi es is always zero. For the same reason, inthe produ t of −u′ ·v′′−u′′ ·v′ in Lemma 4.2.3, the entries for the indi es not in R are ea hzero. So the proofs are valid for an arbitrary y ling example.Handling the la k of su� ien y: Remember that we used su� ien y only in Lemmas4.2.3 and 4.2.4. This latter one used the sign property of su� ient matri es, based on Lemma4.1.2. Therefore, if the algorithm he ks that the required sign property is ful�lled duringevery ex hange pivot ( ases (c) and (C) refer to su h pivots), tableau (c) annot be followedby tableau (C), be ause of the orthogonality theorem. If the required sign stru ture isviolated, the erti� ate that matrix M is not su� ient is provided by the same lemma.There remain the ases of tableaux (a)-(b) and (A)-(B). Lemma 4.2.3 handled this ase.The proof of the lemma is based on the produ t

−u′ · v′′ − u′′ · v′ (4.14)referring to su h subsequent tableaux MB′ and MB′′ , where the same variable moves duringboth pivot operations, and in both ases this variable was hosen a tively (that is, not as these ond variable of an ex hange pivot). Note, that we do not need the whole tableau here,the only information we use is the olumn of q (the a tual omplementary solution) and theset of indi es in the basis. If ve tor (4.14) is stri tly sign reversing, then as in the note afterLemma 4.2.3, the eviden e that matrix M is not su� ient is the ve tor u′ − u′′.Let us introdu e a list Q(p) (p = 1, . . . , n). Two ve tors of dimension n belongs to everyentry of this list. At the beginning,Q(p) :=

[[1, . . . , n]

[0, . . . , 0]

]p = 1, . . . , n.107

When variable ul or vl leaves the basis during a diagonal pivot or su h an ex hange pivotwhere this variable is a tive (variable sele ted �rst), we modify the value of Q(l) in su h away, that we write the indi es of variables in basis to the �rst ve tor before the a tual pivotoperation, while we write the values of variables in basis before the pivot operation to these ond ve tor:Q(l) :=

[[ indi es of variables in basis ]

[ values of variables in basis] . ]If variable ul or vl enters the basis passively (as the se ond variable of an ex hange pivot),we modify the value of Q(l) as:Q(l) :=

[[1, . . . , n]

[0, . . . , 0]

]

l

k

− kdiagonalpivot

negative positivekyes = 0yesno no

the tableau erti� ate fromnot su� ient, +

u”v′ ≥ 0?−u′v”−

k−the tableau erti� ate fromnot su� ient,

omplementarytableauviolated?

tkksign ofsign stru ture

erti� ate from Qnot su� ient,

START

− k

ex hange pivotl

0

k

STOP STOP STOPFigure 4.10: Flow hart of the modi�ed algorithm.We mean by an operation Q(j) = [{I}, {h}] that, to the entry of j in the list Q, we writelist I to the pla e of basis indi es, while the values of the ve tor h in the pla e of q.Before the algorithm performs a pivot operation, it he ks if the a tively sele ted variablethat enters the basis was hosen a tively previously or not when it left the basis. If yes, withthe help of list Q, it he ks ve tor (4.14) and only after this does it modify list Q. Be ausethe omplementary pairs of variables move together during the pivot operations, it is notne essary to provide spa e for both of them in list Q.108

+ + + +... ...0 k

q(u, v)

−M I

1(x, y) −(x, y)i =

−tki/qk if i ∈ IN

−1/qk if i = k

0 otherwise.Figure 4.11: The dual solution when no primal solution exists.Note, that be ause the de�nition of the initial values of Q and the modi� ation of Qduring a passive ex hange pivot, it su� es to he k produ t (4.14) during any pivot. Iftableau (a) (or (b)) is followed by tableau (A) (or (B)), the produ t will always be zero.It would not be ne essary to �ll out list Q every time. With a slight modi� ation of thealgorithm, we would be able to save storage spa e as well. In the worst ase, the storagespa e required by list Q would be the storage spa e required to store n2 integer and n2rational numbers.We have to investigate the ase when (P − LCP ) has no solution. This o urs whenK = ∅. The stru ture of the pivot tableau is shown in Figure 4.11. Consider the ve tor

(x′,y′) = t(k) |JN∪JB.Using the orthogonality theorem, we get that this ve tor is orthogonal to every row of

[−MT | −I], in other words MTx′ + y′ = 0. Applying the orthogonality theorem to the olumn of the right-hand side ve tor q (in the starting basis), we have(x′,y′)T tq |JN∪JB

= (x′,y′)T (q,0) = x′Tq = qk.So the ve tor (x,y) = (x′,y′)/(−qk) is a solution to the problem (D − LCP ), be ausenonnegativity and omplementary follow from the stru ture of the pivot tableau.4.4 SummaryWe have presented a generalized riss- ross type algorithm for linear omplementarity prob-lems with su� ient matri es, using s-monotone pivot rules. For better pra ti al appli ability,we have modi�ed the generalized algorithm so that the a priori information on the su� ien yof the matrix is not ne essary. In ase of la k of su� ien y, if the algorithm annot ensure�niteness, then it terminates and provides a polynomial size erti� ate that the matrix is notsu� ient. We have a hieved our goals using the duality theorem of linear omplementarity109

problems [33℄ and with its EP theorem form [32℄. With the use of �exible s-monotone pivotrules, the algorithm provides signi� ant freedom in hoosing the pivot position (usually dur-ing the �rst part of the algorithm), making it possible to avoid some numeri ally instablepivots.The �niteness of the generalized riss- ross algorithm provides a onstru tive proof tothe LCP duality theorem, while the modi�ed version provides a onstru tive proof to theLCP duality theorem in the EP form as well.

110

Chapter 5The appli ation of linear optimization inthe petroleum industry5.1 Introdu tionThis hapter on erns pra ti al optimization problems arising in the petroleum industry.Modern supply hain management models in orporate nearly every aspe t of petroleumindustry from e onomi al onsiderations to te hni al details su h as blending. While most onstraints of the model may be modelled with linear terms, blending problems result inbilinear ones, where both the amounts and qualities of the yields are unknown.The aim of this hapter is to provide a survey on some pra ti al issues of the solution of themathemati al programming problems arising in the produ tion planning models of petroleumindustry. A survey on the developments of the basi models and solution te hniques used inre�nery optimization is presented in [85℄.Although several formalizations are possible for blending problems [5℄, usually the so- alled �ow model is used in the petroleum industry.We have already seen the basi modelling on epts of blending in Se tion 2.7 and Se tion3.4. There we have assumed that the quality of the inner blends are known, and that the ratioof the amounts in whi h these blends are further mixed is also known. Both assumptions areunrealisti in pra ti e. If these quantities are onsidered as variables, then we get a (rathergeneral) bilinear programming problem.The bilinear onstraints of the model make the solution of the problem pra ti ally hal-lenging. From the theoreti al point of view, the problem is NP-hard. In pra ti e, this isrealized in omputational hallenges and in the existen e of lo al optimal solutions. Unfor-tunately there are no omputationally e� ient general algorithms to solve su h problems.111

In pra ti e however, the solution s heme known as sequential linear programming proved tobe omputationally very e�e tive. The pra ti ally implemented version of this te hnique isknown in petroleum industry optimization as distributive re ursion [52℄.The sequential programming approa h to bilinear programming resolves the problem ofbilinear terms by �xing the unknown quality parameters and �ow ratios to some guessed val-ues. In ea h iteration, based on the urrent solution, a new, hopefully better approximationof these values are generated. This pro ess is dis ussed in detail in Se tion 5.3.1.Pooling problems re eived a onsiderable amount of interest sin e the in reasing ompu-tational apa ities made the use of mathemati al programming approa hes possible. The�rst sequential linear programming method was proposed in [37℄, alled that time as math-emati al approximation programming. Sin e then, several other authors have developedand dis ussed sequential linear programming s hemes [7, 54, 53, 99℄. These methods do notguarantee the identi� ation of a global optimal solution. Although several attempts weremade to de�ne algorithms that are either apable of �nding global optimal solutions or anapproximation of it in [7, 30, 89℄ and [90℄; these methods are not yet apable of solving realworld instan es, with often several thousands, or even tens of thousands of variables and onstraints.One of the widely used state of the art modelling system in petroleum industry, thatemploys distributive re ursion is the PIMS system [67℄ of ASPEN Te h. In . [44℄. Thissystem and its a essories o�er a omplex solution form modelling and solving to detailedreporting of the solution.The sequential linear programming te hnique used by PIMS is based on the initial valuesof the quality parameters in the model that has to be �guessed� by the user. These values areknown as PGUESS values. The solution methods are highly sensitive to the orre t hoi eof PGUESS. However, it is very demanding to �nd the proper values.A small resear h group at ELTE, onsisting of my supervisor Tibor Illés, his PhD studentMarianna Nagy and the author of this thesis, was onta ted by the experts of MOL Pl . [68℄to help to analyze the pra ti al problems arising during the solution, and to help to developmethods that may speed up the solution pro ess.The main ontribution presented here is the analysis of the model and solution pra ti e,revealing methods that may be used in pra ti e to in rease reliability, e� ien y and mayhelp to �nd feasible solutions of hard problems. Several analyti pro edures are presentedthat he k model onsisten y and help to keep the model free of redundan ies. Also, severalmethods are identi�ed that may help the experts to �nd feasible solutions even for demandingproblems. A ording to the experts, a speed up of about 30 − 40% may be a hieved withthese simple omplementary te hniques. 112

It must be noted, that our resear h group did not dire tly use the PIMS system, dueto li ensing limitations. All the experiments were run by the experts of MOL Pl ., and theresults were dis ussed through an extensive series of onsultations.The hapter is organized as follows. Se tion 5.2 summarizes some results and te hniquesused in the pra ti e of bilinear programming. Se tion 5.3 fo uses on the bilinear program-ming problem in the spe ial ase of oil re�nery optimization and the solution pra ti e usingPIMS. Se tion 5.4 presents the main ontributions, in luding the throughout analysis of themodel and solution te hniques, introdu ing several omplementary methods that may helpto in rease reliability and e� ien y. The hapter is losed with des ribing some tenden iesin the solution methods that may fundamentally in�uen e the future of petroleum industryoptimization.5.2 Bilinear programming in pra ti eSeveral real world appli ations yield bilinear optimization problems. Some examples alongwith the blending problem of oil re�neries in lude trim loss problems in paper mills where thestru ture of the ut patterns and the number of uts arried out are also de ision variables[40, 41, 76, 94, 95℄, manure management in farms whi h also in lude blending type problemswith nutrition values as qualitative variables, [12℄, hemi al pro ess ontrol problems withunknown total �ow rates and adjustment values [74℄, supply hain management problems inmulti-national ompanies, where transfer pri es between the divisions should be determinedas well as the transported amount of goods [88℄, hemi al equilibrium problems [60℄ andquadrati assignment problems [84℄. It is lear that bilinearity arises in a wide area of realworld appli ations, making it an extremely important optimization problem.The asso iated bilinear programs of form (1.4) in most pra ti al instan es are non onvex,making the development of spe ialized algorithms ne essary.The most straightforward way to solve bilinear problems is to use the widely availablegeneral nonlinear solvers like GAMS or MINOS. Unfortunately, sin e it is impossible toprepare the solvers for ea h unique problem, usually only problems of limited size or well- onditioned problems are managed su essfully. Generally, the solvers onverge only to alo al optima or may even not onverge. Therefore, it is useful to restart the algorithm fromseveral initial points and then hoose the best from the feasible solutions (optimal solution andidates). Some promising results are presented in [12, 74℄.A similar approa h is to try less exa t, but more generally appli able approximations hemes like simulated annealing or geneti algorithms. Su h methods were used in [76℄.113

However, the most widely spread method is the sequential linear programming (SLP).It is used su essfully in many real world appli ations, like in petroleum industry re�nerymodels.The general framework of SLP is presented in [8℄. However, for bilinear programmingproblems, the SLP approa h is quite straightforward. The basi idea of SLP is that thevariables in the bilinear terms are divided into two lasses (with respe t to bilinearity, so ifvariables from one lass are �xed, the obje tive fun tion and all onstraints be ome linear).In ea h su essive iteration the values of variables in one of the lasses are �xed, and a linearprogramming problem is solved. The method may be formalized for (1.4) as follows. If apoint y is �xed, the resulting problemmax cTx + xT Dy + dT y

aTi x + xT Fiy + bT

i y ≤ bi i = 1, . . . ,m (5.1)Nx + M y ≤ d,is linear, and an be solved by a linear programming solver. If the above problem is feasible,then set the solution x := x and solve the symmetri linear problem

max cT x + xT Dy + dTy

aTi x + xT Fiy + bT

i y ≤ bi i = 1, . . . ,m (5.2)N x + My ≤ d.If the problem is again feasible, set y := y and iterate until some stopping riteria is ful�lled,or an iteration limit is ex eeded.One may observe, that a ne essary ondition for (1.4) being feasible is that the linear onstraintsNx + My ≤ d (5.3)should have a solution. Problem (5.3) is usually referred to as the ore LP.Although the above approa h is natural and simple, it has some obvious disadvantages.If in any iteration, the asso iated linear program is infeasible, a model-spe i� method mustbe applied to modify the values of the �xed variables to help a hieving feasibility in the nextiteration. We must note, that on e in a subsequent iteration the problem be omes feasiblein both the x and y variables, the solutions remain feasible in any further iterations.The method of SLP is summarized in Figure 5.1.114

LP (y)solvesolve the ore LP infeasible → model is infeasible!feasible

feasibleoptimalswit h role of x and y

unbounded → model is unbounded?Che k terminating onditionsTerminate with solutioninfeasible → modify x or terminateif time limit ex eeded

LP (x)Figure 5.1: A possible SLP algorithm for bilinear programming.For general bilinear problems the approa h based on SLP an be very disappointing, asdemonstrated by the example in [12℄.max 20x1 + 70x2 + (y1 + y2)(70x1 + 95x2)

x1 + x2 ≤ 10,

x1 + 1.5x2 ≤ 12,

y1 + y2 ≤ 1,

y1(x1 + 5x2) ≤ x1, y2(x1 + 5x2) ≤ x2,

y1(4x1 + x2) ≤ 5x1, x24x1 + x2) ≤ 4x2,

y1(x1 + x2) ≤ x1, y2(x1 + x2) ≤ x2,

x1, x2, y1, y2 ≥ 0.For this example, the SLP stops after at most 2 iterations and �nds the global optimum(x,y) = (10, 0, 1, 0) only if it was started from y = (1, 0).However, the main drawba k is that the outlined pro edure has some un ertainty ininfeasible iterations. A possible approa h to over ome this is the use of penalty fun tions,making it possible to simultaneously optimize in x and y. In the method proposed in [8℄ asear h for a KKT point using �rst order approximations is organized into an SLP frameworkwith ea h subsequential linear programming problem being a dire tion �nding problem witha trust-region step size. The method is usually referred to as penalty SLP or PSLP.115

Suppose, that the urrent solution (whi h may be infeasible) is x, y, and the urrentve tor ontaining the trusted step sizes is ∆. The dire tion d = (dx,dy) �nding problemsimpli�ed to the bilinear ase may be formalized asmax cT (x + dx) + (x + dx)Dy + xD(y + dy) + dT (y + dy) − µ

(m∑

i=1

zi

)

ai(x + dx) + (x + dx)T Fiy + xT Fi(y + dy) + bi(y + dy) − bi ≤ zi, i = 1, . . . ,m

N(x + dx) + M(y + dy) ≤ d,

−∆ ≤ d ≤ ∆,

z ≥ 0,where µ is the penalty parameter and the ve tor z measures the infeasibility rate of thesolution. If (dx,dy) = (0,0) and µ was sele ted properly, then x, y is a KKT point, otherwisedx,dy is added to the urrent solution, and the trusted step sizes ∆ are modi�ed a ordingto ratio of the predi ted and a tualized hange in the penalty obje tive fun tion [8℄.As one may observe, both te hniques require the solution of many linear programs,although the size and nature of these linear programs may be very di�erent.Both methods are proven to onverge to a globally optimal solution in some spe ial ases.For example, if the obje tive fun tion is onvex (i.e. D is positive semide�nite), there areno bilinear onstraints, moreover the feasible set is de�ned by the artesian produ t of twopolyhedra (i.e. {(x,y) | Ax ≤ b, By ≤ d}), then any limit point of the SLP pro eduresglobally solve the problem [93℄.However, for general bilinear programming problems - like the majority of the pra ti alinstan es - having no favorable hara teristi s, spe ialized methods have to be developed.5.3 Bilinear optimization in the petroleum industryIn this se tion, we brie�y summarize the main on epts of the blending model, and explainhow those guessed quality parameters and the �ow ratio values are updated, whi h ausethe bilinearity.The bilinear optimization problem arising in petroleum industry may be divided into twofundamentally di�erent parts. The �rst onsists of the linear onstraints like transportationequations, resour e limits, e onomi al side onstraints (e.g. ontra ts) and other linear terms,while the se ond is typi ally bilinear and is on erned with blending and quality standards.The unknowns of the model are mainly also of two kinds: quantitative and qualitativevariables. The quantitative variables are only bounded by te hni al and e onomi al on-116

straints, while the range of the qualitative variables are de�ned by (often stri t) industrialstandards.The input parameters of the model are the pri es, the quantities and the qualities of theraw materials available on sto k or on the market, as well as the list of possible outputs,des ribed in standards by prede�ned qualities. With the aim of maximizing pro�ts (orminimizing ost), the typi al linear terms in lude apa ity and required minimal produ tion onstraints, weight balan e onstraints, volumetri balan e onstraints, utility usage (likeele tri ity or steam onsumption, not in luded in the equations for the hydro arbonids) ortransfer equations (in ase of multi-period or multi-re�nery models).However, from the optimization point of view, the most interesting onstraints are theblending onstraints. The main produ tion pro esses in a re�nery all in orporate some kindof blending. The di� ulty with blending is that the values of the qualitative variablesare expressed nonlinearly, that is the produ t of the quantitative and qualitative variablesdetermine the quality of the blend, making the asso iated onstraint bilinear.�The produ tionpro esses start with rude distillation, where rude oils are blended together and distilledinto uts or fra tions based on boiling range. The yields and qualities of these uts anbe ontrolled by operating parameters. The streams then are blended together to reatesuitable quality feeds to the various units. Intermediate streams in the re�nery are blendedtogether to make �nal produ ts that onform with produ t quality spe i� ations� [45℄.It is important to note, that the iterative solution method used to solve these bilinearprograms is not a lassi al sequential programming approa h, sin e in every linear program-ming problem solved, the quality and �ow ratio variables are �xed and only the quantityvariables are unknowns. The other variables are updated between subsequent iterations,based on the distributive re ursion, des ribed in the next se tion.5.3.1 Distributive re ursionWe now explain how the guessed quality parameters and the �ow ratio values used to dis-tribute the error terms are updated between two subsequent iterations. We use the method-ology and notations introdu ed in Se tion 2.7.Consider a re ursed row of the problem, i.e. the quality onstraints for some inner blendand given quality Qj.−qj1x1 − · · · − qjnxn + pjxF + RjF = I.In the �rst iteration of the su essive pro ess, the quality estimates pj equals the PGUESSvalues, given (guessed) by the experts. Now suppose that a solution x1, . . . , xn is known.117

Now, the error variable RjF ontains the di�eren e between the amount of the spe i� omponent � orresponding to quality Qj � �owing in to the blend, and ontained a ordingto the guess. In the next iteration, the guessed quality parameter is updated in su h away, that it tries to minimize the error term in the next iteration. In other words, the newquality parameter is su h, that if the �ow values would not hange, then the error termwould be ome zero.pupdated

j :=qj1x1 + · · · + qjnxn − RjF + I

xF

. (updating guess)To dire t the solution to a feasible region, when ne essary, these values are then adjustedto (or lose to) their feasible region. The thus arising imbalan e in the equation will besmoothed by the error ve tor. To limit the size of the error ve tors, in ea h subsequentiteration, both their lower and upper bounds (symmetri around zero) are tightened. Thispro ess may lo alize the model around an infeasible solution. To prevent this, the followingrule is applied: if in several subsequent iterations the number of quality variables inside rangedo not de rease, some (or all) of the bounds on the error ve tors are in reased signi� antly.The update of the ratios de�ning how the error variables are distributed between theblends is quite straightforward. Suppose that an inner blend (with re ursed rows des ribingits qualities) is mixed into k further blends, with �ow values of xi, i = 1, . . . , k. Then theerror distribution ratio Di for blend i is de�ned byDupdated

i :=xi

x1 + · · · + xn

. (updating error distribution)If the linear programming problem orresponding to the a tual guessed quality parame-ters and error distribution ratios is infeasible, then this pro ess is arried out with a solutionthat is as lose to a feasible one as possible. Noti e, that the �rst phase of the lassi alsimplex algorithm terminates with su h a solution on infeasible problems where the sum ofinfeasibility is minimal. This observation will be vital later, when the use of interior pointalgorithms will be dis ussed in Se tion 5.4.5.3.2 Some further properties of the modelIt must also be mentioned, that the re�nery models may also in lude trilinear terms as well[45℄. Trilinear terms may arise for example for qualities, for whi h the spe i� ations arede�ned on weight basis, instead of volumetri basis (sin e in this ase, the produ t of thequality and quantity variables must also be multiplied with the unknown spe i� gravityvalues). 118

In the onsidered bilinear model, the obje tive fun tion is linear and depends only onthe quantitative variables. The bilinear programming problem (1.4) may be spe ialized todes ribe the typi al re�nery problem asmax cTx

aix + xT Fiy ≤ bi, i = 1, . . . ,m (5.4)Nx ≤ d.

v ≤ y ≤ w.The lower and upper bounds on the qualitative variables y are presented separately toemphasize their spe ial role in the optimization. On the quantitative variables x usuallyonly nonnegativity bounds are set, while typi ally 0 < wi − vi < ǫ, that is the range of thequality variables are stri tly bounded.One may observe, that the error distribution variables do not �t perfe tly into (5.4).This is not surprising however, sin e those error variables were only introdu ed to handlethe error terms arising in imperfe t quality guesses in the linearization of the model, and areunne essary in the exa t bilinear model.The optimization of this non onvex model is NP-hard, having no spe ial properties that ould be exploited, thus global optimization te hniques are required, and the existen e oflo ally optimal solutions result in serious hallenges in pra ti e.A usual model of two re�neries and a single period onsists of about 4000 − 6000 on-straints and 7000 − 8000 variables with a onstraint matrix density of less than 0.1%. Themodel is thus very sparse, but also very degenerate: only about 1% of the onstraints havenonzero right-hand side value, and usually no more than 1% of the variables have nonzero oe� ient in the obje tive fun tion, making the model very degenerate both from primaland dual side. This degenera y may ause numeri al instability1, while also gives rise toalternate solutions even for the same lo ally optimal obje tive fun tion values.Requirements de�ned by standards often range in a very tight interval, and several so- alled tight- one onstraints exists in the model, of the forma11x1 + · · · + a1kxk ≤ 0 (5.5)a21x1 + · · · + a2kxk ≥ 0 (5.6)where ||a1 − a2|| is numeri ally small.1While dual degenera y may help a primal simplex method (and primal degenera y the dual ones) to onverge qui kly, degenera y in the right hand side may ause the phenomena of stalling for the primalsimplex method. 119

Even that the model is sparse, it still ontains tens of thousands of data entries even forthe simplest ases, many of whi h have to be modi�ed if the time-span hanges, making ithard to keep it dependently up-to-date. Some pra ti al methods that proved to be e� ientin pra ti e while tra king model hanges are presented in Se tion 5.4.The solution of the model an also be divided into two main steps. First, a feasiblesolution has to be found, whi h already prove to be very hard in pra ti e. As a se ond step,while optimizing the obje tive fun tion the best solution is to be found, that is hard due tothe several lo ally optimal feasible solutions.A omprehensive des ription of the model and the history of oil re�nery models is foundin [85℄, while several aspe ts of the urrent state of the art models and methods in the ompetitive market was presented in [45℄.5.3.3 Solution in pra ti e (using PIMS)One of the major, widely used software pa kages on the market for re�nery produ tionoptimization is PIMS [67℄. The applied optimization te hnique was fundamentally the samein the last de ade, and is based on the spe ial SLP te hnique distributive re ursion. Theevolution of the model is explored in detail in [85℄.The aim of this se tion is to provide a brief introdu tion to some modelling and pra ti alissues of re�nery optimization. The framework in whi h this is done is the use of the stateof the art modeler, Aspen PIMS.In the method based on DR2, the quality parameters in the bilinear terms arising in themodelling of blending has to be guessed apriori. After setting these initial values for thequalitative variables, usually referred as PGUESS values, the resulting model is linear in x,and a onventional linear programming solver (XPRESS-MP [98℄ in our ase) is used to solveit. The hoi e of PGUESS values is possibly the most vital part of the optimization. Slightlydi�erent hoi es may lead to di�erent lo al optimal solutions, or may result in feasible orinfeasible models.After setting the PGUESS values and the parameters of PIMS (like the sele tion of thelinear programming algorithm used by XPRESS-MP), the orresponding linear program issolved. Sin e the guessed quality parameters usually yield an infeasible starting model, theDR te hnique is used to obtain new qualitative variables. These new qualitative valuesmay be outside the range given by the standards, referred in PIMS as properties not within onvergen e toleran e. If so, after optimizing the linearized model in the qualitative variables2It must be noted, that the distributive re ursion is not mu h dis ussed in the literature. This des riptionof DR is mainly based on the experiments of the experts at MOL, and the resear h team.120

Basic modelingcollection of data, converting to PIMS

PIMS - Modeling, model generation

- Parameter settings (PGUESS etc.)

- Choice of algorithm (primal, dual, Newton-barrier)

- Options for the algorithm (pricing strategies etc.)

- LP generation

XPESSMPSolution of the LP problem

PIMS:

The LP solution

is optimal

Solution documentation

PIMS: - Distributive recursion

- Modifying (some) data

- Building a new LP problem

yes

no

Figure 5.2: Solution methodology using PIMS.x, new quality variables y values will be generated (as well as the error distribution ratiosbased on x), until both the linearized model is optimal, and all qualitative variables arein range, or no improvement is a hieved. This pro ess is referred as a re ursion. Onere ursion onsists of the solution of the linear program with �xed qualitative variables, andthe modi� ation of the quality parameters and error distribution rations based on the urrentsolution.Figure 5.2. summarizes the main steps of optimization using the PIMS framework. First,the de�nition of the main model des ribing the physi al stru ture of the re�nery, and thespe i� ation of input and output onditions and requirements take pla e. As a omplemen-tary, but equally important step, the model data is olle ted and imported to PIMS. Thelarge size ombined with the frequent hanges in the model data makes it a rather hal-lenging task to keep all information in the model up-to date, and to tra k possible mistakesnaturally arising with su h large models. A ording to our experien es, an automated model he king greatly help to in rease reliability.The hanges of the qualitative variables throughout the re ursions in the qualitativevariables are saved in a log �le. The typi al size of a set of PGUESS values is around 1000-4000 entries, usually organized to a sparsely �lled tableau of 100-300 rows and around 100 olumns, and only around 30%−60% of these values are hanged in a typi al run. Moreover,only a few per entages of them hanges heavily or several times during the re ursions.121

If after some re ursions (it is very rare and pra ti ally nearly impossible that the �rstlinearization of a real word model is optimal), the obtained model is feasible with everyqualitative variables in range, then the re ursion is stopped. The example of Table 5.1 sum-marizes the hange of the properties not within onvergen e toleran e for a relatively simpleproblem. The olumn denoted by Re. ontains the re ursion number. To de rease size,only every se ond iteration is presented. The olumn Its presents the number of iterationsmade by the primal simplex method to solve the linear programming problem, while in thenext olumn the per entage of the a tual obje tive fun tion is shown, with respe t to thefound �nal solutions obje tive fun tion value. The last three olumns show the number ofqualitative variables outside the bounds, de omposed to the three submodels of the model.The solution of this well behaving, single period model took 83 re ursion, that means thesolution of 83 linear programs with altogether 62723 simplex iterations, and took about 19minutes on a 3000 MHz omputer.Several notes are due at this point. First observe, that with the ex eption of the �rstiterates, the obje tive fun tion value doesn't hange mu h, and shows an in reasing tenden y.A ording to the experien es, this is the typi al ase when the number of qualitative variableoutside the bounds de reases steadily, and may be onsidered to the fa t that the qualitativevariables don't dire tly a�e t the obje tive fun tion in (5.4). However, in di� ult ases,the number of quality variables outside range may �u tuate heavily, and in su h ases theobje tive fun tion may also hange signi� antly. Su h example ases will be presented inSe tion 5.4.Although the number of qualitative variables outside range �u tuates sometimes, theirsum shows a de reasing tenden y. It must be mentioned, that for hard models, it's typi althat the sum of the infeasible qualitative variables de rease for some iterations, then suddenlyjumps up, then begins de reasing again, and the phenomenon is repeated several times.Moreover, if the number of re ursions is higher than 100 (for models of this size, largermodels may have more re ursions), and still no feasible model is a hieved, then pra ti esuggest that no feasible solution, or only a very poor feasible solution will be obtained bythe method. Usually, the solution pro ess is terminated in su h ases, and is started againafter some modi� ations in the input data.We should note that a ording to the experien es, it may happen only a few per entage ofthe PGUESS values are modi�ed during the total optimization phase. It may however meantwo ompletely di�erent things. Either, those qualitative parameters that do not hangemay be properly and optimally set, or the model may be heavily lo alized around a lo aloptima or even around an infeasible region.Two kinds of infeasibilities are distinguished. The �rst one is when the linear program122

Qualitative varsoutside rangeRe. Its. Obj. A B C1 2481 83.83% 2453 547 283 499 97.77% 1446 241 135 371 98.61% 704 89 147 831 99.10% 902 157 89 8576 99.43% 785 166 811 1140 99.44% 531 133 613 1709 99.38% 441 81 52215 2880 99.27% 358 90 817 4818 99.23% 177 67 24419 1635 99.25% 184 60 621 3433 99.40% 197 70 823 625 99.66% 330 160 825 2810 99.87% 264 117 827 2652 99.90% 260 148 629 1630 99.94% 205 128 231 570 99.95% 181 81 26233 2963 99.95% 114 54 16835 830 99.95% 117 53 437 1404 99.94% 96 33 239 1148 99.94% 72 27 041 1835 99.94% 80 33 113

Qualitative varsoutside rangeRe. Its. Obj. A B C43 804 99.95% 78 23 10145 2114 99.95% 65 5 7047 909 99.94% 69 21 449 633 99.95% 74 26 351 1500 99.95% 44 14 253 1446 99.95% 53 25 7855 817 99.95% 53 23 7657 547 99.96% 63 17 8059 2217 99.96% 53 13 061 2229 99.96% 34 20 063 2401 99.96% 29 14 065 691 99.96% 20 18 067 82 99.96% 30 31 069 72 99.96% 30 32 071 51 99.96% 41 8 073 0 99.96% 13 12 075 23 99.96% 17 2 077 26 99.96% 11 3 079 29 99.96% 17 7 081 40 99.96% 13 9 083 0 100.00% 0 0 0Table 5.1: Re ursion passes in a relatively simple ase, that yields a feasible linear programin the �rst iteration.with �xed qualitative variables is infeasible, while the se ond is the infeasibility of the qualityparameters y. Usually the optimization begins with �nding su h qualitative values y forwhi h the linear program be omes feasible. On e the linear programs to be solved arefeasible, the aim of the re ursions is to make the qualitative variables feasible, while keepingthe model feasible in variables x.The experts' prime tool in handling infeasibility and lo al optimal solutions is to hangethe PGUESS values, and rerun the modi�ed model. The time load for a single optimization an be hours even in the simplest ases. Unfortunately, after an optimization that yielded an123

infeasible model only, PIMS o�ers no e� ient help in how to modify the PGUESS values, sothe hanges in the PGUESS has to be done a ording either to the experien es of the experts,or by statisti al learning based on previously found promising PGUESS values. Althoughthe hanges in the qualitative variables starting from the initial PGUESS values are reportedin a log-�le during the solution pro ess, the analysis of the hanges is not in luded in PIMS.Another possible way to alter the solution pro ess and the solution is to hange the pa-rameters of the linear optimization ode used. The sele tion among the linear programmingmethods like the primal simplex, dual simplex or Newton-barrier methods, or even only the hoi e of the parameters of the methods like rash heuristi s or pri ing strategies, the numberof Newton-barrier iterations before rossover usually lead to signi� antly di�erent solutions.Usually, simply by permuting the rows or olumns in the des ription of the model3 ausesthe deterministi solvers to generate di�erent solutions.Although it is possible to run PIMS with di�erent parameters automati ally with the so- alled CASE option, the analysis and sorting of the results has to be done manually. Runningwith di�erent settings, PGUESS values or with permuted data all take the deterministi algorithms to possibly di�erent solutions, greatly assisting the sear h for a feasible solution,or in exploring the set of lo al optimal solutions.For multi period models, it is possible to provide di�erent sets of PGUESS values for ea hperiod. Sin e the initial inventory di�ers in ea h period, di�erent lo alizations of PGUESSvalues are required to be able to su essfully solve the model, greatly in reasing the numberof initially infeasible quality variables.The optimization is losed by a detailed reporting of the solution. The whole pro essmay be des ribed as a modelling, solving and reporting framework. However, it is possibleand very useful to extend it with some extra analyti features, as shown in the next se tion.5.4 Improving the solution te hniqueThis se tion is a olle tion of observations, te hni al notes and some ideas that improvedthe quality or speed of the solution pro ess in pra ti e.The two main pra ti al lasses of optimization methods for linear programming problems,are the lass of simplex like, and the lass of interior point methods. A key question of whatfollows will be the phenomenon of primal and dual degenera y. For the simplex method,highly degenerate basis may ause su h simplex iterations, where the obje tive fun tion re-mains un hanged. This phenomenon is the main reason of pra ti al y ling (sin e numeri al3This approa h was not supported by the version of PIMS, with whi h this study was arried out.124

onsiderations are prior to index sele tion rules in pra ti al implementations [57℄). The mainimpli ation of dual degenera y is that multiple optimal solutions may o ur. While degen-era y is not an issue for interior point methods, they are often harder to apply to infeasibleproblems [43℄.Subse tion 5.4.1 deals with the general stru ture and properties of the model, whileSubse tion 5.4.2 onsists of analysis on erning the optimization phase.All the analysis in luded here are based on sample models provided by MOL, and weredone with a software written by our resear h group in Visual C++.5.4.1 Analysis of the modelThis se tion tries to extend the framework of PIMS with analyti steps. A ording to theexperien es obtained by the experts, these simple methods in reased the numeri al stabilityof the model, and lead up to 30 − 40% speed up in the total time spent on solving themodels. All analysis presented here is based on the initial linear programming problem withthe original PGUESS values, and the �nal linear programming problem obtained by PIMS,both of them made available by PIMS in MPS format.The �rst step inserted to the pro ess is analyzing the model stru ture before attempting tosolve it. The aim of this synta ti al like he king is to �nd those mistakes, that usually arisein the ase of large models with frequent data or stru ture (or both) update requirements.• Dupli ate onstraints are unne essary in any optimization problem, or more pre isely,adding the same onstraint twi e or several times is equivalent with adding it onlyon e form modelling aspe ts. However, omputationally, dupli ate onstraints maymake the problem harder, as it was illustrated in [24℄. Sin e today's state of the artlinear programming solvers usually eliminate su h redundan ies, the main reason of he king for these onstraints is to draw the expert's attention to it, and thus he k theintended purpose of the onstraint. A possible sour e of this phenomenon is opyingthe onstraints for minor modi� ations, but a identally leaving it un hanged.• Rows in the onstraint matrix, where the ratio of oe� ients are very large may ausenumeri al problems while solving the linear programming problem, espe ially for thesimplex based algorithms. Nearly every simplex based solver start with � alling� therows of the matrix, su h reating very small numbers if there are large s aling dif-feren es. Both the simplex and the dual simplex method is sensitive to very smallnumbers. If possible, this should be avoided by either arti� ial res aling, or by adjust-ing units. 125

Basic modelingcollection of data, converting to PIMS

PIMS: Modeling, parameter settings, options

XPESSMPSolution of the LP problem

PIMS:

The LP solution

is optimal

PIMS: Distributive recursionno

yes

LP generation: cases

Model analysis

Consistency checks

Reporting and

classifying changes

Solution analysis

parameter resetting

Back looking

analysisPIMS: Solutions documentation

Figure 5.3: The extended solution methodology using PIMS.• Empty onstraints do not e�e t the model, but their existen e should be noti ed and he ked. They may be forgotten onstraints intended to delete, or onstraints intendedto build into the model.• A onstraint is alled all zero onstraint if it an only be satis�ed if all its variables withnonzero oe� ient are at zero level. These onstraints are typi ally like the following: oe� ients are positive and the variables are sign restri ted, with less or equal to zeroright-hand side. Su h onstraints are not redundant, but �x several variables at a zerolevel, and the intended purpose should be he ked.These onsisten y he ks proved to be an e� ient omplement to PIMS own warningsystem. The main steps, where model analyti investigations may be inserted to the solutionpra ti e is summarized in Figure 5.3.5.4.2 Analysis of the optimization phaseAt the beginning of the solution of a new model, the analysis of the optimization phasefo uses on the PGUESS values and methods to alter the run of the SLP method to lo alizefeasible solutions. After a feasible solution was found, the analysis is on erned with alternatesolutions, and with exploring the stru ture of possible solutions. However, the primary goalof the optimization is to �nd a feasible model.126

As we have already mentioned, the linear programming subproblems are both highly pri-mal and dual degenerate. While primal degenera y auses numeri al problems espe ially forthe simplex based methods, dual degenera y means that there may be alternative solutionsfor the �nal linearization of the model.The main aim of the analysis of the optimization phase is to olle t data on erninggood PGUESS values, and to identify solution parameters that help the experts to omputefeasible solutions.Lo alizing a feasible solutionThe simplest possible ase, when the PGUESS was hosen in the way that the initial linearprogram is feasible, and the qualitative variables y have su essfully onverged to theirfeasible range was already dis ussed in Table 5.1 in the previous se tion.If the initial PGUESS does not yield a feasible solution for the �rst run, the most straight-forward way to try to �nd a feasible solution is to alter the ourse of the SLP method. This an be done with sele ting di�erent linear optimization methods like dual simplex or theNewton-barrier method, or to set new parameters for the solvers. Sin e the linear program-ming subproblems are usually highly dual degenerate, these modi� ations will yield optimalsolutions with di�erent stru tures, and will alter the ourse of the SLP, sin e it will resultin di�erent quality parameter hanges.One of the desirable ases is when although the starting linear programming model isinfeasible, but after some re ursions a feasible linear program is identi�ed, and the propertyparameters onverge to their feasible range. Unfortunately, it does not happen for the �rstmodel in most of the ases.At the beginning of planning, before the PGUESS values are lo alized by the expertsaround a favorable feasible region, the typi al run onsists of either infeasible linear programs,or PIMS is unable to make every qualitative variable feasible, and gives the solution up. Itmay also happen, that although the quality values onverge to their feasible range, theasso iated linear programs remain infeasible. In su h ases, the SLP method stu k in aninfeasible lo al neighborhood of the solution, and the solution pro ess is terminated. Su h a ase is presented in Table 5.2.In su h ases, the qualitative variables of the model has to be pushed into a new regionof attra tion. This may be done by resetting the initial PGUESS, or by trying to make morede�nite modi� ations of the qualitative variables during the re ursions.PIMS do uments the hanges made to the model in its iteration log. From this �le,after an infeasible run, those quality values may be identi�ed that were hanged frequently,127

Qualitative varsoutside rangeRe. Its. Status A B C1 659 infeasible 582 210 82 397 infeasible 711 146 93 625 infeasible 607 126 114 249 infeasible 509 107 85 209 infeasible 286 46 96 639 infeasible 254 87 27 638 infeasible 469 139 88 311 infeasible 540 139 89 57 infeasible 441 74 1110 239 infeasible 373 45 8

Qualitative varsoutside rangeRe. Its. Status A B C11 903 infeasible 606 118 812 235 infeasible 586 139 813 56 infeasible 555 76 814 143 infeasible 400 59 415 16 infeasible 413 69 816 1 infeasible 275 11 28617 1 infeasible 206 19 22518 0 infeasible 174 19 019 0 infeasible 3 1 020 0 infeasible 0 0 0Table 5.2: Re ursion passes where the qualitative variables onverge to their feasible range,but the asso iated linear program remains infeasible.and remained infeasible. The PGUESS values and onstraints asso iated to these qualityparameters an thereafter be given spe ial are, when trying to modify the initial values andsettings to �nd a feasible model. Moreover, as experien es show, while after a simplex basedoptimization phase the quality values are modi�ed relatively slightly, the values are modi�edquite dramati ally after a Newton-barrier based iteration.It is natural to ask, why a Newton-barrier based iteration a ts like that. The answer isnot surprising however, and in reality, it has nothing to do with the method itself, but howit's applied. In Se tion 5.3.1, we have seen that for an infeasible linearization, it is naturalto use a �solution� as lose to feasibility as possible. On the other hand, the Newton-barriermethod used in PIMS is an infeasible interior point method. As for su h methods, thesolution diverges to in�nity on infeasible linear programs. While applying su h a solution inthe distributive re ursion makes no sense whatsoever, it shakes (although quite randomly)the guessed values of the model4. It must be noted, that the ross-over phase after theNewton-barrier method somewhat dampens this e�e t.Table 5.3 ompares the number of hanges in the qualitative variables y in the �rst 10iterations. In both ases, exa tly the same parameters and initial PGUESS values wereused, the only di�eren e that in the se ond run, in the �rst 5 re ursions the Newton-barriermethod was used.4This example shows how even a mistake may sometimes be exploited.128

Re ursion passes with simplex methodSubmodel 1 2 3 4 5 6 7 8 9 10A 1410 1505 1122 1007 1036 833 735 730 641 494B 750 584 376 298 260 207 120 166 89 128C 27 30 18 25 15 5 4 6 0 0Re ursion passes with 5 Newton-barrier re ursionsSubmodel 1 2 3 4 5 6 7 8 9 10A 4620 3287 4700 5480 5754 3212 3101 1821 1932 1495B 1423 1208 1233 2282 2579 1369 623 520 452 326C 94 94 100 100 105 105 42 30 34 29Table 5.3: The number of qualitative variables hanged with and without the use of Newton-barrier based re ursions.30

30.5

31

31.5

32

32.5

33

33.5

34

34.5

35

1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76Figure 5.4: The hanges of a sele ted qualitative variable in a run in luding 20 Newton-barrier iterations.While the large modi� ations of the Newton-barrier method usually don't result in qui k onvergen e on hard problems, they help to move the a tual solution away from a bad lo aloptima, or form an infeasible region. This phenomenon is illustrated by Table 5.4. The basemodel was the same as in the ase of Table 5.2 in whi h no feasible solution was identi�ed.The olumn Cpl. presents the omplementarity of the �nal Newton-barrier solution, whilethe olumn St. ontains the status of the solution. The total running time was 52 minuteson a 3000 MHz omputer.It's possible to set the number of iterations where Newton-barrier is applied in PIMS. Inthis run, it was set to 100, but mu h shorter sequen es are also frequently useful (sin e for129

Newton- Qualitative varsbarrier outside rangeRe. Its. Cpl. St. A B C1 97 106 inf. 5462 1621 793 23 109 inf. 5521 1487 835 20 108 inf. 6022 1416 587 35 109 inf. 5650 2031 269 19 108 inf. 5482 2576 2611 89 1010 inf. 5054 2364 2613 108 1011 inf. 5412 2971 4715 21 108 inf. 5803 3294 55

Newton- Qualitative varsbarrier outside rangeRe. Its. Cpl. St. A B C17 62 1012 inf. 5514 3289 3119 32 106 inf. 5831 3234 6921 147 107 inf. 2578 624 2623 129 1012 inf. 2304 466 2725 108 106 inf. 4995 1072 7527 134 1012 inf. 2401 461 2829 42 1010 inf. 5509 1213 8431 14 106 inf. 6191 2940 58Qualitative varsNewton-barrier Primal-Szimplex outside rangeRe. Its. Obje tive Cpl. Status Its Obje tive Status A B C33 49 99.10% 10−5 rossover 938 99.10% optimal 3619 1141 2841 47 99.98% 10−3 rossover 925 99.98% optimal 834 82 849 57 99.98% 10−3 rossover 18948 99.98% optimal 221 117 681 44 100.00% 10−2 rossover 1391 100.00% optimal 77 35 389 32 100.00% 10−1 rossover 972 100.00% optimal 50 19 6999 45 100.00% 10−2 rossover 663 100.00% optimal 35 32 0Qualitative varsoutside rangeRe. Its. Obje tive A B C100 82 100.00% 30 31 0102 72 100.00% 30 32 0104 51 100.00% 41 8 0106 0 100.00% 13 12 0108 23 100.00% 17 2 0110 26 100.00% 11 3 0112 29 100.00% 17 7 0114 40 100.00% 13 9 0116 36 100.00% 1 10 0

Qualitative varsoutside rangeRe. Its. Obje tive A B C118 10 100.00% 0 8 0120 25 100.00% 3 3 0122 25 100.00% 0 3 0124 25 100.00% 0 7 0126 25 100.00% 2 10 0128 25 100.00% 0 7 0130 25 100.00% 0 2 0132 25 100.00% 0 5 0136 0 100.00% 0 0 0Table 5.4: The Newton-barrier method helps to shake infeasible quality parameters.130

infeasible linear programs, the hange in the guessed parameters is rather random, even afterthe ross-over phase). For feasible problems however, Newton-barrier method is typi allyfaster, and generally more robust than simplex based methods.The hanges in the values for a sele ted qualitative variable are presented in Figure5.4, where the �rst 20 iterations orrespond to hanges made after a Newton-barrier basediteration, while all later hanges orrespond to simplex based iterations. Where no value isindi ated, no hanges in the value of the sele ted qualitative variable are made.As experien e shows, re ursions longer than 150-200 re ursions (based on model size)should generally be avoided, sin e even if they onverge to a lo ally optimal solution, theirquality is usually very low. Based on this observation, the optimization should usually bestopped after 200 re ursions, and rerun with di�erent settings. It may happen in extreme ases, that the SLP based method seems to y le. There are even examples where the simplexalgorithm seems to y le, usually in a ross-over phase of the Newton-barrier method.Qualitative varsoutside rangeRe. Its. Status A B C5 7925 infeasible 780 218 87 253 infeasible 460 130 99 168 infeasible 507 114 1011 613 infeasible 224 45 013 78 infeasible 609 131 815 228 infeasible 340 118 817 2 infeasible 214 19 218 0 infeasible 161 19 019 1 infeasible 3 1 020 1 infeasible 157 19 0

Qualitative varsoutside rangeRe. Its. Status A B C21 0 infeasible 157 19 022 1 infeasible 1 1 023 1 infeasible 157 19 024 0 infeasible 157 19 025 1 infeasible 1 1 0. . .296 1 infeasible 157 19 0297 0 infeasible 157 19 0298 1 infeasible 1 1 0. . .Table 5.5: SLP seems to y le. The �rst 4 iterations in luded Newton-barrier runs.Although it is rare, it sometimes happens that the SLP method seems to y le. Su h anexample is presented in Table 5.5.On the interior point methods and the distributive re ursionWe have seen in the previous se tion that the possibilities in the Newton-barrier methodare not fully exploited in the investigated version of PIMS. There are two straightforward131

approa hes how to in orporate the favorable speed and robustness of the NB methods intothe solution framework based in distributive re ursion.The �rst is to embed the infeasible models into an always feasible one, and then minimizethe sum of infeasibilities. This approa h is just the same as the lassi al �rst phase of thesimplex method, but with the Newton-barrier method.The se ond approa h is based on a sophisti ated stopping riteria for the applied in-feasible Newton-barrier method. Typi ally, on an infeasible problem, before the sequen eof solutions began to diverge, a �u tuating e�e t may be observed, i.e. the quality of thesolution measured in omplementarity, infeasibility and duality gap has lo al minimas. Su h�u tuating may be observed in Table 5.6.It would be more appropriate for the distributive re ursion to stop and use an interme-diate iterate of the Newton-barrier method, rather than the diverged �nal solution. Alsonote, that the ross-over phase is unne essary, sin e the distributive solution may use anysolution, not only basi solutions.5.5 Future tenden iesThe existen e of the lo al optimal solutions with the rigorous numeri al di� ulties leadthe leading ompanies working in re�nery optimization to develop new methods, aiming tolo alize global optimal solutions. Several extension aiming this goal was built into the newPIMS-ONE system, as explained by [45℄. The main ornerstones of the new methodology issummarized here.As a �rst step, a multi�start strategy was in luded into the system to explore a majorityof the lo al optimal solutions in the sear h for the best one. The method �rst generate amesh of starting points (PGUESS values) over the bounds des ribed by the standards (whilede�ning sharp bounds help to de rease the number of initial points, they also stri tly limitthe sear h spa e). The number of these points are redu ed by the following steps:• Be ause of the regions of attra tion (in a vi inity of a lo al optima, all starting points onverge to the same limit point), from initial points lose to ea h other in somemetri s, at most one is kept.• Those points that belongs to an infeasible model, or refer to a solution with a verypoor obje tive fun tion are dis arded (whi h means that only one LP is solved withthe �xed initial quality parameters).This approa h is very similar to those that was used manually with DR.132

Its. P.inf D.inf U.inf P.obj D.obj Compl.0 1.18E+06 9.57E+01 2.79E+04 8769544% 2909585% 3.60E+1010 1.73E+00 2.11E-01 4.08E-02 1.79% 572.39% 3.00E+0520 3.57E-03 5.64E-04 8.43E-05 99.25% 100.88% 9.50E+0230 1.21E-04 1.07E-04 2.86E-06 99.93% 100.06% 6.70E+0140 7.95E-05 9.77E-05 7.69E-07 99.93% 100.06% 6.30E+0150 6.41E-06 3.31E-05 6.15E-08 99.93% 100.06% 9.50E+0052 5.76E-06 2.92E-05 5.26E-08 99.93% 100.06% 8.50E+0053 5.32E-06 2.25E-05 4.38E-08 99.93% 100.06% 7.20E+0054 4.77E-06 2.07E-05 3.82E-08 99.93% 100.06% 6.60E+0055 4.76E-06 2.07E-05 3.80E-08 99.93% 100.06% 6.70E+0057 4.62E-06 1.85E-05 3.33E-08 99.93% 100.06% 9.20E+0059 5.30E-06 1.99E-05 2.86E-08 99.93% 100.06% 1.40E+0160 5.83E-06 2.33E-05 2.61E-08 99.93% 100.06% 1.90E+0170 9.46E-06 1.04E-05 4.66E-09 99.93% 100.06% 7.40E+0080 3.24E-05 8.38E-06 2.79E-09 99.93% 99.93% 8.50E+0090 2.10E-05 3.22E-05 2.56E-09 99.93% 99.93% 2.10E+01100 4.12E-05 3.84E-05 2.33E-09 99.93% 99.79% 5.80E+01110 2.54E-04 1.56E-04 1.86E-09 99.79% 99.38% 1.60E+02130 6.48E-04 3.91E-03 6.98E-10 88.91% 68.38% 2.20E+04140 8.00E-03 1.71E-01 9.31E-10 54.65% -489.46% 1.40E+05150 1.65E-02 3.20E+01 2.33E-10 -1454.54% -82800% 1.40E+07160 2.30E+00 2.09E+04 2.33E-10 -244731% -1.48E+08% 1.30E+10169 6.41E+00 1.01E+07 2.33E-10 -4296397% -9.2E+10% 4.90E+12Table 5.6: The �u tuation of the interior point solutions for infeasible models.The ore LP of a model is the linear part of it, without the bilinear onstraints. Thefundamental role played by this submodel is that if it's infeasible, that implies modellingproblems, and su h models should arefully be revised before a solution attempt is made. Inthe method proposed by [45℄ the following steps are iterated. Let us suppose that an initialpoint (PGUESS) is given.• Take the ore LP, and sele t some of the bilinear onstraints, based on the urrentsolution (take those for example that are violated). Take the LP relaxation envelopeof the sele ted bilinear onstraints a ording to (5.7). Solve the resulting LP. Thissolution provides an upper bound on the optimal obje tive value the original problem.133

LP onvexi� ationLP onstraintsNLP onstraintsNLP te hniqueSLP or DR?

(partly)tighten bounds

starting pointupper boundCore LP

Figure 5.5: The �ow hart of the algorithm.• Solve the NLP problem (using for example the original SLP or DR te hnique), usingthe solution of the LP relaxation as an initial point.• In the �nal step, the bounds on both the quality and quantitative variables are tight-ened, based on observations on the properties of mixing, and on standard LP te h-niques.The aim of sele ting only a set of the bilinear onstraints in the �rst step is to ontrol thesize of LP to be solved. The method is iterated, until either no improvement is a hieved, orthe best known upper bound is su� iently lose to the obje tive value of a feasible solution.The sket h of the algorithm is presented in Figure 5.5.One of the major modi� ations in use that a�e t the experts, is that stri t lower and upperbounds are required on the quality parameters for the method to work e� iently. Good lowerand upper bound simultaneously improve the approximation of the onvexi� ation, and helpde reasing the sear h spa e for the multi-start algorithm.5.5.1 Convex envelopes used in re�nery modelsConvex envelopes are also widely spread in appli ations, and are often ombined eitherwith bran h and bound te hniques [12℄ or in orporated into a more omplex algorithmi s heme. Su h a onvex envelope for oil re�nery appli ations is presented in [45℄. Considerthe bilinear term w := q · x, where q, x ∈ R and lq ≤ q ≤ uq and lx ≤ x ≤ ux. Then the134

following inequalities hold [58℄,[59℄:An example of the approximation.

w ≥ lqx + lxq − lxlq

w ≥ uqx + uxq − uxuq

w ≤ uqx + lxq − lxuq

w ≤ lqx + uxq − uxlq

(5.7)Similar inequalities may be derived for trilinear terms, also important in appli ations asshown in [62℄. Similarly to the other methods, the vital question is the sharpness of thebounds on the variables, determining the approximating value of the onvexitif ation.Sin e every reformulation of the non onvex bilinear programming problems either in reasethe size of the model, or in ludes onvexi� ation relaxations, stri t lower and upper boundson variables involved in bilinear terms greatly in rease the e� ien y of the proposed methods.5.6 Con lusionsIn this Chapter, we have presented omplementary analyti te hniques that in rease thee� ien y and reliability of the urrent solution te hniques used in oil re�nery optimization.The number of re�neries, as well as the number of onstraints and variables in a single re�n-ery model in rease steadily. This in rease makes the presented omplementary te hniquesne essary, in order to de rease the e�e t of the arising omputational di� ulties.Although we have on entrated on the lassi al sequential linear programming approa h,the new te hniques to be introdu ed in the pra ti e in the future will admit similar numeri aldi� ulties, and su h omplementary tools are likely to be of high value in the optimizationpro ess in the future as well.Spe ial a knowledgementWe would like to express our gratitude to the leaders and experts of the SCM Group atMOL Pl ., who provided fundamental help and support in the elaboration of the resultspresented in this hapter. Spe ial thanks are due to Béla Kelemen, János Györ�, MarianSiroky, Gábor Tolvaj, Tibor Bernáth and Csaba Mészöly.

135

SummaryThe thesis is on erned with pivot based solution te hniques of the three lassi al prob-lems of linear optimization: linear feasibility problems, linear programming problems andlinear omplementarity problems. As an important real life appli ation, the blending prob-lem in the petroleum industry is presented.After a short histori al survey and the des ription of the stru ture of the thesis, the linearfeasibility problem is onsidered. From the analysis of a variant of the monotone build upsimplex method � originally developed for linear programming problems by Anstrei her andTerlaky � a new upper bound is dedu ed for the iteration number of the algorithm undera weaker degenera y on ept. For degenerate problems, a new primal-dual type re ursivemethod, that �ts well with the MBU algorithm, is presented.It is proved, that the new upper bound for the iteration number generalizes easily to theoriginal MBU algorithm for linear programming problems, as well as to the �rst and se ondphase of the simplex method.The riss- ross algorithm is generalized for the linear omplementarity problem with�exible pivot sele tion rules (like LIFO or MOSV) to the lass of su� ient matri es. Thismatrix lass is the widest lass of matri es for whi h riss- ross type algorithms are �nite.Unfortunately, there is no e� ient algorithm known to he k whether a matrix is su� ient ornot. The algorithm is generalized in the sense of EP theorems. Su h a modi�ed algorithm is apable of pro essing any linear omplementarity problem, and it either solves it or providesa polynomial erti� ate that the matrix of the problem is not su� ient.For ea h pivot algorithm presented in the thesis, it is shown that degenera y may behandled with the new on ept of s-monotone pivot sele tion rules. This pivot sele tion ruleallows the simultaneous �niteness proof of several pivot sele tion rules, like the MOSV orLIFO, or the lassi al minimal index rule. The �exible pivot sele tion rules may often allowto avoid numeri ally instable pivots.As a real life appli ation, the same aspe ts of the blending problem typi al in thepetroleum industry is presented at the end of the se tions dealing with linear feasibilityand programming. These des riptions and numeri al results are brought together in thelast hapter, by presenting several aspe ts of real life petroleum re�nery optimization. Thenumeri al hallenges fa ed are presented, and several methods that improved the solutionpra ti e are introdu ed.

ÖsszefoglalóA tézis a lineáris optimalizálás három f® klasszikus problémájának, a lineáris megenge-dettségi, a lineáris programozási és a lineáris komplementaritási feladatnak pivot algorit-musokkal történ® megoldásával foglalkozik. A lineáris optimalizálás egy fontos gyakorlatialkalmazása, az olajipari keverési, illetve termelés tervezési feladata szolgál alkalmazási pél-daként.A történeti áttekintés és a dolgozat struktúrájának ismertetése után a dolgozat rátéra lineáris megengedettségi feladat megoldásának vizsgálatára. Az Anstrei her-Terlaky féle,eredetileg lineáris programozási feladatokra kifejlesztett monoton build-up (MBU) szimplexalgoritmusának (egy variánsát fejlesztettük ki megengedettségi feladatra) elemzéséb®l, egya szokásostól gyengébb degeneráltsági feltétel mellett a klasszikustól eltér® fels® korlátotkaptunk a lehetséges lépésszám maximumára. Degenerált feladatok kezelésére bemutattunkegy primál-duál jelleg¶ rekurzív algoritmust, mely jól illeszkedik az MBU algoritmushoz.Megmutattuk, hogy a lépésszámra adott új korlát, az eredeti lineáris programozási MBUalgoritmusra, és a szimplex algoritmus els®, illetve második fázisára is általánosítható.A lineáris komplementaritási feladatok megoldására szolgáló riss- ross algoritmus �ex-ibilis (mint amilyen a LIFO vagy a MOS) index választási szabállyal ellátott változatánakegy általánosítását adtuk az egyik legb®vebb mátrixosztályra, melyre az riss- ross típusúalgoritmusok még végesek. Ez a mátrixosztály az elégséges mátrixok osztálya. Sajnálatosmódon, nem ismert hatékony eljárás az elégségesség ellen®rzésére, azonban az algoritmusáltalánosíthatónak bizonyult az úgynevezett EP tételek szellemében. Az így általánosítottalgoritmus tetsz®leges lineáris komplementaritási feladatra elindítható, és vagy megoldja azt,vagy polinom méret¶ bizonyítékát szolgáltatja annak, hogy a feladat mátrixa nem elégséges.Mind a lineáris megengedettségi, mind a lineáris programozási feladatra bemutatott pivotalgoritmusokra igaz, hogy a degenerá iót képesek az általunk bevezetett, úgynevezett s-monoton indexválasztási szabályok segítségével feloldani, illetve a bemutatott általánosított riss- ross algoritmus végességének biztosítására is s-monoton indexválasztási szabályt alka-lmaztunk. Ezen index választási szabály több ismert szabály egyidej¶ kezelését teszi lehet®vé,mint például a MOS vagy a LIFO index választási szabály, illetve akár a klasszikus min-imálindex szabályt is. A �exibilis indexválasztási szabályok, számos esetben biztosíthatjáka numerikusan instabil bázis serék elkerülését.A lineáris optimalizálási feladatokkal foglalkozó fejezetek végén, egy, az olajiparban jel-legzetes keverés, termelés tervezési feladat révén adtunk példát lehetséges alkalmazásokra.Az alkalmazási példákat az ötödik fejezetben foglaltuk egységes keretbe, melyben az ola-jipari keverési, illetve termeléstervezési feladatok megoldása során felmerül® lineáris optimal-izálási feladatok megoldásának gyakorlatát, a felmerül® nehézségeket, és azok egy lehetségeskezelését mutattuk be.

Bibliography[1℄ I. Adler and N. Megiddo. A simplex algorithm whose average number of steps isbounded between two quadrati fun tions of the smaller dimension. Journal of theAsso iation for Computing Ma hinery, 32(4):871�895, 1985.[2℄ I. Adler, N. Megiddo, and M. J. Todd. New results on the average behavior of sim-plex algorithms. Ameri an Mathemati al So iety. Bulletin. New Series, 11(2):378�382,1984.[3℄ A. A. Akkele³, L. Balogh, and T. Illés. New variants of the riss- ross method forlinearly onstrained onvex quadrati programming. European Journal of OperationalResear h, 157(1):74�86, 2004.[4℄ K. M. Anstrei her and T. Terlaky. A monotoni build-up simplex algorithm for linearprogramming. Operations Resear h, 42(3):556�561, 1994.[5℄ C. Audet, J. Brimberg, P. Hansem, S. Le Digabel, and N. Mladenovi¢. Pooling problem:Alternative formulations and solution methods. Management S ien e, 50:761�776,2004.[6℄ D. Avis and V. Chvatal. Notes on bland's rule. Mathemati al Programming Study,8:24�34, 1978.[7℄ T.E. Baker and L.S. Lasdon. Su essive linear programming at Exxon. ManagementS ien e, 31(3):264�274, 1985.[8℄ M. S. Bazarraa, H. D. Sherali, and C. M. Shetty. Nonlinear Programming: TheoryAnd Appli ations. Series In Dis tere Mathemati s And Optimization. John Wiley &Sons, 1979.[9℄ J. F. Benders. Partitioning pro edures for solving mixed-variables programming prob-lems. Numeris he Mathematik, 4:238�252, 1962.[10℄ F. Bilen, Zs. Csizmadia, and T. Illés. Anstrei her-terlaky type monotoni simplex algo-rithms for linear feasibility problems. A epted to Optimization Methods and Software.138

[11℄ F. Bilen, Zs. Csizmadia, and T. Illés. A new analysis for monotoni type simplexalgorithms for feasibility problems. A epted to Alkalmazott Matematikai Lapok. [inHungarian℄.[12℄ J.M. Bloemhof-Ruwaard and M. T. H. Eligius. Generalized bilinear programming:An appli ation in farm management. European Journal of Operational Resear h,31(3):264�274, 1985.[13℄ K. H. Borgwardt. Probabilisti analysis of the simplex method. In Mathemati aldevelopments arising from linear programming (Brunswi k, ME, 1988), volume 114 ofContemp. Math., pages 21�34. Amer. Math. So ., Providen e, RI, 1990.[14℄ K. Cameron and J. Edmonds. Existentially polytime theorems. In Polyhedral ombi-natori s (Morristown, NJ, 1989), volume 1 of DIMACS Ser. Dis rete Math. Theoret.Comput. S i., pages 83�100. Amer. Math. So ., Providen e, RI, 1990.[15℄ Y. Y. Chang. Least index resolution of degenera y in linear omplementarity problems.(79-14), 1979. Te hni al Report 79-14, Department of Operations Resear h, STanfordUniversity, Stanford, Calfornia, USA.[16℄ S. J. Chung. NP- ompleteness of the linear omplementarity problem. Journal ofOptimization Theory and Appli ations, 60(3):393�399, 1989.[17℄ V. Chvátal. Linear programming. A Series of Books in the Mathemati al S ien es. W.H. Freeman and Company, New York, 1983.[18℄ R. W. Cottle, J. S. Pang, and V. Venkateswaran. Su� ient matri es and the linear omplementarity problem. Linear Algebra and its Appli ations, 114/115:231�249, 1989.[19℄ Zs. Csizmadia and T. Illés. New riss- ross type algorithms for linear omplementarityproblems with su� ient matri es. Optimization Methods & Software, 21(2):247�266,2006.[20℄ Zs. Csizmadia, T. Illés, and M. Nagy. Large s ale bilinear optimization in pra ti e: a ase study. Te hni al report, MOL, 2006.[21℄ G. B. Dantzig. Linear programming and extensions. Prin eton University Press, Prin e-ton, N.J., 1963.[22℄ G. B. Dantzig and P. Wolfe. De omposition prin iple for linear programs. OperationsResear h, 8:101�111, 1960. 139

[23℄ D. den Hertog, C. Roos, and T. Terlaky. The linear omplementarity problem, su� ientmatri es, and the riss- ross method. Linear Algebra and its Appli ations, 187:1�14,1993.[24℄ A. Deza, E. Nematollahi, R. Peyghami, and T. Terlaky. The entral path visits all theverti es of the klee-minty ube. Optimization Methods and Software, 2005. To appear.[25℄ A. Deza, E. Nematollahi, and T. Terlaky. How good are interior point methods?klee-minty ubes tighten iteration- omplexity bounds. AdvOl-Report, M Master U.Hamilton, Ontario, Canada., (20), 2004.[26℄ I. I. Dikin. Iterative solution of problems of linear and quadrati programming. DokladyAkademii Nauk SSSR, 174:747�748, 1967.[27℄ Gy. Farkas. A fourier-féle me hanikai elv alkalmazásai (the appli ations of the me han-i al prin iple of fourier). Mathematikai és Természettudományi Értesít®, 12:457�472,1894.[28℄ Gy. Farkas. A fourier-féle me hanikai elv alkalmazásainak algebrai alapjáról (on thealgebri ba kground of the appli ations of the me hani al prin iple of fourier). Math-ematikai és Fizikai Lapok, 5:49�54, 1896.[29℄ Gy. Farkas. Theorie der einfa hen unglei hungen. Journal für die Reine und Ange-wandte Mathematik, 124:1�27, 1901.[30℄ C. A. Floudas and V. Visweswaran. Primal-relaxed dual global optimization approa h.Journal of Optimization Theory and Appli ations, 78(2):187�225, 1993.[31℄ K. R. Fris h. The logaritmi potential method for solving linear programming problems.University Institute of E onomi s, Oslo, 1955. Memorandum.[32℄ K. Fukuda, M. Namiki, and A. Tamura. EP theorems and linear omplementarityproblems. Dis rete Applied Mathemati s. Combinatorial Algorithms, Optimization andComputer S ien e, 84(1-3):107�119, 1998.[33℄ K. Fukuda and T. Terlaky. Linear omplementarity and oriented matroids. Journal ofthe Operations Resear h So iety of Japan, 35(1):45�61, 1992.[34℄ K. Fukuda and T. Terlaky. Criss- ross methods: a fresh view on pivot algorithms.Mathemati al Programming, 79(1-3, Ser. B):369�395, 1997. Le tures on mathemati alprogramming (ismp97) (Lausanne, 1997).140

[35℄ K. Fukuda and T. Terlaky. On the existen e of a short admissible pivot sequen efor feasibility and linear optimization problems. Pure Mathemati s and Appli ations,10(4):431�447, 1999.[36℄ P. E. Gill, W. Murray, M. A. Saunders, J. A. Tomlin, and M. H. Wright. On proje tedNewton barrier methods for linear programming and an equivalen e to Karmarkar'sproje tive method. Mathemati al Programming, 36(2):183�209, 1986.[37℄ R. E. Gri�th and R. A. Stewart. A nonlinear programming te hnique for the optimiza-tion of ontinuous pro essing systems. Management S ien e, 7:379�392, 1960/1961.[38℄ A. Haar. Die minkowskis he geometrie und die annäherung an stetige funktionen.Mathematis he Annalen, 78:294�311, 1918.[39℄ A. Haar. A lineáris egyenl®tlenségekr®l. Matematikai és Természettudományi értesít®,36:279�296, 1918.[40℄ I. Harjunkoski, T. Westerlund, J. Isaksson, and H. Skrifvars. Di�erent formulationsfor solving trim loss problems in a paper- onverting mill with ilp. Computers andChemi al Engineering, 20:121�126, 1996.[41℄ I. Harjunkoski, T. Westerlund, R. Pörn, and H. Skrifvars. Di�erent transformationsfor solving non- onvex trim-loss problems by minlp. European Journal of OperationalResear h, 105:594�603, 1998.[42℄ P. Huard. Resolution of mathemati al programming with nonlinear onstraints bythe method of enter. In Nonlinear programming, pages 207�219. North Holland,Amsterdam, The Netherlands, 1967.[43℄ T. Illés and T. Terlaky. Pivot versus interior point methods: pros and ons. EuropeanJournal of Operational Resear h, 140(2):170�190, 2002. O.R. for a united Europe(Budapest, 2000).[44℄ AspenTe h In . http://www.aspente h. om/.[45℄ B. Jo�e. Global optimization in re�nery planning. In 17th EURO Mini Conferen e:Continuous Optimization in Industry, 2005.[46℄ N. Karmarkar. A new polynomial-time algorithm for linear programming. Combinator-i a. An International Journal of the János Bolyai Mathemati al So iety, 4(4):373�395,1984. 141

[47℄ E. Klafszky and T. Terlaky. Variants of the Hungarian method for solving linearprogramming problems. Optimization, 20(1):79�91, 1989.[48℄ E. Klafszky and T. Terlaky. The role of pivoting in proving some fundamental theoremsof linear algebra. Linear Algebra and its Appli ations, 151:97�118, 1991.[49℄ V. Klee and P. Kleins hmidt. The d-step onje ture and its relatives. Mathemati s ofOperations Resear h, 12(4):718�755, 1987.[50℄ V. Klee and G. J. Minty. How good is the simplex algorithm? In Inequalities, III(Pro . Third Sympos., Univ. California, Los Angeles, Calif., 1969; dedi ated to thememory of Theodore S. Motzkin), pages 159�175. A ademi Press, New York, 1972.[51℄ M. Kojima, N. Megiddo, T. Noma, and A. Yoshise. A uni�ed approa h to interiorpoint algorithms for linear omplementarity problems, volume 538 of Le ture Notes inComputer S ien e. Springer-Verlag, Berlin, 1991.[52℄ L. Lasdon and B. Jo�e. The relationship between distributive re ursion and su es-sive linear programming in re�ning produ tion planning models. In NPRA ComputerConferen e Seattle, Washington, 1990.[53℄ L. Lasdon, A. Waren, S. Sarkar, and G. F. Pala ios. Solving the pooling problem usinggeneralized redu ed gradient and su essive linear programming algorithms. ACMSigmap Bulletin, 27:9�25, 1979.[54℄ L. S. Lasdon, M. Enquist, and G. F. Pala ios. Nonlinear optimization by su essivelinear programming. Management S ien e, 28(10):1106�1120, 1982.[55℄ C. E. Lemke and J. T. Howson, Jr. Equilibrium points of bimatrix games. J. So .Indust. Appl. Math., 12:413�423, 1964.[56℄ C. E. Lemke and J. T. Howson, Jr. On omplementary pivot theory. In Mathemati sof de ision s ien es, volume Part 1, pages 95�114. AMS, Providen e, Rhode Island,1968.[57℄ I. Maros. Computational te hniques of the simplex method. International Series inOperations Resear h & Management S ien e, 61. Kluwer A ademi Publishers, Boston,MA, 2003. With a foreword by András Prékopa.[58℄ G. P. M Cormi k. Converting general nonlinear programming problems to separa-ble non-linear programming problems. Te hni al Report T-267, George WashingtonUniversity, Washington, D.C, The George Washington University, 1972.142

[59℄ G. P. M Cormi k. Computability of global solutions to fa torable non onvex programs.Mathemati al Programming, 10(2):147�175, 1976.[60℄ C. M. M Donald and C. D. Floudas. Global optimization for the phase and hemi- al equilibrium problem: appli ation to the nrtl equation. Computers and Chemi alEngineering, 19:1111�1139, 1995.[61℄ Nimrod Megiddo and R. Chandrasekaran. On the ǫ-perturbation method for avoidingdegenera y. Operations Resear h Letters, 8(6):305�308, 1989.[62℄ C. A. Meyer and C. A. Floudas. Trilinear monomials with mixed sign domains: fa etsof the onvex and on ave envelopes. Journal of Global Optimization, 29(2):125�155,2004.[63℄ H. Minkowski. Geometrie der Zahlen. Teubner, Leipzig und Berlin, 1896.[64℄ K. G. Murty. Computational omplexity of parametri linear programming. Mathe-mati al Programming, 19(2):213�219, 1980.[65℄ K. G. Murty. Linear and ombinatorial programming. Robert E. Krieger PublishingCo. In ., Melbourne, FL, 1985. Reprint of the 1976 original.[66℄ K. Paparizzos. Pivoting rules dire ting the simplex method through all feasible verti esof klee-minty examples. Opsear h, Journal of the Operational Resear h So iety of India,26(2):77�96, 1989.[67℄ Aspen PIMS. http://www.aspente h. om/produ ts/produ t. fm?Produ tID=129.[68℄ MOL Pl . The Hungarian Oil Company, http://www.mol.hu/en/about_mol/.[69℄ A. Prékopa. On the development of optimization theory. Alkalmazott MatematikaiLapok, 4(3-4):165�191 (1979), 1978.[70℄ A. Prékopa. On the development of optimization theory. The Ameri an Mathemati alMonthly, 87(7):527�542, 1980.[71℄ R. T. Ro kafellar. The elementary ve tors of a subspa e of RN . In CombinatorialMathemati s and its Appli ations (Pro . Conf., Univ. North Carolina, Chapel Hill,N.C., 1967), pages 104�127. Univ. North Carolina Press, Chapel Hill, N.C., 1969.[72℄ C. Roos. An exponential example for Terlaky's pivoting rule for the riss- ross simplexmethod. Mathemati al Programming, 46(1, (Ser. A)):79�84, 1990.143

[73℄ C. Roos, T. Terlaky, and J. P. Vial. Interior point methods for linear optimization.Springer, New York, 2006. Se ond edition of Theory and algorithms for linear opti-mization [Wiley, Chi hester, 1997; MR1450094℄.[74℄ O. J. S hraa and C. M. Corwe. The numeri al solution of bilinear data re on ilia-tion problems using un onstrained optimization methods. Computers and Chemi alEngineering, 22:1215�1228, 1998.[75℄ A. S hrijver. Theory of linear and integer programming. Wiley-Inters ien e Series inDis rete Mathemati s. John Wiley & Sons Ltd., Chi hester, 1986. AWiley-Inters ien ePubli ation.[76℄ R. Östermark. Solving a nonlinear non- onvex trim loss problem with a geneti hybridalgorithm. Computers and Operations Resear h, 26:623�635, 1999.[77℄ Gilbert Strang. Linear algebra and its appli ations. A ademi Press, New York, se ondedition, 1980.[78℄ T. Terlaky. A �nite � riss- ross method� for solving linear programming problems.Alkalmazott Matematikai Lapok, 10(3-4):289�296, 1984.[79℄ T. Terlaky. A onvergent riss- ross method. Optimization, 16(5):683�690, 1985.[80℄ T. Terlaky. A �nite riss- ross method for oriented matroids. Journal of CombinatorialTheory B, 42(3):319�327, 1987.[81℄ T. Terlaky. Optimization in te hnologi al systems. Originally in Hungarian: �Optimal-izálás m¶szaki informatikai rendszerekben�, Do tor of A ademy Thesis, 2005.[82℄ T. Terlaky and S. Z. Zhang. Pivot rules for linear programming: a survey on re enttheoreti al developments. Annals of Operations Resear h, 46/47(1-4):203�233, 1993.Degenera y in optimization problems.[83℄ M. J. Todd. Polynomial expe ted behavior of a pivoting algorithm for linear omple-mentarity and linear programming problems. Mathemati al Programming, 35(2):173�192, 1986.[84℄ A. Torki, Y. Yajima, and T. Enkawa. A low-rank bilinear programming approa hfor sub-optimal solution of the quadrati assignment problem. European Journal ofOperational Resear h, 94:384�391, 1996.144

[85℄ M.A. Tu ker. Lp modelling - past, present and future. Te hni al report, KBC Advan edTe hnologies, In ., Houston, Texas.[86℄ H. Väliaho. P∗-matri es are just su� ient. Linear Algebra and its Appli ations,239:103�108, 1996.[87℄ H. Väliaho. Criteria for su� ient matri es. Linear Algebra and its Appli ations,233:109�129, 1996.[88℄ C.J. Vidal and M. Goets hal kx. A global supply hain model with transfer pri ing andtransportation ost allo ation. European Journal of Operational Resear h, 129:134�158,2001.[89℄ V. Visweswaran and C. A. Floudas. New properties and omputational improvementof the GOP algorithm for problems with quadrati obje tive fun tions and onstraints.Journal of Global Optimization, 3(4):439�462, 1993.[90℄ V. Visweswaran and C. A. Floudas. New formulations and bran hing strategies for theGOP algorithm. In Global optimization in engineering design, volume 9 of Non onvexOptim. Appl., pages 75�109. Kluwer A ad. Publ., Dordre ht, 1996.[91℄ G. F. Voronoi. Nouvelles appli ations des parameters ontinus á la théorie des formesquadratiques. Journal für die Reine und Angewandte Mathematik, 133:97�178, 1909.[92℄ Z. M. Wang. A �nite onformal-elimination free algorithm over oriented matroid pro-gramming. Chinese Annals of Mathemati s. Series B, 8(1):120�125, 1987. A Chinesesummary appears in Chinese Ann. Math. Ser. A 8 (1987), no. 1, 138.[93℄ R. E. Wendell, Jr. Hurter, and P. Arthur. Minimization of a non-separable obje tivefun tion subje t to disjoint onstraints. Operations Resear h, 24(4):643�657, 1976.[94℄ T. Westerlund, I. Harjunkoski, and J. Isaksson. Solving a produ tion optimizationproblem in a paper- onverting mill with milp. Computers Chemi al Engineering, 22(4�5):563�570, 1998.[95℄ T. Westerlund, I. Harjunkoski, and Isaksson J. Solving a two-dimensional trim-lossproblem with milp. European Journal of Operational Resear h, 104:572�581, 1998.[96℄ P. Wolfe. A te hnique for resolving degenera y in linear programming. SIAM Journalof Applied Mathemati s, 11:205�211, 1963.145

[97℄ Margaret H. Wright. The interior-point revolution in optimization: history, re entdevelopments, and long lasting onsquen es. Bulletin of the Ameri an Mathemati alSo iety, 42(1):39�56, 2004.[98℄ XPRSS-MP. http://www.dashoptimization. om/home/produ ts/produ ts_overview.html.[99℄ J. Z. Zhang, N. H. Kim, and L. Lasdon. An improved su essive linear programmingalgorithm. Management S ien e, 31(10):1312�1331, 1985.[100℄ S. Zionts. The riss- ross method for solving linear programming problems. Manage-ment S ien e, 15(7):426�445, 1969.

146