This article was downloaded by: [North Carolina State University]On: 25 November 2014, At: 13:21Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Optimization Methods and SoftwarePublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/goms20

Enhancing the behavior of interior-point methods via identification ofvariablesMaría D. González-Lima aa Departamento de Cómputo Científico y Estadística y Centro deEstadística y Software Matemático (CESMa) , Universidad SimónBolívar , Apdo 89000, Caracas, 1080-A, VenezuelaPublished online: 28 Sep 2007.

To cite this article: María D. González-Lima (2007) Enhancing the behavior of interior-pointmethods via identification of variables, Optimization Methods and Software, 22:6, 937-958, DOI:10.1080/10556780701427654

To link to this article: http://dx.doi.org/10.1080/10556780701427654

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Optimization Methods and SoftwareVol. 22, No. 6, December 2007, 937–958

Enhancing the behavior of interior-point methods viaidentification of variables

MARIA D. GONZÁLEZ-LIMA*

Departamento de Cómputo Científico y Estadística y Centro de Estadística y SoftwareMatemático (CESMa), Universidad Simón Bolívar, Apdo 89000,

Caracas 1080-A, Venezuela

(Received 14 February 2004; revised 30 June 2005; in final form 12 March 2007)

In many linear programming models of real life problems the solution set is not bounded. The presenceof unbounded variables in the solution set can severely hurt the practical performance of primal–dualinterior-point methods for linear programming that generate iterates which follow closely the centralpath or converge to the analytic center. In this work we study the effect of the unbounded variablesby analysing the numerical behavior of the LSSN algorithm proposed by González-Lima, Tapia andPotra [1]. when applied to linear problems with unbounded solution sets. We discuss the numeri-cal behavior of the algorithm and we present a numerical procedure, based on the performance ofthe algorithm, to identify and remove the unbounded variables and related constraints. We developtheoretical support for the procedure and experimental evidence of its performance.

Keywords: Linear programming; Primal–dual interior-point methods; Unbounded solution set

1. Introduction

The theory of primal–dual interior-point methods for linear programming is based on theassumption that the relative interior of the feasibility set is not empty. This assumption impliesthat the solution set of the problem is bounded (and not empty), and, hence the notions ofthe analytic center and central path, which are crucial in the development and behavior of theprimal–dual methods, are well defined.

However, in many models of real life problems the solution set is not bounded since the rela-tive interior of the feasibility set is empty. This is indeed the case for many linear programmingproblems from the Netlib test collection.

Todd [2] studied the effect of the primal variables that are unbounded on the solution setand of null variables (variables that are zero throughout the feasible set) on the computational

*Corresponding author. Email: mgl@cesma.usb.ve

Optimization Methods and SoftwareISSN 1055-6788 print/ISSN 1029-4937 online © 2007 Taylor & Francis

http://www.tandf.co.uk/journalsDOI: 10.1080/10556780701427654

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

938 M. D. González-Lima

performance of variants of Karmarkar’s primal projective algorithm. He pointed out thedetrimental effect that unbounded variables have on the performance of the algorithm.

The presence of unbounded (primal or dual) variables in the solution set also degradesthe performance of primal–dual algorithms that generate iterates which follow closely thecentral path or converge to the analytic center. Different algorithms possess these properties,for example, the short step ones with the best known polynomial complexity bound ([3]),the ones using cutting plane ideas (e.g., Goffin et al. [4, 5]), or the ones, in which we areparticularly interested, designed for solving some specific applications (see González-Limaet al. [6], Thompson et al. [7, 8]). Additionally, iterative methods for solving the linear systemsarising at each iteration of a primal–dual interior-point method, which are efficient alternativesfor very large problems (e.g., [9]), are also affected by the presence of unbounded variables inthe solution set since these iterative methods are very sensitive to the ill-conditioning of thelinear system to be solved. This ill-conditioning can be aggravated when the relative interiorof the feasibility region is empty.

The detrimental effect related to the existence of the unbounded variables may not be fullynoticed when the iterates are generated in a very loose neighborhood of the central path andconvergence to the analytic center is not enforced. This is indeed the case in many implemen-tations of the general primal–dual method, where the perturbation parameter is chosen so asto tend to zero when approaching the solution set, since in this way fast local convergenceis obtained. Besides, in most available interior-point codes for linear programming, as forexample, Lipsol [10] or Hopdm [11], a preprocessing step is performed before the algorithmstarts. For instance see [12]. During this step, the constraint matrix of the linear problem ismodified in such a way that linear dependencies, fixed variables, empty rows or columns etc,are eliminated. This, in many instances, gets rid of the unbounded variables so that the per-formance of the algorithm is not affected by them. But the preprocessing step is done in sucha way that there is not necessarily an explicit knowledge of the variables or the constraintseliminated.

In this paper, we are interested in studying the relationship of the unboundedness of the solu-tion set for linear programming with the performance of primal–dual interior-point methods.Then, this work studies the numerical behavior of the LSSN algorithm proposed by González-Lima, Tapia and Potra [1] when applied to linear programming problems with unboundedsolutions sets. Since the LSSN algorithm was designed for effectively computing the analyticcenter solution, bad behavior is expected. We present a numerical procedure, based on theperformance of the algorithm, to identify and remove the unbounded variables and relatedconstraints. We develop theoretical support for the procedure and experimental evidence ofits effectiveness. We show that after applying the procedure, the problems can be solved byusing the LSSN algorithm.

We need to make use of indicator functions as part of our identification process. Hence, wepresent the functions used in this paper and relate them with others presented in the literature.We study their properties when evaluated at points generated by the LSSN algorithm. Ourprocedure relies on the theoretical results obtained.

The structure of this paper is the following. In the next section, we present some preliminariesand results related with the boundedness of the solution set. Sections 3 and 4 are devoted tothe definition and study of the unbounded variables and their role. We prove that any linearproblem with unbounded primal or dual variables in the solution set can be solved by solvingan equivalent linear programming problem with bounded solution set. In section 5, we presentthe algorithm to be used in our numerical experimentation and we show how it performson some problems from Netlib with unbounded solution sets. In the following section weintroduce the indicator functions, study some of their properties and develop the theoreticalsupport for our numerical identification procedure. Next, the numerical procedure is given and

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

Identification of variable in IPM 939

illustrated by testing a subgroup of Netlib problems. Some comments about the use of otherindicators are also included. Finally, the last section contains remarks and conclusions.

2. Preliminaries

In this paper, we deal with the linear programming problem in the standard form

minimize ctx

subject to Ax = b, x ≥ 0, (1)

where c, x ∈ Rn, b ∈ R

m, A ∈ Rm×n(m < n) and A has full rank m. The dual problem of (1)

can be stated as

maximize bty

subject to Aty + z = c (2)

z ≥ 0,

where y ∈ Rm and z ∈ R

n are the Lagrange multiplier vectors corresponding to the equalityand inequality constraints of problem (1), respectively. We will refer to y as the multipliervariables and z as the dual variables.

The first-order optimality (or Karush–Kuhn–Tucker) conditions for problem (1) are:

F(x, y, z) ≡⎛⎝ Ax − b

Aty + z − c

XZe

⎞⎠ = 0; (x, z) ≥ 0, (3)

where X = diag(x), Z = diag(z) and e is the n-vector of all ones. The feasibility set ofproblem (3) is

F = {(x, y, z) : Ax = b, Aty + z = c, (x, z) ≥ 0}.A feasible point (x, y, z) ∈ F is said to be strictly feasible if x and z are strictly positive. Theset of all the strictly feasible points is denoted by F+.

We denote the solution set of problem (3) by

S = {(x, y, z) : F(x, y, z) = 0, (x, z) ≥ 0}.We use the notation Fp and Sp to denote the feasible set and the solution set, respectively, ofproblem (1). Similar notations are used for the dual problem. Observe that S = Sp × Sd.

The central path associated with problem (3) and parameterized by the parameter μ > 0 isdefined as the collection of points

Pc = {(x(μ), y(μ), z(μ)) ∈ F+ : X(μ)Z(μ)e = μe}.This is equivalent to saying that a strictly feasible point (x, y, z) is on the central path if and

only if it satisfies x1z1 = x2z2 = · · · = xnzn. Observe that a point (x, y, z) is on the centralpath Pc if and only if satisfies

Fμ(x, y, z) =⎛⎜⎝

Ax − b

At y + z − c

XZe − μe

⎞⎟⎠ = 0, with (x, z) > 0. (4)

The nonlinear system (4) corresponds to the nonlinear system (3) perturbed by the vector μe.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

940 M. D. González-Lima

The analytic center (x∗, y∗, z∗) of the solution set S is defined as

(x∗, y∗, z∗) = arg max

⎧⎨⎩

∏i∈I+

x

xi

∏i∈I+

z

zi : (x, y, z) ∈ S

⎫⎬⎭ ,

where I+x = {i : xi > 0, 1 ≤ i ≤ n} and I+

z = {i : zi > 0, 1 ≤ i ≤ n} for all (x, y, z) in therelative interior of the solution set S (we refer to Rockafellar [13] for a definition of relativeinterior).

The following proposition states that under the assumption that the solution set is non-emptyand bounded, the analytic center and the central path are well defined. Moreover, the centralpath converges to the solution set. This result relates the analytic center and central path set,which plays a key role in the development and study of primal–dual interior-point methodsfor linear programming.

PROPOSITION 2.1 Let us assume that S �= ∅ and is bounded. Then.

1. The zero–non zero pattern of points in the relative interior is invariant.2. For each μ > 0, there exists a unique point w(μ) = (x(μ), y(μ), z(μ)) ∈ F+ such that

Fμ(w(μ)) = 0.3. The central path point w(μ) converges to the analytic center of the solution set as μ goes

to zero.

Proof See El–Bakry, Tapia and Zhang [14] for the proof of 1. See McLinden [15],Megiddo [16] or Wright [3] for the proof of 2 and 3. �

In the following we establish a property of the central path that will be useful in our analysis.Define a β-neighborhood of the central path to be the set

Nμ(β) = {(x, y, z) ∈ F+ : ‖XZe − μe‖ ≤ βμ},for any μ > 0. (‖.‖ denotes the Euclidean norm unless otherwise is specified).

Because of Proposition 2.1 it should be intuitively clear that if we have a sequence ofiterates in the feasibility region that belong to a β-neighborhood of the central path for someμ > 0 and β sufficiently small, then this sequence is arbitrarily close to the central-path point(x(μ), y(μ), z(μ)). The next proposition states this. This proposition is a version, tailored toour needs, of Lemma 2.1 from Zhang and Tapia [17].

In order to simplify notation, for any μ > 0 and (x, y, z) ∈ R2n+m with (x, z) > 0 set

ημ = ‖XZe/μ − e‖, w = (x, y, z) and let w(μ) = (x(μ), y(μ), z(μ)) be the central-pathpoint satisfying Fμ(w(μ)) = 0.

PROPOSITION 2.2 Assume that the solution set S is non-empty and bounded, and let {wk} bea sequence in F+. Given μ > 0, if limk→∞ ηk

μ = 0 then limk→∞ wk = w(μ).

Proof The proof can be found in [18]. It is similar to the proof of Lemma 2.1 from [17]. �

Hence, the boundedness of the solution set is relevant for many theoretical results crucialin the context of primal–dual interior point methods. However, many real life problems aremodeled in such a way that their solution set may be unbounded. In these cases, there existvariables which can take arbitrarily large values on the solution set. These are the unboundedvariables. In the following sections we formally define and describe these variables.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

Identification of variable in IPM 941

3. Unbounded/Null Variables and the Solution Set

DEFINITION 3.1 The variable xj (or zj ) for any j ∈ {1, . . ., n} is a primal (or dual) unboundedvariable if, for a given positive constant M and any M ≥ M, there exists an optimal solutionx (or an optimal solution pair (y, z)) of the primal problem (1) (or dual problem (2)) suchthat xj ≥ M (or zj ≥ M).

Alternatively, we can define the unbounded variables as follows.

DEFINITION3.2 The variablexj (or the variable zj ) for any j ∈ {1, . . ., n} is a primal (or dual)unbounded variable if for any M > 0 there exist optimal solutions (x(M), y(M), z(M)) ∈ Ssuch that limM→∞ xj (M) = ∞ (or limM→∞ zj (M) = ∞).

We will denote the sets of primal and dual unbounded variables as Up and Ud respectively.Because in the solution set xjzj = 0 for all j then Up

⋂Ud = ∅.

DEFINITION 3.3 The variable xj for any j ∈ {1, . . ., n} is a primal null variable if xj = 0 forall feasible points x of problem (1). Similarly, the variable zj for any j ∈ {1, . . ., n} is a dualnull variable if zj = 0 for all feasible pairs (y, z) of problem (2).

We will denote the sets of primal and dual null variables as Np and Nd. Observe thatNp

⋂Up = ∅ and Nd

⋂Ud = ∅.

Let A =(

ct

A

)∈ R

(m+1)×n. The following proposition establishes a condition that implies

the existence of unbounded variables.

PROPOSITION 3.4 Let us assume that S is not empty. Let J be a subset of {1, . . ., n}, |J | > 1,such that there exists a vector w ∈ R

|J | satisfying that∑

j∈J aij wj = 0. If all the componentsof w have the same sign (this is, if there exits a positive linear combination of the columns ofA corresponding to the set J which equals zero), then the variables xj with j ∈ J are primalunbounded variables.

Proof Let x be a solution of the primal problem (1). Let us assume that wj > 0 for all j (if notthe argument is similar but considering −wj ). For any λ ≥ 0 let us define the points x(λ) ∈ R

n

as xj (λ) = xj + λwj if j ∈ J and xj (λ) = xj if not. Then, xj (λ) ≥ 0 for all j . Besides,(Ax(λ))i = ∑

j aij x(λ)j = ∑j∈J aij (xj + λwj ) + ∑

j∈J c aij xj . By definition of w and thefeasibility of x, we have that Ax(λ) = b. Additionally, ct x(λ) = ct x + ∑

j∈J λcj wj = ct x.This result is true for any value of λ ≥ 0, therefore the variables xj are primal unbounded forj ∈ J . �

Notice that if there exists a dual unbounded variable zj , then there is an index i ∈ {1, . . ., m}such that for any optimal solution (x(M), y(M), z(M)) we have limM→∞ |yi (M)| = ∞. Thisfollows from the feasibility equations of the dual problem (2), that is,

∑i∈{1,...,m} aji yi(M) +

zj (M) = cj . The variable yi is called a dual multiplier unbounded variable.Thus, the unboundedness of the dual variables is linked to the presence of dual multiplier

variables y that are also unbounded. In this paper we are interested in their role, hence wedefine the dual multiplier null and unbounded variables similarly to definitions 3.1 and 3.2.

DEFINITION 3.5 The variable yi for any i ∈ {1, . . ., m} is a dual multiplier null variable ifyi = 0 for any feasible pair (y, z) of problem (2). The variable yi for any i ∈ {1, . . ., m} is

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

942 M. D. González-Lima

a dual multiplier unbounded variable if given a constant N and any N with |N | ≥ |N | thereexists an optimal solution pair (y, z) of the dual problem (2) such that yi/N ≥ 1.

We denote the sets of dual multiplier null and unbounded variables as Ndm and Udm,respectively. The following proposition is relevant.

PROPOSITION 3.6 Let yio be a dual multiplier unbounded variable. Then, the followings aresatisfied

1. For N > 0 there exist solutions (x(N), y(N), z(N)) ∈ S satisfying limN→∞ |yio (N)|= ∞.

2. Unless all the primal variables are null variables, there exists another dual multiplierunbounded variable.

3. For all j , one of the following is satisfied:(i) zj is a dual unbounded variable (and limN→∞

∑i∈Udm

ajiyi(N) = −∞) or(ii) limN→∞

∑i∈Udm

ajiyi(N) = Rj .

Proof Item 1 follows from the definition of a dual multiplier unbounded variable. Item 2follows from the feasibility equations for the dual problem (2) evaluated at the optimal solutions(x(N), y(N), z(N)) since

∑i∈{1,...,m} ajiyi(N) + zj (N) = cj for all j ∈ {1, . . ., n}. If yio is

the unique dual multiplier unbounded variable, then taking the limit when N goes to infinity,we obtain that zj is a dual unbounded variable for all j . Finally, item 3 is obtained by usingthe same feasibility equations from (2) evaluated at (x(N), y(N), z(N)). These equations canbe written as

∑i∈Udm

ajiyi(N) + ∑i∈Uc

dmajiyi(N) + zj (N) = cj for all j . Taking the limit

when N → ∞ the proof is complete. �

We want to emphasize that, for the purpose of this paper, the role of the dual multiplierunbounded variables is more practical than theoretical and this is the reason why we focus onthe primal and dual slack variables for the definition and results that follow.

DEFINITION 3.7 For any j ∈ {1, . . ., n}, we say that j corresponds to an unbounded variableat the solution set if and only if j ∈ Up

⋃Ud.

Unbounded variables are dual to the variables that have value equal to zero everywherein the feasibility set (the null variables). The next proposition formalizes this. The proof ofthis proposition can be obtained from known results in the literature of primal–dual interior-point methods. However, we prove it here for various reasons. First of all, the proof shownhere is simple and it only requires knowledge of linear programming. Second, it gives a preciserelationship between the unbounded and null variables. Finally, the proof follows a schemethat is similar to other results derived in this paper, so we consider useful to include it.

PROPOSITION 3.8 Let us assume that S �= ∅. Then, Np = Ud and Nd = Up.

Proof Let us prove the first part. Let jo ∈ {1, . . ., n} be such that jo ∈ Np. Then, xjo= 0 for

any feasible vector x for problem (1). Therefore, problem (1) is equivalent to

minimize ctx

subject to Ax = b

xjo= 0 (5)

x ≥ 0.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

Identification of variable in IPM 943

Let us define the matrix A ∈ R(m+1)×n and vectors c ∈ R

n, b ∈ Rm+1 as

aij =

⎧⎪⎪⎨⎪⎪⎩

1 : if i = m + 1, j = jo

0 : if i = m + 1, j �= jo

0 : if i ∈ {1, ..., m}, j = jo

aij : otherwise

and

bi ={

0 : if i = m + 1bi : otherwise

cj ={

0 : if j = jo

cj : otherwise

Let (P ) be a modification of problem (1), where the matrix A and vectors b, c are consideredinstead of A and b, c.

It is easy to prove that S = S with S the solution of Problem (P ). In the dual, we havethe restriction Aty + z = c, which implies that

∑mi=1 aijo

yi + am+1joym+1 + zjo

= cjo. But,

because of the definition of A and c we get the equation

ym+1 + zjo= 0. (6)

On the other hand, since am+1j = 0 for all j �= jo we have∑m

i=1 aij yi + am+1j ym+1 +zj = cj . Hence

∑mi=1 aij yi + zj = cj for all j �= jo. Thus, the variables ym+1 and zjo

appearonly in (6). This implies zjo

= −ym+1. Let M be any positive value. If (y, z) ∈ Sd then (y, z)

∈ Sd with ym+1 = −2M , zjo= −ym+1 and yi = yi, zj = zj for all i ∈ {1, . . . , m}, j �= jo. It

is clear that jo ∈ Ud since zjo> M and M is arbitrary. This implies that Np ⊆ Ud.

To prove the other inclusion, we will use the fact that for any primal–dual feasible point thefollowing equation is satisfied

xtz = xt (c − Aty) = ctx − bty. (7)

Let jo ∈ {1, . . . , n} be such that jo �∈ Np. So, there exists a feasible point x of problem (1)such that xjo

= λ > 0. Let (y∗, z∗) be any solution of problem (2).Using (7) we get

n∑j=1,j �=jo

z∗j xj + λz∗

jo=

n∑j=1

cj xj −m∑

i=1

biy∗i .

Since the right hand side is a finite number, it is clear that jo �∈ Ud. Thus Ud ⊆ Np andfinally we have Ud = Np.

The second part of the proposition comes from the correspondence between primal and dualproblems. �

Let us consider the following two illustrative examples.

Example 3.9 ⎧⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎩

min x1 − x2 − x3

s.a. x1 − x2 − 2x3 = 2

x1 − x2 − 5x3 = 2

x1, x2, x3 ≥ 0

S = {(x, y, z) : x = (2 + λ, λ, 0)t , y = (1 − λ, λ)t , z = (0, 0, 1 + 2λ)t , for all λ ≥ 0}.Thus Up = {1, 2} = Nd, Udm = {1, 2} and Np = {3} = Ud.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

944 M. D. González-Lima

Example 3.10 ⎧⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎩

min 2x2 + 3x5

s.a. x1 + 2x2 − 2x3 = 1

x4 + 2x5 − 2x6 = 2

x1, x2, x3, x4, x5, x6 ≥ 0

S = {(x, y, z) : x = (1 + 2λ, 0, λ, 2 + 2λ, 0, λ)t , y = (0, 0)t , z = (0, 2, 0, 0, 3, 0)t for allλ ≥ 0}. Thus Up = {1, 3, 4, 6} = Nd, Ndm = {1, 2}.

From now on we will denote U = Up = Nd and N = Np = Ud.The previous proposition establishes conditions for boundedness of the solution set. We can

write this proposition in an equivalent way. In order to do so let us consider the conditions

(A1) {x ∈ Rn : Ax = b, x > 0} �= ∅ and

(A2) {(y, z) ∈ Rm+n : Aty + z = c, z > 0} �= ∅.

Observe that conditions (A1) and/or (A2) are satisfied if and only if the sets Np and/or Nd

are not empty, respectively. By Proposition 3.8 this is equivalent to the assertion that the setsSp and Sd are bounded. Conditions (A1) and (A2) are satisfied simultaneously if and only ifthe set F+ is not empty. Moreover, it is known that S �= ∅ if F �= ∅. Therefore, Proposition 3.8can be written as the following equivalent known result.

PROPOSITION 3.11 Suppose that F �= ∅. Then, F+ �= ∅ if and only if the solution set S isnon-empty and bounded. In particular, condition (A1) is satisfied if and only if the set Sd isnon-empty and bounded; condition (A2) is satisfied if and only if the set Sp is non-empty andbounded.

Proof It results from Proposition 3.8. See Wright [3] for an alternative proof. �

Therefore, conditions (A1) and (A2) are satisfied ⇐⇒ F+ �= ∅ ⇐⇒ S �= ∅ andbounded.

In the next section we will show that the unbounded variables can always be removed, sothat the linear problem can be transformed into an equivalent one where the solution set isbounded.

4. Role of Unbounded/Null variables

We will start this section by considering the two examples presented in the previous section.Observe that the problem given in Example (3.9) can be solved by removing all the

unbounded and null primal variables and the constraints corresponding to the unboundedand null dual multiplier variables and considering the constant value of 2 added to the objec-tive function. The values of the original variables can be obtained by setting xj = 0 for allj ∈ N and solving the feasible constraints.

In a similar fashion, problem in Example (3.10) can also be solved by removing all theunbounded primal variables and the constraints corresponding to the null dual multipliervariables and considering the constant value of 0 added to the objective function.

These illustrations suggest that the unbounded variables and null dual multipliers couldbe removed from the original problem which can then be solved by considering a linearprogramming problem with bounded solution set.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

Identification of variable in IPM 945

The following theorem states this result. The precise linear problem to be solved in orderto solve Problem (1) without including unbounded or null variables is described.

First, we present a technical result useful for proving the theorem.We use the following notation: if x ∈ R

n and I is a subset of the index set {1, . . . , n} thenxI is the vector formed by the components xj with j ∈ I. If M ∈ R

k×n then MI is the matrixformed by the columns j of M such that j ∈ I. If R is a subset of the index set {1, . . . , m}then M(R,I) is the submatrix of M formed by the components Mij such that i ∈ R and j ∈ I.

LEMMA 4.1 Let us consider the system

Mt = v (8)

with M ∈ Rk×m. Then, there exists subsets R of {1, . . . , m}, C of {1, . . . , k}, such that t is a

solution of (8) if and only if t satisfies M(C,R)tR + M(C,Rc)tRc = vC . Moreover, |C| = |R| =number of maximal linearly independent rows or columns in M and M(C,R) is invertible.

Proof Applying Gaussian elimination to the augmented matrix (M v) we obtain theaugmented matrix (M v) such that the system (M)y = v is equivalent to system (8).

(M) satisfies the following properties:

• Some rows of (M) may be zero and they correspond to the linearly dependent rows of M .• All the rows that are not equal to zero in (M) form a linearly independent set. The subset C

corresponds to the positions of these rows in the matrix M .• All the columns in M corresponding to the columns in (M) with pivot form a linearly

independent set. The subset R corresponds to the positions of these columns.

By well known results of linear algebra, |C| = |R| = number of maximal linearly independentrows or columns in M . Then, t solves the system M(C,R)tR + M(C,Rc)tRc = vC . But thissystem is equivalent to M(C,R)tR + M(C,Rc)tRc = vC . This completes the proof. �

THEOREM 4.2 Let us assume Problem (1) has a solution. Let At = (At )N cdm

∈ Rn×m, and

b = bN cdm

∈ Rm with 1 ≤ m ≤ m. For notational convenience let us denote A = A and b = b.

Let U and N be the sets of primal unbounded and null variables for (1). Suppose |U | = k ≥ 1,and call B = (U

⋃N)c.

Assume that AU ∈ Rm×k is not the zero matrix. Let C ⊂ U and R ⊂ {1, . . . , m} denote the

index sets corresponding to maximal sets of columns and rows that are linearly independentin AU . Then, we have the following cases:1. If Rc �= ∅.

Let AU1 = (AU)(C,R), AU2 = (AU)(C,Rc), AtB1

= (AtB)R and At

B2= (At

B)Rc .Then, the solutions (x∗

B, x∗U) of problem (1) can be found by solving

minimize (cB − AtB1

DcC)txB + ctCDbR

subject to (AB2 − AU2DtAB1)xB = bRc − AU2D

tbR, xB ≥ 0,(9)

and

A(R,U)

(xCxCc

)= bR − A(R,B)x

∗B, with (xC, xCc) ≥ 0. (10)

Here, D = ((AU)(C,R))−1.

2. If Rc = ∅.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

946 M. D. González-Lima

Then, a solution (x∗B, x∗

U) of problem (1) can be found by solving

x∗B = 0, AUx∗

U = b, with x∗U ≥ 0.

The dual optimal solution is (y∗, z∗B) with y∗ the unique solution of the system At

Uy = cU

and z∗B = cB − At

By∗.

Proof Since xN = 0. Problem (1) can be solved by solving

minimize ctBxB + ct

UxU

subject to ABxB + AUxU = b, (xB, xU ) ≥ 0,(11)

Assume that AU ∈ Rm×k and AB ∈ R

m×l .Then, the corresponding dual problem is

maximize ytb

subject to AtBy + zB = cB,

AtUy + zU = cU ,

(zB, zU ) ≥ 0,

(12)

Since zU = 0 for any feasible solution of (12) we can delete these variables and considerthe equation

AtUy = cU . (13)

Applying Lemma 4.1 with M = AtU , we obtain that there exist subsets R, C of the sets

{1, . . . , m} and {1, . . . , k} and matrices AU1 = AU(C,R)and AU2 = AU(C,Rc)

such that anysolution y of (13) satisfies y1 = D(cC − At

U2y2) with D = (At

U1)−1, if Rc �= ∅ or y the unique

solution of the system AtUy = cU if Rc = ∅. In the first case, we substitute the value of y1

in (12) using a corresponding partition of the matrix AtB and the vector b. Then, the dual

problem (12) has been transformed in a problem with a less or equal number of variables andconstraints

maximize yt2(bRc − AU2D

tbR) + ctCDtbR

subject to (AtB2

− AtB1

DAtU2

)y2 + zB = cB − AtB1

DcC,

zB ≥ 0,

(14)

The dual problem is the Problem (9).If Rc = ∅ let us call y∗ the solution of At

Uy = cU . Substituting y∗ in (12) we getz∗B = cB − At

By∗. We know z∗B ≥ 0 because Problem (1) has a solution. In this case, the

primal problem becomes

minimize xtB(cB − At

By∗)

subject to xB ≥ 0,(15)

It is clear that x∗B = 0 is the unique solution of this problem and x∗

U can be obtained by solvingAUx∗

U = b with x∗U ≥ 0. This completes the proof. �

Some observations are in order.

• The previous result is also true if B is defined as B = Uc. In this case, the null primalvariables are part of Problem (9).

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

Identification of variable in IPM 947

• If m = 0 (i.e., all the rows of the original matrix A correspond to null dual multipliervariables), the solution of Problem (1) is found by solving min ct

BxB subject to xB ≥ 0. Theoptimal values of all the variables can be found by solving AUxU + ABx∗

B = b for xU ≥ 0.• If AB1 = 0 = bR, Problem (14) is equal to Problem (1) without the null and unbounded

primal variables, and the constraints corresponding to the set R⋃

Ndm.• If B = ∅, the optimal objective value of (1) is given by ct

CDbR, null variables arezero and the primal unbounded variables are given by (10) with right hand side equalto bR.

• If AU = 0, Problem (1) can be solved by deleting the positions corresponding to the primalnull variables.

• If cU = 0 and there exists xB ≥ 0 such that ABxB = b, Problem (1) can be solved byremoving the columns of the matrix A corresponding to the unbounded primal and dualvariables.

• The number of constraints of Problem (9) corresponds to the number of the linearlydependent rows of AU .

Let us illustrate the result of Theorem 4.2 in the examples 3.9, 3.10.

In the first example, the set B = ∅ and AU =(

1 −1

1 −1

), then the sets R = C = {1}.

Therefore, the optimal objective value is equal to ctCDbR = 2 and A(R,U)

(xC

xCc

)= b1 with

(xC, xCc ) ≥ 0 impling x1 − x2 = 2 with x2 ≥ 0.In the second example, the problem is easily solved, since all the rows of A are removed

from the beginning. If x∗ denotes an optimal point, then x∗B =

(x∗

2

x∗5

)=

(0

0

). The optimal

value of the unbounded variables is obtained by solving x1 − 2x3 = 1 and x4 = 2 + 2x6 withx3, x6 ≥ 0.

In the next section, we study the computational effect of the unbounded variableswhen applying a primal–dual interior-point algorithm designed for computing the analyticcenter.

5. Algorithm

The basic primal–dual interior-point method for linear programming was originally proposedby Kojima, Mizuno, and Yoshise [19], based on the earlier work by Megiddo [16].A modification of the Kojima–Mizuno–Yoshise algorithm is presented by González-Lima, Tapia and Potra [1]. This algorithm, based in the result given by Proposi-tion 2.2 is called the Long-Step Shrinking-Neighborhood algorithm (or LSSN algorithm)and it has been designed for effectively computing the analytic center of the solu-tion set.

The idea of the algorithm is to consider a sequence of gradually shrinking neighborhoods{N (βj )} of the central path with βj → 0. The subsequence converging to the analytic cen-ter is obtained by considering a sequence of μ′s and for each fixed μ using a dampedNewton method (with a line-search globalization strategy) to approximately solve the systemFμ(x, y, z) = 0.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

948 M. D. González-Lima

A detailed description of the LSSN algorithm follows.

ALGORITHM LSSNGiven w0 = (x0, y0, z0) ∈ F+, σ 0 ∈ (0, 1), and β0 ∈ (0, 1).Choose η ∈ (0, 1/2) and 0 < l < u < 1.k = 0. Do until convergence.Step 1. Set μ = μk = σ 0(xk)t zk/n and β = βk.

Step 2. If fμ(wk) = ‖XkZke/μ − e‖2 ≤ β2 (Proximity Condition)go to step 6.Step 3. Substep 3.1. Find �wk = (�xk, �yk, �zk)

solving F ′(xk, yk, zk)(�wk)t = −F(xk, yk, zk) + μ(0, 0, e)t .

Substep 3.2. Choose τ k ∈ (0, 1). Computeαk = min(1, τ kαk), with αk = −1/(min((Xk)−1�xk, (Zk)−1�zk)).

Substep 3.3. Set wk+1 = (xk+1, yk+1, zk+1) = wk + αk�wk.

Step 4. Substep 4.1. If fμ(wk+1) ≤ fμ(wk) + ηαk∇fμ(wk)t (�wk)t

go to step 5.Substep 4.2. If not, reduce αk := ραk, with ρ ∈ [l, u].Substep 4.3. Set wk+1 = (xk+1, yk+1, zk+1) = wk + αk�wk.

Go to step 4.Step 5. Do k := k + 1, βk = β, μk = μ and go to Step 2.Step 6. Set μ = μk = σ 0(xk)t zk/n.Step 7. Do steps 3.1, 3.2, 3.3.Step 8. Choose βk+1 ≤ βk.Step 9. Do k = k + 1 and go to step 1

We note that while the Proximity Condition at step 2 is not satisfied, the steps 3 and 4 ofthe above algorithm are performed iteratively with the same value of μ.

The next theorem states a convergence result for the LSSN algorithm.

THEOREM 5.1 Let {(xk, yk, zk)} be generated by the LSSN algorithm applied to a linear pro-gramming problem, where conditions (A1)and (A2) are satisfied. Suppose that the parameterchoices have been made so that τ k ≥ τ > 0 for all k and some τ ∈ (0, 1) and βk → 0. Assumefurther that min(XkZke)/xkt

zk ≥ γ /n for all k and some γ ∈ (0, 1). Let I(μ) be the set ofindices defined by

I(μ) = {k : μk = μ}. (16)

This is to say, the set I(μ) is the set of the indices k corresponding to the iterates generatedby the LSSN algorithm, where the Proximity Condition in Step 2 is not satisfied.

Then,

1. For any value of β ∈ (0, 1), there exists ko ∈ I(μ) such that fμ(xk0 , yk0 , zk0) ≤ β.2. There exists a subsequence of {(xk, yk, zk)} that converges to the analytic center of the

solution set.

Proof See [1]. �

Item 1 from the previous theorem says that the LSSN algorithm is well defined. It alsoimplies that if we apply the algorithm removing the Proximity Condition in Step 2, then thealgorithm generates an infinite sequence of iterates that converges to a point in the central path.The next corollary presents this result. For simplicity, we call this version of the algorithm

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

Identification of variable in IPM 949

Table 1. Performance of the LSSN algorithm.

Problem Iteration ||XZe − μe||/μ ||Ax − b|| ||Aty + z − c|| ||(�x, �y, �z)|| xt z

ADLITTLE 1 1.2 × 103 6.0 × 104 4.7 × 105 1.5 × 104 4.6 × 107

13 4.9 × 10−1 6.1 × 10−4 4.7 × 10−3 3.3 × 109 4.6 × 105

25 3.8 × 10−1 5.6 × 10−7 1.0 × 10−4 3.6 × 1012 4.6 × 105

37 4.0 × 102 7.9 × 10−9 5.5 × 10−3 1.2 × 1012 7.6 × 104

49 4.9 × 10−1 6.5 × 10−13 1.6 × 101 1.0 × 1018 2.2 × 103

61 3.8 × 10−1 4.4 × 10−13 1.6 × 101 1.1 × 1021 2.2 × 103

SC50B 1 8.7 × 102 6.4 × 103 1.6 × 102 5.7 × 1002 1.6 × 104

13 6.9 × 10−1 1.5 × 10−5 3.6 × 10−7 1.2 × 107 1.5 × 102

25 4.8 × 10−1 1.9 × 10−8 4.7 × 10−10 9.3 × 109 1.6 × 102

37 3.4 × 10−1 7.5 × 10−10 1.8 × 10−11 2.4 × 1011 1.6 × 102

49 3.4 × 10−1 2.9 × 10−11 7.0 × 10−13 6.2 × 1012 1.6 × 102

61 2.6 × 10−1 2.2 × 10−12 5.6 × 10−14 7.8 × 1013 1.6 × 102

SC105 1 1.3 × 103 1.4 × 104 3.3 × 102 8.0 × 1002 3.3 × 104

13 5.1 × 10−1 2.7 × 10−5 6.6 × 10−7 1.0 × 1007 3.3 × 102

25 4.2 × 10−1 1.7 × 10−8 4.2 × 10−10 1.6 × 1010 3.3 × 102

37 7.7 × 1002 1.3 × 10−10 3.0 × 10−12 4.5 × 109 3.8 × 101

49 5.2 × 10−1 9.1 × 10−13 5.8 × 10−16 9.6 × 1015 6.8 × 10−1

61 3.8 × 10−1 1.2 × 10−12 9.7 × 10−16 1.2 × 1019 6.8 × 10−1

BEACONFD 1 1.7 × 103 1.0 × 107 3.2 × 104 9.9 × 103 2.3 × 107

16 4.0 × 100 2.4 × 101 7.5 × 10−2 9.2 × 106 2.0 × 105

31 2.2 × 100 3.6 × 10−1 1.1 × 10−3 6.1 × 108 2.1 × 105

46 2.2 × 100 6.1 × 10−3 2.2 × 10−5 3.6 × 1010 2.1 × 105

61 1.3 × 100 5.2 × 10−4 1.2 × 10−4 4.3 × 1011 2.2 × 105

76 1.1 × 100 7.7 × 10−5 5.7 × 10−4 2.8 × 1012 2.2 × 105

91 1.1 × 100 1.3 × 10−4 3.7 × 10−3 1.9 × 1013 2.2 × 105

106 1.1 × 100 4.9 × 10−4 3.3 × 10−2 1.3 × 1014 2.2 × 105

121 1.1 × 100 2.6 × 10−7 2.4 × 10−1 8.4 × 1014 2.2 × 105

(excluding the Step 2), the LSSN CP-Algorithm. Observe that for the LSSN CP-Algorithm,I(μ) has an infinite number of elements.

COROLLARY 5.2 Under the conditions of the previous theorem and any μ > 0 consider asequence of iterates generated by the LSSN CP-Algorithm. Then, the sequence of iterates{wk} generated by the algorithm satisfies that limk→∞ ‖wk − w(μ)‖ = 0.

Proof From the previous theorem we obtain that limk→∞ ηkμ = 0. Then, from Proposition 2.2

we obtain the result. �

In [1] the effectiveness of the LSSN algorithm is shown for solving linear programmingproblems when the considered assumptions are satisfied.

However, let us study the performance of the LSSN algorithm on problems ADLIT-TLE, SC50B, SC105 and BEACONFD from the Netlib test collection. Table 1 shows thisperformance.

The experiments were performed in 64 bit arithmetic on a Ultra 10 using codes imple-mented in MATLAB†. The line-search strategy (backtracking) defined in step (4) of theLSSN algorithm was implemented using the value η = 10−4 and a fixed value ρ = 1/2.The parameter choices and starting point are the similar to that used in [1]. This is to say, in allproblems, the parameters τ k were chosen as τ k = 1 − min{0.05, 0.05(xk)t zk}, β0 = 0.25, andσ 0 = 0.01. The starting point w0 is found following Lustig et al. [20] and it does not satisfythe linear constraints but all the components of the primal and dual variables are positive.

†MATLAB is a registered trademark of the MathWorks, Inc.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

950 M. D. González-Lima

In these tests the Proximity Condition in Step 2 is never satisfied. Observe the large valueof the step directions, the small decrease in the measure of the proximity to the central pathand the lack of primal or dual feasibility for problems ADLITTLE and BEACONFD. Thenumerical results shown in Table 1 coincide with the numerical behavior expected from anyprimal–dual interior-point algorithm for computing the analytic center solution applied to aproblem with unbounded solution set.

We analysed the problems considered in table 1 and we found out that none of them satisfycondition (A1). All these problems have constraints of the type xi = 0, for some i = 1, . . . , n.

In the case of problem BEACONFD, there are other constraints of different types but imply-ing that there exist null dual variables. As an illustration, consider the following constraintspresent in BEACONFD.

x3 + 0.05x41 − 0.05x42 = 37

x27 + 0.05x104 − 0.05x105 = 148.

The variables x3, x42, x27 and x105 only appear in these constraints. This means that columns3 and 42, and columns 27 and 105 are, respectively, linearly dependent in the matrix A

(after transforming to standard form as stated in Problem (1)). On the other hand, c(3) =c(42) = 0, which implies that in the corresponding dual constraints, we have yi + z3 = 0 andyi − 0.05z42 = 0, some i. This implies z3 = −z42. Because, the non-negativity constraints forz we get z3 = z42 = 0 in all the feasible set. The same argument applies to the variables z27

and z105. Hence, primal variables x3, x42, x27 and x105 are unbounded and z3, z42, z27 and z105

are dual null variables.Problem BEACONFD shows that there are constraints related with the existence of

unbounded variables that are not so immediate to detect. Therefore, to know if conditions(A1) and (A2) are satisfied, or equivalently if the solution set is not bounded, may not be asimple matter for any linear programming problem.

How to detect the constraints and conditions related with the existence of unboundedvariables is the topic of the next section.

6. Indicator functions

For linear programming problems many indicator functions have been proposed to identifyvariables that are zero at the solution.A thorough study of the indicators for using with interior-point methods is presented by El–Bakry, Tapia and Zhang [14]. This study implies that anindicator proposed by Tapia is particularly effective in the context of primal–dual interior-pointmethods. In [21], the logarithmic Tapia indicators are introduced and their ability to identifyseveral groups of variables, not only the two basic groups (zero and non-zero), is shown.In particular, we are interested in their ability, as well as the ability of the Tapia indicators,to identify unbounded variables in the solution set. In the following, we define the Tapia andlogarithmic Tapia indicators and we study their behavior in the context of the LSSN algorithm.

DEFINITION 6.1 Let {vk} be a real sequence. For each k, consider the function T (vk) =vk+1/vk and Log T (vk) = log |1 − T (vk)|. For wk = {(xk, yk, zk)} ∈ R

2n+m, the Tapia andlogarithmic Tapia indicators are defined as the functions T (wk

j ) and Log T (wkj ), respectively,

for any variable wj .

We will call the Primal, Dual, and Dual Multipliers Log Tapia indicators to the logarithmicTapia indicators corresponding to the primal, dual or dual multiplier variables, respectively.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

Identification of variable in IPM 951

The next proposition studies the behavior of the Log Tapia indicators when the LSSNalgorithm is applied to a problem with bounded solution set. It is clear that any result obtainedfor the Log Tapia indicators is extensible to the Tapia indicators and vice verse.

PROPOSITION 6.2 Let {(xk, yk, zk)} be generated by the LSSN algorithm applied to a linearproblem, where conditions (A1) and (A2) are satisfied.

Let us suppose that the parameter choices have been made so that τ k ≥ τ > 0 for all k andsome τ ∈ (0, 1). Assume further that min(XkZke)/xkt

zk ≥ γ /n for all k and some γ ∈ (0, 1).Then, there exists a subsequence of {(xk, yk, zk)}, where limk→∞ LogT (xk

j ) = −∞ andlimk→∞ Log T (zk

j ) = −∞ for all j .

Proof By Theorem 5.1, there exists a subsequence of {(xk, yk, zk)} converging to the analyticcenter (x∗, y∗, z∗). For any j such that x∗

j �= 0 it is clear that the sequence {xk+1j /xk

j } convergesto 1. The first part of Theorem 5.1 also implies that ηk

μk ≤ βk . From this inequality it is very easy

to show that (1 − βk/1 + βk) ≤ (xk+1j zk+1

j /xkj z

kj ) ≤ (1 + βk/1 − βk) for all j . Therefore,

{zk+1j /zk

j } also converges to 1. The proof is complete by the definition of the Log Tapiaindicators. �

A similar result is true if the algorithm applied is the LSSN CP-Algorithm. In this case,it can be easily proved that the sequence generated by the algorithm satisfies that the Primaland Dual Log Tapia indicators converge to minus infinity. This follows from the fact that‖wk − w(μ)‖ converges to zero with μ = μ0. This, in turn, says that the sequences {xk} and{zk} converge to the vectors x(μ) and z(μ) with xj (μ)zj (μ) = μ > 0 for all j . Therefore, thesequences xk+1

j /xkj and zk+1

j /zkj converge to 1 for all j .

Proposition 6.2 and the consequent discussion say that if the solution set S is bounded, thenthe value of the logarithmic Tapia indicators become arbitrarily small from some iterationk0 ∈ I(μ), if β is small. Moreover, we can always find a subsequence, where these indicatorsconverge to minus infinity. Figure 1 shows this behavior for problem AFIRO from Netlib.

If some logarithmic Tapia indicator converges to minus infinity, it is not known from theprevious propositions whether the corresponding (primal or dual) variable is bounded ornot. However, the next proposition and the consequent discussion give useful information inthis case.

PROPOSITION 6.3 Let {(xk, yk, zk)} be a sequence of iterates generated by applying theLSSN algorithm to a linear problem. Let us suppose that limk→∞ Log T (xk

jo) = −∞ and

limk→∞ Log T (zkjo) = −∞ some jo ∈ {1, . . . , n} and k ∈ I(μ). If {αk} is bounded away from

zero, then the sequence {|xkjozkjo

− μ|} converges to 0.

Proof The proof follows from the way that the search direction is found in the LSSNalgorithm, which implies that {αk(−1 + (μ/xk

jozkjo))} converges to 0. �

This result implies that under these conditions the presence of unbounded variables doesnot affect the behavior of the LSSN algorithm.

Let us compute the (Primal and Dual) Log Tapia indicators for some problems, where thesolution set are not bounded. Figures 2, 3, and 4 show the Primal and Dual Log Tapia indicatorsgenerated for problems ADLITTLE and BEACONFD, when the LSSN algorithm is applied.Notice that the behavior of the indicators is clearly separated in two groups. In one of them theindicators seem to decrease to minus infinity and in the other group the indicators increase or

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

952 M. D. González-Lima

Figure 1. Problem AFIRO.

behave in a zig-zagging way until a constant value is attained. This behavior was also observedfor the Dual Multipliers Log Tapia indicators.

If some logarithmic Tapia indicator converges to a constant value we would like to con-clude that the corresponding variable is unbounded (or null). But without the assumption

Figure 2. Problem ADLITTLE.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

Identification of variable in IPM 953

Figure 3. Problem BEACONFD.

Figure 4. Problem BEACONFD.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

954 M. D. González-Lima

of F+ �= ∅, there is not much that can be proved. However, the next propositions considerdifferent cases that we may have for any sequence {(xk, yk, zk)} ∈ R

2n×m. Their proofs arestraightforward. We present them for the primal variable but similar results apply for the dualand dual multiplier variables.

PROPOSITION 6.4 If limk→∞ Log T (xkjo) = ∞, then the sequence {xk

jo} is unbounded.

PROPOSITION 6.5 If limk→∞ Log T (xkjo) = C < 0, then the sequence {xk

jo} converges Q-

linearly to zero or it is unbounded.

These propositions suggest that, under the conditions that k ∈ I(μ), Axk − b → 0 andAtyk + zk − c → 0, the followings are satisfied (similar results apply to the dual variables):

• Log T (xkjo) → ∞ ⇒ jo ∈ U .

• Log T (xkjo) → C ⇒ jo ∈ U

⋃N : if T (xk

jo) → d > 1 ⇒ jo ∈ U , if T (xk

jo) → d < 1 ⇒

jo ∈ N

Based on the results and observations from this section a numerical procedure for detectingand removing unbounded variables was designed. In the next section we present the procedureand the numerical results obtained by using it. The logarithmic Tapia indicators have thepractical advantage over the Tapia indicators that bounded and unbounded (primal and dual)variables are clearly separated. We will use the Tapia indicators in our numerical procedurein order to separate the unbounded and null variables.

7. Numerical Procedure

1. Consider a starting point (x0, y0, z0) with x0 > 0, z0 > 0.2. Apply the LSSN algorithm with k0 being the allowed maximal number of iterations (usu-

ally k0 is chosen between 15 and 30 iterations, depending on the size of the problem).Alternatively, the LSSN algorithm may be stopped when ill-conditioning is affecting theperformance of the algorithm.

3. Find the variables (primal, dual and dual multipliers) such that the corresponding Log Tapiaindicators are not close to minus infinity and their values have approximately become aconstant. In practice, we look for the variables (xi, yj , zi) such that Log T (x

k0i ) > −ε,

Log T (yk0j ) > −ε and Log T (z

k0i ) > −ε and the slope of the Logarithmic Tapia indi-

cators become close to zero for the last five consecutive iterations. In our numericalexperimentation, ε = 2 has worked fine.

Let us denote U1, U2 and U3 as the sets formed by the primal, dual and dual multipliersvariables found in this step.

4. If cU1⋃

U2 = 0, then remove the set columns in the matrix A corresponding to indices inU1

⋃U2. Remove the corresponding positions in c. Remove the rows in A corresponding

to U3 and the corresponding positions in b. Look for a starting point of the new problemand apply the LSSN Algorithm.

5. If cU1⋃

U2 �= 0, let di , for i ∈ U1⋃

U2, the values of the Tapia indicators at the last five iter-ations. Then, U = {i ∈ U1

⋃U2 : di > 1} and N = {i ∈ U1

⋃U2 : di < 1}. In a similar

fashion the sets Udm ⊆ U3 and Ndm ⊆ U3 are defined.6. Formulate a new linear programming problem using Theorem 4.2 and use the LSSN

algorithm for solving the problem stated.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

Identification of variable in IPM 955

The numerical procedure was applied to a subgroup of problems from Netlib. We say theLSSN algorithm converges (and a solution is found) when

max

( |(xk)t (zk)|1 + |btyk| ,

‖Axk − b‖1 + ‖x‖ ,

‖Atyk + zk − c‖1 + ‖yk‖ + ‖zk‖ ,

‖XkZke − μe‖μ

)≤ 10−8.

The LSSN algorithm stops when convergence is attained or when the number of iterationsreaches 120.

Table 2 shows the results obtained.All the problems were solved without using Theorem 4.2.The optimal objective value coincides with the optimal value for the original problem, and wewere able to always recover the solution of the original problem. Table 2 contains informationabout the proximity reached to the analytic center of the new problem solved (i.e., (‖X∗Z∗e −μe‖/μ) with (x∗, y∗, z∗) the approximate solution found) and the values of the feasibilityconstraints and the relative duality gap at the solution, in order to compare with the resultspresented in Table 1. Table 2 also contains the dimensions of the matrix A before and afterapplying the numerical procedure.

We stress that the linear systems arising at each iteration of the LSSN algorithm weresolved by using the whole Jacobian matrix. We know that this is not an efficient way tosolve the systems, but in this way we obtained fast information from the Logarithmic Tapiaindicators. Alternatively, we also solved the systems by decomposing them and forming thesmaller so-called normal equations system. In this case, the solutions of the systems are foundfaster than when using the whole Jacobian matrix. Therefore, large problems can be tested.However, the numerical procedure seemed to work better, in terms of faster information aboutthe unbounded variables, when the whole Jacobian matrix was considered. Because in thiswork we were interested in illustrating the effect of removing the unbounded variables, wechose this way of solving the systems.

7.1 Other indicator functions

Other indicator functions, besides the logarithmic Tapia indicators and the Tapia indicators,may be used together with the LSSN algorithm to identify unbounded variables. The mostimmediate ones are the variables as indicators. Several tests were performed with problemsfrom Netlib in order to compare between the different indicators. The variable as indica-tors have (also when used for identifying zero variables) the disadvantage that it is noteasy to set the value that these indicators have to have, in order to conclude that the cor-responding variable is unbounded. Therefore, reliable information is not always obtainedearly on in the process, and wrong information can be derived from the indicators. It isimportant to notice that some variables can have large values at the solution without beingunbounded, as in the case of Problem SHARE1B, where the primal variable as indicatorfor one of the variables has the value of 13 × 105 at the solution but the solution set isbounded.

However, the variables as indicators can be used jointly with the logarithmic Tapia indicatorsto help to identify the unbounded variables in the primal and dual problems. Sometimes theinformation obtained by using the variables as indicators is relevant to separate different typesof variables or constraints, and these indicators give faster information than the logarithmicTapia indicators.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

956M

.D.G

onzález-Lim

a

Table 2. Numerical procedure applied to NETLIB problems (σ 0 = 0.01).

Problem ||X∗z∗e − μe||/μ ||Au∗ − b|| ||Aty∗ + z∗ − c|| (x∗)t (z∗)/1 + |bt y∗| Original size of A(m × n) Final size of A(m × n)

BEACONFD (29†) 1.3 × 10−15 7.4 × 10−11 3.8 × 10−15 4.8 × 10−15 173 × 295 121 × 213ISRAEL (63) 1.4 × 10−13 1.4 × 10−10 7.3 × 10−13 6.7 × 10−9 174 × 316 174 × 316STOCFOR1 (38) 1.8 × 10−11 2.2 × 10−12 3.3 × 10−13 2.0 × 10−11 117 × 165 117 × 165SCFXM1 (53) 8.5 × 10−11 6.3 × 10−12 3.3 × 10−14 1.3 × 10−12 330 × 600 319 × 578BANDM (52) 1.6 × 10−15 2.7 × 10−13 6.6 × 10−15 4.1 × 10−9 305 × 472 296 × 451BRANDY (45) 1.4 × 10−15 6.3 × 10−7 4.1 × 10−12 4.1 × 10−9 193 × 303 149 × 259E226 (53) 2.4 × 10−10 9.5 × 10−14 5.9 × 10−15 2.0 × 10−11 223 × 472 205 × 435SC105 (20) 2.1 × 10−15 1.7 × 10−13 2.5 × 10−16 1.6 × 10−9 105 × 163 104 × 162SC205 (32) 1.9 × 10−15 7 × 10−13 9.6 × 10−17 4.7 × 10−9 205 × 317 203 × 315SC50b (17) 8.7 × 10−16 7.2 × 10−14 2.5 × 10−16 4.3 × 10−11 50 × 78 48 × 76ADLITTLE (33) 3.1 × 10−10 5.2 × 10−13 1.1 × 10−12 4.3 × 10−11 56 × 138 55 × 137AGG (72) 6.2 × 10−10 1.4 × 10−9 1.8 × 10−12 2.5 × 10−11 488 × 615 457 × 545

†Number of Iterations required for convergence after the numerical procedure was applied.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

Identification of variable in IPM 957

8. Conclusions

In this paper, we considered the unboundedness of the solution set for linear programmingproblems. We introduced the topic by proving that unbounded variables in the primal problemcorrespond to null variables in the dual problem, and vice verse. Therefore the relative interiorof the feasibility set is not empty if and only if the solution set is bounded. A key assumptionin the theoretical development and study of primal–dual interior-point methods is the non-emptiness of the interior of the feasibility set. We showed that this assumption may not besatisfied in real world problems. We also showed that this assumption is fundamental for thegood performance of primal–dual algorithms that generate iterates converging to the analyticcenter of the solution set as the primal-dual LSSN algorithm. This algorithm, together withthe Tapia and logarithmic Tapia indicator function introduced by El–Bakry, et al. [21], wereused to develop a numerical procedure to detect the variables and constraints related to theunboundedness of the solution set. We presented this procedure and showed how it couldbe effectively used for several problems from Netlib. We supported our numerical procedurewith theoretical results that explain and relate the behavior of the algorithm and the indicatorfunctions considered. Comments on the use of other indicator functions were also included.

Acknowledgement

This research was partially supported by BID-CONICIT (project I-06, subproyecto PAN I),CONICIT G97-000592, and DID-USB (project GID001).

References

[1] González-Lima, M.D., Tapia, R.A. and Potra, F.A., 1998, On effectively computing the analytic center of thesolution set. SIAM Journal on Optimization, 8(1), 1–25.

[2] Todd, M., 1990, The effects of degeneracy and null and unbounded variables on variants of Karmarkar’s linearprogramming algorithm. In: T.F. Coleman and Y. Li (Eds). Large-Scale Numerical Optimization (Philadelphia:SIAM), pp. 81–91.

[3] Wright, S., 1997, Primal-Dual Interior-Point methods, SIAM editors.[4] Goffin, J.-L. and Sharifi Mokhtarian, F., 1999, Using the primal dual infeasible Newton method in the analytic

center method for problems defined by deep cutting planes. Journal of Optimization Theory and Applications,101, 35–58.

[5] Goffin, J.-L., Gondzio, J., Sarkissian, R. andVial, J.P., 1997, Solving Nonlinear Multicommodity Flows Problemsby the Analytic Center Cutting Plane Method. Mathematical Programming, 76(1), 131–154.

[6] González-Lima, M.D., Tapia, R.A. and Thrall, R., 1996, On the construction of strong complementarity solu-tions for DEA linear programming problems using a primal-dual interior-point method. Annals of OperationsResearch, 66, 139–162.

[7] Thompson, R., Brinkmann, E., Dharmapala, P., González-Lima, M.D. and Thrall, R., 1997, DEA/AR profitratios and sensitivity of 100 large US banks. European Journal of Operations Research, 98, 213–229.

[8] Thompson, R., Dharmapala, P., Diaz, J., González-Lima, M.D. and Thrall, R., 1996, DEA analytic center mul-tiplier sensitivity with an illustrative application to independent oil companies. Annals of Operations Research,66, 163–177.

[9] Bergamaschi, L., Gondzio, J. and Silli, G., 2004, Preconditioning indefinite systems in interior point methodsfor optimization. Computational Optimization and Applications, 28, 149–171.

[10] Zhang, Y., 1999, User’s guide to LIPSOL - linear-programming interior point solvers. Special Issue ofOptimization Methods and Software on Interior Point Methods (with software in compact disk), 385–396.

[11] Gondzio, J., 1995, HOPDM (version 2.12) - A fast LP solver based on a primal-dual interior point method.European Journal of Operations Research, 85, 221–225.

[12] Gondzio, J., 1997, Presolve analysis of linear programs prior to applying the interior point method. INFORMSJournal on Computing, 9(1), 73–91.

[13] Rockafellar, R.T., 1972, Convex Analysis (2nd edn). (Princeton, NJ: Princeton University Press).[14] El-Bakry, A., Tapia, R.A. and Zhang, Y., 1994, On the use of indicators in identifying zero variables for interior

point methods. SIAM Review, 36(1), 45–72.[15] McLinden, L., 1980, An analogue of Moreau’s approximation theorem, with application to the nonlinear

complementarity problem. Pacific Journal of Mathematics, 88, 101–161.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014

958 M. D. González-Lima

[16] Megiddo, N., 1989, Pathways to the optimal set in linear programming. In: N. Megiddo (Ed.). Progress inMathematical programming, interior-point and related methods (New York: Springer-Verlag) pp. 131–158.

[17] Zhang, Y. and Tapia, R.A., 1991, On the convergence of interior-point methods to the center of solution set inlinear programming. Technical Report TR91-30, Department of Computational and Applied Mathematics, RiceUniversity.

[18] González-Lima, M.D., 2003, Enhancing the behavior of interior-point methods via identification of variables.Technical Report 2003–04, Universidad Simon Bolivar, Centro de Estadistica y Software Matemático (CESMa).Available online at www.cesma.usb.ve.

[19] Kojima, M., Mizuno, S., and Yoshise, A., 1989, A primal-dual interior point method for linear programming.In: Nimrod Megiddo (Ed.) Progress in Mathematical programming, interior-point and related methods (NewYork: Springer-Verlag) pp. 29–47.

[20] Lustig, I., Marsten, R. and Shanno, D., 1992, On implementing Mehrotra’s predictor-corrector interior pointmethod for linear programming. SIAM Journal on Optimization, 2, 435–449.

[21] El-Bakry, A., González-Lima, M.D., Tapia, R.A. and Zhang,Y., 1997, Identifying groups of variables in interior-point methods. Thechnical Report TR93-35 (revised July 97), Department of Computational and AppliedMathematics, Rice University, U.S.A.

Dow

nloa

ded

by [

Nor

th C

arol

ina

Stat

e U

nive

rsity

] at

13:

21 2

5 N

ovem

ber

2014