Decision ProceduresAn Algorithmic Point of ViewGaussian Elimination and Simplex

Gaussians eliminationGiven a linear system Ax = b

Manipulate A|b to an upper-triangular form

A x = b

Gaussians eliminationThen, solve backwards from the ks row according to:

Gaussian elimination - exampleAnd now x3 = -1, x2 = 3, x1 = 1 problem solved.

Feasibility with SimplexSimplex was originally designed for solving the optimization problem:

s.t.

We are only interested in the feasibility problem.Is this system optimal ? Is this system feasible ?

General simplexWe will learn a variant called general simplex. Very suitable for solving the feasibility problem fast.The input:

A is a m n coefficient matrixThe problem variables:

First step: convert the input to general form

General formGeneral form:

A combination of: Linear equalities of the formLower and upper bounds on variables.

Converting to General FormA: Replace (where ) with and

s1,..., sm are called the additional variables.

Example 1Convert to: It is common to keep the conjunctions implicit

Example 2Convert to:

Simplex basicsLinear inequality constraints, geometrically, define a convex polyhedron.

Wikipedia

Our example from before, geometricallyGeneral Simplex begins in the origin...

Matrix formRecall the general form:Due to the additional variables:now A is an m (n + m) matrix.

xys1s2s3

The tableauThe diagonal part is inherent to the general form

We can instead write:

This is called the tableau

The tableauThe tableau changes throughout the algorithm, but maintains its m n structure

Distinguish between basic and nonbasic variablesInitially, basic variables = the additional variables.Basic variablesNonbasic variables

The tableauDenote by B Basic variables N Nonbasic variablesThe tableau is simply a rewrite of the system:

The basic variables are also called the dependent variables.

The general simplex algorithmSimplex maintains: The tableau, an assignment to all variablesThe bounds

Initially, B = additional variables N = problem variables (xi) = 0 for i 2 {1,...,n+m}

InvariantsTwo invariants are maintained throughout: All nonbasic variables satisfy their bounds

Can you see why these invariants are maintained initially ?We should check that they are indeed maintained

The general simplex algorithmThe initial assignment satisfies If the bounds of all basic variables are satisfied by , return `Satisfiable.

Otherwise... pivot.

PivotingFind a basic variable xi that violates its bounds.Suppose that (xi) < liFind a nonbasic variable xj such that aij > 0 and (xj) < uj, oraij < 0 and (xj) > ljWhy ?

PivotingFind a basic variable xi that violates its bounds.Suppose that (xi) < liFind a nonbasic variable xj such that aij > 0 and (xj) < uj, oraij < 0 and (xj) > ljSuch a variable xj is called suitable.If there is no suitable variable return UnsatisfiableWhy ?

Pivoting xi with xjSolve equation i for xj:From:

To:

Swap xi and xj, and update the i-th row accordingly.From

To:

-ai1 aij...1 aij...-ainaij

Pivoting xi with xj

Update all other rows:Replace xj with its equivalent obtained from row i:

PivotingUpdate as follows: Increase (xj) by Now xj is a basic variable: it can violate its bounds.

Update (xi) accordinglyQ: What is now (xi) ?

Update for all other basic (dependent) variables.

ExampleRecall the tableau and constraints in our example:

Initially assigns 0 to all variablesBounds of s1 and s3 are violated

ExampleRecall the tableau and constraints in our example:

We will solve s1x is a suitable nonbasic variable for pivotingIt has no upper boundSo now we pivot s1 with x

ExampleRecall the tableau and constraints in our example:

Solve 1st row for x: Replace x with s1 in other rows:

ExampleThe new state:

Solve 1st row for x: Replace x with s1 in other rows:

ExampleThe new state:

What about the assignment ? We should increase x by Hence, (x) = 0 + 2 = 2 Now s1 is equal to its lower bound: (s1) = 2 Update all the others

ExampleThe new state:

Now s3 violates its lower boundWhich nonbasic variable is suitable for pivoting ? Thats right y

ExampleThe new state:

We should increase y by

ExampleThe final state:

All constraints are now satisfied

ObservationsThe additional variables: Only additional variables have bounds.These bounds are permanent.Additional variables exit the base only on extreme points (their lower or upper bounds).When entering the base, they shift towards the other bound and possibly cross it (violate it).

ObservationsCan it be that we pivot(xi,xj) and then pivot(xj,xi) and enter a (local) cycle ? No.For example, suppose that aij > 0.We increased (xj) so now (xi) = li.After pivoting, possibly (xj) > ujBut aij = 1 / aij > 0, hence xi is not suitable.

ObservationsIs termination guaranteed ?Not obvious. Perhaps there are bigger cycles.In order to avoid circles, we use Blands rule: determine a total order on the variables. Choose the first basic variable that violates its bounds, and first nonbasic suitable variable for pivoting. It can be proven that this guarantees that no base is repeated, which implies termination.

General-Simplex with Blands rule

For a variable x not necessarily greater than 0, replace it with x x, where x, x >=01. Because x = 0 for all x.2. The initial set of nonbasic variables (original variables) do not have a bound...Such a variable x_j has slack to allow us to fix the assignment to x_iFor example in the first case we can increase x_j up to u_j. This will also increase the value of x_i.There is no way to fix the assignment to x_i.A: \alpha(x_i) = l_i