Simplex Algorithm

Algorithms Design &Techniques

Muhammad Kashif NawazMS(CS) Batch # 3

[email protected]

mailto:[email protected]

Contents Overview• AGENDA

– Introduction

• KEY TERMINOLOGIES

• SIMPLEX ALGORITHM

• Software Simulation

• SIMPLEX ALGORITHM VARIATIONS

• REFERENCES

04/11/2023 Simplex Algorithm 2

Introduction(Background)

• Linear programming is the process of taking various linear inequalities relating to some situation, and finding the "best" value obtainable under those conditions.

• Linear programming is part of a very important area of mathematics called optimization techniques.

• The graphical method is useful only for problems involving two decision variables and relatively few problem constraints.


Introduction

• Question:– Can we solve all LP problems using graphical approach ?

• NO – WHY ???

» Real life systems can have dozens or hundreds of variables, or more.

• What happens when we need more decision variables and more problem constraints?– Thus we need another systematic approach to solve the LP Problem. – Simplex Algorithm


Introduction

• Simplex method which was developed by George B. DANTZIG (1914-2005) in 1947.

• The most popular method used for the solution of Linear Programming Problems (LPP) is the simplex method.

• The journal Computing in Science and Engineering listed it as one of the top 10 algorithms of the twentieth century.

• Simplex Algorithm can solve an equation for many variables (dimensions).


Key Terminologies• Standard Form & Slack Variable• Basic & Non basic Variables• Simple Tableau• Pivoting


Standard Maximization Problems in Standard Form

• A linear programming problem is said to be a standard maximization problem in standard form if its mathematical model is of the following form:

• Maximize the objective function

• Subject to problem constraints of the form

• With non-negative constraints


Slack Variables

• A mathematical representation of surplus resources.

• In real life problems, it’s unlikely that all resources will be used completely, so there usually are unused resources.

• Slack variables represent the unused resources between the left-hand side and right-hand side of each inequality.


x1 + 2x2 < 323x1 + 4x2 < 84

x1 + 2x2 + s1 = 323x1 + 4x2 + s2 = 84

Basic and Nonbasic Variables

• Basic variables are selected arbitrarily with the restriction that there be as many basic variables as there are equations. The remaining variables are non-basic variables.

• This system has two equations, we can select any two of the four variables as basic variables. The remaining two variables are then non-basic variables.

• A solution found by setting the two non-basic variables equal to 0 and solving for the two basic variables is a basic solution. If a basic solution has no negative values, it is a basic feasible solution.


Simple tableau

• Simplex Tableau: Most real-world problems are too complex to solve graphically. They have too many corners to evaluate, and the algebraic solutions are lengthy. A simplex tableau is a way to systematically evaluate variable mixes in order to find the best one.

• Pivot Column: The column of the tableau representing the variable to be entered into the solution mix.

• Pivot Row: The row of the tableau representing the variable to be replaced in the solution mix.

• Pivot Number: The element in both the pivot column and the pivot row.


All Variables Solutions

Basic Variables Coefficients

0

Pivoting

• Pivoting : – To transform a current tableau into the next one and increase the

value of the objective function to the next feasible solutions. To get to that feasible solution several steps are performed and those steps are called pivoting.

– Steps:• First, divide all the entries of the pivot row by the pivot, its entry in the pivot column, to obtain

Rownew • Then, replace each of the other rows, including the objective row, by the difference row −

c .Rownew

• where c is the row’s entry in the pivot column


Simplex Algorithm• Diagram• Steps• Example• Cycling & Bland’s rule• Efficiency


SIMPLEX METHOD


Step-1Write the standard

maximization problem in

standard form, introduce slack

variables to form the initial

system, and write the initial

tableau.

Step-3Select

the pivot

column

Step-5Select

the pivot element

and perform the pivot operatio

n

STOPThe optimal solution has been

found.

STOPThe linear programming problem

has no optimal solution

Step 2Are there

any negative

indicators in the

bottom row?

Step 4Are there

any positive elements in

the pivot column

above the dashed

line?

Simplex algorithm for standard maximization problems

YESYES

NONO

To Solve A Linear Programming Problem In Standard Form, Use The Following Steps

• Convert each inequality in the set of constraints to an equation by adding slack variables.

• Create the initial simplex tableau.• Select the pivot column. ( The column with the “most negative value” element in

the last row.)• Select the pivot row. (The row with the smallest non-negative result when the last

element in the row is divided by the corresponding in the pivot column.)• Use elementary row operations calculate new values for the pivot row so that the

pivot is 1 (Divide every number in the row by the pivot number.)• Use elementary row operations to make all numbers in the pivot column equal to 0

except for the pivot number. If all entries in the bottom row are zero or positive, this the final tableau. If not, go back to step 3.

• If you obtain a final tableau, then the linear programming problem has a maximum solution, which is given by the entry in the lower-right corner of the tableau.


Example(Step 1)

Maximize = 3x + 5y

Subject to:

x + y < 4

x + 3y < 6

x, y > 0


Maximize = 3x + 5y + 0u + 0v

Subject to:

x + y + u = 4

x + 3y + + v = 6

x, y, u, v > 0

Original Form Standard Form with slack

Example(Step 2)


u

v

z

1 1 1 0 4

1 3 0 1 6

-3 -5 0 0 0

Initialized by the coefficients of the objective function. The signs are reversed in this row (objective row)

We have constraints + 1 rows and variables +1 columns

Pivot Column/Entering Variable. In our example, its Y

Example(Step 3)x y u v

u 1 1 1 0 4

v 1 3 0 1 6

z -3 -5 0 0 0


Pivot Column/Entering Variable

The row with the smallest θ-ratio determines the departing variable. For our example, it is variable v. The intersecting element of Pivot Column and Pivot Row is the pivot.

In the next tableau the intersected element will become 1 and the rest of elements will become 0 in the pivot column.

Departing Variable / Leaving Variable/ Pivot Row

Example(Step 4 Iteration 1)x y u v

u 1 1 1 0 4

v 1 3 0 1 6

z -3 -5 0 0 0


x y u v

u 1 1 1 0 4

y 1/3

1 0 1/3

2

z -3 -5 0 0 0

x y u v

u 2/3 0 1 -1/3

2

y 1/3 1 0 1/3 2

z -4/3

0 0 5/3 10

The pivot row will be replaced by the Rownew we calculated.Row 1 and 3 will be calculated using the formula :Row = Row – c.Rownew

Row1 –c.Rownew where c = 11-(1)(1/3), 1-(1)(1), 1-(1)(0), 0-(1)(1/3), 4-(1)(2)

2/3, 0, 1, -1/3, 2

Row3- c.Rownew where c = -5-3-(-5)(1/3), -5-(-5)(1), 0-(-5)(0), 0-(-5)(1/3), 0-(-5)(2)

-4/3, 0, 0, 5/3, 10

All the values in the last row are nonnegative ??

Example(Step 4 Iteration 2)x y u v

u 2/3

0 1 -1/3

2

y 1/3

1 0 1/3

2

z -4/3

0 0 5/3

10


x y u v

x 1 0 3/2

-1/2

3

y 0 1 -1/2

1/2

1

z 0 0 2 1 14

x y u v

x 1 0 3/2

-1/2

3

y 1/3

1 0 1/3

2

z -4/3

0 0 5/3

10

ϴ-Ratio: 3

ϴ-Ratio: 6

Row = Row – c.Rownew

Row2 –c.Rownew where c = 1/31/3-(1/3)(1), 1-(1/3)(0), 0-(1/3)(3/2), 1/3-(1/3)(-1/2), 2-(1/3)(3)

0, 1, -1/2, 1/2, 1

Row3- c.Rownew where c = -4/3-4/3-(-4/3)(1), 0-(-4/3)(0), 0-(-4/3)(3/2), 5/3-(-4/3)(-1/2), 10-(-4/3)(3)

0, 0, 2, 1, 14

All the values in the last row are nonnegative ??

Row new : 1, 0, 3/2, -1/2, 3

Example(Optimal Solution)

• x = 3• y = 1 • u = 0• v = 0• z = 14• We choose u and v equal to 0, because their column has all distinct values.

If they were like x and y, they’d have some value.


Cycling & Bland’s Rule

• When an objective function’s values “stall” for several iterations in a row and that the algorithm cycles back to a previously considered point and hence never terminate. This phenomenon is called cycling.

• Although, it rarely happens, but when it does ,a simple modification of the simplex method, called Bland’s rule, eliminates even the theoretical possibility of cycling.

• Assuming that the variables are denoted by a subscripted letter (e.g., x1, x2,..., xn), this rule can be stated as follows: – Among the columns with a negative entry in the objective row, select the

column with the smallest subscript in choosing pivot column. – Resolve a tie among the smallest θ-ratios by selecting the row labeled by the

basic variable with the smallest subscript in choosing pivot row.


Software Simulation

• Simplex Method has became so efficient that its calculation packages are even available online. One of them is as follows:

• For ease, we will copy the same problem to the online tool and check whether it gives us the same result: maximize z = 3x +5y subject to :x + y = 4x+3y = 6

• Simplex Method Online Tool http://www.zweigmedia.com/RealWorld/simplex.html


http://www.zweigmedia.com/RealWorld/simplex.html

Efficiency of Simplex Algorithm

• Since we know that the algorithm progresses through a sequence of adjacent points of a feasible region. Right?

• So, one should probably expect bad news because the number of extreme points is known to grow exponentially with the problem size.

• The worst-case efficiency of the simplex method has been shown to be exponential.

• But Fortunately!• More than half a century of practical experience with the algorithm has

shown that the number of iterations in a typical application ranges between m and 3m, with the number of operations per iteration proportional to mn, where m and n are the numbers of equality constraints and variables, respectively.


Other Variants of the Algorithm

• Ellipsoid Method:– An important mile- stone in the history of such algorithms was the

proof by L. G. Khachian showing that the ellipsoid method can solve any linear programming problem in polynomial time.

– However, ellipsoid method was much slower than the simplex method in practice

• Karmarker’s Method– Narendra Karmarkar published an algorithm that not only had a

polynomial worst-case efficiency but also was competitive with the simplex method. This method generates a sequence of feasible solutions that lie within the feasible region rather than going through a sequence of adjacent extreme points as the simplex method does. (Interior Point Methods).


Reference(s)

• The material presented in this lecture is adopted from:• Introduction to Design and Analysis of Algorithms, 3rd Edition,

Anany Levitin. • Introduction to Algorithms, Second Edition,

Thomas.H.Cormen• www.iftikharahmad.org/teachings/Lec_16_Simplex_Algorith

m.pdf


http://www.iftikharahmad.org/teachings/Lec_16_Simplex_Algorithm.pdf

http://www.iftikharahmad.org/teachings/Lec_16_Simplex_Algorithm.pdf

Education

Simplex Algorithm