Transcript Chapter 4. The Simplex Method

Chapter 4. Solving Linear Programs
The Simplex Method
LP Model: The Augmented (Canonical) Form
max
s.t.
max
3 x1 + 5 x2
+x3
x1
+x4
2 x2
+x5
3 x1 + 2 x2
x3, x4, x5
x1, x2,
cx
1
1
=4
= 12
= 18
>= 0
+ c2 x 2 +  + cn x n
s.t.
a x
a x
11
1
21
1
+ a12 x 2 +  + a1n x n + X n +1  b1
+ a 22 x 2 +  + a 2n x n + X n + 2  b2


a x + a x ++ a x
x  0 , x  0 ,, x , X
x : Slack Variable.
m1
1
n +i
1
m2
2
2
mn
n
n
n +1
+ X n + m = bm
,  , Xn +m  0
x1
2 x2
3 x1 + 2 x2
+ x3
=4
+ x4
= 12
+ x5 = 18
Augmented Solution: A solution for the original
variables that has been augmented by the corresponding
values of the slack variables.
In the standard form: (3, 2)
In the augmented form: (3, 2, 1, 8, 5)
How does the augmented corner solution look like?
Rank of Matrix A
a
 11

 a21
A
...


a
 m1



a2n 

... 

amn 
a12
... a1n
a22
...
...
am2
...
...
A has full rank if and only if
Ax  0
 x0
and
AT y  0

y  0.
Basis of Matrix A
A have full rank and n  m. Then, B is a
basis of A, if B is a full - rank m  m sub - matrix of A
and its m columns are selected m columns of A.
Let m  n matrix
Example :
a
 11

 a21
B
...


a
 m1



a2m 
.
... 

amm 
a12
... a1m
a22
...
am2
...
...
...
A  (B, N).
Basic Solution of Ax=b
Consider t he matrix equation
For a selected basis
Ax  b.
B of A, the equation can be
decomposed as
BxB  NxN b.
B, is
xN 0.
The basic solution, correspond ing to basis
1
xB  B b
and
A Basis Solution of the Augmented System
x1
+ x3
2 x2
3 x1 + 2 x2
1

A  0
3

0
2
2
1
0
0
=4
+ x4
= 12
+ x5 = 18
0 0
1 0 0 



1 0 . Select B  ( A3 , A4 , A5 )   0 1 0 
0 0 1 
0 1 


x
 x
x

3
xB
4
5




 x1 
and x N   .
 x2 
4 
 
1
x B  B b  12 
 
8 
and
 0
x N   .
 0

Basic Solution: An augmented corner-point solution.
Basic variables are x1, x2 and x5
In the standard form: (4, 6)
In the augmented form: (4, 6, 0, 0, -6)

Basic Feasible Solution: An augmented feasible cornerpoint solution.
Basic variables are x2, x3 and x5
In the standard form: ( 0, 6)
In the augmented form: ( 0, 6, 4, 0, 6)
Basic Solution of the Augmented Form
Basic Variable:
– A basic variable solution has at least n variables, original or
slack, whose value are zero. These variables are called nonbasic variables, and the others are called basic variables.
( 4, 6, 0, 0, -6)
( 0, 6, 4, 0, 6 )
– A basic solution can be obtained directly form the augmented
form by setting n ( non-basic ) variables equal to zero and
solving m equations for m ( basic ) variables
– The coefficient matrix of the m equations is called basis.
– If the values of the m basic variables are nonnegative, then it is
a basic feasible solution corresponding to a feasible cornerpoint in the original form.
Adjacent Basic Solutions
– Two basic solutions are adjacent if all but one of their
basic variables are the same. Two adjacent basic
solutions represent two neighbor corner-point in the
original form.
( 4, 6, 0, 0, -6 )
and
( 4, 3, 0, 6, 0 )
– Two basic feasible solutions are adjacent if all but one
of their basic variables are the same. Two adjacent basic
feasible solutions represent two feasible neighbor
corner-point in the original form.
( 4, 0, 0, 12, 6 )
and
( 4, 3, 0, 6, 0 )
Algebraic Outline of the Simplex Method
Start from a basic feasible solution
Is there a ‘better’ adjacent basic feasible solution exists?
NO
YES
Move to a better adjacent basic feasible solution
– The movement involves converting one non-basic variable into a basic
variable ( called entering basic variable ) and simultaneously converting a
basic variable into a non-basic variable ( called the leaving basic variable ),
and then compute the new basic feasible solution.
Z = 3x1 + 5x2
x1
2x2
3x1 + 2x2
+ x3
=4
+ x4
= 12
+ x5 = 18
1. What is the criterion for selecting the entering basic
variable or to show that the current basic feasible solution
is optimal?
The candidate for the entering basic variables are the n current
nonbasic variables. The one chosen would be changed from nonbasic
into basic, so its value would be increased from zero to some positive
number and the others would be kept at zero. Since we require an
improvement, the rate of change in the objective function must be a
positive one.
Z = 3x1 + 5x2
x1
2x2
3x1 + 2x2
+ x3
=4
+ x4
= 12
+ x5 = 18
1. What is the criterion for selecting the entering basic
variable or to show that the current basic feasible solution
is optimal?
Express the objective function in terms of only nonbasic variables and
also the basic variables in terms of the nonbasic variables---called
Canonical Form. Then, the coefficient of the entering variable must be
positive. If the coefficients are all nonpositive, then no better adjacent
basic feasible solution exists and the current one is optimal.
2. How to select the leaving basic variable?
Z = 5 X2
+ X3
2X2
2X2
X3
=4
+X4
= 12
+ X5 = 18
=4
X4
= 12 - 2X2
X5 = 18 - 2X2
2. How to select the leaving basic variable?
Z = 5 X2
+ X3
2X2
2X2
X3
=4
+X4
= 12
+ X5 = 18
=4
X4
= 12 - 2X2
X5 = 18 - 2X2
3. How to compute the new basic feasible solution?
Z = 3X1
X1
- (5/2) X4 +30
+ X3
=4
X2
+ ( 1/2 ) X4
=6
X1
X4 + X5 = 6
Iteration 2 for the example:
Z = 3 X1
X1
+ X3
X2
3X1
Z=
+ 30
=4
=6
+ X5 = 6
+ 30 + 3X1
X3
X2
= 4 - X1
=6
X5 = 6 - 3X1
Z=
- (5/2 ) X4
- X5 + 36
X3 + ( 1/3 ) X4 - ( 1/3 ) X5 = 2
X2
X1
+ ( 1/2 ) X4
=6
- ( 1/3 ) X4 + ( 1/3 ) X5 =2
Summary of the Simplex Method
1. Select a basic feasible solution
2. Express the basic variables in terms of the nonbasic variables,
and express the objective function in terms of only nonbasic
variables.
If all objective coefficients are non-positive, then stop: the
current basic feasible solution is optimal.
Otherwise, select the entering variable such that its coefficient is
the largest, and among basic variables select the leaving
variable such that it becomes zero first when increasing the
entering variable and keeping the other nonbasic variables at
zero. If no basic variable becomes zero, then stop: the objective
function is unbounded.
3. Goto Step 2.
The Original Simplex Method: Tabular Form

Gaussian Elimination
X1
+ X3 = 4
X2 - X3 = 1
3X1 + 2X2 - 4X3 = -5
1
0
0
2
3
2
Row 3 - 3 x Row 1:
1
0
0
2
0
2
( Row 2 ) / 2:
1
0
0
1
0
2
1
-1
-4
4
1
-5
1
-1
-7
4
1
-17
1
-1/2
-7
4
1/2
-17
Row 3 - 2 x (Row 2):
1
0
0
0
1
0
1
-1/2
-6
4
1/2
-18
0
1
0
1
-1/2
1
4
1/2
3
0
1
0
1
0
1
4
2
3
0
1
0
0
0
1
1
2
3
(Row 3 ) / (-6 ):
1
0
0
Row 2 + ( Row 3 ) / 2:
1
0
0
Row 1 - Row 3:
1
0
0
max
c x +c x
1
1
2
2
+  + cn x n
s.t.
11
1
12
2
+  + a1n x n + X n +1  b1
21
1
22
2
+  + a 2n x n + X n + 2  b2
a x +a x
a x +a x


a x + a x ++ a x
x  0 , x  0 ,, x
m1
1
m2
1
x
n +i
2
mn
2
: Slack Variable.
max 3 x1+ 5 x2
s.t. x1
+ x3
=4
2 x2
+ x4
= 12
3 x1 + 2 x2
+ x5 = 18
x1, x2, x3, x4, x5
>= 0
n
+ X n + m = bm
n
, X n +1 ,  , X n + m  0
z - c x - c x -  -c x
a x + a x ++ a x + X
a x + a x ++ a x + X
1
1
2
2
n
=0
n
11
1
12
2
1n
n
n +1
 b1
21
1
22
2
2n
n
n +2
 b2

a x +a x
m1
1
m2
2

+  + a mn x n + Xn +m = bm
z - 3 x1 -5 x2
=0
x1
+ x3
=4
2 x2
+ x4
= 12
3 x1 + 2 x2
+ x5 = 18
x1, x2, x3, x4, x5
>= 0
Step 0. Initial Table
Basic
Var..
1
Z
x
x
x

x x
n 1
n2
nm
2
- c1 - c2
a
a
11
21
a
a
12
22


a
a
m1
m2
x
...
n +1
Right-Hand
Side
x
n +m

0

0
0

1

1

0

0
b
b

0
 0


b

0
Basic Variable
1
1
RHS
Z
-3
-5
0
0
0
0
3
1
0
1
0
0
4
4
0
2
0
1
0
12
5
3
2
0
0
1
18
2
m

Step 1. Optimality Test
The current basic feasible solution is optimal if and only if every
coefficient in Row 0 is nonnegative. If it is, stop; otherwise, go to
Step 2.

Step 2. Update the Basic Feasible Solution
1. Determine the variable by selecting the variable with the lowest
coefficient in Row 0. The corresponding column is called the pivot
column.
Basic Variable
RHS
Z
-3
-5
0
0
0
0
3
1
0
1
0
0
4
4
0
2
0
1
0
12
5
3
2
0
0
1
18
Step 2. ( Continued )
2. Determine the leaving variable by
a) picking out each coefficient in the pivot column that is strictly
positive, ( if no positive coefficient exists, then stop; the problem
is unbounded. )
b) dividing each of these coefficients into RHS for the same
row,
c) identifying the row that has the smallest of these ratios, and
d) selecting the basic variable for this row, which is called the
pivot row.
Basic Variable
RHS
Ratio
Z
-3
-5
0
0
0
0
3
1
0
1
0
0
4
4
0
2
0
1
0
12
6 = 12/2
5
3
2
0
0
1
16
9 = 18/2

Step 2. ( Continued )
3. The intersection of the pivot column and pivot row is the pivot
number. Replace the leaving basic variable by the entering basic
variable in Column 0. Perform Gaussian elimination for the pivot
number. Goto Step 1.
Basic Variable
Z
3
2
5
-3
1
0
3
-5
0
2
2
0
1
0
0
0
0
1
0
0
0
0
1
RHS
0
4
12
18
0
0
0
1
RHS
0
4
6
18
( Row 2 ) / 2 :
Basic Variable
Z
-3
3
1
4
0
5
3
-5
0
1
2
0
1
0
0
0
0
1/2
0
Row 3 - 2 x Row 2
Basic Variable
Z
3
2
5
-3
1
0
3
-5
0
1
0
0
1
0
0
0
0
1/2
-1
0
0
0
1
RHS
0
4
6
6
Row 0 + 5 x Row 2
Basic Variable
Z
3
2
5
-3
1
0
3
0
0
1
0
0
1
0
0
5/2
0
1/2
-1
0
0
0
1
RHS
30
4
6
6
Iteration 2:
Basic Variable
RHS Ratio
Z
-3
0
0
5/2
0
30
3
2
5
1
0
3
0
1
0
1
0
0
0
1/2
-1
0
0
1
4
6
6
Basic Variable
RHS
Z
-3
0
0
5/2
0
30
3
1
0
1
0
0
4
2
0
1
0
1/2
0
6
1
1
0
0
-1/3
1/3
2
Basic Variable
RHS
Z
0
0
0
3/2
1
36
3
0
0
1
1/3
-1/3
2
2
0
1
0
1/2
0
6
1
1
0
0
-1/3
1/3
2
4
2
Initial BFS for General Form
Artificial Variable for Equality Constrains
maximize
subject to
3x1 + 5x2
x1
=4
2x2 <= 12
3x1 + 2x2 <= 18
x1, x2 >= 0
maximize
subject to
3x1 + 5x2
x1
=4
2x2 + x4
= 12
3x1 + 2x2
+ x5 = 18
x1, x2, x3, x4
>= 0
maximize
subject to
3x1 + 5x2
x1
+ x3
=4
2x2
+ x4
= 12
3x1 + 2x2
+ x5 = 18
x1, x2, x3, x4, x5
>= 0
maximize
subject to
Basic Variable
Z
3
4
5
Basic Variable
Z
3
4
5
3x1 + 5x2
x1
- Mx3
+ x3
=4
2x2
+ x4
= 12
3x1 + 2x2
+ x5 = 18
x1, x2, x3, x4, x5
>= 0
-3
1
0
3
-103
1
0
3
-5
0
2
2
-105
0
2
2
100
1
0
0
0
1
0
0
0
0
1
0
0
0
1
0
0
0
0
1
RHS
0
4
12
18
0
0
0
1
RHS
-400
4
12
18
The Big M Method
1. Change every > = constraint to <= constraint by multiplying -1
2. Add every <= constraint a slack variable. If the right side of a <=
constraint is nonnegative, then the slack variable is the basic
variable for this row.
3. Multiply -1 to every constraint with the negative right side, and add
artificial variables to be the basic variables for these rows.
4. Add Big M as the objective coefficient for each artificial variable
in the original objective function.
5. Start the simplex method to optimize the new objective function.
The Two-Phase Method
1. Change every >= constraint to <= constraint by multiplying -1.
2. Add every <= constraint a slack variable. If the right side of a <= constraint is
nonnegative, then the slack variable is the basic variable for this row.
3. Multiply -1 to every constraint with the negative right side, and add artificial
variables to be the basic variables for these rows.
4. Minimize the sum of the artificial variables as the Phase-1 linear program.
minimize
subject to
x1
x3
+ x3
=4
2x2
+ x4
= 12
3x1 + 2x2
+ x5 = 18
x1, x2, x3, x4, x5
>= 0
5. After find a BFS, optimize the original objective function as the Phase-2 linear
program.
Tie Breaking
Degeneracy: The number of zero-value variables (including slacks) is
always greater than or equal to the number of original variables. In other
words, the number of positive-value variables is always less than or equal
to the number of inequality constraints. When they are not equal, we have
a degenerate corner or basic feasible solution.
Left-Right and Top-Bottom Rule
Max
s.t.
2x1 - 3x2 + 5x3
8x1 + 5x2 + 4x3 <= 24
x1 - 7x2
<= 17
5x1 + 3x2 + 10x3 <= 60
x1,
x2,
x3 >= 0
Simplex Method:
Basic Variable
RHS Ratio
Z
-2
3
-5
0
0
0
0
4
5
6
8
1
5
-5
-7
3
4
0
10
1
0
0
0
1
0
0
0
1
24
17
60
Basic Variable
6
6
RHS Ratio
Z
8
-13/4
0
5/4
0
0
30
3
2
-5/4
1
1/4
0
0
6
5
1
-7
0
0
1
0
17
6
-15
31/2
0
-5/2
0
1
0
Basic Variable
0
RHS Ratio
Z
301/62
0
0
45/62
0
13/62
30
3
49/62
0
1
3/62
0
5/62
6
5
-179/31 0
0
-35/31
1
14/31
17
2
-30/31
0
-5/31
0
2/31
0
1
Cycling:
Degeneracy can cause the phenomenon called cycling, the
infinite repetition of a finite sequence of basic feasible
solutions producing non change in the value of the objective
and thereby preventing the finite termination of the
Simplex method.
Min
s.t.
x1
x2
x1,
x2,
x3
x3,
- (3/4)x4 +20x5 - (1/2)x6 +
+ (1/4)x4 - 8x5 x6 +
+ (1/2)x4 - 12x5 - (1/2)x6 +
+
x6
x4,
x5,
x6,
6x7
9x7 = 0
3x7 = 0
=1
x7 >= 0
Methods Handling Degeneracy

Random Choice
– Chose the row randomly from among all those
that yield the minimum ratio

Perturbation of the RHS
– bi + i,
i=1,…,m
Lexicographic Ordering
 Double Least Index Rule

– Least indexed variable among all eligible
entering variables
– Least indexed variable among all rows that yield
the minimum ratio