Transcript Part 3. Linear Programming - National Cheng Kung University

Part 3. Linear Programming
3.2 Algorithm
General Formulation
min f  x1 , x2 ,
x
r
, xr    Ci xi
Convex function
i 1
subject to
r
Convex
region
a
i 1
r
a
i 1
x  bj
j  1, 2,
x  bj
j  m  1, m  2,
ji i
ji i
xi  0
i  1, 2,
,m
,p
,r
 A local minimum is a global minimum.
 The minimum must lie at the intersection
of several constraints, but not in the interior
of the convex region.
Example
5
 4

max  V p q p   Cc xc 
c 1
 p 1

s.t.
Profit
q p  Y p1 x1  Y p 2 x2  Y p 3 x3  Y p 4 x4  Y p 5 x5
0  qp  Dp
p  1, 2,3, 4
xc  S c
c=1,2,3
x4  x5  S4
0  xc
Amount of product p
c=1,2,3,4,5
Amount of crude c
Graphical Solution
min 40 x1  36 x2
s.t.
5 x1  3 x2  45
8  x1  0
10  x2  0
Degenerate Problems
Non-unique solutions
Unbounded minimum
Degenerate Problems
– No feasible region
min f  x    x1  x2
s.t.
x1  x2  2
x1  2 x2  0
x1 , x2  0
Simplex Method
– The standard form
min (or max ) f  x   c1 x1  c2 x2 
 cn xn
s.t.
a11 x1  a12 x2 
 a1n xn  b1
a21 x1  a22 x2 
 a2 n xn  b2
am1 x1  am 2 x2 
 amn xn  bm
x1  0, x2  0,
, xn  0
b1  0, b2  0,
, bm  0
or
min (or max ) f  x   cT x
s.t.
A m n x  b
x0
b0
Simplex Method
- Handling inequalities
Ex. slack variables
Ex. surplus variables
x1  2 x2  3x3  4 x4  25
x1  5 x2  x3  7


x1  2 x2  3x3  4 x4  s1  25
where
x1  5 x2  x3  s2  7
where
s1  0
s2  0
Simplex Method
- Handling unrestricted variables
(1) Let
x  unrestricted variable = s   s 
(i.e.,    x   )
where s   0 and s   0
(2) Use one of the equations in the standard form
1
x1   bi  ai 2 xi 2 
ai1
 ain xin 
ans then substitute it into other equations.
Simplex Method
- Calculation procedure
min f  x   3x1  2 x2
s.t.
x1  2 x2  4
3x1  2 x2  14
x1  x2  3
x1 , x2  0
Calculation Procedure
- Step 0
Convert all inequality constraints to the form
in which the RHSs are positive.
 x1  2 x2  4
3x1  2 x2  14
x1  x2  3
Calculation Procedure
- Step 1
Introduce slack/surplus variables and convert
the inequality constraints to equality constraints
 x1  2 x2  x3
3x1  2 x2
x1  x2
 4
 x4
 14
 x5  3
Calculation Procedure
Step 2: find a basic solution corresponding to a
corner of the feasible region.
A m n x  b
 Gauss-Jordan Elimination
 Canonocal Form
x1
x2
xm
x1 , x2 ,
xm1 ,
 a1,m1 xm 1
 a2,m1 xm 1
 a1,n xn
 a2,n xn
 b1
 b2
 am ,m 1 xm 1
 a m , n xn
 bm
, xm : basic variables (dependent variables)
, xn : nonbasic variables (independent variables)
Remarks
• The solution obtained from a cannonical system
by setting the non-basic variables to zero is
called a basic solution (particular solution).
• A basic feasible solution is a basic solution in
which the values of the basic variables are
nonnegative.
• Every corner point of the feasible region
corresponds to a basic feasible solution of the
constraint equations. Thus, the optimum solution
is a basic feasible solution.
Full Rank Assumption
The m rows of Amn (m  n) are
linearly independent, i,e,, the rank
of A mn is m.
Given the set of m simultaneous linear equations
in n unkowns, i.e.
A mn x  b
(1)
Let B be any nonsingular m  m submatrix made
of columns of A. Then, if all n - m components of
x NOT associated with the columns of B are set
equal to zero, the solution to the resulting set of
equations is said to be a basic solution to (1) w.r.t.
the basis B. The components of x associated with
the columns of B are called the basic variables.
A mn  B mm | N m( n  m ) 
x   xTB | xTN 
x N  0  Bx B  b
Fundamental Theorem of Linear
Programming
Given a linear program in standard form
where A is an mxn matrix of rank m.
1. If there is a feasible solution, there is a
basic feasible solution;
2. If there is an optimal solution, there is an
optimal basic feasible solution.
Implication of Fundamental
Theorem
Search in
n
n!
 
 m  m!(n  m)!
basic feasible solutions
Extreme Point
A point x in a convex set C is said to be
an extreme point of C if there are no two
distinct points x1 and x 2 in C such that
x   x1  (1-  )x 2 for some  and 0    1.
Theorem (Equivalence of
extreme points and basic
solutions)
Let A be an m  n matrix of rank m and b an m-vector.
Let K be the convex polytope consists of all n-vectors
x satisfying
Ax = b
x0
A vector is an extreme point iff x is a basic feasible solution.
Corollary
If there is a finite optimal solution to
a linear programming problem,
there is a finite optimal solution
which is an extreme point of the
constraint set.
Step 2
m  number of basic variables
= number of constraints
n  m  number of non-basic variables
x3  x1  2 x2  4
x4  3 x1  2 x2  14
x5  x1  x2  3
f  3 x1  2 x2  0

x1 and x2 are selected as
non-basic variables
x3  4, x4  14, x5  3, f  0
Step 3: select new basic and
non-basic variables
f  3x1  2 x2  0
new basic variable
Exame the coefficients in f to determine which variable
(x1 or x2 ) decreases the value of the objective function when
the variable is increased from zero. As a general rule, it
is advantageous to select as a new basic variable which has
the largest positive coefficient in the objective function. Thus,
x2 remains at zero
x1 increases from zero to a positive number
Which one of x3, x4, x5 should be
selected as the new non-basic
variables?
 x1  2 x2  4  x1,max   

3x1  2 x2  14  x1,max  14 / 3 x1  3
x1  x2  3

x1,max  3 
 x5  0
The limiting value for x1 can be calculated analytically
for each equation by bi /ai1 , i.e. - 4, 14 / 3 and 3. The limiting
constraint is the one for which the ratio is positive and
has the smallest value.
Step 4: Transformation of the
Equations
New basic variables: ( x1, x3 , x4 )
New non-basic variables: ( x2 , x5 )
The transformation can be achieved by substituting x1
as a function of x2 and x5 (the new non-basic variables).
Let
x1  3  x2  x5
x3  x1  2 x2  4
 x3  x2  x5  7
x4  3x1  2 x2  14  x4  5 x2  3x5  5
x5  x1  x2  3  x1  x2  x5  3
f  3x1  2 x2  0  f  5 x2  3x5  9
=0
Repeat step 4 by Gauss-Jordan
elimination
Original System:
-1 2
3 2

 1 1

3 2
1 0 0
0 1 0
0 0 1
0 0 0
 x1 
0   x2   4 
 



0  x3  14
    
0   x4   3 
   
1  x5
0
 
 f 
Write in terms of the augmented matrix
N
N
-1 2
3 2

 1 1

3 2
B
B
B
4

0 1 0 0 14

0 0 1 0 3

0 0 0 1 0
1 0 0 0
Step 3: Pivot Row
Select the smallest positive ratio
bi/ai1
Step 3: Pivot Column
Select the largest positive element in the objective function.
Pivot element
By elementary row operation
B N B B N
0 1 1
0 5 0

1 -1 0

0 5 0
7
5

0 1 0 3

0 3 1 9 
0 1 0
1 3 0
To do a better job in book-keeping,
a "tableau" can be used
x1 x2 x3 x4 x5 f
Basic variables
x3 0 1
x4 0 5
x1 1 -1
0 5
1
0
0
0
0 1
1 3
0 1
0 3
b
0 7
0 5
0 3
1 9
Step 5: Repeat Iteration
x1 x2 x3
x4
x5
x3 0 0 1 -1/5 8/5
x2 0 1 0 1/ 5 3/ 5
x1 1 0 0 1/ 5 2 / 5
0 0 0 1
0
f
b
0 6
0 1
0 4
1 14
An increase in x4 or x5 does not reduce f
x  4, x  1, x  6, f  14
*
1
*
2
*
3
*
Note that the coefficient of x5 is zero.
x1 x2
x3
x4 x5 f
b
x5 0 0 5/8 -1/8 1 0 15/4
x2 0 1 3/ 8 1/ 8 0 0 13/ 4
x1 1 0 1/ 4 1/ 4 0 0 5 / 2
0 0
0
1 0 1 14
x  5 / 2, x  13/ 4, x  15 / 4, f  14
*
1
*
2
Alternative solution!
*
5
*
It is necessary to obtain a first
feasible solution!
[Example]
min f ( x)  x1  x2
s.t.
3x1  x2  7
x1  2 x2  4
STEP 0:
STEP 1:
3x1  x2 - x3  7
x1  2 x2  x4  4

x1  x2  0, x3  -7, x4  4
Infeasible!
Phase I – Phase II Algorithm
• Phase I: generate an initial basic feasible
solution;
• Phase II: generate the optimal basic
feasible solution.
Phase-I Procedure
• Step 0 and Step 1 are the same as before.
• Step 2: Augment the set of equations by
one artificial variable for each equation to
get a new standard form.
New Basic Variables
a11 x1  a12 x2
a21 x1  a22 x2
am1 x



 a1n xn
 a 2 n xn

 amn xn
3x1
 x2
x1
2 x2
 x3
 xn 1
 b1
 b2
 xn  2
 xn m
 x5
 x4
 7
 x6
 4
One feasible solution can always be found:
 basic variables: x5  7, x6  4;
 nonbasic variables: x1  x2  x3  x4  0.
 bm
New Objective Function
If the minimum of this objective function is reached,
then all the artificial variables should be reduced to 0.
m
f   xn k
k 1
f  x5  x6
or
 m

 m

f    ai1  x1    ai 2  x2 
 i 1 
 i 1 
f  4 x1  3x2  x3  x4  11
m
 m

   ain  xn   bi
i 1
 i 1 
Step 3 – Step 5
N
N
N
N
B
B
x1
x2
x3
x4
x5
x6
b
x5
3
1
1
0
1
0
7
x6
1
2
0
1
0
1
4
4
3
1
1
0
0
11
B
N
x1
x6
x1
x2
N
N
N
B
x1 x2
x3
x4
1 1/ 3 1/ 3 0
x5
1/ 3
x6
0
b
7/3
0
0
5/3
5/3
1/ 3
1/ 3
1
1
1/ 3
4 / 3
1
0
5/3
5/3
B
x1
B
x2
N
x3
N
x4
N
x5
1
0
0
0
1
0
N
x6
Initial feasible solution:
x1  2, x2  1
2 / 5 1/ 5 2 / 5 1/ 5 2 
x3  x4  0
1/ 5
3 / 5 1/ 5 3 / 5 1
0
0
1
1 0
b