Transcript Linear Programming (Optimization)

Integer Programming (정수계획법)
max c' x
Ax  b
x  0 and integer
( x  B n , 0  x  1 and integer)
 Formulations
 Plane scheduling, Crew scheduling
(leg, route)
Let
1
xj  
0
if route j selected
otherwise
1
aij  
0
if leg i is part of route j
otherwise
c j : cost of using route j
OR-1 2009
1
Plane scheduling
min
nj1c j x j
s. t.
nj 1 aij x j  1, i  1,2,...,m
x j {0,1} for all j
m : total number of legs,
n : total number of possible routes (big number)
This formulations is a set partitioning type formulation
Crew scheduling
min
nj1c j x j
s. t.
nj 1 aij x j  1, i  1,2,...,m
x j {0,1} for all j
Set covering type formulation.
Note that the routes and costs can be different from the plane scheduling
case.
OR-1 2009
2
 Set partitioning, set covering, set packing :
N = { 1, 2, 3, 4, 5 }
S1 = {1, 2}, S2 = {1, 3, 4}, S3 = {3, 4}, S4 = { 1, 4, 5},
S5 = {3, 4, 5}, S6 = {1, 2, 3}, S7 = {2, 3}
Set partitioning : { S1, S5 }, { S4, S7 }, …
Set covering : { S1, S2, S4 }, { S4, S6 }, …
Set packing : { S1, S3 }, { S5 }, …
{max, min} nj1 c j x j
1
1
0 
1
0 
1
0 
1
1
0 
0 
0 
0 
1
1
1

 
 
 
 
 
 
    
0 x1  1 x2  1 x3  0 x4  1 x5  1 x6  1 x7  1
0 
1
1
1
1
0 
0  1
 
 
 
 
 
 
    
0
0
0
1
1
0
0
1
x j {0, 1} for all j
OR-1 2009
3
 Traveling salesman problem
(Formulation in the handout is correct, but not a good formulation. We
will consider a different formulation.)
 Fixed cost charge problem
 0
cost function c( x)  
 K  cx
if x  0
if x  0
 x  0  y  1
Introduce binary decision variable y that satisfies 

x

0

y

0


and use cost function c( x)  Ky  cx
i.e., we want to enforce the condition ( x  0  y  1) in the formulation
If K  0 and want min cx, can use the formulation
min Ky  cx, s.t. x  uy, x  0, y {0,1}
(u : upper bound on x)
(Note that y = 1, x = 0 cannot be an optimal solution since K > 0.)
OR-1 2009
4
Nonlinear Objective functions
• May approximate a nonlinear objective function by a continuous
piecewise linear objective function.
f(x)
c4
c2
c3
c1
K
x
OR-1 2009
L1
L2
L3
(x1)
(x2)
(x3)
5
Formulation:
Want to express x as x = x1 + x2 + … + xk
Ljwj  xj  Ljwj-1
w0 = 1
wj  {0, 1}
xj  0
j = 1, 2, … , k
j = 1, 2, … , k
j = 1, 2, … , k
 note that we have wj  wj-1 for j = 1, 2, … , k.
If wj = 0 for some j  wk = 0 for k  j+1
If wj = wj-1 = 1  xj = Lj
If wj = wj-1 = 0  xj = 0
If wj = 0, wj-1 = 1  0  xj  Lj
Write the piecewise linear function as K + c1x1 + c2x2 + … + ckxk .
OR-1 2009
6
 Knapsack problem
max
s. t.
nj1c j x j
nj 1 a j x j  b
x j  0 and integer ( x j binary)
(c j , a j , b  0 and integer)
(general knapsack, binary knapsack problem)
 either-or constraints
either 3 x1  2 x2  18
or
x1  4 x2  16
must hold
OR-1 2009
3x1  2 x2  18  yM
x1  4 x2  16  (1  y ) M
y  {0,1}
7
 Functions with N possible values
f ( x1 , x2 ,...,xn )  d1, or d 2 ,..., or d N

f ( x1 , x2 ,...,xn )  iN1 di yi
iN1 yi  1
yi  {0,1} for all i
OR-1 2009
8
 Quadratic 0-1 function
ex) quadratic knapsack problem
max  i  j cijxixj +  djxj
Ax  b
x binary
 max  i  j cijyij +  djxj
Ax  b
yij  xi , yij  xj
xi + xj – 1  yij
x, y binary
for all i, j
Express xixj by yij and add constraints that ensure yij = 1 if xi = xj = 1, and
yij = 0 otherwise.
( xi = xj = 1
OR-1 2009

yij = 1 )
9
Models with many constraints
 Traveling salesman problem (undirected)
Undirected graph G = (N, E), edge cost ce , e  E
N: set of nodes, E: set of edges (edge e = (i, j), i, j  N)
Find a minimum cost tour
Cutset formulation
minimize
subject to
e  E ce xe
e   ({i}) xe = 2 ,
e  (S) xe  2 ,
xe  { 0, 1 },
iN
S  N, S  , N,
where (S) = { e = ( i, j )  E : i  S, j  S} (called a cut defined by S,
set of edges with one end node in S and the other end node not in S)
OR-1 2009
10
Subtour elimination formulation
minimize
subject to
e  E ce xe
e   ({i}) xe = 2 ,
e  E(S) xe  |S| - 1 ,
xe  { 0, 1 },
iN
S  N, S  , N,
where E(S) = { e = ( i, j )  E : i  S, j  S} (set of edges with both
end nodes in S)
 Both formulations are correct and LP relaxations of both formulations
give the same solution set.
We will consider how to solve the formulation later.
 Although the formulation given in the handout is not a good one, it
may be used if we want to solve small TSPs using a generic IP
software, not the sophisticated algorithm that can handle the cutset or
subtour elimination constraints.
OR-1 2009
11
 Integer programming significantly increases the modeling capability,
but usually takes much more time to solve than linear programming.
Theoretically, it is generally believed that there does not exist a
polynomial time algorithm to solve general integer programming
problems.
However, recent advances in theory and software, combined with
powerful computers makes it possible to solve many real world
problems using integer programming.
OR-1 2009
12
Branch-and-Bound Algorithm
 Solution set is non-convex. Difficult to solve.
 Idea : divide and conquer.
Divide feasible solutions set into many disjoint sets. Find the best
solution for each subset. Then choose the best one. To divide the
feasible solutions set, we add linear inequalities to the problem. (e.g.
x1  1, and x1  0 )
But the divided problem is still integer programming problem. So we
need a means to find the best integer solution for each integer
programming problem.
 Linear programming relaxation :
(IP)
max c' x
Ax  b
x  0 and integer
( x  B n , 0  x  1 and integer)
OR-1 2009
(LPR)
max c' x
Ax  b
x0
( 0  x 1 )
13
 Let z be the optimal value of (IP) and zLP be the optimal value of (LPR)
If x* is a feasible solution to (IP)  x* feasible solution to (LPR)
Hence {set of feasible solutions to (IP) }  { set of feasible solutions to
(LPR) } and objective function is the same.
If x* is an optimal solution to (IP) (i.e. z = c’x*), x* may not be an optimal
solution to (LPR), but it is a feasible solution to (LPR). Hence
 z  zLP
( zLP provides an upper bound on z)
Also a feasible solution to (IP) provides a lower bound on z.
( If x0 is a feasible solution to (IP), then c’x0  z)
Hence if we solve (LPR) and obtain optimal solution which happens to
be integral, then the solution provides a lower bound which is the same
as the upper bound.
It implies that the solution is optimal to (IP).
 If the obtained solution is not integral, we only know an upper bound on
optimal value to (IP), but don’t know an optimal solution. Then we
divide the feasible solution set and repeat the procedure again for each
divided problem.
OR-1 2009
14
 Results of solving LP relaxation.
1. unbounded  integer program unbounded
2. infeasible  integer program infeasible
3. optimal solution which is integer  it is optimal to integer program
4. optimal solution not integer  only obtain upper bound.
Need to branch
 How to divide the solution set in case of 4.
Suppose x* is optimal solution to LP relaxation and
x*j  k   , k  0 integer, 0    1 fractional
Then consider 2 sets
(a) Ax  b, x  0, x j  k , x integer
(b)
Ax  b, x  0, x j  k  1, x integer
Any feasible solution to integer program is contained in one of (a), (b).
So we do not miss any feasible solution. Then we solve LP relaxation
of (a), (b) again. (Search procedure with tree structure). Note that the
optimal solution x* to the initial LP relaxation is not a feasible solution to
the LP relaxation of (a) and (b).
OR-1 2009
15
Example
max 17 x1  12 x2
s. t. 10 x1  7 x2  2
x1 
x2  5
x1 , x2  0
x1 , x2 integer
OR-1 2009
16
x2
5
4
3
2
P0
1
0
OR-1 2009
1
2
3
4
x1
17
P0 : x1  1.76, x2  3.33
  68.33
x1  1
P1 : x1  1,
x1  2
x2  4
P2 : x1  2,
  65
x2  2.86
  68.29
x2  3
x2  2
P3 : x1  2.6, x2  2
Infeasible
  68.2
x1  3
x1  2
P4 : x1  2, x2  2
  58
OR-1 2009


P5 : x1  3, x2  1.43
  68.14
18
x2
5
4
3
P1
2
P2
1
0
OR-1 2009
1
2
3
4
x1
19
x2
5
4
3
P1
2
P3
1
0
OR-1 2009
1
2
3
4
x1
20
 Useful tool to facilitate the search:
Suppose x* is the best solution currently known and its objective value
is z*.
If the LP relaxation of a subproblem gives optimal value z’ such that
z’  z*, we know that there does not exist an integer solution in this
subproblem which gives objective value greater than z*.
Hence we do not need to explore the subproblem any further and
prune the subproblem (node).
 Obtaining a good (small) upper bound in each subproblem is important
to reduce the number of subproblems examined. Modern approach
adds more valid inequalities, (which are satisfied by feasible integer
points but not satisfied by the current optimal solution to the LP
relaxation), to tighten the feasible region (cutting plane approach).
 Effectiveness of B-and-B alg. largely depends on our ability to prune
many nodes in the B-and-B tree in early stage of the B-and-B algorithm,
hence cutting plane approach improves the performance of the alg. on
many cases, and it is employed in modern commercial S/W.
OR-1 2009
21
Special techniques
 Solving models with many constraints: TSP problem
As before, we solve the LP relaxation of the problem. If integer optimal
solution (tour) found, it is optimal to IP. Otherwise, we branch.
Procedure to solve the LP relaxation: see the next flow chart.
Branching : If xij*  0, 1, solve two subproblems after setting xij = 0, and
xij = 1.
 TSP site: http://www.tsp.gatech.edu/
OR-1 2009
22
 Procedure to solve the LP relaxation (with many constraints)
Solve LP relaxation
(w/o cut-set constraints)
If x* tour, stop.
O/w find violated cut-set
Solve LP after adding the
Cut-set constraint.
 violated cut-set?
Y
N
Stop, optimal sol. to
LP relaxation obtained
OR-1 2009
23
 Previous procedure identifies constraints of the model as we need
them, while we start the procedure with a few of the constraints.
To solve the LP again after adding a violated cut set constraint, we
can use the dual simplex method.
Note that, after adding a violated constraint to the tableau, the tableau
gives a dual feasible and primal infeasible basis if we take the surplus
variable for the added cut set constraint as additional basic variable
(the additional dual variable ym+1 takes value 0).
 The dual of the previous formulation has many variables. Similarly, it
is possible that we can start solving the problem with a few variables
and identifies a variable to be added to the current LP to obtain a
better solution. That procedure is called the (delayed) column
generation method. Recall the case of cutting stock problem we
considered earlier (Chapter 13).
OR-1 2009
24
OR-1 2009
25