Transcript Linear Programming (Optimization)

Chap 10. Integer Prog. Formulations
 Mixed Integer Programming Problem
minimize
c’x + d’y
subject to
Ax + By = b
x, y  0
x integer
 Integer programming problem: no continuous variables y
 Zero-one (or binary) integer programming problem:
no continuous variables y and x restricted to be 0 or 1
 Note that the feasible solution set is no longer convex compared to
the linear programming problem. Powerful modeling tool, but usually
difficult to solve.
Linear Programming 2009
1
10.1 Modeling Techniques
 Binary choice
Example: zero-one knapsack problem
Given n items, j th item has weight wj and value cj. Given capacity of
the sack is K. Select the items to carry in the sack which maximizes
the total value.
maximize
j = 1, …, n cjxj
subject to j = 1, … , n wjxj  K
xj  {0, 1}, j = 1, … , n
(only one constraint. typical case in resource allocation problems.
Has many applications in theory and algorithms.)
Variations: integer knapsack problem, min form, precedenceconstrained knapsack.
Linear Programming 2009
2
 Forcing constraints
decision A can be made ( x = 1 ) only if decision B also has been
made ( y = 1 )
( If y = 0 (decision B not made), then x = 0 (decision A cannot be
made)
 xy,
x, y  {0, 1}
Example: Facility location problem (uncapacitated)
n potential facility locations, m clients who needs to be serviced from
these locations.
fixed cost cj of opening a facility at location j.
cost dij of serving client i from facility j.
Select a set of facility locations and assign each client to one facility ,
while minimizing the total cost.
Linear Programming 2009
3
 Formulation
minimize
j = 1, …, n cjyj + i = 1, …, m j = 1, …, n dijxij
subject to
j = 1, … , n xij = 1 ,
for all i
xij  yj ,
for all i, j
xij , yj  {0, 1},
for all i, j
( or i xij  myj for all j )
yj = 1 if facility j is open, 0 otherwise.
xij = 1 if client i is served by facility j, 0 otherwise.
(or we may set 0  xij  1, and interpret it as the fraction of
customer i’s demand serviced by facility j)
Note that even without the binary constraints on xij , each client will be
served entirely from a location which can serve it most cheaply. Then
the meaning of xij is changed as the fraction of client i’s demand
satisfied from facility j.
Linear Programming 2009
4
 Relations between variables
Generalized upper bound constraints (GUB constraints)
j = 1, … , n xj  1
xj  {0, 1}
Linear Programming 2009
for all j
5
 Disjunctive constraints
at least one of two constraints (a’x  b, c’x  d) needs to be satisfied.
Let M be a sufficiently large number
a’x  b - My,
c’x  d - M(1 – y)
y  {0, 1}
In the text, following form is used assuming a, c  0 and x  0.
a’x  yb,
c’x  (1 – y)d
y  {0, 1}
 Note that the feasible solution set is not convex for disjunctive
constraints. Extension for more than 2 constraints.
Linear Programming 2009
6
 Restricted range of values
x  { a1, a2, … , am }
x = j = 1, …, m ajyj
j = 1, …, m yj = 1,
yj  {0, 1}
Linear Programming 2009
7
 Arbitrary piecewise linear cost functions
f(x)
x = i = 1, … , k i ai
f(x) = i = 1, … , k i f(ai)
0
a1
1
Linear Programming 2009
a2
y1
2
a3
y2
3
a4
y3
x
4
8
minimize
i = 1, … , k i f(ai)
subject to
i = 1, … , k i = 1 ,
1  y1 ,
i  yi – 1 + yi ,
i = 2, … , k – 1
k  yk - 1 ,
i = 1, … , k-1 yi = 1 ,
i  0 ,
yi  {0, 1}
Linear Programming 2009
9
Alternative formulation
f(x)
c4
c2
c3
c1
K
x
OR-1 2009
L1
L2
L3
(x1)
(x2)
(x3)
10
Formulation:
Want to express x as x = x1 + x2 + … + xk
Ljwj  xj  Ljwj-1
j = 1, 2, … , k
w0 = 1
wj  {0, 1}
j = 1, 2, … , k
xj  0
j = 1, 2, … , k
 note that we have wj  wj-1 for j = 1, 2, … , k.
If wj = 0 for some j  wk = 0 for k  j+1
If wj = wj-1 = 1  xj = Lj
If wj = wj-1 = 0  xj = 0
If wj = 0, wj-1 = 1  0  xj  Lj
Write the piecewise linear function as K + c1x1 + c2x2 + … + ckxk .
OR-1 2009
11
 Set covering, set packing, set partitioning problems
M = { 1, … , m}, N = { 1, … , n}
M1 , M2 , … , Mn  M
weight cj for each Mj
F  N is a cover of M if j  F Mj = M
F  N is a packing of M if Mj  Mk =  for all j, k  F, j  k.
F  N is a partition of M if it is both a cover and a packing of M.
Find a cover F of minimum weight.
Find a packing F of maximum weight.
Find a partitioning of (min, max) weight.
Linear Programming 2009
12

A : m  n,
(0, 1) matrix
if i  Mj
Let aij = 1 ,
0,
otherwise
define xj = 1
if j  F. ( i.e. Mj is selected)
0, otherwise
constraints
Ax  e ,
Ax  e ,
Ax = e,
where e is an m-dimensional vector with all components
equal to 1.
 Has many practical applications in scheduling, graphs, ...
Linear Programming 2009
13
 Boolean Quadratic Function
min f(x) = i = 1n dixi + i, j, i  j cijxixj ,
xi  {0, 1} for all i
Let yij = xixj
 min i = 1n dixi + i, j, i  j cijyij
xi + xj – yij  1
-xi + yij  0
-xj + yij  0
for all i, j, i  j
xi, yij  {0, 1} for all i, j
 constraints ensure that yij = 1  xi = 1, xj = 1
Linear Programming 2009
14
 Ex: quadratic knapsack problem, max cut of a graph, lift-and-project
algorithm for 0-1 IP.
 Def: Given a graph G = (V, E), and subset S  V of vertices, the set of
edges with exactly one endpoint in S is called a cut (relative to S).
 Given G = (V, E), and edge weights cij, e = (i, j)  E, find a maximum
weight cut of G.
max (i, j)  E cij ( xi(1-xj) + (1-xi)xj )
xi  {0, 1} for all i
(may add constraint x1 = 1 )
 Note: max cut problem is difficult to solve (NP-hard), but min cut
problem is easy (max-flow min-cut theorem).
Linear Programming 2009
15
10.2 Guidelines for strong formulations
 Time to solve LP generally depends on the number of constraints
(m) and the number of variables (n).
 Empirically, number of iterations in simplex method:
O(m), O(log n)
Each iteration: O(m2)
 However, for integer programming problems, running time is
usually exponential (NP-hard problems) of the problem size and
very erratic.
 Strong formulation is important. A formulation which describes the
feasible integer points closely is desirable. (Why?)
Linear Programming 2009
16
 Branch-and-Bound Algorithm for IP
(IP) minimize
subject to
c’x
Ax  b
x  0 and integer
 Let zIP be the optimal value of IP and zLP be the optimal value of the LP
relaxation of IP, i.e. problem with integer requirement dropped.
Then
zLP  zIP holds, i.e. zLP provides a lower bound on zIP .
Usually upper bound can be obtained from a feasible solution.
If lower bound = upper bound, then optimal.
 If LP relaxation gives an integer optimal solution, then it is optimal.
Otherwise, divide the feasible solution set of IP into two (or more)
disjoint subsets and continue to solve the LP relaxation for each subset.
Linear Programming 2009
17
 Ex: 0-1 IP
Let S be the set of feasible solutions to IP. If xj* = , 0 <  <1, in an
optimal solution x* to LP relaxation, divide S into two sets S1, S2, where
S1 = S  { x: xj = 0}, S2 = S  { x: xj = 1}.
Then solve LP relaxation for S1 and S2 respectively.
 Important tool to facilitate search:
If we have current best integer solution (incumbent) with value z0 and
LP relaxation of a subproblem gives value z’ with z0  z’, then the
subproblem does not include a better solution than the current
incumbent and the subproblem can be pruned.
Therefore, if we can find a strong lower bound (high value), it is more
likely that the subproblem can be pruned earlier in the search procedure,
hence mitigating the need to seach the subproblem further (prevent the
explosion of the search tree).
Linear Programming 2009
18
 Why strong formulation?
Let P1, P2 be the set of feasible solutions to the LP relaxation of two
different formulations. ( P1  P2, S  P1, S  P2 )
Then the optimal values of the LP relaxations of the two formulations
give zP2  zp1 . Thus formulation for P1 gives stronger bound.
ex) formulations for the facility location problem
P2
P1
Linear Programming 2009
19
10.3 Modeling with exponentially many
constraints
 Minimum spanning tree problem
Graph G = (N, E), |N| = n, |E| = m
Edge weight ce , e  E
Find min weight spanning tree. (Recall Kruskal, Prim’s alg.)
Tree : connected acyclic graph (spanning tree: all nodes of G
included in the tree)
Forest: acyclic graph
A spanning tree has n – 1 edges and connected ( iff condition)
Linear Programming 2009
20
Linear Programming 2009
21
 For S  N, define
E(S) = { ( i, j )  E : i, j  S }
(S) = { ( i, j )  E : i  S, j  S} ( cut defined by S)
 Subtour elimination formulation (no subtour allowed)
minimize
e  E ce xe
subject to
e  E xe = n – 1
e  E(S) xe  |S| - 1 ,
S  N, S  , N,
xe  { 0, 1 }.
Has exponential number of constraints ( 2n – 1 )
Linear programming relaxation can be obtained by replacing
xe  { 0, 1 } with 0  xe  1.
If not have e  E xe = n – 1, obtain formulation for min weight forest
(negative arc costs may be allowed)
Linear Programming 2009
22
 Cutset formulation ( use connectedness)
Use
e  (S) xe  1 ,
S  N, S  , N.
 Comparing two formulations:
The polyhedron of LP relaxation of subtour elim. formul. is properly
contained in the one for cutset formulation, hence it is better.
It can be shown that LP relaxation of subtour elim. formul. gives integer
optimal solutions.
 Why consider IP formulation although there exist good algorithms?
Algorithms fail if problem structure changed a little bit: degree
constrained spanning tree problem, Shortest total path length spanning
tree problem, Steiner tree problem, capacitated spanning tree problem,
…
Linear Programming 2009
23
 Traveling salesman problem (undirected)
Undirected graph G = (N, E), edge cost ce , e  E
Find a minimum cost tour
Cutset formulation
minimize
e  E ce xe
subject to
e   ({i}) xe = 2 ,
iN
e  (S) xe  2 ,
S  N, S  , N,
xe  { 0, 1 }.
Linear Programming 2009
24
 Subtour elimination formulation
minimize
e  E ce xe
subject to
e   ({i}) xe = 2 ,
iN
e  E(S) xe  |S| - 1 ,
S  N, S  , N,
xe  { 0, 1 }.
 LP relaxations of both formulations give the same solution set.
Linear Programming 2009
25
Remarks
 For directed version of the TSP problem, the following formulation is
possible, which is smaller in size. But it is a bad formulation. (refer
exercise 10.15 in text page 477)
ui – uj + nyij  n – 1 ,
( i, j )  A, i, j  1,
{ i : ( i, j )  A} yij = 1 ,
jN
{ j : ( i, j )  A} yij = 1 ,
iN
yij  { 0, 1 },
i, j  N
 Note that, uj’s are continuous variables in the above formulation.
 Undirected TSP is a special case of directed case, we may replace
each edge by two directed arcs with opposite direction and having the
same costs as the edge.
Linear Programming 2009
26
 Is the formulation correct?
The formulation has u, y variables. If (u*, y*) feasible, we only read y*
values ( projection of (u*, y*) to y space)
We need to show that (1) any tour solution y* satisfies the constraints
and (2) any non-tour solution does not satisfy the constraints.
(1) For any tour y*, if node i is k-th node in the tour, assign ui = k.
(2) If y* is 0,1 and satisfies degree constraints, it is either a tour or
consists of subtours. If subtours exist, there is one that does not
include node 1. Add the constraints ui – uj + nyij  n – 1 along the
arcs in the subtour.
Linear Programming 2009
27
 Comparing the LP relaxation of the cutset formulation (say A) (in
directed case version) and the LP relaxation of the previous formulation
(B): It can be shown that the projection of the polyhedron B onto y
space gives a polyhedron which completely contains A (the inclusion
can be strict), hence cutset formulation (or subtour elimination
formulation) is stronger.
( Let P = {(x, y)Rn  Rp : Ax + Gy  b}, then Projx (P) = { x: (x, y)  P for
some y} )
 Although the previous formulation is not strong, it can be an alternative
to use if you only have a generic IP software to use, not the
sophisticated one to handle the subtour elimination constraints.
Linear Programming 2009
28
How to Solve the LP relaxation of the Cut-Set
Formulation? (many constr.)
Solve LP relaxation
(w/o cut-set constraints)
If y* tour, stop.
O/w find violated cut-set
Solve LP after adding the
Cut-set constraint.
 violated cut-set?
Y
N
Stop
Linear Programming 2009
29
 If the obtained solution is not a tour, branch and apply the same
procedure again. Choose the best solution
 Branching : If xij*  0, 1, solve two subproblems after setting xij = 0,
and xij = 1.
 Branch-and-cut approach ( cutting plane alg.)
 Ideas for TSP formulation can be used for various routing,
sequencing problems.
 Branch-and-cut Ideas useful to solve many difficult IP problems.
 What can we do for the LP with many variables? For the LP with
many vars. and constraints?
 TSP site: http://www.tsp.gatech.edu/
Linear Programming 2009
30