Integer Programming ( 정수계획법 )

 max 𝑐 ′ 𝑥 𝐴𝑥 ≤ 𝑏 𝑥 ≥ 0 and integer ( 𝑥 ∈ 𝐵 𝑛 , or 0 ≤ 𝑥 ≤ 1 and integer) 

Formulations

 Plane scheduling, Crew scheduling leg : one flight taking off from somewhere and landing somewhere else. route : a set of legs that an airplane can serve sequentially and return home.

Let 𝑥 𝑗 = 1, if route 𝑗 selected 0, otherwise 𝑎 𝑖𝑗 = 1, if leg 𝑖 is part of route 𝑗 0, otherwise 𝑐 𝑗 : cost of using route 𝑗 .

OR-1 2014 1

Plane scheduling: min 𝑛 𝑗=1 𝑐 𝑗 𝑥 𝑗 s. t. 𝑛 𝑗=1 𝑎 𝑖𝑗 𝑥 𝑗 = 1, 𝑖 = 1,2, … , 𝑚 𝑥 𝑗 ∈ {0,1} for all 𝑗

m

: total number of legs,

n

: total number of possible routes (big number). Each route should satisfy timing and regulatory constraints. Assume that we already generated enough routes. This formulations is a set partitioning type formulation Crew scheduling: min 𝑛 𝑗=1 𝑐 𝑗 𝑥 𝑗 s. t. 𝑛 𝑗=1 𝑎 𝑖𝑗 𝑥 𝑗 ≥ 1, 𝑖 = 1,2, … , 𝑚 𝑥 𝑗 ∈ {0,1} for all 𝑗 Crews may ride some airplanes (legs) as passengers (dead heading).

Set covering type formulation.

Note that the routes and costs can be different from the plane scheduling case.

OR-1 2014 2

 Set partitioning, set covering, set packing :

ex)

𝑁 = {1, 2, 3, 4, 5} 𝑆 1 = {1, 2} , 𝑆 2 = {1, 3, 4} , 𝑆 3 = {3, 4} , 𝑆 4 = {1, 4, 5} , 𝑆 5 = {3, 4, 5} , 𝑆 6 = {1, 2, 3} , 𝑆 7 = {2, 3} Let 𝑀 be the index set of chosen subsets.

Set partitioning

: {𝑆 1 , 𝑆 5 } , {𝑆 4 , 𝑆 7 } , … ( 𝑗∈𝑀 𝑆 𝑗 = 𝑁, 𝑆 𝑖 ∩ 𝑆 𝑗 = ∅, 𝑖 ≠ 𝑗, 𝑖, 𝑗 ∈ 𝑀)

Set covering

: {𝑆 1 , 𝑆 2 , 𝑆 4 } , {𝑆 4 , 𝑆 6 } , … ( 𝑗∈𝑀 𝑆 𝑗 = 𝑁)

Set packing

: {𝑆 1 , 𝑆 3 } , {𝑆 5 } , … ( 𝑆 𝑖 ∩ 𝑆 𝑗 = ∅, 𝑖 ≠ 𝑗, 𝑖, 𝑗 ∈ 𝑀) {max, min} 

n j

 1

c j x j

        1   1   0   0 0

x

1   1        0   1    1 0  

x

2          0   0   1   1 0

x

3   1        0   0    1 1  

x

4          0   0   1   1 1

x

5          1   1   1   0 0

x

6          0   1   1   0 0

x

7                1  1   1    1 1  

x j

 { 0 , 1 } for all

j

OR-1 2014 3

  Traveling salesman problem (Formulation in the handout considers the directed case. It is correct, but not a strong formulation. We will consider a different formulation for undirected case. Similar idea can be used for directed case too.) Fixed cost charge problem cost function 𝑐 𝑥 = 0, if 𝑥 = 0 𝐾 + 𝑐𝑥, if 𝑥 > 0 Introduce binary decision variable 𝑦 that satisfies 𝑥 > 0 ⇒ 𝑦 = 1 𝑥 = 0 ⇒ 𝑦 = 0 and use cost function 𝑐 𝑥 = 𝐾𝑦 + 𝑐𝑥 i.e., we want to enforce the condition ( 𝑥 > 0 ⟺ 𝑦 = 1) in the formulation.

If 𝐾 > 0 and want to minimize 𝑐(𝑥) , can use the formulation min 𝐾𝑦 + 𝑐𝑥 , s.t.

𝑥 ≤ 𝑢𝑦, 𝑥 ≥ 0, 𝑦 ∈ {0,1} ( 𝑢 : upper bound on 𝑥 ) (Note that 𝑦 = 1, 𝑥 = 0 cannot be an optimal solution since 𝐾 > 0 .) OR-1 2014 4

 Nonlinear Objective functions • May approximate a nonlinear objective function by a continuous piecewise linear objective function.

𝑓(𝑥) 𝑐 𝑖 : slope 𝑐 4 𝑐 2 𝑐 3 𝑐 1 𝐾 x ( 𝐿 𝑥 1 1 ) ( 𝐿 𝑥 2 2 ) 𝐿 3 ( 𝑥 3 ) OR-1 2014 5

Express 𝑥 as 𝑥 = 𝑥 1 + 𝑥 2 + ⋯ + 𝑥 𝑘 , where each 𝑥 𝑗 𝑗 -th interval is used to express 𝑥 . denote how much of the Want to express 𝑐 𝑡 𝑥 𝑡 . 𝑥 as 𝑥 = 𝐿 1 + 𝐿 solution and objective value as 2 + ⋯ + 𝐿 𝑡−1 𝑓 𝑥 = 𝐾 + 𝑐 1 𝐿 + 𝑥 𝑡 , for some 𝑡 1 + 𝑐 2 𝐿 2 in a feasible + ⋯ + 𝑐 𝑡−1 𝐿 𝑡−1 + Formulation: 𝐿 𝑗 𝑤 𝑗 ≤ 𝑥 𝑗 ≤ 𝐿 𝑗 𝑤 𝑗−1 , 𝑤 0 = 1 𝑤 𝑗 ∈ {0, 1} , 𝑗 = 1, 2, … , 𝑘 𝑗 = 1, 2, … , 𝑘 𝑥 𝑗 ≥ 0, 𝑗 = 1, 2, … , 𝑘  note that we have If 𝑤 𝑗 = 0 𝑤 𝑗 for some 𝑗 ≤ 𝑤 𝑗−1  𝑤 𝑘 for 𝑗 = 1, 2, … , 𝑘 .

= 0 for 𝑘 ≥ 𝑗 + 1 If If 𝑤 𝑤 If 𝑤 𝑗 𝑗 𝑗 = 𝑤 𝑗−1 = 𝑤 = 0 𝑗−1 , 𝑤 = 1 = 0 𝑗−1 = 1    𝑥 𝑗 𝑥 𝑗 = 𝐿 = 0 𝑗 0 ≤ 𝑥 𝑗 ≤ 𝐿 𝑗  Write the piecewise linear function as 𝐾 + 𝑐 1 𝑥 1 + 𝑐 2 𝑥 2 + … + 𝑐 𝑘 𝑥 𝑘 .

OR-1 2014 6

 Knapsack problem max 𝑛 𝑗=1 𝑐 𝑗 𝑥 𝑗 s. t. 𝑛 𝑗=1 𝑎 𝑗 𝑥 𝑗 ≤ 𝑏 𝑥 𝑗 ≥ 0 and integer (or 𝑥 𝑗 binary) (general knapsack, binary knapsack problem)  satisfying at least one of two constraints (disjunctive constraints) If 𝑥 satisfies at least one of the two constraints, 𝑥 is feasible.

ex) one machine job scheduling problem.

Let 𝑀 denote a large positive value either or 3𝑥 1 + 2𝑥 2 ≤ 18 𝑥 1 + 4𝑥 2 ≤ 16 must hold (or both hold)  3𝑥 1 + 2𝑥 2 𝑥 1 + 4𝑥 2 ≤ 18 + 𝑦𝑀 ≤ 16 + 1 − 𝑦 𝑀 𝑦 ∈ {0,1} OR-1 2014 For computational purpose, it is not desirable to use big 𝑀 in the formulation.

The technique can be extended to handle 𝑘 out of 𝑚 constraints case.

7

 ex) scheduling jobs on a machine 𝑛 jobs, processing time 𝑑 𝑖 for job 𝑖 , any additional constraints and obj. fn.

Let 𝑡 𝑖 denote the starting time of job 𝑖 . Then, for each pair of jobs 𝑖, 𝑗, either or 𝑡 𝑗 ≥ 𝑡 𝑖 + 𝑑 𝑖 𝑡 𝑖 ≥ 𝑡 𝑗 + 𝑑 𝑗  𝑡 𝑡 𝑗 𝑖 − 𝑡 − 𝑡 𝑖 𝑗 ≥ 𝑑 ≥ 𝑑 𝑖 𝑗 − 𝑀𝑦 𝑖𝑗 − 𝑀(1 − 𝑦 𝑖𝑗 ) must hold 𝑦 𝑖𝑗 ∈ 0,1 , ∀ 𝑖, 𝑗 𝑦 𝑖𝑗 = 0 𝑦 𝑖𝑗 = 1   job job 𝑗 𝑖 precedes job precedes job 𝑖 𝑗  Functions with

N

possible values 𝑓 𝑥 1 , 𝑥 2 , … , 𝑥 𝑛  𝑓 𝑥 1 , 𝑥 = 𝑑 2 1 , 𝑜𝑟 𝑑 , … , 𝑥 𝑛 2 , … , 𝑜𝑟 𝑑 = 𝑁 𝑖=1 𝑑 𝑖 𝑁 𝑦 𝑖 𝑁 𝑖=1 𝑦 𝑖 = 1 𝑦 𝑖 ∈ {0, 1} for all 𝑖 .

OR-1 2014 8

 Quadratic 0-1 function ex) quadratic knapsack problem max 𝑖,𝑗,𝑖≠𝑗 𝑐 𝑖𝑗 𝑥 𝑖 𝑥 𝑗 𝑛 𝑗=1 𝑎 𝑗 𝑥 𝑗 + ≤ 𝑏 𝑛 𝑗=1 𝑑 𝑗 𝑥 𝑗 (or 𝐴𝑥 ≤ 𝑏 ) 𝑥 𝑗 ∈ {0, 1} for all 𝑗  max 𝑖,𝑗,𝑖≠𝑗 𝑐 𝑖𝑗 𝑦 𝑖𝑗 + 𝑛 𝑗=1 𝑑 𝑗 𝑥 𝑗 𝑛 𝑗=1 𝑎 𝑗 𝑥 𝑗 ≤ 𝑏 𝑦 𝑖𝑗 ≤ 𝑥 𝑖 , 𝑦 𝑖𝑗 ≤ 𝑥 𝑗 , for all 𝑖, 𝑗 𝑥 𝑖 + 𝑥 𝑗 − 1 ≤ 𝑦 𝑖𝑗 , for all 𝑖, 𝑗 𝑥 𝑗 ∈ {0, 1} for all 𝑗 , 𝑦 𝑖𝑗 ∈ {0,1} for all 𝑖, 𝑗 Express 𝑥 𝑖 𝑥 𝑗 𝑦 𝑖𝑗 = 0 by 𝑦 𝑖𝑗 otherwise.

and add constraints that ensure 𝑦 𝑖𝑗 (If 𝑥 𝑖 = 𝑥 𝑗 = 1  𝑦 𝑖𝑗 = 1 , else (at least one of 𝑥 𝑖 , 𝑥 𝑗 = 1 = 0 if ) 𝑥  𝑖 ( 𝑥 𝑖 = 𝑥 𝑗 = 1  𝑦 𝑖𝑗 = 1 ) OR-1 2014 = 𝑥 𝑗 𝑦 𝑖𝑗 = 1 = 0 ) , and 9

Models with many constraints



Traveling salesman problem (undirected case)

Undirected graph 𝐺 = (𝑁, 𝐸) , edge cost 𝑐 𝑒 , 𝑒 ∈ 𝐸 𝑁 : set of nodes, 𝐸 : set of edges (edge 𝑒 = 𝑖, 𝑗 , 𝑖, 𝑗 ∈ 𝑁 ) Find a minimum cost tour that visits each node exactly once and returns home

Cutset formulation

minimize 𝑒∈𝐸 𝑐 𝑒 𝑥 𝑒 subject to 𝑒∈𝛿( 𝑖 ) 𝑥 𝑒 = 2 , 𝑒∈𝛿(𝑆) 𝑥 𝑒 ≥ 2 , 𝑥 𝑒 ∈ {0,1} for all 𝑒 ∈ 𝐸 , 𝑖 ∈ 𝑁 𝑆 ⊂ 𝑁 , 𝑆 ≠ ∅, 𝑁 where 𝛿 𝑆 = {𝑒 = (𝑖, 𝑗) ∈ 𝐸: 𝑖 ∈ 𝑆, 𝑗 ∉ 𝑆 or 𝑗 ∈ 𝑆, 𝑖 ∉ 𝑆} defined by 𝑆 , the set of edges with one end node in 𝑆 (called a cut and the other end node not in 𝑆 ) (Note that the number of constraints can be very large.) OR-1 2014 10

Subtour elimination formulation

minimize 𝑒∈𝐸 𝑐 𝑒 𝑥 𝑒 subject to 𝑒∈𝛿( 𝑖 ) 𝑥 𝑒 = 2 , 𝑖 ∈ 𝑁 𝑒∈𝐸(𝑆) 𝑥 𝑒 ≤ 𝑆 − 1 , 𝑥 𝑒 ∈ {0,1} for all 𝑒 ∈ 𝐸 , 𝑆 ⊂ 𝑁 , 𝑆 ≠ ∅, 𝑁 where 𝐸 𝑆 = {𝑒 = (𝑖, 𝑗) ∈ 𝐸: 𝑖 ∈ 𝑆, 𝑗 ∈ 𝑆} (set of edges with both end nodes in 𝑆 )  Both formulations are correct and LP relaxations of both formulations give the same solution set. But there are enormous number of constraints . We will consider how to solve the formulation later.

 Although the formulation given in the handout is not a good one, it may be used if we want to solve small TSPs using a generic IP software, not the sophisticated algorithm that can handle the cutset or subtour elimination constraints.

OR-1 2014 11

 Integer programming significantly increases the modeling capability, but usually takes much more time to solve than linear programming. Theoretically, it is generally believed that there does not exist a polynomial time algorithm to solve general integer programming problems.

However, recent advances in theory and software, combined with powerful computers makes it possible to solve many real world problems using integer programming. OR-1 2014 12

Branch-and-Bound Algorithm

 Solution set is

non-convex

. Difficult to solve.

 Idea :

divide and conquer

. Divide feasible solutions set into many disjoint sets. Find the best solution for each subset. Then choose the best one. To divide the feasible solutions set, we add linear inequalities to the problem. (e.g.

x

1  1, and

x

1  0 ) But the divided problem is still integer programming problem. So we need a means to find the best integer solution for each integer programming problem.

 Linear programming relaxation : max 𝑐 ′ 𝑥 max 𝑐 ′ 𝑥

(IP)

𝐴𝑥 ≤ 𝑏 𝑥 ≥ 0 and integer

(LPR)

(𝑥 ∈ 𝐵 𝑛 , or 0 ≤ 𝑥 ≤ 1 and integer) 𝐴𝑥 ≤ 𝑏 𝑥 ≥ 0 (0 ≤ 𝑥 ≤ 1) OR-1 2014 13

 Let

z

If 𝑥 ∗ be the optimal value of (IP) and

z LP

is a feasible solution to (IP)  𝑥 ∗ be the optimal value of (LPR) feasible solution to (LPR) Hence {set of feasible solutions to (IP) } ⊆ { set of feasible solutions to (LPR) } and objective function is the same.

If 𝑥 ∗ is an optimal solution to (IP) (i.e. 𝑧 = 𝑐′𝑥 ∗ ), 𝑥 ∗ may not be an optimal solution to (LPR), but it is a feasible solution to (LPR). Hence  𝑧 ≤ 𝑧 𝐿𝑃 ( 𝑧 𝐿𝑃 provides an

upper bound

on 𝑧 ) Also

a feasible solution to (IP) provides a lower bound on

𝑧 .

( If 𝑥 0 is a feasible solution to (IP), then 𝑐′𝑥 0 ≤ 𝑧 ) Hence if we solve (LPR) and obtain optimal solution which happens to be integral, then the solution provides a lower bound which is the same as the upper bound.

It implies that the solution is optimal to (IP).

 If the obtained solution is not integral, we only know an upper bound on optimal value to (IP), but don’t know an optimal solution. Then we

divide the feasible solution set

and repeat the procedure again for each divided problem.

OR-1 2014 14

 Results of solving LP relaxation.

1.

2.

3.

4.

unbounded  integer program unbounded infeasible  integer program infeasible optimal solution which is integer  it is optimal to integer program optimal solution not integer  only obtain upper bound. Need to branch  How to divide the solution set in case of

4

.

Suppose 𝑥 ∗ is an optimal solution to LP relaxation and 𝑥 𝑗 ∗ = 𝑘 + 𝛼 , 𝑘 ≥ 0 integer, 0 < 𝛼 < 1 fractional.

Then consider 2 sets (a) 𝐴𝑥 ≤ 𝑏, 𝑥 ≥ 0, 𝑥 𝑗 ≤ 𝑘 , 𝑥 integer (b) 𝐴𝑥 ≤ 𝑏, 𝑥 ≥ 0, 𝑥 𝑗 ≥ 𝑘 + 1 , 𝑥 integer Any feasible solution to integer program is contained in one of (

a

), (

b

). So we do not miss any feasible solution.

Then we solve LP relaxation of (

a

), (

b

) again

. (Search procedure with tree structure). Note that the optimal solution 𝑥 ∗ to the initial LP relaxation is not a feasible solution to the LP relaxation of (a) and (b).

OR-1 2014 15

OR-1 2014 max 17𝑥 1 + 12𝑥 2 s.t.

10𝑥 1 + 7𝑥 2 ≤ 2 𝑥 1 + 𝑥 2 ≤ 5 𝑥 1 , 𝑥 2 ≥ 0 𝑥 1 , 𝑥 2 integer

Example

16

OR-1 2014

x

2 3 2 5 4 1 0

P

0 1 2 3 4

x

1 17

OR-1 2014

x

1  1

P

1 :

x

1   1 ,  65

x

2

P

0 :

x

1   1 .

76 , 

x

2  68.33

3 .

33  4

x

1  2

P

2 :

x

1   2 ,

x

2  68.29

 2 .

86

x

2  2

x

2  3

P

3 :

x

1   2 .

6 ,

x

2  68.2

 2

x

1  2

P

4 :

x

1   2 ,

x

2  58  2  

x

1  3

Infeasible P

5 :

x

1   3 ,

x

2   68.14

1 .

43 18

OR-1 2014

x

2 3 2 5 4

P

1 1 0

P

2 1 2 3 4

x

1 19

OR-1 2014

x

2 3 2 5 4

P

1 1 0

P

3 1 2 3 4

x

1 20



Useful tool to facilitate the search:

Suppose 𝑥 ∗ is the best solution currently known and its objective value is 𝑧 ∗

.

If the LP relaxation of a subproblem gives optimal value 𝑧′ such that 𝑧′ ≤ 𝑧 ∗

,

we know that there does not exist an integer solution in this subproblem which gives objective value greater than 𝑧 ∗

.

Hence we do not need to explore the subproblem any further and

prune

the subproblem (node).

 Obtaining a

good (small) upper bound

in each subproblem is important to reduce the number of subproblems examined. Modern approach adds more valid inequalities, (which are satisfied by feasible integer points but not satisfied by the current optimal solution to the LP relaxation), to tighten the feasible region (

cutting plane approach

).

 Effectiveness of B-and-B alg. largely depends on our ability to prune many nodes in the B-and-B tree in early stage of the B-and-B algorithm, hence cutting plane approach improves the performance of the alg. on many cases, and it is employed in modern commercial S/W.

OR-1 2014 21

Special techniques



Solving models with many constraints: TSP problem

(model with cut-set constraints)  As before, we solve the LP relaxation of the problem. If integer optimal solution (tour) found, it is optimal to IP. Otherwise, we branch.

 Procedure to solve the LP relaxation: see the next flow chart.

 Branching : If 𝑥 ∗ 𝑖𝑗 𝑥 𝑖𝑗 = 1 .

≠ 0, 1 , solve two subproblems after setting 𝑥 𝑖𝑗 = 0 , and  TSP site: http://www.math.uwaterloo.ca/tsp/ OR-1 2014 22

 Procedure to solve the

LP relaxation

(with many constraints) Solve LP relaxation (w/o cut-set constraints) If 𝑥 ∗ tour, stop.

o/w find violated cut-set  violated cut-set?

N Stop, optimal sol. to LP relaxation obtained Solve LP after adding the Cut-set constraint.

Y OR-1 2014 23

 Previous procedure identifies constraints of the model as we need them, while we start the procedure with a few of the constraints.

To solve the LP again after adding a violated cut set constraint, we can use the dual simplex method. Note that, after adding a violated constraint to the tableau, the tableau gives a dual feasible and primal infeasible basis if we take the surplus variable for the added cut set constraint as additional basic variable (the additional dual variable 𝑦 𝑚+1 takes value 0).  The dual of the previous formulation has many variables. Similarly, it is possible that we can start solving the problem with a few variables and identifies a variable to be added to the current LP to obtain a better solution. That procedure is called the (delayed)

column generation

method. Recall the case of cutting stock problem we considered earlier (refer Chapter 13).

OR-1 2014 24