Transcript Linear Programming (Optimization)

Chapter 1. Formulations
1
 Mixed Integer Optimization Problem (or (Linear) Mixed Integer Program,
MIP)
min 𝑐 ′ 𝑥 + 𝑑 ′ 𝑦
𝐴𝑥 + 𝐵𝑦 = 𝑏 (or 𝐴𝑥 + 𝐵𝑦 ≤ 𝑏)
𝑥 ∈ 𝑍+𝑛 , 𝑦 ∈ 𝑅+𝑘 ,
where 𝐴 ∈ 𝑍 𝑚×𝑛 , 𝐵 ∈ 𝑍 𝑚×𝑘 , 𝑏 ∈ 𝑍 𝑚 , 𝑐 ∈ 𝑍 𝑛 , 𝑑 ∈ 𝑍 𝑘
 Integer Optimization Problem (IP)
min 𝑐 ′ 𝑥
𝐴𝑥 = 𝑏
𝑥 ∈ 𝑍+𝑛
 Binary (or zero-one) Integer Optimization Problem (BIP)
min 𝑐 ′ 𝑥
𝐴𝑥 = 𝑏
𝑥 ∈ 0, 1
Integer Programming 2013
𝑛
( or 𝑥 ∈ 𝐵𝑛 )
2
 Combinatorial Optimization Problem
Given a finite set 𝑁 = 1, … , 𝑛 , weights 𝑐𝑗 for each 𝑗 ∈ 𝑁, and a set 𝐹 of
feasible subsets of 𝑁.
Want to find a minimum (or maximum) weight feasible subset in 𝐹.
(COP)
min
𝑆⊆𝑁
𝑗∈𝑆 𝑐𝑗
:𝑆 ∈ 𝐹
 Almost all COPs can be formulated as IP or BIP
Knapsack problem, Traveling Salesman Problem, Min/Max cut of graph, Stable
set, Steiner Tree, …
Integer Programming 2013
3
Applications
 Numerous applications
Transportation (train scheduling)
Airline crew scheduling, plane scheduling
Production planning, distribution, SCM, logistics
Energy, Eletricity generation planning
Telecommunicaton network design, operation
Buses for the handicapped
Ground holding of aircraft
Cutting problems
….
Too many to list.
 Provides very strong modeling capabilities (much better than LP alone), but
usually difficult to solve (The solution set is not convex)
linear  nonlinear, convex  non-convex
 Recent advances in theory and software makes IP a practical option.
Integer Programming 2013
4
1.1 Modeling Techniques
 Steps
Define necessary variables
Define (construct) constraints so that feasible points correspond to the feasible
solutions of the problem
Define objective function
 May exist many correct, but different formulations. Needs creativity.
Which formulation is better?
 Frequently, use incidence vectors to denote sets
𝑆 ⊆ 𝑁, incidence vector of 𝑆 is 𝑛 −dimensional vector 𝑥𝑆 such that 𝑥𝑗𝑆 = 1 if
𝑗 ∈ 𝑆 and 𝑥𝑗𝑆 = 0 otherwise.
Integer Programming 2013
5
Binary choice
 The 0-1 knapsack problem
𝑛 items
𝑤𝑗 is the weight of item 𝑗, and 𝑐𝑗 is its value
Bound 𝑏 on the weight that can be carried in a knapsack
Select items to be put in the knapsack so that the total value is maximum.
𝑥𝑗 = 1 if item 𝑗 is selected, and 𝑥𝑗 = 0 otherwise
The capacity bound cannot be exceeded:
Variables are 0-1:
Total value is maximized:
Integer Programming 2013
𝑛
𝑗=1 𝑎𝑗 𝑥𝑗
≤𝑏
𝑥𝑗 ∈ 0, 1 for 𝑗 = 1, … , 𝑛
max
𝑛
𝑗=1 𝑐𝑗 𝑥𝑗
6
 Variations
Integer knapsack problem
Precedence constrained knapsack problem
Partial orders on projects
project 𝑖 must be done to perform project 𝑗 ( 𝑖 → 𝑗 )
( can be represented as directed graph, use 𝑥𝑖 ≥ 𝑥𝑗 in constraints (𝑥: binary))
ex) repair kit selection, selecting tools for FMS tool magazine, open pit mining,
…
Quadratic knapsack problem: Objective is quadratic function
Integer Programming 2013
7
 Boolean Quadratic Function
max 𝑓 𝑥 =
𝑛
𝑖=1 𝑑𝑖 𝑥𝑖
+
𝑖,𝑗,𝑖≠𝑗 𝑐𝑖𝑗 𝑥𝑖 𝑥𝑗
, 𝑥𝑖 ∈ 0,1 for all 𝑖.
( use additional variables 𝑦𝑖𝑗 , such that 𝑦𝑖𝑗 = 𝑥𝑖 𝑥𝑗 for binary 𝑥. Extended
formulation.)
 max
𝑛
𝑖=1 𝑑𝑖 𝑥𝑖
+
𝑖,𝑗,𝑖≠𝑗 𝑐𝑖𝑗 𝑦𝑖𝑗
𝑥𝑖 + 𝑥𝑗 − 𝑦𝑖𝑗 ≤ 1
−𝑥𝑖 + 𝑦𝑖𝑗 ≤ 0
−𝑥𝑗 + 𝑦𝑖𝑗 ≤ 0
for all 𝑖, 𝑗, 𝑖 ≠ 𝑗
𝑥𝑖 , 𝑦𝑖𝑗 ∈ 0, 1 for all 𝑖, 𝑗
 constraints ensure that 𝑥𝑖 = 𝑥𝑗 = 1
⟺
𝑦𝑖𝑗 = 1
 The technique will be used in lift and project
Integer Programming 2013
8
 Ex: quadratic knapsack problem, max cut of a graph
Def: Given a graph 𝐺 = (𝑉, 𝐸), and subset 𝑆 ⊆ 𝑉 of vertices, the set of edges
with exactly one endpoint in 𝑆 is called a cut (relative to 𝑆).
 Given 𝐺 = (𝑉, 𝐸), and edge weights 𝑐𝑖𝑗 , 𝑒 = (𝑖, 𝑗) ∈ 𝐸, find a maximum
weight cut of 𝐺.
max
(𝑖,𝑗)∈𝐸 𝑐𝑖𝑗
𝑥𝑖 1 − 𝑥𝑗 + 1 − 𝑥𝑖 𝑥𝑗
𝑥𝑖 ∈ 0,1 for all 𝑖
(may add constraint 𝑥1 = 1 )
 Note: max cut problem is difficult to solve (NP-hard), but min cut problem is
easy (max-flow min-cut theorem).
Integer Programming 2013
9
Forcing constraints
 Decision A (𝑥 = 1) can be made only if decision B (𝑦 = 1) also has been made.
𝑥≤𝑦
 Uncapacitated Facility Location (UFL)
Given potential depots 𝑁 = 1, … , 𝑛 , and a set 𝑀 = 1, … , 𝑚 of clients.
Fixed cost 𝑐𝑗 to open depot 𝑗, and transportation cost 𝑑𝑖𝑗 if all of client 𝑖 ′ 𝑠
order is delivered from depot 𝑗.
Determine which depots to open, and which depot serves each client so as to
minimize the total cost.
Define 𝑦𝑗 = 1 if depot 𝑗 is open, and 𝑦𝑗 = 0 otherwise
𝑥𝑖𝑗 is fraction of the demand of client 𝑖 satisfied from depot 𝑗.
Satisfaction of demand of client 𝑖 :
Integer Programming 2013
𝑛
𝑗=1 𝑥𝑖𝑗
= 1 for 𝑖 = 1, … , 𝑚.
10
if depot 𝑗 not opened, then 𝑥𝑖𝑗 = 0.
⟹ 𝑥𝑖𝑗 ≤ 𝑦𝑗 for 𝑖 ∈ 𝑀, 𝑦𝑗 ∈ 0, 1
min
s.t.
𝑛
𝑗=1 𝑐𝑗 𝑦𝑗 +
𝑛
𝑗=1 𝑥𝑖𝑗
𝑚
𝑖=1
𝑛
𝑗=1 𝑑𝑖𝑗 𝑥𝑖𝑗
=1
for 𝑖 ∈ 𝑀
𝑥𝑖𝑗 ≤ 𝑦𝑗 for 𝑖 ∈ 𝑀, 𝑗 ∈ 𝑁
0 ≤ 𝑥𝑖𝑗 ≤ 1 for 𝑖 ∈ 𝑀, 𝑗 ∈ 𝑁, 𝑦𝑗 ∈ 0, 1 for 𝑗 ∈ 𝑁
 Note that there exists an optimal solution with 𝑥𝑖𝑗 = 0 or 1.
Capacitated version?
Integer Programming 2013
11
Relation between variables
𝑛
𝑗=1 𝑥𝑖𝑗
≤ = 1, 𝑥 ∈ 𝐵𝑛 : Generalized upper bound (GUB) constraint
Integer Programming 2013
12
Disjunctive constraints
 Assume 𝑥 ≥ 0
𝑎′ 𝑥 ≥ 𝑏, 𝑐 ′ 𝑥 ≥ 𝑑,
𝑎, 𝑐 ≥ 0
at least one of the two constraints needs to be satisfied
 𝑎′ 𝑥 ≥ 𝑦𝑏, 𝑐 ′ 𝑥 ≥ 1 − 𝑦 𝑑, 𝑦 ∈ 0, 1
 General form:
𝑎𝑖′ 𝑥 ≥ 𝑏𝑖 , 𝑖 = 1, … , 𝑚 ( 𝑎𝑖 ≥ 0 for all 𝑖)
(at least 𝑘 out of 𝑚 constraints needs to be satisfied)
𝑚
′

𝑦𝑖 ∈ 0, 1 , 𝑖 = 1, … , 𝑚.
𝑖=1 𝑦𝑖 ≥ 𝑘, 𝑎𝑖 𝑥 ≥ 𝑏𝑖 𝑦𝑖 ,
 Ex) machine scheduling: two jobs must be processed on the same machine
and cannot be processed simultaneously.
𝑝𝑖 : processing time of job 𝑖,
𝑡𝑖 : start time of job 𝑖,
 either 𝑡2 ≥ 𝑡1 + 𝑝1 or 𝑡1 ≥ 𝑡2 + 𝑝2 should hold.
(need a different formulation from above, using big 𝑀)
Note that the feasible solution set is not convex.
Integer Programming 2013
13
Restricted range of values
 Want to restrict a variable x to take values in a set 𝑎1 , … , 𝑎𝑚 .

𝑥=
𝑚
𝑗=1 𝑎𝑗 𝑦𝑗
Integer Programming 2013
,
𝑚
𝑗=1 𝑦𝑗
= 1 , 𝑦𝑗 ∈ 0, 1 for all 𝑗
14
Arbitrary piecewise linear cost functions
 Piecewise linear, not necessarily convex, cost function
(separable piecewise linear convex cost function can be modeled as LP (min
problem, but non-convex (or non-concave) function cannot be modeled as LP)
, 𝑓 𝑥 = 𝑘𝑖=1 𝜆𝑖 𝑓 𝑎𝑖 ,
𝑘
𝑖=1 𝜆𝑖 = 1 , 𝜆1 , … , 𝜆𝑘 ≥ 0
𝑓(𝑥)
𝑥=
𝑎1 𝑦 𝑎2
1
2
1
Integer Programming 2013
𝑦2 𝑎 3
3
𝑦3
𝑎4
4
𝑘
𝑖=1 𝜆𝑖 𝑎𝑖
𝑥
15
 Formulation:
𝑘
𝑖=1 𝜆𝑖 𝑓
min
s. t.
𝑎𝑖
𝜆1 ≤ 𝑦1 , 𝜆𝑘 ≤ 𝑦𝑘−1
𝜆𝑖 ≤ 𝑦𝑖−1 + 𝑦𝑖 , 𝑖 = 2, … , 𝑘 − 1,
𝑘
𝑖=1 𝜆𝑖
= 1,
𝑘−1
𝑖=1 𝑦𝑖
=1
𝜆𝑖 ≥ 0, for all 𝑖,
𝑦𝑖 ∈ 0, 1 , for all 𝑖
Integer Programming 2013
16
 Alternative formulation
𝐿𝑗 𝑤𝑗 ≤ 𝑥𝑗 ≤ 𝐿𝑗 𝑤𝑗−1 , 𝑗 = 1,2, … , 𝑘
𝑤0 = 1
𝑤𝑗 ∈ 0, 1 , 𝑗 = 1,2, … , 𝑘
𝑥𝑗 ≥ 0, 𝑗 = 1,2, … , 𝑘
𝑓(𝑥)
𝑓 = 𝐾 + 𝑐1𝑥1 + 𝑐2𝑥2 + ⋯ + 𝑐𝑘𝑥𝑘
Note that constraints imply 𝑤𝑗 ≤ 𝑤𝑗−1
𝐾
𝑤0
𝑤1
𝐿1
Integer Programming 2013
𝑤3
𝑤2
𝐿2
𝑥
𝐿3
17
Set covering, set packing, set partitioning
 𝑀 = {1, … , 𝑚}, 𝑁 = {1, … , 𝑛}
𝑀1 , 𝑀2 , … , 𝑀𝑛 are collection of subsets of 𝑀.
cost 𝑐𝑗 for each subset 𝑀𝑗
 𝐹 ⊆ 𝑁 is a cover of 𝑀 if
𝑗∈𝐹 𝑀𝑗
=𝑀
𝐹 is a packing of 𝑀 if 𝑀𝑗 ⋂𝑀𝑘 = ∅ for all 𝑗, 𝑘 ∈ 𝐹, 𝑗 ≠ 𝑘
𝐹 is a partition of 𝑀 if it is both a cover and a packing of 𝑀.
weight of a subset 𝐹 of 𝑁 is defined as
 Let 𝐴: 𝑚 × 𝑛 with 𝑎𝑖𝑗 = 1,
𝑗∈𝐹 𝑐𝑗
if 𝑖 ∈ 𝑀𝑗 ,
= 0, otherwise
 𝐴𝑥 ≥ 𝑒, 𝐴𝑥 ≤ 𝑒, 𝐴𝑥 = 𝑒, 𝑥 ∈ 𝐵𝑛
Note that 𝐴𝑥 =
Integer Programming 2013
𝑛
𝑗=1 𝐴𝑗 𝑥𝑗 ,
( 𝑒 : unit vector)
where 𝐴𝑗 is 𝑗 − 𝑡ℎ column of 𝐴.
18
Sequencing problem with setup times
 One machine, 𝑚 operations
operation 𝑗 requires unique tool 𝑗
capacity of tool magazine is 𝐵 < 𝑚
loading or unloading tool 𝑗 into the magazine requires 𝑠𝑗 units of setup time
𝑛 jobs need to be performed by the machine and each job 𝑖 requires multiple
operations 𝐽𝑖 , 𝐽𝑖 ≤ 𝐵
(setup time required prior to each job is sequence dependent)
 Determine the optimal job sequence that minimizes total setup time. Assume
the magazine is empty initially.
Integer Programming 2013
19
 Let
𝑥𝑖𝑟 = 1,
= 0,
if job 𝑖 is the 𝑟 −th job processed
otherwise
𝑦𝑗𝑟 = 1,
if tool 𝑗 is on the magazine while the 𝑟 −th job is processed
= 0,

otherwise
𝑦𝑗0 = 0, for all j. (or may use initial magazine setting)
𝑛
𝑟=1 𝑥𝑖𝑟 = 1,
𝑛
𝑖=1 𝑥𝑖𝑟 = 1,
𝑥𝑖𝑟 ≤ 𝑦𝑗𝑟 ,
𝑚
𝑗=1 𝑦𝑗𝑟
Integer Programming 2013
≤ 𝐵,
for all 𝑖.
for all r.
(job 𝑖 must be done)
(𝑟 − 𝑡ℎ job must be done)
for all 𝑗 ∈ 𝐽𝑖 , for all 𝑟, 𝑖.
for all 𝑟.
20
𝑚
𝑗=1
minimize
𝑛
𝑟=1 𝑠𝑗
𝑚
𝑗=1
 minimize

𝑚
𝑗=1
minimize
𝑦𝑗𝑟 − 𝑦𝑗,𝑟−1
𝑛
𝑟=1 𝑠𝑗 𝑧𝑗𝑟
𝑧𝑗𝑟 ≥ 𝑦𝑗𝑟 − 𝑦𝑗,𝑟−1 ,
for all 𝑗, 𝑟,
𝑧𝑗𝑟 ≥ 𝑦𝑗,𝑟−1 − 𝑦𝑗𝑟 ,
for all 𝑗, 𝑟.
𝑛
𝑟=1 𝑠𝑗 𝑧𝑗𝑟
𝑧𝑗𝑟 ≥ 𝑦𝑗𝑟 − 𝑦𝑗,𝑟−1 ,
for all 𝑗, 𝑟,
𝑧𝑗𝑟 ≥ 𝑦𝑗,𝑟−1 − 𝑦𝑗𝑟 ,
for all 𝑗, 𝑟,
𝑛
𝑟=1 𝑥𝑖𝑟 = 1,
𝑛
𝑖=1 𝑥𝑖𝑟 = 1,
𝑥𝑖𝑟 ≤ 𝑦𝑗𝑟 ,
𝑚
𝑗=1 𝑦𝑗𝑟
Integer Programming 2013
≤ 𝐵,
for all 𝑖.
for all 𝑟.
for all 𝑗 ∈ 𝐽𝑖 , for all 𝑟, 𝑖.
for all 𝑟.
21
Uncapacitated lot sizing (ULS) (NW)
 Production plan for a 𝑇 −period horizon for a single product.
𝑐𝑡 is the fixed cost (set-up) of producing in period 𝑡.
𝑝𝑡 is the unit production cost in period 𝑡.
ℎ𝑡 is the unit storage cost in period 𝑡.
𝑑𝑡 is the demand in period 𝑡.
Variables:
𝑦𝑡 is the amount produced in period 𝑡.
𝑠𝑡 is the stock at the end of period 𝑡.
𝑥𝑡 = 1 if production occurs in 𝑡, and 𝑥𝑡 = 0 otherwise.
Integer Programming 2013
22
 Formulation:
min
𝑇
𝑡=1
𝑝𝑡 𝑦𝑡 + ℎ𝑡 𝑠𝑡 + 𝑐𝑡 𝑥𝑡
𝑦1 = 𝑑1 + 𝑠1
𝑠𝑡−1 + 𝑦𝑡 = 𝑑𝑡 + 𝑠𝑡
𝑦𝑡 ≤ 𝜔𝑥𝑡
𝑠𝑇 = 0
𝑠, 𝑦 ∈ 𝑅+𝑇 , 𝑥 ∈ 𝐵𝑇 ,
where 𝜔 =
𝑇
𝑡=1 𝑑𝑡
Integer Programming 2013
for 𝑡 = 2, … , 𝑇
for 𝑡 = 1, … , 𝑇
is an upper bound on 𝑦𝑡 for all 𝑡.
23
 Alternative formulation:
Define 𝑞𝑖𝑡 as the quantity produced in period 𝑖 to satisfy demand in period 𝑡 ≥ 𝑖.
min
𝑇
𝑡=1
𝑡
𝑖=1
𝑝𝑖 + ℎ𝑖 + ℎ𝑖+1 + ⋯ + ℎ𝑡−1 𝑞𝑖𝑡 +
𝑡
𝑖=1 𝑞𝑖𝑡
= 𝑑𝑡 ,
𝑇
𝑡=1 𝑐𝑡 𝑥𝑡
for 𝑡 = 1, … , 𝑇
𝑞𝑖𝑡 ≤ 𝑑𝑡 𝑥𝑖 ,
for 𝑖 = 1, … , 𝑇 and 𝑡 = 𝑖, … , 𝑇
𝑇(𝑇+1)/2
𝑥 ∈ 𝐵𝑇 ,
𝑞 ∈ 𝑅+
,
 If we replace 𝑥 ∈ 𝐵𝑇 , by 0 ≤ 𝑥𝑡 ≤ 1 for all 𝑡, then the LP has an optimal
solution with 𝑥 ∈ 𝐵𝑇 . Hence this is a stronger formulation.
 The formulation is a special case of the uncapacitated facility location problem.
Substitute 𝑦𝑖𝑡 = 𝑞𝑖𝑡 /𝑑𝑡 for all 𝑖 and 𝑡 ≥ 𝑖.
 Extensions: capacitated lot sizing, multiple item lot sizing, …
Integer Programming 2013
24