Transcript Linear programming
Linear Programming and Simplex Algorithm Reference: Numerical Recipe Sec. 10.8
Content • Linear programming problems • Simplex algorithm • QSOpt solver • LP Applications – Collision detection – Melodic similarity – Underdetermined linear systems 2
The Diet Problem • Dietician preparing a diet consisting of two foods, A and B.
A B Cost 0.60
0.40
Protein 20 30 Fat Carbohydrate 12 6 30 15 Minimum requirement: protein 60g; fat 24g; carbohydrate 30g Looking for
minimum cost diet
3
Linear Programming min
s
.
t
.
20
x
1 12
x
1 30
x
2 6
x
2
C
30
x
1 0 .
60
x
1 15
x
2 0 .
40
x
2 60 24 30
where x
1 ,
x
2 0 Cp. NLP min
f
(
x
) subject to
h
(
x
) 0
g
(
x
) 0 4
Maximize
Subject to The Problem N: dimension of
x
M: number of constraints (M = m 1 +m 2 +m 3 ) 5
Theory of Linear Optimization Terminology: Feasible vector Feasible basic vector Optimal feasible vector 6
Theory (cont) •
Feasible
region is convex and bounded by hyperplanes • Feasible vectors that satisfy
N
of the original constraints as equalities are termed
feasible basic vectors.
• Optimal occur at boundary (gradient vector of objective function is always nonzero) • Combinatorial problem: determining which
N
constraints (out of the
N+M
constraints) would be satisfied by the optimal feasible vector 7
Simplex Algorithm (Dantzig 1948) • A series of combinations is tried to increase the objective function • Number of iterations less than O(max(
M,N
)) • Worst case complexity is exponential (in number of variables); yet has polynomial
smoothed
complexity (and works well in practice) 8
I.
II.
III.
The Canonical Problem Maximize Subject to N: dimension of
x
M: number of constraints (M = m 1 +m 2 +m 3 ) 9
QSopt [ ref ] • Contains a GUI and a callable library 11
LP Format Example min
C
s
.
t
.
20
x
1 0 .
60
x
1 30
x
2 0 .
40
x
2 60 12
x
30
x
1 1 6
x
2 15
x
2 24 30
where x
1 ,
x
2 0 Somehow x1 >= 0 x2 >= 0 Doesn’t work!?
Problem smallExample Minimize obj: 0.60x1 + 0.40x2
Subject to c1: 12x1 + 6x2 >= 24 30x1 + 15x2 >= 30 Bounds 0 <= x1 0 <= x2 End
for QSOpt scripts 12
QSOpt Examples 13
• Primal problem Duality • Dual problem 14
15
Duality • If a linear program has a finite optimal solution then so does the dual. Furthermore,
the optimal values of the two programs are the same.
• If either problem has an unbounded optimal solution, then the other problem has no feasible solutions.
16
Diet Problem min
C
s
.
t
.
20
x
1 0 .
60
x
1 30
x
2 12
x
30
x
1 1 6
x
2 15
x
2 0 .
40
x
2 60 24 30
where x
1 ,
x
2 0 max
s
.
t
.
C
20 12 30
x x x
1 1 1 0 .
60
x
1 30
x
2
s
1 0 .
40
x
2 60 6
x
2 15
x
2
s s
2 3 24 30
where x
1 ,
x
2 ,
s
1 ,
s
2 ,
s
3 0 17
Applications
Ax
b
min
x
The Problem Under-determined system (infinite number of solutions) Find the solution with minimum magnitude (i.e., closest to origin) 19
Example
x
1
x x
2 2
x
3 2 0 (-2,0,0) (1,-1,1) Approach 1: find complete solution 1 0 1 1 0 1 0 2
x n x c
1
c
1 1 ,
x p
x n x p
2 0 0 (-2,0,0) subtract its projection along (1,-1,1)
q
0 0 2 1 3 1 1 ,
qq T
1 3 1 1 1 1 1 1 1 1 3 1 1 1 1 1 1 0 0 2 1 1 1 1 1 1
d
0 0 2 2 3 3 2 2 3 3 4 3 2 2 3 24 9 20
Linear Programming Solution min ,
x a
where
Ax
x i
0
b
• Most simplex algorithm assume nonnegative variables • Set
x i
y i
where
y i z i
0 and
z i
0
x i
R
21
min 3 subject to
z
1
z z i z i
2
y
1
y i y i y
2 :
z
2 ,
z
3
i
,
i
y
2
y
3 1 , 2 , 3 1 , 2 , 3 2 0
x i
0 ,
z i y i
y i
0 ,
z i
0
x
1
x x
2 2
x
3 2 0 Example min 3
y
1
z
1 0
y
2
z
2 0
y
3
z
3 0
y
1
z
1 0
y
2
z
2 0
y
3
y
1
y
2
z
3
z
1
z
2
y
2
y
3 0
z
2
z
3 2 0 22
Solved by QSopt
d
3 The real minimum (-4/3, -2/3, 2/3) The LP constraint is not exactly ||x|| 2 (but ||x|| ) This problem can also be solved by SVD (singular value decomposition) 23
LP in CD (narrow phase, ref ) • A pair of convex objects: each facet represented by the plane inequality a i x+b i y+c i z d i • If two sets of inequality (from two convex objects) define a feasible region, then a collision is detected • The type of optimization (min|max) and the objective function used is irrelevant. 24
LP for CD Convert to
dual
problem for better efficiency(?!) 25
Transportation Distance as a Measure to Melodic Similarity Ref: Typke et al. 2003
Example Partially matched 27
Earth Mover Distance
A a i W B b j U
a
1
x i
b
1 , , ,
w i w i
, , ,
a b n m x i
weighted
R
2 ,
w i
weighted point
R
point set { 0 } set
y j
,
j
,
j
R
2 ,
u j
R
{ 0 }
u j f ij
: flow of weight from
x i
to
y j
over the distance
d ij
28
EMD as Linear Program Solve by simplex algorithm 29
EMD for Melodic Similarity • Ground distance: Euclidean distance
d ij
t i
t j p i
p j
2 • Scale time coordinate (so that they are comparable) • Transposition: transpose one of the melodies so that the weighted average pitch is equal (not optimum but acceptable) 30
HW: Verify planar CD using LP Warning: Non-negative assumption (for some solvers)!!
31
1 2 6 Verify planar CD using LP.
5 1 3 .
.
y x
2 .
x
3 3 [II] [II]
y
11 [I]
y x x
s
1 3
s
2
y
3
s
3 11 4 4 6 .
.
y x
5 [ II]
y
0 [ II] 5 .
x
10 [ I]
x x y
s
4 5
s
5
y
10
s
5 0 3 32
LP Solvers • Lp_solve: MILP • QSopt • GLPK 33
QSopt [ ref ] • Contains a GUI and a callable library 35
LP Format Example min
C
s
.
t
.
20
x
1 0 .
60
x
1 30
x
2 0 .
40
x
2 60 12
x
30
x
1 1 6
x
2 15
x
2 24 30
where x
1 ,
x
2 0 [weird] Somehow x1 >= 0 x2 >= 0 Doesn’t work!?
Problem smallExample Minimize obj: 0.60x1 + 0.40x2
Subject to c1: 12x1 + 6x2 >= 24 30x1 + 15x2 >= 30 Bounds 0 <= x1 0 <= x2 End 36
LP format BNF definition 1 37
LP format BNF definition 2 38
QSOpt API
Scenario: Read and Solve 40
Accessing a Solution • value: objective function • x: solution vector (columns) • pi: dual variables • slack: slack variables • rc: reduced cost • In our applications, we don’t really care about pi, slack, and rc.
• Set them to NULL 41
QS_MAX or QS_MIN
cmat
3 5 .
1 .
0 2 .
3 1 .
1 1 .
4 0 cmatind 0 0 0 1 1 # of nonzeros in jth column Location of start entry 42
Modify an LP Problem • QSnew_row • QSadd_rows • QSnew_col • QSadd_cols • QSdelete_row • QSdelete_col • QSchange_coef • QSchange_objcoef • QSchange_rhscoef • QSchange_sense • See reference manuals for more details (Ch.5) • Use QSwrite_prob to verify the change 43