Transcript Lecture 8: Linear Programming - A Mathematical Optimization Technique

ELEC 5270/6270 Fall 2007
Low-Power Design of Electronic Circuits
Linear Programming – A Mathematical
Optimization Technique
Vishwani D. Agrawal
James J. Danaher Professor
Dept. of Electrical and Computer Engineering
Auburn University, Auburn, AL 36849
[email protected]
http://www.eng.auburn.edu/~vagrawal/COURSE/E6270_Fall07/course.html
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
1
What is Linear Programming
Linear programming (LP) is a
mathematical method for selecting the
best solution from the available solutions
of a problem.
 Method:

 State
the problem and define variables whose
values will be determined.
 Develop a linear programming model:


Write the problem as an optimization formula (a linear
expression to be minimized or maximized)
Write a set of linear constraints
 An
available LP solver (computer program) gives
the values of variables.
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
2
Types of LPs
LP – all variables are real.
 ILP – all variables are integers.
 MILP – some variables are integers,
others are real.
 A reference:


S. I. Gass, An Illustrated Guide to Linear
Programming, New York: Dover, 1990.
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
3
A Single-Variable Problem
Consider variable x
 Problem: find the maximum value of x
subject to constraint, 0 ≤ x ≤ 15.
 Solution: x = 15.

Constraint satisfied
0
15
x
Solution
x = 15
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
4
Single Variable Problem (Cont.)


Consider more complex constraints:
Maximize x, subject to following constraints:




x≥0
5x ≤ 75
6x ≤ 30
x ≤ 10
0
(1)
(2)
(3)
(4)
5
10
15
x
(2)
(3)
(1)
(4)
All constraints
satisfied
Solution, x = 5
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
5
A Two-Variable Problem

Manufacture of chairs and tables:

Resources available:



Profit:



Material: 400 boards of wood
Labor: 450 man-hours
Chair: $45
Table: $80
Resources needed:

Chair



Table



5 boards of wood
10 man-hours
20 boards of wood
15 man-hours
Problem: How many chairs and how many tables should be
manufactured to maximize the total profit?
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
6
Formulating Two-Variable Problem
Manufacture x1 chairs and x2 tables to
maximize profit:
P = 45x1 + 80x2 dollars
 Subject to given resource constraints:

boards of wood,
5x1 + 20x2 ≤ 400
 450 man-hours of labor, 10x1 + 15x2 ≤ 450
 x1 ≥ 0
 x2 ≥ 0
 400
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
(1)
(2)
(3)
(4)
7
Solution: Two-Variable Problem
40
Tables, x2
30
Best solution: 24 chairs, 14 tables
Profit = 45×24 + 80×14 = 2200 dollars
(1) 20
(24, 14)
10
(3)
(4)
0
0
Copyright Agrawal, 2007
10
20
30
40
50
Chairs, x1
ELEC6270 Fall 07, Lecture 8
60
70
(2)
decresing
80
90
increasing
8
Change Profit of Chair to $64/Unit
Manufacture x1 chairs and x2 tables to
maximize profit:
P = 64x1 + 80x2 dollars
 Subject to given resource constraints:

boards of wood,
5x1 + 20x2 ≤ 400
 450 man-hours of labor, 10x1 + 15x2 ≤ 450
 x1 ≥ 0
 x2 ≥ 0
 400
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
(1)
(2)
(3)
(4)
9
Solution: $64 Profit/Chair
40
Tables, x2
30
Best solution: 45 chairs, 0 tables
Profit = 64×45 + 80×0 = 2880 dollars
(1) 20
(24, 14)
10
(3)
(4)
0
0
Copyright Agrawal, 2007
10
20
30
40
50
Chairs, x1
decresing
ELEC6270 Fall 07, Lecture 8
60
(2)
70
80
90
increasing
10
Primal-Dual Problems

Primal problem



Fixed resources
Maximize profit




x1 (number of chairs)
x2 (number of tables)
Fixed profit
Minimize value
Variables:


w1 ($ value/board of wood)
w2 ($ value/man-hour)
Maximize profit 45x1+80x2  Minimize value 400w1+450w2
Subject to:
 Subject to:





Dual Problem

Variables:



5x1 + 20x2
10x1 + 15x2
x1
x2
≤ 400
≤ 450
≥0
≥0




5w1 + 10w2
20w1 + 15w2
w1
w2
Solution:
Solution:




x1 = 24 chairs, x2 = 14 tables
Profit = $2200
Copyright Agrawal, 2007

ELEC6270 Fall 07, Lecture 8
≥ 45
≥ 80
≥0
≥0
w1 = $1, w2 = $4
value = $2200
11
The Duality Theorem

If the primal has a finite optimal solution,
so does the dual, and the optimum values
of the objective functions are equal.
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
12
LP for n Variables
n
minimize
Σ
cj xj
Objective function
j =1
n
subject to
Σ aij xj
≤ bi,
i = 1, 2, . . ., m
= di,
i = 1, 2, . . ., p
j =1
n
Σ cij xj
j =1
Variables: xj
Constants: cj, aij, bi, cij, di
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
13
Algorithms for Solving LP

Simplex method


Ellipsoid method


L. G. Khachiyan, “A Polynomial Algorithm for Linear
Programming,” Soviet Math. Dokl., vol. 20, pp. 191-194, 1984.
Interior-point method


G. B. Dantzig, Linear Programming and Extension, Princeton,
New Jersey, Princeton University Press, 1963.
N. K. Karmarkar, “A New Polynomial-Time Algorithm for Linear
Programming,” Combinatorica, vol. 4, pp. 373-395, 1984.
Course website of Prof. Lieven Vandenberghe (UCLA),
http://www.ee.ucla.edu/ee236a/ee236a.html
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
14
Basic Ideas of Solution methods
Extreme points
Constraints
Extreme points
Objective
function
Constraints
Simplex: search on extreme points.
Copyright Agrawal, 2007
Objective
function
Interior-point methods: Successively
iterate with interior spaces of
analytic convex boundaries.
ELEC6270 Fall 07, Lecture 8
15
Integer Linear Programming (ILP)
Variables are integers.
 Complexity is exponential – higher than LP.
 LP relaxation

Convert all variables to real, preserve ranges.
 LP solution provides guidance.
 Rounding LP solution can provide a nonoptimal solution.

Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
16
Solving TSP: Five Cities
Distances (dij) in miles (symmetric TSP, general TSP is asymmetric)
City
j=1
j=2
j=3
j=4
j=5
i=1
0
18
10
12
27
i=2
18
0
5
12
20
i=3
10
5
0
15
19
i=4
12
12
15
0
6
i=5
27
20
19
6
0
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
17
Search Space: No. of Tours

Asymmetric TSP tours
Five-city problem: 4 × 3 × 2 × 1 = 24 tours
 Nine-city problem: 362,880 tours
 14-city problem: 87,178,291,200 tours
 50-city problem: 49! = 6.08×1062 tours
Time for enumerative search assuming 1 μs
per tour evaluation
=
1.93×1055 years

Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
18
A Greedy Heuristic Solution
Tour length = 10 + 5 + 12 + 6 + 27 = 60 miles (non-optimal)
City
j=1
j=2
j=3
j=4
j=5
i=1
(start)
0
18
10
12
27
i=2
18
0
5
12
20
i=3
10
5
0
15
19
i=4
12
12
15
0
6
i=5
27
20
19
6
0
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
19
ILP Variables, Constants and
Constraints
x14 ε [0,1]
d14 = 12
4
5
x15 ε [0,1]
d15 = 27
x12 ε [0,1]
d12 = 18
1
Integer variables:
xij = 1, travel i to j
xij = 0, do not travel i to j
x13 ε [0,1]
d13 = 10
2
Real variables:
dij = distance from i to j
3
x12 + x13 + x14 + x15 = 2
four other similar equations
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
20
Objective Function and ILP Solution
5 i-1
Minimize ∑ ∑ xij × dij
i=1 j=1
∑ xij = 2
j≠i
Copyright Agrawal, 2007
xij j=1 2 3
i=1 0 0 1
2
1 0 0
3
0 1 0
4
0 0 0
5
0 0 0
for all i
ELEC6270 Fall 07, Lecture 8
4
0
0
0
0
1
5
0
0
0
1
0
21
ILP Solution
d54 = 6
4
5
d45 = 6
1
d21 = 18
d13 = 10
2
3
d32 = 5
Total length = 45
but not a single tour
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
22
Additional Constraints for Single Tour

Following constraints prevent split tours.
For any subset S of cities, the tour must
enter and exit that subset:
∑ xij ≥ 2 for all S, |S| < 5
iεS
jεS
Remaining
set
At least two
arrows must cross
this boundary.
Subset
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
23
ILP Solution
d41 = 12
4
d54 = 6
5
1
d25 = 20
d13 = 10
2
3
d32 = 5
Total length = 53
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
24
ILP Example: Test Minimization


A combinational circuit has n test vectors that
detect m faults. Each test detects a subset of
faults. Find the smallest subset of test vectors
that detects all m faults.
ILP model:


Assign an integer variable ti ε [0,1] to ith test vector
such that ti = 1, if we select ti, otherwise ti= 0.
Define an integer constant fij ε [0,1] such that fij = 1, if
ith vector detects jth fault, otherwise fij = 0. Values of
constants fij are determined by fault simulation.
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
25
Test Minimization by ILP
n
minimize
Σ ti
Objective function
i=1
n
subject to
Σ fij ti
≥ 1,
j = 1, 2, . . ., m
i=1
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
26
3V3F: A 3-Vector 3-Fault Example
Test vector i
fij
i=1
i=2
i=3
Variables: t1, t2, t3 ε [0,1]
Fault j
Minimize t1 + t2 + t3
j=1
1
1
0
j=2
0
1
1
Subject to:
t1 + t2 ≥ 1
t2 + t3 ≥ 1
j=3
Copyright Agrawal, 2007
1
0
1
ELEC6270 Fall 07, Lecture 8
t1 + t3 ≥ 1
27
3V3F: Solution Space
t3
ILP solutions
(optimum)
1
Non-optimum
solution
1st LP solution
(0.5, 0.5, 0.5)
Rounding and
2nd ILP solution
(1.0, 0.5, 0.5)
t2
1
Rounding and
3rd LP solution
(1.0, 1.0, 0.0)
1
t1
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
28
Characteristics of ILP



Worst-case complexity is exponential in number
of variables.
Linear programming (LP) relaxation, where
integer variables are treated as real, gives a
lower bound on the objective function.
Recursive rounding of relaxed LP solution to
nearest integers gives an approximate solution
to the ILP problem.

K. R. Kantipudi and V. D. Agrawal, “A Reduced
Complexity Algorithm for Minimizing N-Detect Tests,”
Proc. 20th International Conf. VLSI Design, January
2007, pp. 492-497.
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
29
3V3F: LP Relaxation and Rounding
ILP – Variables: t1, t2, t3 ε [0,1]
LP relaxation: t1, t2, t3 ε (0.0, 1.0)
Minimize t1 + t2 + t3
Solution: t1 = t2 = t3 = 0.5
Subject to:
Recursive rounding:
t1 + t2 ≥ 1
t2 + t3 ≥ 1
t1 + t3 ≥ 1
(1) round one variable, t1 = 1.0
Two-variable LP problem:
Minimize t2 + t3
subject to t2 + t3 ≥ 1.0
LP solution t2 = t3 = 0.5
(2) round a variable, t2 = 1.0
ILP constraints are satisfied
solution is t1 = 1, t2 = 1, t3 = 0
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
30
Recursive Rounding Algorithm
1. Obtain a relaxed LP solution. Stop if
each variable in the solution is an
integer.
2. Round the variable closest to an integer.
3. Remove any constraints that are now
unconditionally satisfied.
4. Go to step 1.
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
31
Recursive Rounding
ILP has exponential complexity.
 Recursive rounding:

ILP is transformed into k LPs with
progressively reducing number of variables.
 Number of LPs, k, is the size of the final
solution, i.e., the number of non-zero
variables in the test minimization problem.
 Recursive rounding complexity is k × O(np),
where k ≤ n, n is number of variables.

Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
32
Four-Bit ALU Circuit
ILP
Recursive rounding
Initial
vectors
Vectors
CPU s
Vectors
CPU s
285
14
0.65
14
0.42
400
13
1.07
13
1.00
500
12
4.38
13
3.00
1,000
12
4.17
12
3.00
5,000
12
12.95
12
9.00
10,000
12
34.61
12
17.0
16,384
12
87.47
12
37.0
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
33
ILP vs. Recursive Rounding
100
ILP
CPU s
75
Recursive
Rounding
50
25
0
0
Copyright Agrawal, 2007
5,000
10,000
ELEC6270 Fall 07, Lecture 8
15,000 Vectors
34
N-Detect Tests (N = 5)
Relaxed LP/Recur. rounding
ILP (exact)
Circuit
Unoptimized
vectors
c432
608
196.38
197
1.0
197
1.0
c499
379
260.00
260
1.2
260
2.3
c880
1,023
125.97
128
14.0
127
881.8
c1355
755
420.00
420
3.2
420
4.4
c1908
1,055
543.00
543
4.6
543
6.9
c2670
959
477.00
477
4.7
477
7.2
c3540
1,971
467.25
477
72.0
471
20008.5
c5315
1,079
374.33
377
18.0
376
40.7
c6288
243
52.52
57
39.0
57
34740.0
c7552
2,165
841.00
841
52.0
841
114.3
Copyright Agrawal, 2007
Lower
bound
Min.
Min.
CPU s
vectors
vectors
ELEC6270 Fall 07, Lecture 8
CPU s
35
Finding LP/ILP Solvers



R. Fourer, D. M. Gay and B. W. Kernighan, AMPL: A
Modeling Language for Mathematical Programming,
South San Francisco, California: Scientific Press, 1993.
Several of programs described in this book are available
to Auburn users.
B. R. Hunt, R. L. Lipsman, J. M. Rosenberg, K. R.
Coombes, J. E. Osborn and G. J. Stuck, A Guide to
MATLAB for Beginners and Experienced Users,
Cambridge University Press, 2006.
Search the web. Many programs with small number of
variables can be downloaded free.
Copyright Agrawal, 2007
ELEC6270 Fall 07, Lecture 8
36