Transcript CPLEX 6.5 - informs Houston

Linear & Integer
Programming:
A Decade of Computation
Robert E. Bixby
ILOG, Inc. and Rice University
Co-Authors
Mary Fenelon, Zongao Gu, Irv Lustig, Ed Rothberg, Roland Wunderling
ILOG, Inc
Machine Comparison
Model: PILOTS
constraints: 1141, variables: 3652, nonzeros: 43167
Machine
Sun 3/150
Pentium PC (60 MHz)
IBM 590 Powerstation
SGI R8000 Power Challenge
Athlon (650 MHz)
Compaq (667 Mhz EV67)
Seconds
44064.0
222.6
65.0
44.8
22.2
6.8
~5000x improvement
Outline
• Linear programming (LP)
– Introduction
– Computational history 1947 to late 1980s
– The last decade
• Mixed-integer programming (MIP) – Closing
the gap between theory and practice
– Introduction
– Computation history 1954 to late 1990s
– Features in modern codes
– The last year
– A new kind of cutting planes
• The future
LP
A linear program (LP) is an optimization
problem of the form
Minimize cTx
Subject to Ax = b
lxu
Computational History:
1950 –1990
• 1947 Dantzig invents simplex method
• 1951 SEAC (Standards Eastern Automatic Computer) 48 cons. & 72 vars.
– “One could have started an iteration, gone to lunch and returned before it
finished” William Orchard-Hays, 1990.
• 1963 IBM 7090, LP90
1024 cons.
– Oil companies used LP.
• 1973 IBM 360, MPSX & MPSIII
32000 cons.
– These codes lasted into the mid 80s with little fundamental change.
• Mid 1980s –1990
– 1984 Karmarkar, Interior-point (barrier) methods
– 1991 Grötschel: “Some linear programs were hard to solve, even for
highly praised commercial codes like IBM’s MPSX”
– Late 1980s: OSL, XPRESS, CPLEX …
1989 – 1998
An Example
LAU2: A Fleet Assignment Model (1989)
4420 cons, 6711 vars, 101377 nzs
(Tests run on 500 MHz EV6 & Cray Y/MP)
• Idea 0: Run CPLEX
– 7 hours on a Cray Y/MP,
stalled in phase I.
• Idea 1: Handling degeneracy:
Perturbation
– 12332 secs, 1548304 itns
• Idea 2: Barrier methods
–
656 secs (Cray Y/MP)
• Idea 3: Better pricing:
hybrid = partial pricing + devex
(1973)
–
307 secs
• Idea 4: Use the dual
– Explicit dual: 103 secs
– Dual simplex: 345 secs
• Idea 5: Steepest edge
– Dual steepest edge: 21 secs
1999 – 2000
• Examine larger models:  10000 rows
• Bottleneck: Linear solves for systems with very sparse
input and very sparse output (BTRAN, FTRAN)
• Eliminating the Bottleneck: These solves can be
done in “linear time” (Linear-Algebra folklore:
reachability)
Bottleneck Removed:
Exploiting It
• Dual: Variables with two finite bounds usually do
not need to be binding in the ratio test.
• Sparse pricing: Fast updates for minimum when
few reduced-costs change.
• …
BIG Test Set
Model Rows Cols
NZs
Model Rows Cols
NZs
Model Rows
Cols
NZs
M1
M2
M3
M4
M5
M6
M7
M8
M9
M10
M11
M12
M13
M14
M15
M16
M17
M18
M19
M20
M21
M22
M23
M24
M25
M26
M27
150110
361567
151383
103866
132295
162709
225514
86753
88340
203112
690273
136324
276895
5392558
124280
152952
170369
728432
337746
172018
154822
169045
81245
2139096
1067713
288421
96118
M28
M29
M30
M31
M32
M33
M34
M35
M36
M37
M38
M39
M40
M41
M42
M43
M44
M45
M46
M47
M48
M49
M50
M51
M52
M53
M54
261968
161640
505253
2632880
2001102
1013168
196524
623730
829040
161831
208179
557466
208769
246006
1508199
381259
370839
1928534
2110518
189864
722066
4966017
321663
228565
617683
1807146
391081
M55
M56
M57
M58
M59
M60
M61
M62
M63
M64
M65
M66
M67
M68
M69
M70
M71
M72
M73
M74
M75
M76
M77
M78
M79
M80
M81
M82
191233
100078
144693
157496
111891
612608
316800
197489
326504
154699
112955
487427
93288
159109
467192
377918
358239
707556
189867
23732
363092
1205640
920198
638512
559428
1533590
1169910
1685236
840986
485414
717229
418321
305125
1722112
1899600
749771
2102273
358171
602948
974854
459680
457198
1025706
930166
1211488
1570514
2787708
397080
1061495
6481640
5936426
1231403
1909649
5327318
2511088
3370472
10295
13005
14738
15014
15051
15349
15455
15540
16223
16768
17681
18262
19103
19374
19519
19844
19999
21019
22513
22797
23610
23700
23712
24377
26618
27349
27441
50040
77133
33025
37372
34553
35215
59942
23752
28568
39474
165188
23211
33490
180670
45832
55528
85191
115761
99785
63995
44063
23005
31680
46592
38904
97710
15128
27899
28240
28420
29002
29017
29147
29724
30190
31770
33440
34994
35519
35645
36400
38782
39951
41340
41344
41366
43387
43687
44150
44211
47423
48548
54447
55020
26243
55200
164024
111722
20074
9984
98124
57000
272372
56624
87510
43582
34675
92878
261079
125000
64162
163569
78750
107164
164831
200077
37199
81915
163200
326504
117910
55463
60384
63856
66185
67745
69418
84840
95011
99578
105127
108393
118158
123964
125211
129181
155265
175147
179080
185929
186441
209760
269640
280756
319256
344297
589250
716772
1000000
LP Performance Improvement
1999 – 2000
Segment
Biggest
Biggest
Biggest
Biggest
Biggest
Biggest
Biggest
All
10
20
30
40
50
60
70
Primal
Dual
Best
8.5
7.9
7.4
6.4
5.5
5.2
5.1
4.4
22.3
18.8
20.2
14.0
11.9
10.1
9.1
8.0
18.0
12.2
11.3
8.0
6.7
6.2
5.6
5.2
Barrier: All 3.6
Algorithm Comparison
• Dual vs. Primal: 2.6x (7 not solved by primal)
• Dual vs. Barrier: 1.2x
• And barrier is now faster anyway:
CPLEX 7.0 barrier 1.6x faster
MIP:
Closing the Gap between
Theory and Practice
A mixed-integer program (MIP) is an
optimization problem of the form
Minimize
c Tx
Subject to Ax=b
lxu
some or all xj integral
Branch and Bound (Cut) Tree
Solve LP relaxation:
v=3.5 (fractional)
Root
Upper Bound
G
A
P
Lower Bound
Integer
Infeas
Integer
Remarks:
(1) GAP = 0  Proof of optimality
(2) Practice: Often good enough to have good Solution
MIP really is HARD
A Customer Model: 44 cons, 51 vars, 167 nzs, maximization
51 general integer variables
Branch-and-Cut: Initial integer solution -2586.0
Initial upper bound
-1379.4
…after 120,000 seconds, 370,000,000 B&C nodes, 45 Gig tree
Integer solution and bound: UNCHANGED
Electrical Power Industry, ERPI GS-6401, June 1989:
Mixed-integer programming is a powerful modeling tool,
“They are, however, theoretically complicated and
computationally cumbersome”
California Unit Commitment:
7 Day Model
UNITCAL_7: 48939 cons, 25755 vars (2856 binary)
Previous attempts (by model formulator)
2 Day model: 8 hours, no progress
7 Day model: 1 hour to solve initial LP
CPLEX 7.0 on 667 MHz Compaq EV67
Running defaults ...
Model: UNITCAL_7
Problem 'unitcal_7.sav.gz' read.
Reduced MIP has 39359 rows, 20400 columns, and 106950 nonzeros.
Root relaxation solution time =
5.22 sec.
Nodes
Node Left
*
*
0
0
100
152
1157
87
135
396
Objective
IInf
1.9396e+07
1.9487e+07
1.9561e+07
1.9565e+07
1.9566e+07
1.9567e+07
1.9921e+07
1.9958e+07
1.9636e+07
797
315
377
347
346
354
110
0
0
Cuts/
Best Node
ItCnt
Gap
1.9396e+07
Cuts: 1992
Cuts: 1178
Cuts: 247
Flowcuts: 68
Cuts: 105
1.9571e+07
1.9958e+07
1.9571e+07
1.9636e+07
1.9626e+07
13871
17948
22951
24162
24508
25024
34065
34973
178619
1.94%
0.05%
Best Integer
GUB cover cuts applied: 2
Cover cuts applied: 7
Implied bound cuts applied: 1186
Flow cuts applied: 1120
Flow path cuts applied: 7
Gomory fractional cuts applied: 147
Integer optimal solution: Objective =
1.9635558244e+07
Solution time = 1234.05 sec. Iterations = 310699 Nodes = 6718
Computational History:
1950 –1998
•
1954 Dantzig, Fulkerson, S.
Johnson: 49 city TSP
– Solved to optimality using cutting
planes and solving LPs by hand
•
1957 Gomory
– Cutting plane algorithm: A
complete solution
•
1960 Land, Doig
– B&B
•
1966 Beale, Small
– B&B, Depth-first-search
•
1972 UMPIRE, Forrest, Hirst,
Tomlin
– SOS, pseudo-costs, best
projection, …
•
1972 – 1998 Good B&B remained
the state-of-the-art in commercial
codes, in spite of
– 1983 Crowder, Johnson, Padberg:
PIPX, pure 0/1 MIP
– 1987 Van Roy and Wolsey:
MPSARX, mixed 0/1 MIP
1998…
A new generation of MIP codes
• Linear programming
– Usable, stable, robust
performance
• Variable selection
– pseudo-costs, strong branching
• Primal heuristics
– 5 different tried at root, one
selected
• Node presolve
– Fast, incremental bound
strengthening
• Probing
– Three levels
• Auto disaggregation
–  xj  ( uj) y, y = 0/1
preferred
• Cutting planes
– Gomory, knapsack covers,
flow covers, mix-integer
rounding, cliques, GUB
covers, implied bounds, path
cuts, disjunctive cuts
x + y  3.5, x  0, y  0 integral
y
y  3.5
4.0
(0, 3.5)
3.0
2.0
y  3.5
1.0
1.0
2.0
3.0
4.0
x
x + y  3.5, x  0, y  0 integral
y
y  3.5
4.0
3.0
CUT:
2x + y  4 2.0
y  3.5
1.0
1.0
2.0
3.0
4.0
x
Gomory Mixed Cut
• Given y, xj  Z+, and
y +  aijxj = d = d + f, f > 0
• Rounding: Where aij = aij + fj, define
t = y + (aijxj: fj  f) + (aijxj: fj > f)  Z
• Then
(fj xj: fj  f) + (fj-1)xj: fj > f) = d - t
• Disjunction:
t  d  (fjxj : fj  f)  f
t  d  ((1-fj)xj: fj > f)  1-f
• Combining:
((fj/f)xj: fj  f) + ([(1-fj)/(1-f)]xj: fj > f)  1
BIGTEST – 80 Models
CPLEX 6.0 versus CPLEX 7.0
• Running defaults (7200 second limit)
– CPLEX 6.0 FAILS on 31
– CPLEX 7.0 FAILS on 2
• CPLEX 6.0 tuned versus CPLEX 7.0 defaults
– CPLEX 7.0 5.1x
faster
Performance Impact: Individual Cuts
• Cliques
%
• Implied
%
• Path
%
• GUB covers
%
• Disjunctive
%
• Flow covers
%
• MIR
-3
-1
0
14
17
41
56
Local Cuts for MIP:
An idea from the TSP
(Applegate, Chvátal, Cook, Bixby)
Membership Problem
Given:
a. A polytope P  Rn.
b. An oracle for optimizing linear
objectives over P.
c. x*  Rn
Question: Find p1,…,pk P giving x* as a convex
combination, or find (a, ) such that
aTx* >  and aTx   xP
Solving Membership:
Delayed Column Generation
Step 1: Find 1,…, k such that
jpj = x*
j = 1
j  0 (all j)
Succeed: Done
Fail: LP code gives (a, ) such
that
aTx* >  and aTpj  
for j = 1,…,k.
Step 2: Use the oracle to solve
 = Maximize aTy
Subject to yP
Let y*P satisfy  = aTy*.
Case 1:     done
Case 2: Append y* to the list
p1,…,pk and return to Step 1.
Application to MIP
For the given MIP:
• Let x* be a solution of an LP relaxation
minimize {cTx: Ax=b, l  x  u}
and assume x* is not integral feasible.
• Let P = polyhedron defined by a subset of Ax=b together
with l  x  u.
Prototypical choice: A single constraint. Apply delayed
column generation using MIP code itself as the
optimization oracle.
Future?
• Linear programming
– The improvements in the last 2 years were completely
unanticipated!
– Examine behavior of even larger and more-difficult
models.
• Mixed-integer programming
– Increasingly exploit special structure.