Transcript IBM ILOG CPLEX Advanced Performance Tuning Optimization Technologies Workshop John Gregory CPLEX Product Manager IBM [email protected] Copyright © ILOG 2008 CPLEX Performance Tuning Overview  Speed  Continuous Models (LP,

IBM ILOG CPLEX Advanced
Performance Tuning
Optimization Technologies Workshop
John Gregory
CPLEX Product Manager
IBM
[email protected]
Copyright © ILOG 2008
CPLEX Performance Tuning Overview
 Speed
 Continuous Models (LP, QP, QCP/SOCP)
 MIP
 Memory
 Operating System
 Deployment Application
 CPLEX
 Numerical
 Ill conditioning
 Numerical Instability
 Summary
2
Copyright © ILOG 2008
Tuning for Speed: Continuous Models
 Try each algorithm
 Primal Simplex
 Dual Simplex
 Network Simplex
 Barrier (only option for QCP/SOCP)
 Sifting (LP only)
 Concurrentopt (LP only, parallel processors)
3
Copyright © ILOG 2008
Tuning for Speed: Continuous Models
 Other tactics
 Presolve dual on (apply presolve, then solve the dual model)
 LP only
Constraints
: 1972170
[Greater: 1972170]
Variables
:
[Free: 275036]
275036
Constraint nonzeros: 9234686
Objective
nonzeros:
156817
RHS
nonzeros: 1487481
 Parallel barrier method
 Check for numerical difficulties
4
Copyright © ILOG 2008
Tuning for Speed: Continuous Models
 Iteration logs
 Degeneracy (simplex methods)
 Excessive time per iteration
5
Copyright © ILOG 2008
Tuning for Speed : Continuous Models
 Degeneracy, sample log
Iter:
4751
Infeasibility = 8.000000
Iter:
4870
Infeasibility = 8.000000
Iter:
4976
Infeasibility = 6.999999
Iter:
5083
Infeasibility = 6.000000
Iter:
5191
Infeasibility = 6.000000
Iter:
5862
Infeasibility = 6.000000
Iter:
5968
Infeasibility = 6.000000
...
Perturbation started.
6
Copyright © ILOG 2008
Tuning for Speed: Continuous Models
 Excessive time per iteration, sample log
Elapsed time =
12
sec.
138.23 sec. (37000 iterations)
Iter: 37969
Infeasibility = 387849.999786
Iter: 39121
Infeasibility = 379979.999768
Iter: 40295
Infeasibility = 375639.999998
Elapsed time =
150.41 sec. (41000 iterations)
...
Elapsed time = 24318.58 sec. (138000 iterations)
827
sec.
Iter: 138958
Infeasibility = 23.754244
Elapsed time = 24736.98 sec. (139000 iterations)
Elapsed time = 25145.43 sec. (140000 iterations)
7
Copyright © ILOG 2008
Tuning for Speed: Continuous Models
 Degeneracy
 Try each algorithm
 Instruct CPLEX to perturb the model
 Excessive time per iteration
 Try each algorithm
 Check for virtual memory usage
 Run with more memory available
 Turn on memory emphasis parameter
 Consider alternate pricing (pgradient/dgradient)
 Time per iteration depends on pricing scheme
8
Copyright © ILOG 2008
Tuning for Speed: MIP
 Try the Tuning Tool (CPLEX 11 or later)
 Primarily for MIPs, but works for all problem types
 Works with all CPLEX interfaces, including
interactive optimizer
 Can make use of internal algorithm information
 Let CPLEX and your computer do the work
9
Copyright © ILOG 2008
Tuning for Speed: MIP
 Input: Model(s) to tune, tuning instructions
 Problem in memory or set of problems in files
 Fixed parameter set, parameters which should not be
tuned, e.g., tolerances, threads
 Parameters, metrics to control tuning process (e.g.
total tuning time, average or worst case run time)
 Output: Tuned and fixed parameter settings
10
Copyright © ILOG 2008
Tuning for Speed: MIP
 Examine node log (if Tuning Tool doesn’t help)
 Excessive iteration counts for node relaxations.
 Use different start/sub algorithm.
 Barrier method doesn’t support restarts.
 Set dual gradient to exact steepest edge norms, file
parameter to store node file on disk and compress it.
 More progress needed in best integer or best node
value?
11
Copyright © ILOG 2008
Tuning for Speed: MIP
 Excessive iteration counts for node LPs, sample
node log #1 (dual simplex to solve node LPs)
Nodes
Node Left
0
0
0
0
0
2
1
1
2
2
3
1
…
8
2
9
1
…
Solution time =
Objective
IInf
-89.0000
-89.0000
-89.0000
infeasible
-89.0000
infeasible
6
6
6
-89.0000
infeasible
8
Best Integer
5
Cuts/
Best Node
ItCnt
-89.0000
Fract: 4
-89.0000
-89.0000
-89.0000
-89.0000
5278
12799
12799
20767
27275
32502
-89.0000
-89.0000
65717
73714
177.33 sec. Iterations = 73714 Nodes = 10 (1)
12
Copyright © ILOG 2008
Tuning for Speed: MIP
 Excessive iteration counts for node LPs, sample
node log #2 (primal simplex to solve node LPs)
Nodes
Node Left
0
0
0
0
0
2
1
1
2
2
3
1
…
8
2
9
1
…
Solution time =
Objective
IInf
-89.0000
-89.0000
-89.0000
infeasible
-89.0000
infeasible
5
5
5
-89.0000
infeasible
8
Best Integer
5
Cuts/
Best Node
ItCnt
-89.0000
Fract: 5
-89.0000
-89.0000
-89.0000
-89.0000
6603
7120
7120
9621
10616
12963
-89.0000
-89.0000
21522
23891
54.37 sec. Iterations = 23891 Nodes = 10 (1)
13
Copyright © ILOG 2008
Tuning for Speed: MIP
 Progress in best node but not best integer:
Nodes
Node Left
Objective
IInf
Best Integer
Cuts/
Best Node
22.0000
22.3333
22.6667
4433
5196
6164
29.03%
27.96%
26.88%
25.6667
25.6667
25.7778
55739
56424
57267
17.20%
17.20%
16.85%
ItCnt
Gap
...
300
400
500
229
309
387
22.6667
cutoff
26.5000
40
31
31.0000
31.0000
31.0000
7800
7900
8000
5260
5324
5385
28.5000
28.2500
27.3750
23
26
30
31.0000
31.0000
31.0000
...
14
Copyright © ILOG 2008
Tuning for Speed: MIP
 Progress in best integer but not best node:
Nodes
Node Left
Objective
IInf
Best Integer
Cuts/
Best Node
2018.0000
560.0000
810.0000
27
0
0
291
175
3780.0000
2626.0000
2590.0000
2590.0000
2590.0000
560.0000
560.0000
560.0000
560.0000
560.0000
3703
3703
4405
4553
5747
85.19%
78.67%
78.38%
78.38%
78.38%
1544.0000
944.0000
1468.0000
0
110
176
93
1710.0000
1710.0000
1710.0000
1710.0000
560.0000
560.0000
560.0000
560.0000
66026
66279
66801
67732
67.25%
67.25%
67.25%
67.25%
ItCnt
Gap
…
300
300+
393
400
500
296
296
368
372
472
7740+
7800
7900
8000
5183
5240
5325
5424
*
*
...
*
15
Copyright © ILOG 2008
Tuning for Speed: MIP
 Tactics for lack of progress in best integer
 Use especially if customer accepts a good, perhaps
sub optimal solution.
 MIP emphasis
 Set to 1 (emphasize integer feasibility)
 Set to 4 (emphasize finding hidden integer solutions)
 Solution Polishing
 Requires an integer feasible solution
 Use to improve sub optimal solutions at end of MIP search
 Or use as replacement for majority of MIP search when
- local search works well
- node LPs are time consuming
- improving best node appears hopeless
16
Copyright © ILOG 2008
Tuning for Speed: MIP
 Lack of progress in best integer (continued)
 RINS Heuristic
 Local search heuristic
- Needs an integer feasible solution
 MIP Starts and Solution Repair
 CPLEX 10.0 and later accepts partial MIP starts
- Solution repair for (partial) infeasible MIP starts
 May make RINS, Solution Polishing more effective
 Feasibility Pump Heuristic (CPLEX 11 and later)
 Applied first at root node
17
Copyright © ILOG 2008
Tuning for Speed: MIP
 Lack of progress in best integer (continued)
 Probing
 Set to 3 (maximum probing)
 Cuts
 Set to 2 (aggressive cut generation)
 Cuts and Probing don’t explicitly help find feasible
solutions, but the resulting tighter formulation may
help.
 Can be useful when combined with setting MIP emphasis to
1 (emphasize integer feasibility)
18
Copyright © ILOG 2008
Tuning for Speed: MIP
 Tactics for lack of progress in best node
 MIP emphasis
 Set to 2 or 3 to emphasize proving optimality and moving
best node respectively.
- May need Partial/Infeasible MIP start, Feasibility Pump or
RINS Heuristic to compensate for fewer integer feasible
solutions
 Probing
 Set to 3 (maximum probing)
 Cuts
 Set to 2 (aggressive cut generation)
 Cuts and Probing often tighten the model formulation
19
Copyright © ILOG 2008
Tuning for Speed: MIP
 Tactics for lack of progress in best node and
best integer
 Set MIP emphasis to 3
 May cause difficulties finding integer feasible solutions
 Compensate by using (partial or infeasible) MIP starts,
Feasibility Pump, RINS Heuristic, Solution Polishing to help
find more integer feasible solutions
20
Copyright © ILOG 2008
Tuning for Speed: MIP
 Tactics for lack of progress in best node and
best integer (continued)
 Provide a starting solution, regardless of objective
value
 Can help CPLEX’s local search heuristics
 Can yield additional tightening of the model
 Consider external methods (e.g. Constraint Programming,
Metaheuristics like Simulated Annealing, model specific
heuristics) to find a starting solution
 Try CPLEX’s feasopt feature
21
Copyright © ILOG 2008
Tuning for Speed: MIP
 Other tactics
 Specific model knowledge determines parameter
settings
n
x
i 1
i 1

xi binary
Try branching up first
22
Copyright © ILOG 2008
Tuning for Speed: MIP
 Other tactics (continued)
 Specific model knowledge determines priority order.
 Time period models
- Higher priority to earlier time periods.
 Dependencies among integer variables
xi -Uz i  0
xi integer
zi binary
23
Copyright © ILOG 2008
Tuning for Speed: MIP
 Other tactics (continued)
 Specific model knowledge determines changes to
model
 Adding model specific cuts
- Tighten formulation
- Remove symmetry
 Sherali, Smith “Improving Zero-One Model Representations
Via Symmetry Considerations”
 Model reformulation
http://www.ilog.com/products/cplex/news/newsletters/cplex_newsletter_9410
.cfm
24
Copyright © ILOG 2008
Memory Tuning: Operating System
 Virtual memory slows performance
 Task Manager (Windows)
 Top, ps, vmstat under Unix
 Make sure you can allocate all available memory
 Java VM limitations
 Use –Xmx option to increase heap size
 On a 32-bit machine, the real limit is 2GB
 Limits on individual accounts
 Compiler/Linker options
 32 bit AIX has default of 256 MB for memory allocations.
 Write a simple program to test available memory (or ask us for
one).
25
Copyright © ILOG 2008
Memory Tuning: Deployment Considerations
 Compare memory usage
 Solve the Deployment application model(s) with:
 Interactive CPLEX
 Simple example programs
 Purify, Insure, Bounds Checker, Valgrind (Linux
only) can help with memory leaks
 Make sure you solve the correct model
 Display problem statistics
 Data check parameter
 Diagnostic routines in check.c
26
Copyright © ILOG 2008
Memory Tuning: CPLEX Considerations
 Sometimes CPLEX really uses all available
memory
 Default settings emphasize speed over memory
conservation.
 Is CPLEX memory usage reasonable?
 1 GB per million constraints of an LP
- Exceptions: large aspect ratios, very dense constraints
 1 GB per million (constraints + quadratic variables) of a QP
- Exception: dense Q matrix lacking sparse factorization
 At least that much for MIP, MIQP
27
Copyright © ILOG 2008
Memory Tuning: CPLEX Considerations
 Memory conservation techniques
 General
 Turn memory emphasis on (slight extra run time)
 Use different algorithm (LP and QP)
 As a last resort, turn presolve off (potential big increase in
run time)
 LP
 Set presolve dual indicator on
28
Copyright © ILOG 2008
Memory Tuning: CPLEX Considerations
 Memory conservation techniques (continued)
 MIP
 MIP search tree often dominates memory usage
- Performance tuning to reduce node count.
- Setting node file parameter to 3 efficiently writes nodes to
disk.
- Setting variable select parameter to 3 (strong branching).
Fewer nodes required to make the same amount of
progress.
29
Copyright © ILOG 2008
Memory Tuning: Other Tactics
 Buy more memory (but only after accurately
estimating requirements)
 Specialized algorithms
 Example: Column generation for models with large
aspect ratios.
30
Copyright © ILOG 2008
Memory Tuning: Other Tactics
 Review of model development process
 Model reformulation
 Reduce problem dimensions
 Reduce “difficulty” while increasing problem size.
 MIP reformulations for column generation.
 Increase in problem size in exchange for sparser problem
data
T
T
min x Qx st
Q L L
Ax  b 
T
min y y st
y  Lx  0
Ax  b
31
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Sensitivity of computed solutions to perturbations in
problem data
 Condition number of a matrix
 CPLEX solves systems of equations of the form
Bx  b
 Exact solution is
1
xB b
32
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Condition number of a matrix (continued)
 To measure effects of perturbations, consider
B( x  x)  b  b
 Exact solution is
1
x  x  B (b  b)
 Combine with exact solution of x:
1
x  B b
33
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Condition number of a matrix (ctd).
 Using Cauchy - Schwarz inequality
1
|| x ||  || B || || b ||
 This gives max. absolute change in x relative to
change in b. Now consider relative change by using
Cauchy - Schwarz inequality in original system:
|| b ||  || B || || x ||
34
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Condition number of a matrix (continued)
 Combine and rearrange to get
1
|| x || / || x ||  || B || || B || (|| b || / || b ||)
 Similar procedure for perturbing B yields
1
|| x || / || x  x ||  || B || || B || (|| B || / || B ||)
1
cond(B)  || B || || B ||
35
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Ill conditioning can occur under perfect
arithmetic
 But, finite precision can perturb the exact
system of equations we wish to solve, resulting
in significant changes to the computed solution
36
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Sources of perturbations due to finite precision
 Rounding of data
 Near singular matrices
 Avoid computing data in single precision
 Round off error in the simplex method computations,
particularly the basis factorization.
 Round off error due to scaling of large data values
 Floating point calculations involving values of greatly
different orders of magnitude
 Avoid mixtures of very large and small coefficients
37
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Rounding of data
Maximize x1 + x2
c1: 1/3 x1 + 2/3 x2= 1
c2: x1 + 2 x2 = 3
Solution: x1 = 3.0
Maximize x1 + x2
c1: .333333 x1 + .666667 x2 = 1
c2: x1 + 2 x2 = 3
Solution: x1 = x2 = 1.0, cond(B) = 6.66667e+06
Maximize x1 + x2
c1: .333333333 x1 + .666666667 x2 = 1
c2: x1 + 2 x2 = 3
Solution: x1 = 3.0, cond(B) = 6.333333
38
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Avoid rounding whenever possible
Exact formulation:
Maximize x1 + x2
c1: 1/3 x1 + 2/3 x2= 1
c2: x1 + 2 x2 = 3
Imprecisely rounded:
Maximize x1 + x2
c1: .333333 x1 + .666667 x2 = 1
c2: x1 + 2 x2 = 3
Scale to integral value whenever possible:
Maximize x1 + x2
c1: x1 + 2 x2 = 3
c2: x1 + 2 x2 = 3
39
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Rounding of data (continued)
 Changes in right hand side can have similar effect
Maximize x1 + x2
c1: .333333 x1 + .666667 x2 = 1
c2: x1 + 2 x2 = 3
Solution: x1 = x2 = 1.0
Maximize x1 + x2
c1: .333333 x1 + .666667 x2 = .999999
c2: x1 + 2 x2 = 3
Solution: x1 = 3.0
40
Copyright © ILOG 2008
Numerical Tuning: Numerical Instability
 Large values in problem data
 Bounds can magnify round off error:
BxB  b - ( AN  AN ) xN
 Large coefficient ratios in matrix rows or columns:
x  1000000000 0 z  0
x  5000, z binary
41
Copyright © ILOG 2008
Symptoms of Numerical Instability
 Sample iteration log
Iter: 6389
Objective
= 13137.039899
Iter: 6390
Objective
= 13137.039899
Iter: 6391
Objective
= 13726.011591
Markowitz threshold set to 0.3.
Iter: 6456
Scaled infeas = 300615.030682
...
Iter: 6752
Scaled infeas = 0.000002
Iter: 6754
Objective
= -23870.812630
42
Copyright © ILOG 2008
Numerical Tuning Tactics
 General
 Turn numerical emphasis on
 Rounding of data
 Compute data as accurately as possible
 Round off error in simplex method
 Set Markowitz tolerance to .90 (or higher if
necessary)
 Large values in problem data
 Reduce large coefficients if possible
 Set scaling parameter to 1 (aggressive scaling)
43
Copyright © ILOG 2008
CPLEX Performance Tuning Summary
 CPLEX solves most problems with no trouble
 Organized tactics for the outliers can yield dramatic savings in
time.
 Different categories of performance problems.
 Use all available information
 Iteration and node logs for LP and MIP
 Interactive CPLEX
 Simple test programs (CPLEX and your own)
 OS memory management tools
 Purify/Insure/Bounds Checker/Valgrind
 ILOG Technical Support and Consulting
44
Copyright © ILOG 2008