Transcript CPLEX Performance Tuning

IBM ILOG CPLEX Advanced
Performance Tuning
Optimization Technologies Workshop
John Gregory
CPLEX Product Manager
IBM
[email protected]
Copyright © ILOG 2008
CPLEX Performance Tuning Overview
 Speed
 Continuous Models (LP, QP, QCP/SOCP)
 MIP
 Memory
 Operating System
 Deployment Application
 CPLEX
 Numerical
 Ill conditioning
 Numerical Instability
 Summary
2
Copyright © ILOG 2008
Tuning for Speed: Continuous Models
 Try each algorithm
 Primal Simplex
 Dual Simplex
 Network Simplex
 Barrier (only option for QCP/SOCP)
 Sifting (LP only)
 Concurrentopt (LP only, parallel processors)
3
Copyright © ILOG 2008
Tuning for Speed: Continuous Models
 Other tactics
 Presolve dual on (apply presolve, then solve the dual model)
 LP only
Constraints
: 1972170
[Greater: 1972170]
Variables
:
[Free: 275036]
275036
Constraint nonzeros: 9234686
Objective
nonzeros:
156817
RHS
nonzeros: 1487481
 Parallel barrier method
 Check for numerical difficulties
4
Copyright © ILOG 2008
Tuning for Speed: Continuous Models
 Iteration logs
 Degeneracy (simplex methods)
 Excessive time per iteration
5
Copyright © ILOG 2008
Tuning for Speed : Continuous Models
 Degeneracy, sample log
Iter:
4751
Infeasibility = 8.000000
Iter:
4870
Infeasibility = 8.000000
Iter:
4976
Infeasibility = 6.999999
Iter:
5083
Infeasibility = 6.000000
Iter:
5191
Infeasibility = 6.000000
Iter:
5862
Infeasibility = 6.000000
Iter:
5968
Infeasibility = 6.000000
...
Perturbation started.
6
Copyright © ILOG 2008
Tuning for Speed: Continuous Models
 Excessive time per iteration, sample log
Elapsed time =
12
sec.
138.23 sec. (37000 iterations)
Iter: 37969
Infeasibility = 387849.999786
Iter: 39121
Infeasibility = 379979.999768
Iter: 40295
Infeasibility = 375639.999998
Elapsed time =
150.41 sec. (41000 iterations)
...
Elapsed time = 24318.58 sec. (138000 iterations)
827
sec.
Iter: 138958
Infeasibility = 23.754244
Elapsed time = 24736.98 sec. (139000 iterations)
Elapsed time = 25145.43 sec. (140000 iterations)
7
Copyright © ILOG 2008
Tuning for Speed: Continuous Models
 Degeneracy
 Try each algorithm
 Instruct CPLEX to perturb the model
 Excessive time per iteration
 Try each algorithm
 Check for virtual memory usage
 Run with more memory available
 Turn on memory emphasis parameter
 Consider alternate pricing (pgradient/dgradient)
 Time per iteration depends on pricing scheme
8
Copyright © ILOG 2008
Tuning for Speed: MIP
 Try the Tuning Tool (CPLEX 11 or later)
 Primarily for MIPs, but works for all problem types
 Works with all CPLEX interfaces, including
interactive optimizer
 Can make use of internal algorithm information
 Let CPLEX and your computer do the work
9
Copyright © ILOG 2008
Tuning for Speed: MIP
 Input: Model(s) to tune, tuning instructions
 Problem in memory or set of problems in files
 Fixed parameter set, parameters which should not be
tuned, e.g., tolerances, threads
 Parameters, metrics to control tuning process (e.g.
total tuning time, average or worst case run time)
 Output: Tuned and fixed parameter settings
10
Copyright © ILOG 2008
Tuning for Speed: MIP
 Examine node log (if Tuning Tool doesn’t help)
 Excessive iteration counts for node relaxations.
 Use different start/sub algorithm.
 Barrier method doesn’t support restarts.
 Set dual gradient to exact steepest edge norms, file
parameter to store node file on disk and compress it.
 More progress needed in best integer or best node
value?
11
Copyright © ILOG 2008
Tuning for Speed: MIP
 Excessive iteration counts for node LPs, sample
node log #1 (dual simplex to solve node LPs)
Nodes
Node Left
0
0
0
0
0
2
1
1
2
2
3
1
…
8
2
9
1
…
Solution time =
Objective
IInf
-89.0000
-89.0000
-89.0000
infeasible
-89.0000
infeasible
6
6
6
-89.0000
infeasible
8
Best Integer
5
Cuts/
Best Node
ItCnt
-89.0000
Fract: 4
-89.0000
-89.0000
-89.0000
-89.0000
5278
12799
12799
20767
27275
32502
-89.0000
-89.0000
65717
73714
177.33 sec. Iterations = 73714 Nodes = 10 (1)
12
Copyright © ILOG 2008
Tuning for Speed: MIP
 Excessive iteration counts for node LPs, sample
node log #2 (primal simplex to solve node LPs)
Nodes
Node Left
0
0
0
0
0
2
1
1
2
2
3
1
…
8
2
9
1
…
Solution time =
Objective
IInf
-89.0000
-89.0000
-89.0000
infeasible
-89.0000
infeasible
5
5
5
-89.0000
infeasible
8
Best Integer
5
Cuts/
Best Node
ItCnt
-89.0000
Fract: 5
-89.0000
-89.0000
-89.0000
-89.0000
6603
7120
7120
9621
10616
12963
-89.0000
-89.0000
21522
23891
54.37 sec. Iterations = 23891 Nodes = 10 (1)
13
Copyright © ILOG 2008
Tuning for Speed: MIP
 Progress in best node but not best integer:
Nodes
Node Left
Objective
IInf
Best Integer
Cuts/
Best Node
22.0000
22.3333
22.6667
4433
5196
6164
29.03%
27.96%
26.88%
25.6667
25.6667
25.7778
55739
56424
57267
17.20%
17.20%
16.85%
ItCnt
Gap
...
300
400
500
229
309
387
22.6667
cutoff
26.5000
40
31
31.0000
31.0000
31.0000
7800
7900
8000
5260
5324
5385
28.5000
28.2500
27.3750
23
26
30
31.0000
31.0000
31.0000
...
14
Copyright © ILOG 2008
Tuning for Speed: MIP
 Progress in best integer but not best node:
Nodes
Node Left
Objective
IInf
Best Integer
Cuts/
Best Node
2018.0000
560.0000
810.0000
27
0
0
291
175
3780.0000
2626.0000
2590.0000
2590.0000
2590.0000
560.0000
560.0000
560.0000
560.0000
560.0000
3703
3703
4405
4553
5747
85.19%
78.67%
78.38%
78.38%
78.38%
1544.0000
944.0000
1468.0000
0
110
176
93
1710.0000
1710.0000
1710.0000
1710.0000
560.0000
560.0000
560.0000
560.0000
66026
66279
66801
67732
67.25%
67.25%
67.25%
67.25%
ItCnt
Gap
…
300
300+
393
400
500
296
296
368
372
472
7740+
7800
7900
8000
5183
5240
5325
5424
*
*
...
*
15
Copyright © ILOG 2008
Tuning for Speed: MIP
 Tactics for lack of progress in best integer
 Use especially if customer accepts a good, perhaps
sub optimal solution.
 MIP emphasis
 Set to 1 (emphasize integer feasibility)
 Set to 4 (emphasize finding hidden integer solutions)
 Solution Polishing
 Requires an integer feasible solution
 Use to improve sub optimal solutions at end of MIP search
 Or use as replacement for majority of MIP search when
- local search works well
- node LPs are time consuming
- improving best node appears hopeless
16
Copyright © ILOG 2008
Tuning for Speed: MIP
 Lack of progress in best integer (continued)
 RINS Heuristic
 Local search heuristic
- Needs an integer feasible solution
 MIP Starts and Solution Repair
 CPLEX 10.0 and later accepts partial MIP starts
- Solution repair for (partial) infeasible MIP starts
 May make RINS, Solution Polishing more effective
 Feasibility Pump Heuristic (CPLEX 11 and later)
 Applied first at root node
17
Copyright © ILOG 2008
Tuning for Speed: MIP
 Lack of progress in best integer (continued)
 Probing
 Set to 3 (maximum probing)
 Cuts
 Set to 2 (aggressive cut generation)
 Cuts and Probing don’t explicitly help find feasible
solutions, but the resulting tighter formulation may
help.
 Can be useful when combined with setting MIP emphasis to
1 (emphasize integer feasibility)
18
Copyright © ILOG 2008
Tuning for Speed: MIP
 Tactics for lack of progress in best node
 MIP emphasis
 Set to 2 or 3 to emphasize proving optimality and moving
best node respectively.
- May need Partial/Infeasible MIP start, Feasibility Pump or
RINS Heuristic to compensate for fewer integer feasible
solutions
 Probing
 Set to 3 (maximum probing)
 Cuts
 Set to 2 (aggressive cut generation)
 Cuts and Probing often tighten the model formulation
19
Copyright © ILOG 2008
Tuning for Speed: MIP
 Tactics for lack of progress in best node and
best integer
 Set MIP emphasis to 3
 May cause difficulties finding integer feasible solutions
 Compensate by using (partial or infeasible) MIP starts,
Feasibility Pump, RINS Heuristic, Solution Polishing to help
find more integer feasible solutions
20
Copyright © ILOG 2008
Tuning for Speed: MIP
 Tactics for lack of progress in best node and
best integer (continued)
 Provide a starting solution, regardless of objective
value
 Can help CPLEX’s local search heuristics
 Can yield additional tightening of the model
 Consider external methods (e.g. Constraint Programming,
Metaheuristics like Simulated Annealing, model specific
heuristics) to find a starting solution
 Try CPLEX’s feasopt feature
21
Copyright © ILOG 2008
Tuning for Speed: MIP
 Other tactics
 Specific model knowledge determines parameter
settings
n
x
i 1
i 1

xi binary
Try branching up first
22
Copyright © ILOG 2008
Tuning for Speed: MIP
 Other tactics (continued)
 Specific model knowledge determines priority order.
 Time period models
- Higher priority to earlier time periods.
 Dependencies among integer variables
xi -Uz i  0
xi integer
zi binary
23
Copyright © ILOG 2008
Tuning for Speed: MIP
 Other tactics (continued)
 Specific model knowledge determines changes to
model
 Adding model specific cuts
- Tighten formulation
- Remove symmetry
 Sherali, Smith “Improving Zero-One Model Representations
Via Symmetry Considerations”
 Model reformulation
http://www.ilog.com/products/cplex/news/newsletters/cplex_newsletter_9410
.cfm
24
Copyright © ILOG 2008
Memory Tuning: Operating System
 Virtual memory slows performance
 Task Manager (Windows)
 Top, ps, vmstat under Unix
 Make sure you can allocate all available memory
 Java VM limitations
 Use –Xmx option to increase heap size
 On a 32-bit machine, the real limit is 2GB
 Limits on individual accounts
 Compiler/Linker options
 32 bit AIX has default of 256 MB for memory allocations.
 Write a simple program to test available memory (or ask us for
one).
25
Copyright © ILOG 2008
Memory Tuning: Deployment Considerations
 Compare memory usage
 Solve the Deployment application model(s) with:
 Interactive CPLEX
 Simple example programs
 Purify, Insure, Bounds Checker, Valgrind (Linux
only) can help with memory leaks
 Make sure you solve the correct model
 Display problem statistics
 Data check parameter
 Diagnostic routines in check.c
26
Copyright © ILOG 2008
Memory Tuning: CPLEX Considerations
 Sometimes CPLEX really uses all available
memory
 Default settings emphasize speed over memory
conservation.
 Is CPLEX memory usage reasonable?
 1 GB per million constraints of an LP
- Exceptions: large aspect ratios, very dense constraints
 1 GB per million (constraints + quadratic variables) of a QP
- Exception: dense Q matrix lacking sparse factorization
 At least that much for MIP, MIQP
27
Copyright © ILOG 2008
Memory Tuning: CPLEX Considerations
 Memory conservation techniques
 General
 Turn memory emphasis on (slight extra run time)
 Use different algorithm (LP and QP)
 As a last resort, turn presolve off (potential big increase in
run time)
 LP
 Set presolve dual indicator on
28
Copyright © ILOG 2008
Memory Tuning: CPLEX Considerations
 Memory conservation techniques (continued)
 MIP
 MIP search tree often dominates memory usage
- Performance tuning to reduce node count.
- Setting node file parameter to 3 efficiently writes nodes to
disk.
- Setting variable select parameter to 3 (strong branching).
Fewer nodes required to make the same amount of
progress.
29
Copyright © ILOG 2008
Memory Tuning: Other Tactics
 Buy more memory (but only after accurately
estimating requirements)
 Specialized algorithms
 Example: Column generation for models with large
aspect ratios.
30
Copyright © ILOG 2008
Memory Tuning: Other Tactics
 Review of model development process
 Model reformulation
 Reduce problem dimensions
 Reduce “difficulty” while increasing problem size.
 MIP reformulations for column generation.
 Increase in problem size in exchange for sparser problem
data
T
T
min x Qx st
Q L L
Ax  b 
T
min y y st
y  Lx  0
Ax  b
31
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Sensitivity of computed solutions to perturbations in
problem data
 Condition number of a matrix
 CPLEX solves systems of equations of the form
Bx  b
 Exact solution is
1
xB b
32
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Condition number of a matrix (continued)
 To measure effects of perturbations, consider
B( x  x)  b  b
 Exact solution is
1
x  x  B (b  b)
 Combine with exact solution of x:
1
x  B b
33
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Condition number of a matrix (ctd).
 Using Cauchy - Schwarz inequality
1
|| x ||  || B || || b ||
 This gives max. absolute change in x relative to
change in b. Now consider relative change by using
Cauchy - Schwarz inequality in original system:
|| b ||  || B || || x ||
34
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Condition number of a matrix (continued)
 Combine and rearrange to get
1
|| x || / || x ||  || B || || B || (|| b || / || b ||)
 Similar procedure for perturbing B yields
1
|| x || / || x  x ||  || B || || B || (|| B || / || B ||)
1
cond(B)  || B || || B ||
35
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Ill conditioning can occur under perfect
arithmetic
 But, finite precision can perturb the exact
system of equations we wish to solve, resulting
in significant changes to the computed solution
36
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Sources of perturbations due to finite precision
 Rounding of data
 Near singular matrices
 Avoid computing data in single precision
 Round off error in the simplex method computations,
particularly the basis factorization.
 Round off error due to scaling of large data values
 Floating point calculations involving values of greatly
different orders of magnitude
 Avoid mixtures of very large and small coefficients
37
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Rounding of data
Maximize x1 + x2
c1: 1/3 x1 + 2/3 x2= 1
c2: x1 + 2 x2 = 3
Solution: x1 = 3.0
Maximize x1 + x2
c1: .333333 x1 + .666667 x2 = 1
c2: x1 + 2 x2 = 3
Solution: x1 = x2 = 1.0, cond(B) = 6.66667e+06
Maximize x1 + x2
c1: .333333333 x1 + .666666667 x2 = 1
c2: x1 + 2 x2 = 3
Solution: x1 = 3.0, cond(B) = 6.333333
38
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Avoid rounding whenever possible
Exact formulation:
Maximize x1 + x2
c1: 1/3 x1 + 2/3 x2= 1
c2: x1 + 2 x2 = 3
Imprecisely rounded:
Maximize x1 + x2
c1: .333333 x1 + .666667 x2 = 1
c2: x1 + 2 x2 = 3
Scale to integral value whenever possible:
Maximize x1 + x2
c1: x1 + 2 x2 = 3
c2: x1 + 2 x2 = 3
39
Copyright © ILOG 2008
Numerical Tuning: Ill conditioning
 Rounding of data (continued)
 Changes in right hand side can have similar effect
Maximize x1 + x2
c1: .333333 x1 + .666667 x2 = 1
c2: x1 + 2 x2 = 3
Solution: x1 = x2 = 1.0
Maximize x1 + x2
c1: .333333 x1 + .666667 x2 = .999999
c2: x1 + 2 x2 = 3
Solution: x1 = 3.0
40
Copyright © ILOG 2008
Numerical Tuning: Numerical Instability
 Large values in problem data
 Bounds can magnify round off error:
BxB  b - ( AN  AN ) xN
 Large coefficient ratios in matrix rows or columns:
x  1000000000 0 z  0
x  5000, z binary
41
Copyright © ILOG 2008
Symptoms of Numerical Instability
 Sample iteration log
Iter: 6389
Objective
= 13137.039899
Iter: 6390
Objective
= 13137.039899
Iter: 6391
Objective
= 13726.011591
Markowitz threshold set to 0.3.
Iter: 6456
Scaled infeas = 300615.030682
...
Iter: 6752
Scaled infeas = 0.000002
Iter: 6754
Objective
= -23870.812630
42
Copyright © ILOG 2008
Numerical Tuning Tactics
 General
 Turn numerical emphasis on
 Rounding of data
 Compute data as accurately as possible
 Round off error in simplex method
 Set Markowitz tolerance to .90 (or higher if
necessary)
 Large values in problem data
 Reduce large coefficients if possible
 Set scaling parameter to 1 (aggressive scaling)
43
Copyright © ILOG 2008
CPLEX Performance Tuning Summary
 CPLEX solves most problems with no trouble
 Organized tactics for the outliers can yield dramatic savings in
time.
 Different categories of performance problems.
 Use all available information
 Iteration and node logs for LP and MIP
 Interactive CPLEX
 Simple test programs (CPLEX and your own)
 OS memory management tools
 Purify/Insure/Bounds Checker/Valgrind
 ILOG Technical Support and Consulting
44
Copyright © ILOG 2008
Practical Guidelines for Solving
Difficult Mixed Integer Programs
Objective
 CPLEX solves most MIPs to optimality
(including many considered unsolvable only a
few years ago) effectively with default settings
 With recent advances in MIP heuristics, CPLEX finds
good solutions to many of the MIPs it cannot solve to
optimality
 For the remaining difficult MIPs, parameter
tuning may suffice to solve them
 Details in ILOG Optimization Workshops
46
Copyright © ILOG 2008
Objective
 If not, finding good or optimal solutions may be
a time consuming process
 Finding good cuts that strengthen the
formulation
 Simple, straightforward guidelines help derive these cuts or
determine which other approaches are suitable
 Column Generation or other decomposition methods
 Basis Reduction
 Other Reformulations
 Constraint Programming
47
Copyright © ILOG 2008
Overview
 Objective
 Review of Branch and Bound
 Identifying Weak Formulations
 Tuning CPLEX Parameters
 Guidelines for Tightening Difficult MIPs
 Examples
 Methods for Adding Cuts to a Model
 Sanity checking/debugging
 Summary
48
Copyright © ILOG 2008
Review of Branch and Bound
A mixed integer (quadratic) program
(MIP/MIQP) is an optimization problem
of the form
c x  x Qx
Ax  b
l  xu
some or all xj int eger
Minim ize
Subjectto
T
T
49
Copyright © ILOG 2008
Review of Branch and Bound
Branch and Bound for MIP
Upper Bound
Root;
v=3.5
x=2.3
z=0.1
Fathomed
y=0.6
Integer
z=0.3
G
A
P
Lower Bound
Infeas
Integer
Copyright © ILOG 2008
50
Review of Branch and Bound
 Progress of the algorithm depends on:
 Ability to find integer feasible solutions
 # of integer infeasibilities at each node
 Ability to prune nodes
 Objective value of best integer feasible solution
 Ability to move lower bound
 # of other node relaxations with same objective value
 # of active nodes remaining
 Strength of the model formulation
51
Copyright © ILOG 2008
Review of Branch and Bound
 Optimizer Node Log shows algorithm progress:
Nodes
Node Left
...
300
229
400
309
500
387
...
7800
7900
8000
5260
5324
5385
Objective
IInf
Best Integer
Cuts/
Best Node
22.6667
cutoff
26.5000
40
31
31.0000
31.0000
31.0000
22.0000
22.3333
22.6667
4433
5196
6164
29.03%
27.96%
26.88%
28.5000
28.2500
27.3750
23
26
30
31.0000
31.0000
31.0000
25.6667
25.6667
25.7778
55739
56424
57267
17.20%
17.20%
16.85%
ItCnt
Gap
52
Identifying Weak Formulations
 Convex hull of the integer feasible solutions provides the
strongest formulation
 Add valid cuts to strengthen formulation
c1T x  d1
P1  conv({x  Z n : Ax  b, x  0})
c2T x  d 2
P2  {x  R n : Ax  b, x  0}
P3  P2  {x  R n : Cx  d }
P2
Cuts must satisfy
1) ciT x  d i x  P1
(validity)
2) x  P 2 : ciT x  d i
(separation)
P3
c3T x  d3
P1
53
Copyright © ILOG 2008
Identifying Weak Formulations (example)
 A difficult knapsack problem
Minimize
COST:
Subject To
c1: 12228 x1 + 36679 x2 + 36682 x3 + 48908 x4
+ 61139 x5 + 73365 x6 = 89716837
Bounds
x1 >= 0
…
x6 >= 0
Generals
x1 x2 x3 x4 x5 x6
End
54
Copyright © ILOG 2008
Identifying Weak Formulations (example)
 LP solutions are trivial to find (even after many branches
on the variables), but MIP solutions may not be.
c1: 12228 x1 + 36679 x2 + 36682 x3 + 48908 x4 +
61139 x5 + 73365 x6 = 89716837
Domain of x1: [0, 89716837/12228]
Domain of x2: [0, (89716837 – 12228 x1)/36679]
…
Domain of x6: [0, (89716837 – (12228 x1 + 36679 x2
+ 36682 x3 + 48908 x4
+ 61139 x5)/73365]
55
Copyright © ILOG 2008
Identifying Weak Formulations (example)
c1:12228 x1 + 36679 x2 + 36682 x3 +
48908 x4 + 61139 x5 + 73365 x6 = 89716837
MIP:
12228
12228
36679
36679
36682
36682
48908
61139
73365
89716837
48908
61139
73365
56
Copyright © ILOG 2008
Identifying Weak Formulations (example)
c1:12228 x1 + 36679 x2 + 36682 x3 +
48908 x4 + 61139 x5 + 73365 x6 = 89716837
LP:
12228
36679
36682
48908
61139
12228
73365
36679
36682
48908
89716837
61139
73365
57
Copyright © ILOG 2008
Tuning CPLEX parameters
 Try the automatic Tuning Tool (starting with CPLEX 11)
 Use node log to identify whether lack of progress is in
best integer, best node or both.
 Too many parameters for arbitrary experimentation
 Set parameters based on lack of progress
 MIP Emphasis parameter sets multiple other parameters at once
 When neither best integer and best node make progress, take
advantage of MIP heuristics while emphasizing best node
 Solution Repair
 RINS Heuristic
 Solution Polishing
 Attend ILOG Optimization Workshops for more
information
58
Copyright © ILOG 2008
Guidelines for Tightening Difficult MIPs
 Identify the parts of the model that make it
difficult
 Simplify the model if necessary
 Remove any constraints and integrality restrictions not
involved in the performance trouble
 Try to reproduce the trouble in a smaller data instance
 Determine how fractional solutions in the node
relaxations allow the objective to improve
 Use fractional solutions to identify the constraints and
variables that will motivate additional cuts
59
Copyright © ILOG 2008
Guidelines for Tightening Difficult MIPs
 Common tactics for deriving additional cuts
 Linear or logical combinations of constraints
 Associate a graph with the model structure
 Disjunctions
 Solve one or more related models
 Use infeasibility
 CPLEX Conflict Refiner
 Use solution objective value
 Feasibility on model with constraint on objective value
added
 Especially true for models with soft constraints
60
Copyright © ILOG 2008
Guidelines: Linear, Logical Combinations
 Linear combinations of constraints
P  {x  Z 3 : x1  x2  1,
x1  x3  1,
x2  x3  1, 0  xi  1}
Sum theconstraints, divide by 2 : x1  x2  x3  3 / 2
Use integrality toround down : x1  x2  x3  1
Cut off fractionalsolutionslike (.5,.5,.5)
61
Copyright © ILOG 2008
Guidelines: Linear, Logical Combinations
 But how do we find the constraints to combine in
a practical model?
 10000constraints 
x1  x 2  1
 20000constraints 
x1  x3  1
 4000constraints 
x 2  x3  1
 8000constraints 
62
Copyright © ILOG 2008
Guidelines: Linear, Logical Combinations
 Conflict graph
 Associate a node with each binary variable
 Create an edge between the nodes of any pair of binary variables
that cannot both be 1
 At most one binary in a clique can be 1
 Conflict graph identifies a useful linear combination of constraints
 CPLEX creates the conflict graph and by default finds clique cuts
P  {x  Z 3 : x1  x2  1,
x1  x3  1,
x2  x3  1, 0  xi  1}
x1
x2
x3
Clique Cut : x1  x2  x3  1
63
Copyright © ILOG 2008
Guidelines: Disjunctions
Gomory cut:
P  {x  Z 3 : 4 x1  3x 2  5 x3  10}
P  {x  Z : x1  3 4 x 2  5 4 x3  5 2}
3

P  {x  Z 3 : x1  x 2  x3  1 4 x 2  1 4 x3  2  1 2}
 3  12
t
t  2   1 4 x 2  1 4 x3  1 2  x3  2
t  3   1 4 x 2  1 4 x3   1 2  x 2  2
Combining,x 2  x3  2; cuts off (2, 13 , 15 )
Copyright © ILOG 2008
64
Guidelines: Disjunctions
Given aj , d , xj  0, S 1, S 2
Suppose S 1  S 2  
Consider the disjunctive constraint
 a x  d or
j j
jS 1
a x  d
j j
jS 2
T hen
a x  d
j j
jS 1 S 2
65
Copyright © ILOG 2008
Guidelines: Disjunctions
Given xj  0, j ,  j , d 1, d 2 , S
Consider the disjunct ive const raint
 x  d
j j
1
or
jS
 x  d
j j
2
jS
T hen
 max(
j,
 j )xj  min(d d )
1,
2
jS
66
Copyright © ILOG 2008
Guidelines: Disjunctions
 More general disjunctions involve solving LP
subproblem to generate cuts
 CPLEX does disjunctions on x <= k and x >= k+1 for
fractional integer variables at the root node LP
 Can also do disjunctions on arbitrary constraints
 Write a program to solve the LP, calculate the cut
 See “Integer Programming” by Wolsey for the details
67
Copyright © ILOG 2008
Guidelines: When to derive cuts
 CPLEX generates generic cuts, including
 Clique cuts
 Disjunctive cuts based on individual fractional variables
 Gomory cuts
 See manual for complete list
 You don’t need to generate these yourself
 CPLEX is unlikely to generate cuts more specific to
particular models
 Concepts used for Clique, Disjunctive and Gomory cuts can often
be customized to particular models with more special structure
68
Copyright © ILOG 2008
Example 1: A small, pathological model
Minimize
COST:
Subject To
c1: 12228 x1 + 36679 x2 + 36682 x3 + 48908 x4
+ 61139 x5 + 73365 x6 = 89716837
Bounds
x1 >= 0
…
x6 >= 0
Generals
x1
x2
x3
x4
x5
x6
End
69
Copyright © ILOG 2008
Example 1: A small, pathological model
CPLEX node log with defaults:
2098000000
102
0.0000
1
0.0000 32220321
2099000000
103
0.0000
1
0.0000 32235726
MIP - Node limit exceeded, no integer solution.
Current MIP best bound =
infinite)
0.0000000000e+00 (gap is
Solution time = 62804.13 sec. Iterations =
32251097 Nodes = 2100000000 (105)
70
Copyright © ILOG 2008
Example 1: A small, pathological model
 Review the guidelines and tactics
 Simplify the model if necessary
 Determine how fractional solutions affect
objective
 Use fractional solutions to motivate additional
cuts
 Linear or logical combinations of constraints
 Disjunctions
 Solve one or more related models
 Use infeasibility
 Use solution objective value
71
Copyright © ILOG 2008
Example 1: A small, pathological model
Determine how fractional solutions affect the
objective:
CPLEX> display solution variable -
Variable Name
x1
Solution Value
7337.000082
(x1 coefficient almost an
integral multiple of the right
hand side)
CPLEX> change bound x1 u 7337
New bounds on variable 'x1':
0 <= x1 <= 7337
...
CPLEX> display solution variable Variable Name
Solution Value
x1
7337.000000
x4
0.000020
72
Copyright © ILOG 2008
Example 1: A small, pathological model
Determine how fractional solutions affect
objective(ctd):
CPLEX> change bound x4 l 1
New bounds on variable 'x4':
x4 >= 1
…
Variable Name
Solution Value
x1
7333.000409
x4
1.000000
(x4
coefficient
almost an integral
multiple of x1
coefficient)
73
Copyright © ILOG 2008
Example 1: A small, pathological model
Determine how fractional solutions affect
objective(ctd):
Rewrite
c1:12228 x1 + 36679 x2 + 36682 x3 +
48908 x4 + 61139 x5 + 73365 x6 = 89716837
as
c1:12228 x1 + (3*12228 - 5) x2 +
(3*12228 - 2) x3 + (4*12228 - 3)x4 +
(5*12228 - 1) x5 + (6*12228 - 3) x6 =
7337*12228 + 1
74
Copyright © ILOG 2008
Example 1: A small, pathological model
Use a disjunction similar to Gomory cuts:
c1: 12228 x1 + (3*12228 - 5) x2 + (3*12228 - 2) x3
+ (4*12228 - 3) x4 +(5*12228 - 1) x5
+ (6*12228 - 3) x6 = 7337*12228 + 1
Separate by integer multiples of 12228:
t
c1: 12228 (x1 + 3 x2 + 3 x3 + 4 x4 + 5 x5 + 6 x6)
- 5 x2 - 2 x3 - 3 x4 - x5 - 3 x6 = 7337*12228 + 1
= 7338*12228 - 12227
75
Copyright © ILOG 2008
Example 1: A small, pathological model
Use a disjunction similar to Gomory cuts (ctd):
t
c1: 12228 (x1 + 3 x2 + 3 x3 + 4 x4 + 5 x5 + 6 x6)
- 5 x2 - 2 x3 - 3 x4 - x5 - 3 x6 = 7337*12228 + 1
= 7338*12228 – 12227
t <= 7337
- 5 x2 - 2 x3 - 3 x4 - x5 - 3 x6 >= 1
t >= 7338
- 5 x2 - 2 x3 - 3 x4 - x5 - 3 x6 <= -12227
x1 + 3 x2 + 3 x3 + 4 x4 + 5 x5 + 6 x6 >= 7338
5 x2 + 2 x3 + 3 x4 +
x5 + 3 x6 >= 12227
Cuts
76
Copyright © ILOG 2008
Example 1: A small, pathological model
CPLEX node log with heuristics disabled, cuts added:
Nodes
Node
Cuts/
Left Obj
0
0
*3
1
0.0000
IInf
Best Int Best Node ItCnt Gap
1
0
0.0000
0.0000
1
0.0000
4
0.00%
MIP …- Integer optimal solution:
Objective =
0.0000000000e+00
Solution time = 0.00 sec.
Iterations = 4
Nodes = 3
77
Copyright © ILOG 2008
Example 1: A small, pathological model
CPLEX node log with defaults, cuts added:
Nodes
Node
Cuts/
Left Obj
0
0
*0+
0
0.0000
IInf
Best Int Best Node ItCnt Gap
1
0
0.0000
0.0000
1
0.0000
1
0.00%
MIP - Integer optimal solution:
Objective =
0.0000000000e+00
Solution time = 0.00 sec.
Iterations = 1
Nodes = 0
78
Copyright © ILOG 2008
Example 1: A small, pathological model
 Summary
 Determine how fractional solutions affect
objective
 Use fractional solutions to motivate additional
cuts
 Disjunctions
 Separated coefficients based on multiples of 12228 instead
of multiples of 1 as in standard Gomory cut
 Use infeasibility
 Split Cut
79
Copyright © ILOG 2008
Example 2: Penalty variables
Statistics: 559 constraints, 1066 variables (516 binary, 516 general integer)
Node log:
Nodes
Cuts/
Node Left Objective IInf Best Int
Best Node ItCnt Gap
0
0 101984.7744 28
101984.7744 35
*0+
0
0 4.10026e+08 101984.7744 35 99.98%
153036.9306 35 4.10026e+08 Cuts: 41
151 99.96%
*0+
0
0 4.00022e+08 153036.9306 151 99.96%
…
*55950+
0 1.02822e+07 202475.0432
98.03%
56000
infeasible
1.02822e+07 202518.1842
98.03%
Elapsed time = 186.20 sec. (tree size = 13.36 MB).
80
Copyright © ILOG 2008
Example 2: Penalty variables
Node log (ctd):
Nodes
Node
Left
Cuts/
Objective IInf
7149e4 7726073 infeas
7150e4 7727024 309418.1
Best Int
Best Node
Gap
1.02822e+07 307724.1416 97.01%
33
1.02822e+07 307728.0479 97.01%
Elapsed time = 161631.76 sec. (tree size = 9072.93 MB).
Nodefile size = 8945.58 MB (3124.04 MB after compression)
…
7151e4 7727720 357983.4
22
1.02822e+07 307731.4823 97.01%
81
Copyright © ILOG 2008
Example 2: Penalty Variables
 Review the guidelines and tactics
 Simplify the model if necessary
 Determine how fractional solutions affect the
objective
 Use fractional solutions to motivate additional cuts
 Linear or logical combinations of constraints
 Disjunctions
 Solve one or more related models
 Use infeasibility
 Use solution objective value
82
Copyright © ILOG 2008
Example 2: Penalty Variables
Determine how fractional solutions affect the objective:
Min obj: 10000000 id134 + 10000000 id135 + ...
+ 10000000 id161 +
10000 id168 + 10000 id169
+ 1000 id170 + 1000 id171 + 34.299999237 id200
+ … + 10000 id2309
(Implied integer by integrality of other variables in the constraint)
id78:
id134 - id135 + 3 id200 + 3 id204 + 3 id220
+ 3 id228 + 3 id248 + ... + 3 id2096 + 3 id2144
+ 2 id2148
= 4
83
Example 2: Penalty variables
Determine how fractional solutions affect the objective(ctd):
Node
Nodes
Left
Objective IInf
Best Int
Cuts/
Best Node
Itcnt Gap
…
*55950+ 11356
0 1.02822e+07 202475.0432 861218 98.03%
56000 11367 infeasible
1.02822e+07 202518.1842 862287 98.03%
Elapsed time = 186.20 sec.
Comparing the best integer and best node values, we see that
removing integrality enables solutions with the sum of the
expensive penalty variables << 1. But, we don't know yet whether
an integer solution exists with all such penalty variables set to 0.
Can we answer that question?
84
Copyright © ILOG 2008
Example 2: Penalty variables
Yes we can, by solving a related problem:
Add a constraint that sets all the expensive penalty
variables to 0:
conobj: id134 + id135 + id136 + id137 + … + id161 = 0
Results:
MIP - Integer infeasible or unbounded.
Current MIP best bound is infinite.
Solution time =
18.80 sec. Iterations = 409663
Nodes = 38384
85
Copyright © ILOG 2008
Example 2: Penalty variables
Solve another related problem, using solution objective value:
Nodes
Left
Cuts/
Best Node
Node
Objective IInf Best Int
Itcnt
Gap
…
*55950+ 11356
0
1.02822e+7 202475.0432 861218 98.03%
56000 11367 infeasible
1.02822e+7 202518.1842 862287 98.03%
Elapsed time = 186.20 sec.
conobj:
id134 + id135 + id136 + id137 + id138 + … + id161 >= 1
conobj:
id134 + id135 + id136 + id137 + id138 + … + id161
= 1
86
Example 2: Penalty variables
Solve another related problem, using solution objective value:
Node
Nodes
Left
0
0
*205
58200
58300
…
160
…
319
226
Objective IInf
1.01576e+07 16
1.01851e+07 20
infeasible
cutoff
Best Int
Cuts/
Best Node
Itcnt
1.01576e+07
Cuts: 42
54
82
0 1.02922e+07 1.02098e+07
1233
Gap
0.80%
1.02822e+07 1.02806e+07 440316 0.02%
1.02822e+07 1.02811e+07 440441 0.01%
MIP - Integer optimal, tolerance (0.0001/1e-06):
Objective =1.0282191250e+07
Current MIP best bound = 1.0281164221e+07 (gap = 1027.03, 0.01%)
Solution time =
38.51 sec. Iterations = 440476 Nodes = 58326
(201)
87
Example 2: Penalty Variables
 Summary
 Determine how fractional solutions affect the
objective
 Involved both integer feasible solutions and fractional
solutions to relaxations
 Solve one or more related models
 Use infeasibility
 Use solution objective value
 More generally, don’t try to combine multiple
objectives into a single objective
 Solve separate problems with each individual objective
88
Copyright © ILOG 2008
Example 3: A nonconvex MIQP
Model description:
≥0
n
Maxim ize
n
 d
ij
j 1
xi  xj
i 1
i j
n
Subjectto
x  k
j
j 1
xj binary
Copyright © ILOG 2008
89
Example 3: A nonconvex MIQP
Node Log with n = 28, k = 10:
Node
Nodes
Left Objective IInf
Best Integer
Cuts/
Best Node
ItCnt Gap
0
0
13.0405
28
13.0405
29
*0+
0
0
5.0000
13.0405
29 160.81%
*0+
0
0
5.6250
13.0405
29 131.83%
100
82
11.7307
25
5.6250
11.7307
129 108.55%
…
2222400 329
5.6256
5
5.6250
5.6256 2371768
0.01%
MIP - Integer optimal, tolerance (0.0001/1e-06):
Objective = 5.6250000000e+00
Solution time = 408.94 sec.
90
Example 3: A nonconvex MIQP
 Review the guidelines and tactics
 Simplify the model if necessary
 Determine how fractional solutions affect
objective
 Use fractional solutions to motivate additional cuts
 Linear or logical combinations of constraints
 Disjunctions
 Solve one or more related models
 Use infeasibility
 Use solution objective value
91
Copyright © ILOG 2008
Example 3: A nonconvex MIQP
Determine how fractional solutions affect objective
Simplify the model if necessary
max 3x1*x2 + 4x1*x3 + 5x2*x3
subject to
x1 + x2 + x3 <= 2
x1, x2, x3 binary
Extreme point
MIQP optimal solution: x2 = x3 = 1; obj = 5
QP optimal solution: x1 = x2 = x3 = 2/3; obj = 16/3
Non-vertex, fractional
We cannot tighten this formulation with linear
constraints since the integer solutions are extreme
points of the relaxation polyhedron
92
Copyright © ILOG 2008
Example 3: A nonconvex MIQP
Solve one or more related models
Let x1*x2 = z12
z12
z12
z12
z12
<= x1
<= x2
+ 1 >= x1 + x2
binary
This linearization transforms the nonconvex
MIQP into an MIP
93
Copyright © ILOG 2008
Example 3: A nonconvex MIQP
Node log with n=28, k=10 for the linearized problem:
Nodes
Node Left
0
0
*0+
0
*0+
0
…
*85
74
…
137900 23
Objective IInf
12.8720
cutoff
Cuts/
Best Integer
Best Node ItCnt Gap
402
0
0
-0.0000
0.1250
12.8720
12.8720
12.8720
617
617
617
0
5.0417
11.8962
2345 135.96%
5.6250
5.6333
2707056 0.15%
-----
MIP - Integer optimal solution: Objective = 5.6250000000e+00
Solution time = 251.66 sec. Iterations = 2707079 Nodes = 137923
Copyright © ILOG 2008
94
Example 3: A nonconvex MIQP
Determine how fractional solutions affect objective
max 3z12 + 4z13 + 5z23
subject to
x1 + x2 + x3 <= 2
z12 <= x1
z12 <= x2
z12 + 1 >= x1 + x2
z13 <= x1
z13 <= x3
z13 + 1 >= x1 + x3
z23 <= x2
zij = 1
xi = xj = 1 (true for MIP, LP)
z23 <= x3
zij < 1
xi = 0 or xj = 0 (true for MIP)
z23 + 1 >= x2 + x3
All variables binary
MIP optimal: z23 = x2 = x3 = 1, obj = 5.
LP optimal: all variables = 2/3, obj = 8
95
Example 3: A nonconvex MIQP
Use Infeasibility
max 3z12 + 4z13 + 5z23
subject to
x1 + x2 + x3 <= 2
z12 <= x1
z12 <= x2
z12 + 1 >= x1 + x2
z13 <= x1
z13 <= x3
z13 + 1 >= x1 + x3
z23 <= x2
zij = 1
xi = xj = 1
z23 <= x3
z23 + 1 >= x2 + x3
All variables binary
z23 = z12 = 1  x1 + x2 + x3 = 3
 z12 + z13 + z23 <= 1 (cuts off z12 = z13 = z23 = 2/3 in LP)
96
Example 3: A nonconvex MIQP
Extend result to general model with arbitrary n, k:
zij = 1
xi = xj = 1
x1 …
x1
=1
zji
=0
xk x(k+1) …
z1k
xn
zij
Z=
xk
zk1
x(k+1)
Count the number of
blue elements below
the diagonal: k*(k-1)/2
xn
97
Example 3: A nonconvex MIQP
Tightened linearized model:
n
n
 d
Minim ize
j 1
zij
i 1
i j
n
Subjectto
ij
x
j
k
j 1
zij  xj  0
zij  xi  0
xi  xj  1  zij
n
n
 z
j 1
ij
 k * ( k  1) / 2
i 1
i j
Cut
xj , zij binary
Copyright © ILOG 2008
98
Example 3: A nonconvex MIQP
Node log with n=28, k=10 for the linearized problem with cut:
Nodes
Cuts/
Node Left Objective IInf Best Integer
Best Node ItCnt
Gap
0
0
6.4792 110
6.4792
312
*0+
0
0
-0.0000
6.4792
312
--…
*50+
19
0
5.5000
6.0833
2277 10.61%
…
300
20
cutoff
5.6250
5.6919
9882
1.19%
…
MIP - Integer optimal solution: Objective = 5.6250000000e+00
Solution time =
2.89 sec. Iterations = 10154 Nodes = 330
Copyright © ILOG 2008
99
Example 3: A nonconvex MIQP
 Summary
 Simplify the model if necessary
 Determine how fractional solutions affect
objective
 Use fractional solutions to motivate additional
cuts
 Linear or logical combinations of constraints
 Solve one or more related models
 Use infeasibility
 QPs need not have an optimal vertex solution
 MIQPs harder to tighten than MIPs
100
Copyright © ILOG 2008
Methods for Adding Cuts to a Model
 Explicit Cuts
 More MIP preprocessing to tighten the model
 Larger problem may increase memory, run time
 User Cuts at start of optimization
 Added to cut pool rather than problem
 Less memory, node relaxation solve time
 MIP preprocessing not applied to user cuts
 Cut Callback
 Use when need to add cuts at child nodes
 Child node information needed to derive cuts
 Too many cuts to add at start of optimization
 More work to implement
101
Copyright © ILOG 2008
Debugging/Sanity checking
 Sufficient (but not necessary) conditions that the cut(s)
tightened the MIP too much
 Adding the cuts makes a feasible MIP infeasible
 Use CPLEX’s conflict refiner to obtain minimal set
of infeasible constraints
Original MIP
Tightened MIP
Upper Bound = 100
G
A
P
cuts
Infeasible
Lower Bound = 50
102
Debugging/Sanity checking
 Sufficient (but not necessary) conditions …
 Best node of tightened MIP inconsistent with
objective of a feasible solution to original MIP
 Fix variables at values of the feasible solution to
original MIP in the tightened MIP. Run conflict
refiner on resulting infeasible model
Original MIP
Tightened MIP
Upper Bound = 100
Upper Bound = 150
G
A
P
Lower Bound = 50
cuts
G
A
P
Lower Bound = 120
103
Debugging/Sanity checking
 Sufficient (but not necessary) conditions …
 CPLEX can improve on the “optimal” solution from
the tightened MIP when it is given to the original MIP
as an advanced start
 Solution Polishing
Original MIP
Upper Bound = 100
G
A
P
cuts
Lower Bound = 50
Tightened MIP
Original MIP
Upper Bound = 70
Upper Bound = 65
G
A
P
solution
Lower Bound = 70
G
A
P
Lower Bound = 50
104
Debugging/Sanity checking
 Sufficient (but not necessary) conditions that the cut(s)
didn’t tighten the MIP enough
 Fractional solutions that we used to identify the cuts are not cut
off in the node LPs of the tightened MIP
105
Copyright © ILOG 2008
Conclusions
 Most MIPs easy to solve (defaults or parameter tuning)
 The remaining MIPs can be very time consuming for the
practitioner
 I hope the presentation didn’t just illustrate how to
improve performance on 3 particular MIPs
 Use the guidelines to help quickly identify the good cuts
on your own difficult MIPs
 Sometimes a sharp cut found in an hour’s work is better
than the sharpest cut found in a week’s (or more) work
106
Copyright © ILOG 2008
Additional Reading
 Wolsey, L. Integer Programming.
 Polya, How to Solve It.
 Bixby, Fenelon, Gu, Rothberg, Wunderling. Mixed
Integer Programming: A Progress Report.
Available in Grotschel, M.(editor), The Sharpest
Cut.
107
Copyright © ILOG 2008
Infeasibility Analysis with CPLEX
Objective
 Increase knowledge regarding use of CPLEX’s
tools for infeasibility analysis
 Increase awareness regarding use of these
tools beyond analysis of infeasible models
 Performance improvements
 Debugging
 MIP Sensitivity Analysis
109
Outline
 Objective
 Background and Fundamentals
 CPLEX Tools for Infeasibility Analysis
 Interpreting the Results
 Correcting Conflicting Constraints
 Use Beyond Infeasibility
 Computational Aspects
 Conclusions
110
Background and Fundamentals
 A mathematical programming model is infeasible
when no solution can satisfy all the constraints
(including bounds and integrality)
 Example 1:
≤ 90
x1 + x2 + x3 >= 100
<other constraints>
Bounds
0 <= x1 <= 25
0 <= x2 <= 30
0 <= x3 <= 35
111
Background and Fundamentals
Example 2:
Minimize
obj: cost
Subject To
c1: - cost + 80 x1 + 60 x2 + 55 x3 + 30 x4 + 25 x5 +
80 x6 + 60 x7 + 35 x8 + 80 x9 + 55 x10 = 0
c2: x1 + x2 + 0.8 x3 + 0.6 x4 + 0.4 x5 >= 2.1
c3: x6 + 0.9 x7 + 0.5 x8 >= 1.2
c4: x9 + 0.9 x10 >= 0.8
c5: 0.2 x2 + x3 + 0.5 x4 + 0.5 x5 + 0.2 x7 + 0.5 x8 +
x10 - service = 0
c6: x1 + x6 + x9 >= 1
c7: x1 + x2 + x3 + x6 + x7 + x9 >= 2
c8: x2 + x3 + x4 + x5 <= 0
c9: x4 + x5 + x8 <= 1
c10: x1 + x10 <= 1
<other constraints>
Bounds
service >= 3.2
Binaries
x1 x2 x3 x4 x5 x6 x7 x8 x9 x10
112
Background and Fundamentals
Example 3:
c1:12223x + 12224y + 36672z = 149389505
<other constraints>
Bounds
x >= 0
y >= 0
z >= 0
Generals
x y z
113
Background and Fundamentals
 Certificates of infeasibility
 LP (Farkas Lemma):
Ax = b, x ≥ 0
has a solution, or
yTA ≤ 0, yTb > 0
has a solution, but not both
y, if it exists, is a certificate of infeasibility
for Ax = b, x ≥ 0; it provides a linear combination
of the rows of A that prove infeasibility:
≤0
yTAx = yTb > 0
114
Background and Fundamentals
 Certificates of infeasibility
 MIP (Lattices, Hermite Normal Form):
Ax = b, x integral
has a solution, xor
yTA integral, yTb non integral
has a solution,
y, if it exists, is a certificate of infeasibility
for Ax = b, x integer; it provides a linear combination
of the rows of A that give an integral left hand side
and a non integral right hand side:
integral
non-integral
yTAx = yTb
115
Background and Fundamentals
 Example 1:
-x1 - x2 - x3 <= -100
Bounds
0 <= x1 <= 25
0 <= x2 <= 30
0 <= x3 <= 35
Certificate of infeasibility:
y = (1,1,1,1):
0 <= -10
116
Infeasibility Analysis
 The model you want to solve may be infeasible
 The model (or associated data) is incorrect and needs to
be fixed
 The model and data are correct, but the associated data
represents a real-world conflict in the system being
modeled
 You create an infeasible model to test properties of (or
extract information about) a similar model
 You want an explanation of the infeasibilities
 You want a solution that satisfies a relaxed version
of the problem
117
Infeasibility Analysis
 ILOG CPLEX offers two methods for
analyzing and resolving infeasibilities:
 Conflict Refiner
 Use when you want an explanation of the
infeasibilities
 Feasopt
 Use to minimize the infeasibilities
- Find a minimal relaxation
118
Infeasibility Analysis
Two
functionalities / APIs
 Conflict
Refiner
 Find a set of conflicting constraints
 Conflict: Infeasible subset of constraints
 Minimal Conflict: Removal of any single constraint
from the conflict resolves the infeasibility
 Allows constraint grouping and prioritization
 Accessible from interactive CPLEX, all
CPLEX APIs, ILOG OPL, and some of the 3rd
party modeling languages
119
Finding Conflicts
Conflict
Refiner - Inputs
 Conflict
members
 Individual constraints
 Variable bounds
 Integrality restrictions
 Groups of constraints and/or bounds
 Priorities
 Exclude certain constraints
 Put certain constraints in background:
include in conflict, but don’t test them
 Steer Conflict Refiner to certain members
120
Finding Conflicts
Conflict
Refiner - Solutions
 Member
statuses
 Proved to be part of conflict
 Proved not to be in conflict
 Unknown / background
121
Infeasibility Analysis
Two functionalities / APIs
 Feasopt
 Find minimal relaxation of constraints for model to
become feasible
 Choose constraints that can be relaxed
 Choose relaxation measure
 Grouping
 Accessible from interactive CPLEX, all CPLEX APIs,
ILOG OPL, and some third party modeling languages
122
Infeasibility Analysis
(relax the constraints, bounds)
public IloBool feasOpt(const IloRangeArray rngs,
rnglb,
IloNumArray
IloNumArray rngub, const
IloNumVarArray vars,
IloNumArray
Min
cx varub)
b1  Ax  b 2
l  xu
IloNumArray varlb,
Min v1s1  v2 s 2  w  wuu
b1  s1  Ax  b 2  s 2
l    x  u  u
s1, s 2, , u  0
Relaxed constraints in rngs; Relaxed vars in vars
v and w specify priorities (often v = w = 1)
Parameter IloCplex::FeasOptMode = 0
123
Infeasibility Analysis
Optional second
of Feasopt:
Feasiblephase
Optimization
Let zˆ be t heopt imalobject ivefor t hepreviousLP
Min cx
Constrain to solutions with
v1s1  v2 s 2  w  wuu  zˆ best relaxation objective
b1  s1  Ax  b 2  s 2
l    x  u  u
s1, s 2, , u  0
In other words, among all optimal solutions to the problem
on the previous slide, minimize the objective of the
original problem
(IloCplex::FeasOptMode=1)
124
Infeasibility Analysis
 Settings for Feasopt Mode parameter
 0: minimizes the sum of the slacks in the relaxation
 1: find optimal minimum sum relaxation relative to
original objective
 2: minimizes the number of nonzero slacks (MIP)
 3: find optimal minimum number relaxation relative
to original objective
 4: minimizes the sum of squares of the slacks (QP)
 5: find optimal minimum sum of squares relaxation
relative to original objective
125
Interpreting Feasopt output
Nodes
Node Left Objective IInf Best Integer
Cuts/
Best Node
*
*
0.0000
0.0000
Fract: 3
0.0000
0.1600
0.3214
0.3662
0.5775
0+
0+
0
0
100
200
300
400
0
0
0
2
30
60
84
39
0.0000 16
0.0000 16
0.6200 1
0.5070 5
cutoff
0.8169 3
9.0000
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
ItCnt
Gap
41 100.00%
41 100.00%
50 100.00%
50 100.00%
515 84.00%
857 67.86%
1135 63.38%
1525 42.25%
Feasopt - Feasible relaxed sum of infeasibilities: Relaxation =
1.0000000000e+00
Solution time = 0.34 sec.
126
Interpreting the Results (Conflict Refiner)
Example 2:
Minimize
obj: cost
Subject To
c1: - cost + 80 x1 + 60 x2 + 55 x3 + 30 x4 + 25 x5 +
80 x6 + 60 x7 + 35 x8 + 80 x9 + 55 x10 = 0
c2: x1 + x2 + 0.8 x3 + 0.6 x4 + 0.4 x5 >= 2.1
c3: x6 + 0.9 x7 + 0.5 x8 >= 1.2
c4: x9 + 0.9 x10 >= 0.8
c5: 0.2 x2 + x3 + 0.5 x4 + 0.5 x5 + 0.2 x7 + 0.5 x8 +
x10 - service = 0
c6: x1 + x6 + x9 >= 1
c7: x1 + x2 + x3 + x6 + x7 + x9 >= 2
c8: x2 + x3 + x4 + x5 <= 0
c9: x4 + x5 + x8 <= 1
c10: x1 + x10 <= 1
<other constraints>
Bounds
service >= 3.2
Binaries
x1 x2 x3 x4 x5 x6 x7 x8 x9 x10
127
Interpreting the Results (Conflict Refiner)
 What about larger models?

Depends on size of conflict

Tactics for modest sized conflicts
 Look at the easiest constraints
 Find other constraints intersecting the variables in the easiest
constraints
 Combine
 Repeat the previous 3 steps as needed (using combined
constraints when appropriate)

Tactics for larger (or more complicated) conflicts
 Look for constraint groupings
128
Interpreting the Results (Conflict Refiner)
Problem name
: gosh.mps
Variables
: 10733
Objective nonzeros :
26
Linear constraints
:
3792
Nonzeros
: 97231
RHS nonzeros
:
202
129
Interpreting the Results (Conflict Refiner)
.clp file output:
c812: - x74 + x192 - x7193 <= 0
c1589: - x75 + x193 - x9256 <= 0
Shares variables with
easiest constraints
c2527: 0.45 x7191 - 0.46 x7193 >= 0
c2602: 0.45 x9254 - 0.46 x9256 >= 0
c2964: x7191 - x7193 = 0
c3094: x9254 - x9256 = 0
c3178: x74 <= 0.001
c3179: x75 <= 0.001
Easiest constraints
c3300: x192 + x193 = 4
130
Interpreting the Results (Conflict Refiner)
.clp file output (ctd):
\Sum of equality rows in the conflict:
\ sum_eq: x192 + x193 + x7191 - x7193 + x9254 x9256 = 4
Bounds
x74 Free
…
x9256 Free
End
131
Interpreting the Results (Conflict Refiner)
 Look at the easiest constraints, combine with other
constraints sharing the same variables:
c3178: x74 <= 0.001
c3179: x75 <= 0.001
c812: - x74 + x192 - x7193 <= 0
c1589: - x75 + x193 - x9256 <= 0
Combining c3178 with c812 and c3179 with c1589:
x192 - x7193 <= x74 <= 0.001
x193 - x9256 <= x75 <= 0.001
132
Interpreting the Results (Conflict Refiner)
 Repeat as needed:
x192 - x7193 <= 0.001
// from previous
x193 - x9256 <= 0.001
// slide
c3300: x192 + x193 = 4
// easiest intersecting
constraint left
Adding the first two, substituting the third:
x7193 + x9256 >= 3.998
133
Interpreting the Results (Conflict Refiner)
 Repeat as needed (second time):
x7193 + x9256 >= 3.998
// from previous slide
c2964: x7191 - x7193 = 0
// easiest intersecting
c3094: x9254 - x9256 = 0
// constraints left
Adding c2964, c3094, substituting:
x7191 + x9254 = x7193 + x9256 >= 3.998
134
Interpreting the Results (Conflict Refiner)
 Repeat as needed (final time):
x7191 + x9254 = x7193 + x9256 >= 3.998
c2527: 0.45 x7191 - 0.46 x7193 >= 0 // only ones
c2602: 0.45 x9254 - 0.46 x9256 >= 0 // left
Combine c2527 and c2602, replace x7193 + x9256:
0.45(x7191 + x9254 ) - 0.46( x7193 + x9256 ) >= 0
>= 3.998
x7191 + x9254
-.01(x7191 + x9254) >= 0;
<= -.3.998e-02
135
Interpreting the Results (Conflict Refiner)
 Compare final constraint in conflict with Feasopt output:
Iteration: 1
Objective
=
...
Iteration: 27952 Objective
=
Iteration: 28208 Objective
=
Removing perturbation.
Iteration: 28458 Scaled infeas =
Iteration: 28483 Objective
=
1212.649235
0.224527
0.037972
0.000116
0.040583
Feasopt - Feasible relaxed sum of infeasibilities: Relaxation =
3.9980000000e-02
-.001(x7191 + x9254) >= 0;
<=- 3.998e-02
136
Interpreting the Results (Larger Conflicts)
 How can we interpret larger conflicts?
 Look for constraint groupings
Problem name
: mondou2.mps
Variables
:
604
Objective nonzeros :
415
Linear constraints
:
312
Nonzeros
:
1208
RHS nonzeros
:
195
137
Interpreting the Results (Larger Conflicts)
c125: - x51 - x52 - x346 + x378 + x379 - x380 = -1825
c126: - x379 + x380 + x381 - x382 = -2535
…
c293: - x491 + x492 + x494 = 0
c294: x493 - x494 = 0
(conflict contains 45 constraints, all equalities, 4
lower bounds of 0, 11 finite upper bounds)
138
Interpreting the Results (Larger Conflicts)
\Sum of equality rows in the conflict:
\ sum_eq: - x15 - x30 - x50 - x64 - x65 - x69 - x76 - x78 - x79 + x90
+ x92 - x346 + x378 - x475 + x477 = -60342
Bounds
-Inf <= x15 <= 0
x90 >= 0
…
-Inf <= x30 <= 12509
-Inf <= x50 <= 12509
-Inf <= x69 <= 14434
-Inf <= x475 <= 14434
-(x30 + x50 + x69 + x475) ≥ -53886
Sum of all other variables ≥ 0
Left hand side >= -53886, r.h.s = -60342;
139
Interpreting the Results (Larger Conflicts)
 What if sum of equalities doesn’t help?
 Other constraint groupings
 Group individual constraints together by type
or purpose
 Decompose into manageable parts
 Focus on group(s) of constraints that force some activity to
occur
 Then examine their interaction with other groups of
constraints
 If necessary, solve auxiliary models involving subsets of the
constraints
140
Interpreting the Results (Larger Conflicts)
 Example: misc03inf.mps
 Constraint groups:
(all variables binary except COL160)
ROW001: - 205 COL004 - 205 COL005 - 205 COL006 - 550 COL007
- 550 COL008 - ... - 75 COL159 + COL160 = 0
c97: COL160 <= 2000
(Knapsack constraint)
ROW041: COL004 + COL007 + COL010 + COL013 + COL016
+ COL019 - COL022 - COL043 - COL064 - COL085
- COL106 - COL127 = 0
...
ROW058: - COL012 - COL033 - COL054 + COL066 + COL069
+ COL072 + COL078 + COL081 + COL084 - COL096
- COL117 - COL138 = 0 (Balance constraints)
141
Interpreting the Results (Larger Conflicts)
 Constraint groups (ctd):
ROW007: 100 COL001 + 100 COL004 + 100 COL007 + ... +
100 COL100 + 100 COL103 <= 200
ROW008: 100 COL002 + 100 COL005 + 100 COL008 + ... +
100 COL101 + 100 COL104 <= 200
ROW009: 100 COL003 + 100 COL006 + 100 COL009 + ... +
100 COL102 + 100 COL105 <= 200
(Generalized upper bound (GUB) constraints)
142
Interpreting the Results (Larger Conflicts)
 Constraint groups (ctd):
ROW002: COL001 + COL002 + COL003 + ... + COL021 = 1
ROW003: COL022 + COL023 + COL024 + ... + COL042 = 1
ROW032: COL010 + COL011 + COL012 + COL031 + COL032 +
COL033 + COL052 + COL053 + COL054 + COL094 +
COL095 + COL096 + COL115 + COL116 + COL117 +
COL136 + COL137 + COL138 >= 1
...
ROW039: COL022 + COL023 + COL024 + COL043 + COL044 +
COL045 + COL064 + COL065 + COL066 + COL085 +
COL086 + COL087 + COL106 + COL107 + COL108 +
COL127 + COL128 + COL129 >= 1
(Set partitioning/Set covering constraints)
143
Interpreting the Results (Larger Conflicts)
 Constraint groups
 Single knapsack constraint
 12 balance constraints
 3 GUB constraints
 9 Set partition/covering constraints
144
Interpreting the Results (Larger Conflicts)
 Decompose into manageable parts
 Focus on groups that force some activity to occur
 Partition/Covering constraints force activity
 Other groups permit setting all their variables to 0
 If necessary, solve auxiliary models involving subsets
of the constraints
 Determine level of activity forced by Partition/Covering
constraints
 Minimize sum of knapsack variables s.t. partitioning/cover
constraints. At least 4 of the knapsack variables must be 1
 Examine interaction with other groups of constraints
 Incrementally add other groups to auxiliary model
 GUB and balance constraints limit our ability to select the
lighter objects to put into the knapsack
145
Interpreting the Results (Larger Conflicts)
 Interaction with other groups of constraints (ctd):
 Knapsack constraint coefficients range from 75 to
955 with a capacity of 2000
 Either need 4 heavy objects (Set
Partitioning/Covering), or 5 or more objects
(Balance/GUB) if we include any light objects
 Either way, we violate the knapsack:
205 COL004 + 205 COL005 + 205 COL006 + 550 COL007
+ 550 COL008 - ... + 75 COL159 <= 2000
146
Interpreting the Results
Example 3:
c1:12223x + 12224y + 36672z = 149389505
Bounds
x ,y, z >= 0
Generals
x y z
Rewrite c1 as
integral
c1:12224( x + y + 3z ) – x = 12224*12221 + 1
= 12224*12222 - 12223
Disjunction on t = x + y + 3z:
t <= 12221: -x >= 1, contradicting x >= 0
t >= 12222: x >= 12223  l.h.s >= 12223*12223 = 149401729
147
Interpreting the Results
 Partial list of tactics (in order of usefulness) for
interpreting conflicts
 Model knowledge
 Linear combinations of constraints to provide a
certificate of infeasibility (for all model types)
 Other combinations for certificate of infeasibility for
models with binary/integer variables
 Look for groups of constraints that are easier to
interpret
 Conflict file provides sum of equality constraints
 Group constraints together, then decompose
 Disjunctions
148
Fixing Conflicts
 Now that I understand the conflict, how should I
fix it?
 Removal of any constraint or bound makes a minimal
conflict feasible
 Numerous ways to make the model feasible, but only a few
will properly correct the model
 Need to make use of model knowledge to determine
the right way to correct the infeasibility
 Loosening bounds or right hand side values
 Changing matrix coefficients
 Removal of constraints
149
Fixing Conflicts
Example 2: A project staffing problem
Minimize
obj: cost Cost of using employee #1
Subject To
c1: - cost + 80 x1 + 60 x2 + 55 x3 + 30 x4 + 25 x5 +
80 x6 + 60 x7 + 35 x8 + 80 x9 + 55 x10 = 0
c2: x1 + x2 + 0.8 x3 + 0.6 x4 + 0.4 x5 >= 2.1
skills c3: x6 + 0.9 x7 + 0.5 x8 >= 1.2
c4: x9 + 0.9 x10 >= 0.8
c5: 0.2 x2 + x3 + 0.5 x4 + 0.5 x5 + 0.2 x7 + 0.5 x8 +
x10 - service = 0
// service quality
c6: x1 + x6 + x9 >= 1
rules, c7: x1 + x2 + x3 + x6 + x7 + x9 >= 2
c8: x2 + x3 + x4 + x5 <= 0 // unavailable at the time
policies c9: x4 + x5 + x8 <= 1
c10: x1 + x10 <= 1
Bounds
service >= 3.2
Binaries
x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 // 10 employees
150
Fixing Conflicts
Example 2: A project staffing problem
…
c2: x1 + x2 + 0.8 x3 + 0.6 x4 + 0.4 x5 >= 2.1
…
c8: x2 + x3 + x4 + x5 <= 0
…
Binaries
x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 // 10 employees

Changing bounds on binaries fixes the conflict, but makes
no sense within the context of the model
 Any errors in the coefficients for x2 through x5 in c2 have
no effect as long as c8 is present
 Relaxing right hand side of c2 to 1, or r.h.s. of c8 to 2,
resolves the infeasibility
 Use model knowledge to select the appropriate change
151
Use beyond infeasible problems
 Conflict Refiner and Feasopt can do more than
fix infeasible problems
 Save time debugging
 Faster model development
 MIP performance improvements
 Best integer (Feasopt)
 Best node (Conflict Refiner)
 MIP sensitivity analysis
152
Use beyond infeasible problems
 Conflict Refiner
 Debugging
 Customer claimed another optimizer found a better solution
than the one CPLEX claimed was optimal
- Fixed the variable bounds at the “better” solution values
- Conflict revealed that other optimizer had default variable
lower bound of –infinity instead of 0
 Infeasible MIP starts
- Fix the variable bounds at the MIP start values
- Conflict will reveal why the MIP start was rejected
 Model Development
 Fix variable bounds at a known feasible solution. If
infeasible, use conflict refiner to identify incorrect
constraints
153
Use beyond infeasible problems
 Conflict Refiner (ctd)
 MIP Sensitivity Analysis
 Given the optimal solution to a MIP, we want to identify the
constraints that prevent the objective from improving
 Readily available from optimal basis of an LP, not so
obvious with a MIP
 Let z* be the optimal objective value. Add a constraint that
the objective is better than z*. Compute the conflict on the
resulting infeasible MIP to identify constraints that prevent
the objective from improving. Repeat if necessary.
 Tightening MIP formulation
 Conflict refiner on related problem can help identify
constraints from which cuts are derived.
- Example: Fix penalty variables to 0 on models with soft
constraints
154
Use beyond infeasible problems
 Feasopt
 Use to find a feasible solution
 Primarily for MIPs and QCPs; simplex based optimizers
already do this
 Especially advantageous for MIPs due to recently added
local search heuristics that need an integer feasible solution
to operate
 Feasible solution available when Feasopt objective = 0
 Objective focused on minimizing infeasibilities
 Use to prove infeasibility faster
 Proving infeasibility with branch and cut while solving
relaxations associated with the original objective can be
time consuming
 Typically not necessary for continuous models
155
Computational Aspects
 Computation time relative to the original model
 Conflict Refiner solves sub problems
 Conflict Refiner for MIP solves multiple sub problems
 Design tries to minimize the number of sub problems
 Makes use of presolve whenever possible
 Feasopt solves one or two sub problems, depending
on mode
 Mode also affects problem type
 Minimizing number of relaxed constraints for a continuous
model results in a MIP sub problem
 Minimizing sum of squares of relaxed constraints for a
model results in a quadratic objective
156
Computational Aspects
 So, how long will it take?
 Solve times can vary dramatically relative to original
model, especially for MIPs




Less variation for continuous models
MIP performance tuning strategies apply
Numerical tuning strategies apply
Times are usually reasonable when presolve detects
infeasibility on the original model
 Sometimes a non minimal conflict will suffice, saving most
of the run time
157
Infeasibility Analysis with CPLEX
 Summary and Conclusions
 CPLEX offers tools to explain (Conflict Refiner) and
relax (Feasopt) an infeasible model
 Conflict sizes can vary dramatically
 Majority are modest in size, straightforward to interpret
- For significant minority of larger ones, process groups of
constraints rather than individual ones
 Many ways to correct conflicting constraints
 Typically need model knowledge to identify the right one
 Conflict Refiner and Feasopt aren’t just for
infeasibility analysis
 Debugging, model development, MIP performance
improvements, MIP sensitivity analysis
158
Infeasibility Analysis with CPLEX
 Additional Reading (Conflict Refiner algorithms)
 Chinneck, J., Dravnieks, E. “Locating Minimal
Infeasible Constraint Sets in Linear Programs”.
ORSA Journal on Computing, Vol. 3, No. 2, 1991
 Gleeson J., Ryan J. “Identifying Minimally Infeasible
Subsystems of Inequalities”. ORSA Journal on
Computing, Vol. 2, No. 1, 1990
 Junker, U. QuickXPlain: Conflict Detection for
Arbitrary Constraint Propagation Algorithms. IJCAI'01
Workshop on Modelling and Solving problems with
constraints (CONS-1), 2001. Available at
http://citeseer.ist.psu.edu/junker01quickxplain.html
159