Transcript Chapter 1 An Introduction to Model Building

Optimisation Methods
Integer Programming
Lecture Outline
1 Introduction
2 Integer Programming
3 Modeling with 0-1 (Binary) Variables
4 Goal Programming
5 Nonlinear Programming
2
Introduction

Integer programming is the extension of LP that solves
problems requiring integer solutions.

Goal programming is the extension of LP that permits more
than one objective to be stated.

Nonlinear programming is the case in which objectives or
constraints are nonlinear.

All three above mathematical programming models are used
when some of the basic assumptions of LP are made more or
less restrictive.
3
Summary: Linear Programming Extensions

Integer Programming
 Linear, integer solutions

Goal Programming
 Linear, multiple objectives

Nonlinear Programming
 Nonlinear objective and/or constraints
4
Integer Programming
 Solution values must be whole numbers in integer
programming .
 There are three types of integer programs:
 pure integer programming;
 mixed-integer programming;
and
 0–1 integer programming.
5
Integer Programming
•
The Pure Integer Programming problems are cases in
which all variables are required to have integer
values.
•
The Mixed-Integer Programming problems are cases
in which some, but not all, of the decision variables
are required to have integer values.
•
The Zero–One Integer Programming problems are
special cases in which all the decision variables must
have integer solution values of 0 or 1.
6
9.3 The Branch-and-Bound Method for Solving Pure Integer
Programming Problems
 In practice, most IPs are solved by some
versions of the branch-and-bound procedure.
Branch-and-bound methods implicitly
enumerate all possible solutions to an IP.
 By solving a single subproblem, many possible
solutions may be eliminated from
consideration.
 Subproblems are generated by branching on an
appropriately chosen fractional-valued variable.
7
 Suppose that in a given subproblem (call it old
subproblem), assumes a fractional value
between the integers i and i+1. Then the two
newly generated subproblems are
New Subproblem 1 Old subproblem + Constraint x  i
i
New Subproblem 2
Old subproblem + Constraint
x  i 1
i
 Key aspects of the branch-and-bound method
for solving pure IPs
 If it is unnecessary to branch on a subproblem, we
say that it is fathomed.
8


These three situations (for a max problem) result in a
subproblem being fathomed

The subproblem is infeasible, thus it cannot yield the optimal
solution to the IP.

The subproblem yield an optimal solution in which all variables have
integer values. If this optimal solution has a better z-value than any
previously obtained solution that is feasible in the IP, than it
becomes a candidate solution, and its z-value becomes the
current lower bound (LB) on the optimal z-value for the IP.

The optimal z-value for the subproblem does not exceed (in a max
problem) the current LB, so it may be eliminated from
consideration.
A subproblem may be eliminated from consideration in
these situations

The subproblem is infeasible.

The LB is at least as large as the z-value for the
subproblem
9
 Two general approaches are used to determine
which subproblem should be solved next.
 The most widely used is LIFO.
 LIFO leads us down one side of the branch-andbound tree and quickly find a candidate solution and
then we backtrack our way up to the top of the other
side
 The LIFO approach is often called backtracking.
 The second commonly used approach is
jumptracking.
 When branching on a node, the jumptracking method
solves all the problems created by branching.
10
 When solving IP problems using Solver you can
adjust a Solver tolerance setting.
 The setting is found under the Options.
 For example a tolerance value of .20 causes
the Solver to stop when a feasible solution is
found that has an objective function value
within 20% of the optimal solution.
11
Harrison Electric Company

The Company produces two products popular with home
renovators: old-fashioned chandeliers and ceiling fans.

Both the chandeliers and fans require a two-step production
process involving wiring and assembly.

It takes about 2 hours to wire each chandelier and 3 hours to
wire a ceiling fan. Final assembly of the chandeliers and fans
requires 6 and 5 hours, respectively.

The production capability is such that only 12 hours of wiring
time and 30 hours of assembly time are available.
12
Harrison Electric Company
If each chandelier produced nets the firm $7 and each fan
$6, Harrison’s production mix decision can be formulated
using LP as follows:
maximize profit = $7X1 + $6X2
subject to: 2X1 + 3X2 ≤ 12 (wiring hours)
6X1 + 5X2 ≤ 30 (assembly hours)
X1, X2 ≥ 0 (nonnegative)
X1 = number of chandeliers produced
X2 = number of ceiling fans produced
13
Harrison Electric Company
With only two variables and two constraints, the graphical LP
approach to generate the optimal solution is given below:
6X1 + 5X2 ≤ 30
+ = Possible Integer Solution
Optimal LP Solution
(X1 = 33/4, X2 = 11/2,
Profit = $35.25
2X1 + 3X2 ≤ 12
14
Integer Solution to Harrison Electric
Co.
Optimal
solution
Solution if
rounding off
15
Integer Programming
 Rounding off is one way to reach integer
solution values, but it often does not yield the
best solution.
 An important concept to understand is that an
integer programming solution can never be
better than the solution to the same LP problem.
 The integer problem is usually worse in terms
of higher cost or lower profit.
16
Branch and Bound Method

Branch and Bound breaks the feasible solution region into
sub-problems until an optimal solution is found.

There are Six Steps in Solving Integer Programming
Maximization Problems by Branch and Bound.

The steps are given over the next several slides.
17
Branch and Bound Method: The Six Steps
1.
2.
Solve the original problem using LP.

If the answer satisfies the integer constraints, it is done.

If not, this value provides an initial upper bound.
Find any feasible solution that meets the integer
constraints for use as a lower bound.

Usually, rounding down each variable will accomplish
this.
18
Branch and Bound Method Steps:
3.
4.
Branch on one variable from Step 1 that does not have an
integer value.

Split the problem into two sub-problems based on integer
values that are immediately above and below the noninteger value.

For example, if X2 = 3.75 was in the final LP solution,
introduce the constraint X2 ≥ 4 in the first sub-problem
and X2 ≤ 3 in the second sub-problem.
Create nodes at the top of these new branches by solving
the new problems.
19
Branch and Bound Method Steps:
5.
a)
If a branch yields a solution to the LP problem that is not
feasible, terminate the branch.
b)
If a branch yields a solution to the LP problem that is
feasible, but not an integer solution, go to step 6.
20
Branch and Bound Method Steps:
5. (continued)
c)
If the branch yields a feasible integer solution, examine the
value of the objective function.
If this value equals the upper bound, an optimal solution
has been reached.
If it is not equal to the upper bound, but exceeds the lower
bound, set it as the new lower bound and go to step 6.
Finally, if it is less than the lower bound, terminate this
branch.
21
Branch and Bound Method Steps:
6.
Examine both branches again and set the upper bound
equal to the maximum value of the objective function at
all final nodes.

If the upper bound equals the lower bound, stop.

If not, go back to step 3.
Minimization problems involve reversing
the roles of the upper and lower bounds.
22
Harrison Electric Co:
Figure 11.1 shows graphically that the optimal, noninteger solution is
X1 = 3.75 chandeliers
X2 = 1.5 ceiling fans
profit = $35.25
 Since X1 and X2 are not integers, this solution is not
valid.
 The profit value of $35.25 will serve as an initial
upper bound.
 Note that rounding down gives X1 = 3, X2 = 1, profit =
$27, which is feasible and can be used as a lower
23
bound.
Integer Solution: Creating Sub-problems

The problem is now divided into two sub-problems: A and B.

Consider branching on either variable that does not have an
integer solution; pick X1 this time.
Subproblem A
maximize profit = $7X1 + $6X2
Subject to:
2X1 + 3X2 ≤ 12
6X1 + 5X2 ≤ 30
X1
≥4
Subproblem B
maximize profit = $7X1 + $6X2
Subject to:
2X1 + 3X2 ≤ 12
6X1 + 5X2 ≤ 30
X1
≤3
24
Optimal Solution for Sub-problems
Optimal solutions are:
Sub-problem A: X1 = 4; X2 = 1.2, profit=$35.20
Sub-problem B: X1=3, X2=2, profit=$33.00
(see figure on next slide)
 Stop searching on the Subproblem B branch because it has
an all-integer feasible solution.
 The $33 profit becomes the lower bound.
 Subproblem A’s branch is searched further since it has a
non-integer solution.
 The second upper bound becomes $35.20, replacing $35.25
from the first node.
25
Optimal Solution for Sub-problem
26
Sub-problems C and D
Subproblem A’s branching yields Subproblems C and D.
Subproblem C
maximize profit = $7X1 + $6X2
Subject to:
2X1 + 3X2 ≤ 12
6X1 + 5X2 ≤ 30
X1
≥4
X2 ≥ 2
Subproblem D
maximize profit = $7X1 + $6X2
Subject to:
2X1 + 3X2 ≤ 12
6X1 + 5X2 ≤ 30
X1
≥4
X2 ≤ 1
27
Sub-problems C and D
(continued)
 Subproblem C has no feasible solution at all because
the first two constraints are violated if the X1 ≥ 4 and
X2 ≥ 2 constraints are observed.
 Terminate this branch and do not consider its
solution.
 Subproblem D’s optimal solution is
X1 = 4 , X2 = 1, profit = $35.16.
 This non-integer solution yields a new upper bound
of $35.16, replacing the original $35.20.
 Subproblems C and D, as well as the final branches
for the problem, are shown in the figure on the next
slide.
28
Harrison Electric’s Full
Branch and Bound Solution
29
Subproblems E and F
 Finally, create subproblems E and F and solve for X1
and X2 with the added constraints X1 ≤ 4 and X1 ≥ 5.
The subproblems and their solutions are:
Subproblem E
maximize profit = $7X1 + $6X2
Subject to:
2X1 + 3X2 ≤ 12
6X1 + 5X2 ≤ 30
X1
≥4
X1
≤4
X2 ≤ 1
Optimal solution for E:
X1 = 4, X2 = 1, profit = $34
30
Subproblems E and F
(continued)
Subproblem F
maximize profit = $7X1 + $6X2
Subject to:
2X1 + 3X2 ≤ 12
6X1 + 5X2 ≤ 30
X1
≥4
X1
≥5
X2 ≤ 1
Optimal solution for F:
X1 = 5, X2 = 0, profit = $35
31
Goal Programming

Firms usually have more than one goal. For example,







maximizing total profit,
maximizing market share,
maintaining full employment,
providing quality ecological management,
minimizing noise level in the neighborhood, and
meeting numerous other non-economic goals.
It is not possible for LP to have multiple goals unless they are all
measured in the same units (such as dollars),
 a highly unusual situation.

An important technique that has been developed to supplement LP is
called goal programming.
32
Goal Programming
(continued)
 Goal programming “satisfices,”
 as opposed to LP, which tries to “optimize.”
 Satisfice means coming as close as possible to
reaching goals.
 The objective function is the main difference
between goal programming and LP.
 In goal programming, the purpose is to
minimize deviational variables,
 which are the only terms in the objective
function.
33
Example of Goal Programming
Harrison Electric Revisited
Goals Harrison’s management wants to
achieve, each equal in priority:

Goal 1: to produce as much profit above $30 as possible
during the production period.

Goal 2: to fully utilize the available wiring department hours.

Goal 3: to avoid overtime in the assembly department.

Goal 4: to meet a contract requirement to produce at least
seven ceiling fans.
34
Example of Goal Programming
Harrison Electric Revisited
Need a clear definition of deviational variables, such as :
d1– = underachievement of the profit target
d1+ = overachievement of the profit target
d2– = idle time in the wiring dept. (underused)
d2+ = overtime in the wiring dept. (overused)
d3– = idle time in the assembly dept. (underused)
d3+ = overtime in the wiring dept. (overused)
d4– = underachievement of the ceiling fan goal
d4+ = overachievement of the ceiling fan goal
35
Ranking Goals with Priority Levels
A key idea in goal programming is that one goal is
more important than another. Priorities are
assigned to each deviational variable.
Priority 1 is infinitely more important than Priority 2, which is
infinitely more important than the next goal, and so on.
36
Analysis of First Goal
37
Analysis of First and Second Goals
38
Analysis of All Four Priority Goals
39
Goal Programming Versus Linear
Programming

Multiple goals (instead of one goal)

Deviational variables minimized (instead of maximizing
profit or minimizing cost of LP)

“Satisficing” (instead of optimizing)

Deviational variables are real (and replace slack variables)
40
9.2 Formulating Integer Programming
Problems
 Practical solutions can be formulated as IPs.
 The basics of formulating an IP model
41
Example 1: Capital Budgeting IP
 Stockco is considering four investments
 Each investment
 Yields a determined NPV
 Requires a certain cash flow at the present time
 Currently Stockco has $14,000 available for
investment.
 Formulate an IP whose solution will tell Stockco
how to maximize the NPV obtained from the
four investments.
42
Example 1: Solution
 Begin by defining a variable for each decision
that Stockco must make.
 The NPV obtained by Stockco is
Total NPV obtained by Stocko = 16x1 + 22x2 + 12x3 + 8x4
 Stockco’s objective function is
max z = 16x1 + 22x2 + 12x3 +8x4
 Stockco faces the constraint that at most
$14,000 can be invested.
 Stockco’s 0-1 IP is
max z = 16x1 + 22x2 + 12x3 +8x4
s.t.
5x1 + 7x2 + 4x3 +3x4 ≤ 14
xj = 0 or 1 (j = 1,2,3,4)
43
Set Covering as an IP

In a set-covering problem, each member of a given
set must be “covered” by an acceptable member of some
set.

The objective of a set-covering problem is to minimize
the number of elements in set 2 that are required to
cover all the elements in set 1.
44
Piece-wise linear functions as IP
 0-1 variables can be used to model
optimization problems involving piecewise
linear functions.
 A piecewise linear function consists of
several straight line segments.
 The graph of the piecewise linear function is
made of straight-line segments.
 The points where the slope of the piecewise
linear function changes are called the break
points of the function.
 A piecewise linear function is not a linear
function so linear programming cannot be used
to solve the optimization problem.
45
Piece-wise linear functions as IP

By using 0-1 variables, however, a piecewise linear
function can be represented in a linear form. Suppose the
piecewise linear function f (x) has break points b1,b2 ,...,bn .
For some k , bk  x  bk 1. T hen,for some
number z k (0  z k  1), x may be writ t enas :
x  z k bk  (1  z k )bk 1
Because f ( x) is linear for bk  x  bk 1 , we
may writ e f ( x)  z k f (bk )  (1  z k ) f (bk 1 )
46
Piece-wise linear functions as IP
 Suppose the piecewise linear function f (x) has
break points b1,b2 ,...,bn .
 Step 1 Wherever f (x) occurs in the optimization
problem, replace f (x) by z1 f b1 z2 f b2 ... zn f bn.
 Step 2 Add the following constraints to the problem:
z  y ,z
y  y , z  y  y ,..., z  y  y , z
y  y  ...  y  1
z  z  ...  z  1
y  0 or1 i  1,2,...,n  1; z  0 i  1,2,...,n 
1
1
2

1
2
1
2
1
i
2
3
2
3
n
n 1
n 1
n

y
n 1
n 1
n
i
47
Piece-wise linear functions as IP
 If a piecewise linear function f(x) involved in a
formulation has the property that the slope of
the f(x) becomes less favorable to the decision
maker as x increases, then the tedious IP
formulation is unnecessary.
 LINDO can be used to solve pure and mixed
IPs.
 In addition to the optimal solution, the LINDO
output also includes shadow prices and
reduced costs.
 LINGO and the Excel Solver can also be used to
solve IPs.
48
9.4 The Branch-and-Bound Method for Solving Mixed
Integer Programming Problems
 In mixed IP, some variables are required to be
integers and others are allowed to be either
integer or non-integers.
 To solve a mixed IP by the branch-and-bound
method, modify the method by branching only
on variables that are required to be integers.
 For a solution to a subproblem to be a
candidate solution, it need only assign integer
values to those variables that are required to
be integers
49
9.5 Solving Knapsack Problems by the
Branch-and-Bound Method
 A knapsack problem is an IP with a single
constraint.
 A knapsack problem in which each variable
must be equal to 0 or 1 may be written as
max z = c1x1 + c2x2 + ∙∙∙ + cnxn
s.t.
a1x1 + a2x2 + ∙∙∙ + anxn ≤ b
x1 = 0 or 1 (i = 1, 2, …, n)
 When knapsack problems are solved by the
branch-and-bound method, two aspects of the
method greatly simplify.
 Due to each variable equaling 0 or 1, branching on xi
will yield in xi =0 and an xi =1 branch.
 The LP relaxation may be solved by inspection.
50
9.6 Solving Combinatorial Optimization Problems by the
Branch-and-Bound Method
 A combinatorial optimization problem is
any optimization problem that has a finite
number of feasible solutions.
 A branch-and-bound approach is often the
most efficient way to solve them.
 Examples of combinatorial optimization
problems
 Ten jobs must be processed on a single machine. It is
known how long it takes to complete each job and
the time at which each job must be completed. What
ordering of the jobs minimizes the total delay of the
10 jobs?
51
 A salesperson must visit each of the 10 cities before
returning to his home. What ordering of the cities
minimizes the total distance the salesperson must
travel before returning home? This problem is called
the traveling sales person problem (TSP).
 In each of these problems, many possible
solutions must be considered.
52
 When using branch-and-bound methods to
solve TSPs with many cities, large amounts of
computer time is needed.
 Heuristic methods, or heuristics, can be
used to quickly lead to a good solution.
 Heuristics is a method used to solve a problem
by trial and error when an algorithm approach
is impractical.
 Two types of heuristic methods can be used to
solve TSP; nearest neighbor method and
cheapest-insertion method.
53

Nearest Neighbor Method




Begin at any city and then “visit” the nearest city.
Then go to the unvisited city closest to the city we have
most recently visited.
Continue in this fashion until a tour is obtained. After
applying this procedure beginning at each city, take the
best tour found.
Cheapest Insertion Method (CIM)




Begin at any city and find its closest neighbor.
Then create a subtour joining those two cities.
Next, replace an arc in the subtour (say, arc (i, j) by the
combinations of two arcs---(i, k) and (k, j), where k is not
in the current subtour---that will increase the length of the
subtour by the smallest (or cheapest) amount.
Continue with this procedure until a tour is obtained. After
applying this procedure beginning with each city, we take
the best tour found.
54
 Three methods to evaluate heuristics
 Performance guarantees
 Gives a worse-case bound on how far away from
optimality a tour constructed by the heuristic can be
 Probabilistic analysis
 A heuristic is evaluated by assuming that the location
of cities follows some known probability distribution
 Empirical analysis
 Heuristics are compared to the optimal solution for a
number of problems for which the optimal tour is
known
55

An IP formulation can be used to solve a TSP but can
become unwieldy and inefficient for large TSPs.
n
minimize:
s.t.
n
 cij xij
i 1 j 1
n
 xij  1
1 i  n
 xij  1
1 j  n
j 0
n
i 0
ti  t j  nxij  n  1 2  i, j  n
•
•
•

cij = distance from city i to city j
xij = 1 if tour visits i then j, and 0 otherwise (binary)
ti = arbitrary real numbers we need to solve for
LINGO can be used to solve the IP of a TSP.
56
9.7 Implicit Enumeration
 The method of implicit enumeration is often
used to solve 0-1 IPs. Many IP problems can be
converted to 0-1 IP problems.
 Implicit enumeration uses the fact that each
variable must be equal to 0 or 1 to simplify
both the branching and bounding components
of the branch-and-bound process and to
determine efficiently when a node is infeasible.
 The tree used in the implicit enumeration
method is similar to those used to solve 0-1
knapsack problems.
 Some nodes have variable that are specified
called fixed variables.
57
 All variables whose values are unspecified at a
node are called free variables.
 For any node, a specification of the values of
all the free variables is called a completion of
the node.
 Three main ideas used in implicit enumeration
 Suppose we are at any node with fixed variables, is
there an easy way to find a good completion of the
node that is feasible in the original 0-1 TSP?
 Even if the best completion of a node is not feasible,
the best completion gives us a bound on the best
objective function value that can be obtained via
feasible completion of the node. This bound can be
used to eliminate a node from consideration.
58
 At any node, is there an easy way to determine if all
completions of the node are infeasible?
 In general, check whether a node has a feasible
completion by looking at each constraint and
assigning each free variable the best value for
satisfying the constraint.
59
9.8 Cutting Plane Algorithm

An alternative method to the branch-and-bound method
is the cutting plane algorithm.

Summary of the cutting plane algorithm
Step 1 Find the optimal tableau for the IP’s programming
relaxation. If all variables in the optimal solution assume
integer values, we have found an optimal solution to the
IP; otherwise, proceed to step2.
Step 2 Pick a constraint in the LP relaxation optimal tableau
whose right-hand side has the fractional part closest to
1/2. This constraint will be used to generate a cut.
Step 2a For the constraint identified in step 2, write its righthand side and each variable’s coefficient in the form [x]+ f,
where 0 <= f < 1.
60
9.8 Cutting Plane Algorithm
Step 2b Rewrite the constraint used to generate the cut as
All terms with integer coefficients = all terms with fractional coefficients
Then the cut is
All terms with fractional coefficients <= 0
Step 3 Use the simplex to find the optimal solution to the LP
relaxation, with the cut as an additional constraint.

If all variables assume integer values in the optimal
solution, we have found an optimal solution to the IP.

Otherwise, pick the constraint with the most fractional
right-hand side and use it to generate another cut, which is
added to the tableau.

We continue this process until we obtain a solution in which
all variables are integers. This will be an optimal solution to
the IP.
61
9.8 Cutting Plane Algorithm
 A cut generated by the method above
has the following properties:
 Any feasible point for the IP will satisfy the
cut
 The current optimal solution to the LP
relaxation will not satisfy the cut
62