Transcript IOE 466 Statistical Quality Control (TU/TH 1:30

56:171 Operations Research
Instructor: Prof. Yong Chen
TA: Qingyu Yang
M/W/F 12:30 - 1:20
Fall 2005
1
Introduction
• INSTRUCTOR—Yong Chen
- Background
- Availability
• TEXT
- Text Book
- Lecture Notes
• COURSE
- Web site, Email list
- Computing
- Attendance Policy
- Prerequisites
- Grading Policy
• TA
• Lab/Recitation
2
Background
B. E. in computer science, Tsinghua
University, China, 1998
 Ph. D., 2003, Industrial and Operations
Engineering, University of Michigan
 Assistant professor, Dept. of Mechanical
and Industrial Engineering, 2003—now
 Research area: Quality and reliability
engineering; Sensor system design and
analysis

3
Prerequisites



Basic linear algebra and calculus knowledge
Basic probability and statistics knowledge
Basic computing skills
4
Review Questions
How many “Yes” do you get?








Do you know how to solve a system of linear
equations using Gaussian elimination?
Can you draw a line on x-y plane given its equation?
Do you know the meaning of convex and concave
functions?
Can you calculate the first and second derivatives of a
given function?
Do you know what is conditional probability?
Do you know what is probability density function?
Do you know the definition of expectation and
variance of a random variable?
Do you know how to use Excel to do some simple
calculation?
5
Grading Policy





Attendance/participation
Homework
Exam 1
Exam 2
Exam 3
5%
20%
25%
25%
25%
- Grade for attendance is based on random quizzes in lectures
and labs
- Homework should be submitted in-class on the due date;
- No late homework is acceptable;
- but one HW grade is not counted in the final grading.
6
Course Coverage
Deterministic







Linear “programming”
Transportation models
Assignment models
Integer programming
Nonlinear programming
Network models
PERT/CPM
Stochastic


Markov chains
Queueing theory
7
What is Operations
Research?


From military: research on (military) operations
Today, operations research means a scientific
approach to decision making, which seeks to
determine how best to design and operate a
system, usually under conditions requiring the
allocation of scarce resources.
8
Decision-Making
We all make decisions, all of the time
 Day-to-day choices
 Business decisions
 Public policy decisions
 Feasibility vs. optimality decisions
 Easy vs. difficult

9
Course Selection Example
You have to decide your choice of courses
for all the semesters that you are here
 Your choices are restricted by a number of
rules

You have to achieve a minimum number of
credit hours
 You have to take each of the required courses
at some point during the program

10
Course Selection
Example, Cont.
 You can take a course only if you have taken
its pre-reqs
 You can not take two courses with time
conflict
 You can not take more than a maximum
number of credits each semester
 You have to take enough GEC courses from
specified categories
 You have to satisfy the EFA requirements
 You can not take classes before 9 because you
hate getting up early
11
Course Selection Example,
Cont.
The rules limit the possible choices you
have, but there are still a large number of
choices!
 How can you pick the “best” set of courses
for yourself?

12
Course Selection Example,
Cont.
You need to decide your choice of courses
 There are a large number of choices which
satisfy the given rules
 You need to find the courses that maximize
some measure of your performance, e.g.,
expected GPA
 This is an optimization problem

13
Another Example: The Diet
Problem
Given a collection of foods (e.g. milk,
chocolate, orange juice, pizza), determine
how much of each food to eat in a given day
 Goal is to minimize cost or calories, or
maximize “satisfaction”
 Have to satisfy rules limiting our choices

14
The Diet Problem, Cont.
An example:
Minimize the cost of my diet, subject to
satisfying minimum requirements of
protein, and maximum limits on calories
and fat
15
The Diet Problem, Cont.
Decisions:
Milk
Orange juice
Chocolate
Pizza
Objective:
Minimize cost
Rules:
Minimum protein requirement
Maximum calories limit
Maximum fat limit
16
The Diet Problem, Cont.

Stigler 1945
Posed problem
 Solved heuristically


Dantzig 1963, 1990

Solved optimally in 1947 using simplex method
Appears in many Operations Research texts
 New journal articles still appearing

17
The Diet Problem, cont.
IDEAL DIET
1.31 cups wheat flour
16 oz. milk
7.28 tbsp lard
1.77 bananas
0.707 cups cabbage
0.387 potatoes
1.32 cups rolled oats
3.86 tbsp peanut butter
0.0108 oz. beef
0.0824 oranges
0.314 carrots
0.53 cups pork and beans
18
Optimization Problem

Optimization problem
Decisions
 Means of comparing decisions
 Rules governing interactions between decisions

19
Mathematical Models
Once an optimization problem is defined in
words, we need to find an appropriate
mathematical model to analyze/solve it
 A mathematical model captures the essence
of the problem


An idealized version/approximation of the
problem
20
Formulation / Model

Formulation or model – mathematical
representation of an optimization problem
Decision variables
 Objective function
 Constraints
 Parameters (Constants)

21
The Diet Problem—Decision
Variables
Decision variables: a number of variables
whose values are to be determined
 Define the decision variables as:
xm = Gallons of milk consumed daily
xc = Bars of chocolate consumed daily
xo = Gallons of orange juice consumed daily
xp = Pizzas consumed daily

22
The Diet Problem—Objective
Function




Objective: minimize daily cost
Let ci , for i = m, c, o, p be the current cost per
unit for item i
Objective function: cmxm + ccxc + coxo + cpxp
Objective function: the measure of performance
expressed as a function of the decision variables
23



The Diet Problem—
Constraints
Constraints: Restrictions on the values that can be
assigned to the decision variables
We may want to limit our diet so that:
 total fat content in the diet does not exceed
some limit
 total calories do not exceed some limit
 total protein intake is at least some minimum
amount
We need the data for the fat, calorie, and protein
value per unit of each item and the limits we want
to meet
24
The Diet Problem—
Constraints, Cont.
Suppose fi , wi , pi , for i = m, …, p are the
values of fat, calories, and proteins per unit
of item i, respectively
 Suppose F, W, and P are the daily limits on
fat, calories, and protein, respectively
 We have all the data to write our constraints

25
The Diet Problem—
Constraints, Cont.

We want
Daily fat intake: fmxm + fcxc + foxo + fpxp  F
 Daily calorie intake: wmxm + wcxc + woxo + wpxp  W
 Daily protein intake: pmxm + pcxc + poxo + ppxp  P


And, none of the intake amounts for each food is
negative
26
The Diet Problem—
Parameters
Parameters: the constants used to specify
the objective function and the constraints
 For the diet problem, ci, fi, wi, pi, F, W, and
P are parameters
 In practice, it is difficult to determine the
parameters exactly
 Sensitivity analysis is needed

27
The Diet Problem—Complete
Model
Minimize cmxm + ccxc + coxo + cpxp
 Subject to

f x + fcxc + foxo + fpxp  F
 wmxm + wcxc + woxo + wpxp  W
 pmxm + pcxc + poxo + ppxp  P
 xm  0, xc  0, xo  0, xp  0
 m m
28
Other Applications
Optimization has been used in a number of
serious applications yielding huge profits
 A good place to find information about
these applications is the Interfaces journal
 Available electronically

29
Applications from Interfaces





Recovering from major airline disruptions
(Horner 2002)
Reducing travel costs and player fatigue in NBA
(Bean and Birge 1980)
Assigning managers to construction projects
(LeBlanc et al. 2000)
Managing consumer credit delinquency (Makuch
et al. 1992)
Manufacturing of beer cans (Katok and Ott 2000)
30
More Applications






Scheduling prototype vehicle testing at Ford
(Chelst et al. 2001)
Portfolio construction (Bertsimas et al. 1999)
Scheduling police patrol officers (Taylor and
Huxley 1989)
Planning closure and realignment of army bases
(Dell 1999)
Assigning restoration capacity in a
telecommunication network (Ambs et al. 2000)
Crew scheduling for airlines (Butchers 2000)
31
Ch. 3 Introduction to Linear
Programming
The Linear Programming Model
 Examples
 Assumptions of Linear Programming
 Graphic Solutions of LP
 Excel Solver
 Suggested Readings: 3.1-3.6 of H&L

32
Start of Linear Programming

George Dantzig (1914 – 2005),
“Father” of Linear Programming





Junior Statistician U.S.
Bureau of Labor Statistics
(1937-39)
Head of USAF Combat
Analysis Branch (1941-46)
PhD Mathematics, Cal Berkeley (1946)
Invented “Simplex” method for solving linear
programs (1947)
Medal of Science, 1975, for his work in LP
33
Linear Programs
Programming means planning
 Linear programs (LPs) have:

Linear objective function
 Linear constraints
 Continuous variables


Typically very easy to solve, even when
quite large
34
Why Linear Programs?
Many real-world problems can be modeled
as LPs
 Many other problems can be approximated
as LPs
 LP solution techniques provide the
foundation for solution methods for many
other structures of Mathematical Programs
(MPs)

35
Function Types
Linear functions have the general form:
f(x1, x2, …, xn) = c1x1 + c2x2+ …+cnxn
where c1, c2, …, cn are constants
 Linear functions are simple

36
Function Types (Cont.)

Examples of non-linear functions
Polynomial: f(x, y,z) = x2 + y2 + z2
 Cross terms: f(x, y, z) = xy
 Exponential: f(x) = ex
 Maximum: f(x, y, z) = max {x, y, z}
 Absolute: f(x) = |x|


More complex
37
Linear Constraints

Linear constraints are of three types

“”


“”


pmxm + pcxc + poxo + ppxp  P, must have a minimum
quantity of proteins
fmxm + fcxc + foxo + fpxp  F, at most F units of fat in
the diet
“=”

P. 46 of textbook—radiation therapy example
38
General Form of Linear
Constraints
Any linear constraint has the following general form:
a linear
function of
decision
variables


=
a constant
39
Not “linear Constraints”

Examples
x2 + y 2  1
 xy + y + 2z  1
 (x + y + z) is integer


Again, more complex
40
Example 1: Wyndor Glass
Wyndor makes doors and windows. They
have three plants. A batch of doors requires
1 hour at Plant 1 plus 3 hours at Plant 3. A
batch of windows requires 2 hours at Plant
2 plus 2 hours at Plant 3. Plant 1 is
available for 4 hours per week, Plant 2 for
12 hours per week, and Plant 3 for 18 hours
per week. The profit per door batch is
$3000 and the profit per window batch is
$5000. What should Wyndor manufacture
to maximize profits?
41
Wyndor Glass—Resource
Consumption
Production Time
per Batch, hours
Product
Production time
Plant
1
2
3
Door
1
0
3
Window Available hours per
week
0
4
2
12
2
18
42
Wyndor Glass (Cont.)

What are our decisions?
How many doors should we make?
 How many windows should we make?


What is our goal?


Maximize profits
What are our rules?
Don’t exceed capacity at plant 1
 Don’t exceed capacity at plant 2
 Don’t exceed capacity at plant 3

43
Wyndor Glass—LP Model

What are our decisions—Decision Variables
x1 > 0: # batches of doors per week
 x2 > 0: # batches of windows per week


What is our goal—Objective Function


Maximize 3000 x1 + 5000 x2
What are our rules—Constraints
(Plant 1)
(Plant 2)
(Plant 3)
(Non-negativity)
x1
+ 2x2
3x1 + 2x2
x1  0, x2  0
4
 12
 18
44
Example 2: Lego Chair and
Table
Make tables and chairs to maximize profits.
Profit: $16 for each Table, $10 for each Chair
Each table uses 2 large blocks and 2 small blocks
Each chair uses 1 large block and 2 small blocks
You are limited by the availability of material. You only have 6
large blocks and 8 small blocks.
How many tables and chairs should you make to maximize
profit?
45
How to Make the Chair and
Table?
46
Model


Decision variables:

t: # of tables

c: # of chairs
Objective function:


Maximize 16*t+10*c
Constraints:
6 (large legos)
8 (small legos)
2*t + c <= 6
2*t + 2*c <= 8
47
Optimal Solution of Lego
Game
# of tables:
2
# of chairs:
2
Total profit:
$52
48
Product Mix
There are n different products that I can
produce. Each of these products consumes
certain quantity of m different resources.
Every unit of a product i, for i = 1,…, n
uses aij units of resource j, for j = 1,…, m. I
can make a profit of $pi per unit of item i.
The amount of resource j available is uj.
What should I do to maximize my profit?
49
Connection

Product mix is a general version of Wyndor
Glass example and Lego chair and table
example
Wyndor glass: n = 2, m = 3
 Lego chair and table: n=2, m=2


Many business resource allocation
problems take this form
50
Product Mix: General Model

Decision variables:


Objective function:


xi = Amount of item i produced daily
maximize profit = p1x1 + p2x2 + … + pnxn
Constraints:
(Resource Availability)
a1j x1 + a2j x2 + … + anj xn  uj for j = 1,…, m
(Non-negativity)
xi  0 for i = 1,…, n
51
Assumption of LPs

When we write a problem as a linear
program, we are making a few assumptions
about the underlying process

Proportionality: The contribution of a decision
variable to the objective function or any one of
the constraints is proportional to its value, e.g.,



The daily fat in-take from the pizza is proportional
to the amount of pizza eaten daily
# of small blocks used is proportional to the number
of chairs made
The total profit is proportional to the number of
chairs or tables made
52
Assumptions (Cont.)

Additivity: The total contribution to the
objective and left hand side of each constraint
is the sum of individual contributions of each
activity



The total cost of daily food is the sum of costs from
individual food items
The total profit is the sum of profits from chair and
table
Divisibility: Each decision can take any real
value


Daily amount of milk consumption
Amount invested in a one-year CD at the beginning
of year 1
53
Assumptions (Cont.)

Big assumption – Certainty
The value of each parameter needed in the
linear programming is known with certainty
 This assumption is almost never satisfied
 Sensitivity analysis can help to find out the
robustness of our optimal solutions to
uncertainty of data

54
LP Model—A Standard Form
Max c1x1 + c2x2 + … + cnxn
subject to
(functional constraints)
a11x1 + a12x2 + … + a1nxn  b1
a21x1 + a22x2 + … + a2nxn  b2
…
am1x1 + am2x2 + … + amnxn  bm
(nonnegativity constraints)
x1  0, x2  0, …, xn  0
The number of variables is n and the number of constraints is
m+n
55
Notation


We can use some notation to write linear
programs compactly
We can write c1x1 + c2x2 + … + cnxn as


n
c x
i 1 i i
We can write each constraint aj1x1 + aj2x2 + … +
ajnxn  bj as
n
a
i 1
x  b j for j  1, ..., m
ji i
56
Standard Form

Using the previous notation, the standard
form of LP can be written as
n
max
cx

i 1
i i
st
n
a
i 1
x  b j for j  1, ..., m
ji i
xi  0, for i  1, ..., n
57
Some Terminology





Any specification of values for the decision
variables (x1,…, xn) is called a solution.
A feasible solution is a solution for which all the
constraints are satisfied.
An infeasible solution is a solution for which at
least one constraint is violated.
A feasible solution is called an optimal solution if
there is no other feasible solution with objective
function value better than it
The optimal value of a problem is the objective
value of an optimal solution to the problem
58
Example
Min
2x1 + 3x2
subject to x1 + 5x2 = 3
3x1
=2
x1  0, x2  0
In this case, x1 = 2/3 and x2 = (3-2/3)/5 = 7/15 is a feasible
solution for the problem
In fact, it is the only solution to the problem
So it is also the optimal solution to the problem as
well
Optimal value of the problem is 2(2/3)+3(7/15)=2.73
59
Wyndor Glass Co. Problem
max
s.t.




Z=3x1 + 5x2 (in
x1
+ 2x2
(2)
3x1
+ 2x2
(3)
x1  0, x2  0
$K)
4
 12
(1)
 18
x1 = 0, x2 = 0 is a feasible solution with Z=0
Is it optimal?
x1 = 0, x2 = 4 is a feasible solution with Z=20, better
than (0, 0)
Is this the optimal solution?
60
Feasible Region
The feasible region is the collection of all feasible solutions
x2
x1  4
(0,9)
(0,6)
(2,6)
2x2  12
(4,3)
3x1+2x2  18
(0,0)
(4, 0) (6, 0)
x1
61
Moving Isovalue Line


Now that we have the feasible region, how do we find the
best solution?
(0,4) has value 20, all solutions with value 20 lie on the
isovalue line 3x1 + 5x2 = 20
x2
3x1 + 5x2 = 36
3x1 + 5x2 = 20
3x1 + 5x2 = 10
?
x1
62
x2
Finding the Optimal Point
3x1 + 5x2 = 36
3x1 + 5x2 = 20
?
3x1 + 5x2 = 10
x1
The optimal point occurs at the intersection of these two lines:
2x2=12
Plant 2
3x1+2x2 = 18 Plant 3
x1 = 2
x2 = 6
Optimal value: Z=3x1+5x2=36—Maximal profit is $36K
63
Graphical Solutions
Min -2x - y
St
x+y<3
x<2
y<2
y
x, y > 0
-2x - y = -1
x
64
Graphical Solutions
Min -2x - y
St
x+y<3
x<2
y<2
y
x, y > 0
-2x - y = -2
x
65
Graphical Solutions
Min -2x - y
St
x+y<3
x<2
y<2
y
x, y > 0
-2x - y = -4
x
66
Graphical Solutions
Min -2x - y
St
x+y<3
x<2
y<2
y
x, y > 0
-2x - y = -5
x
67
Solving LP’s Graphically

Procedures:
Identify feasible region
 Plot an isovalue line corresponding to a
feasible solution
 Move line in improving direction and find the
last isovalue line touching the feasible region
 Any point(s) on the intersection of the last
isovalue line and feasible region are optimal
solutions

68
Finding Optimal Solutions
Min3x1 – x2
s.t. x1 + x2 < 6
x1
<4
x2 < 4
x1, x2 > 0
x2
(0,4)
(2,4)
(4,2)
(0,0)
(4,0)
x1
Which point is optimal?
69
Finding Optimal Solutions
Min -2x1 – x2
s.t. x1 + x2 < 6
x1
<4
x2 < 4
x1, x2 > 0
x2
(0,4)
(2,4)
(4,2)
(0,0)
(4,0)
x1
Which point is optimal?
70
Finding Optimal Solutions
Max x1 + x2
s.t. x1 + x2 < 6
x1
<4
x2 < 4
x1, x2 > 0
x2
(0,4)
(2,4)
(4,2)
(0,0)
(4,0)
x1
Which point is optimal?
71
Graphical LP Solutions
Works well for 2 decision variables
 “Possible” for 3 decision variables
 Impossible for 4+ variables



Other solution approaches necessary
Good to illustrate concepts, aid in
conceptual understanding
72
Example
3.2-2(a) of H&L: The colored area in the following graph
represents the feasible region of a LP problem whose objective
function is to be maximized. Label the following statement as
True or False and give an example of an objective function that
illustrates your answer.
(a) If (3,3) produces a larger value of the objective function than
(0, 2) and (6, 3), then (3,3) must be an optimal solution.
x2
(3, 3)
(6, 3)
(0, 2)
(0, 0)
(6, 0)
x1
73
Property of Optimal Solution


In all cases there is a corner point of the feasible
solution region that is an optimal solution
Is this a coincidence?


NO!
Important result: Whenever a linear program has
an optimal solution and it has a corner, there is
always an optimal solution on one of the corners
of the feasible region

Note that the statement does not say that a linear
program always has an optimal solution, it does not say
that all optimal solutions have to be on the corners, in
fact it does not presume that there will be corner
points!
74
Pathological Cases
What are the possibilities that a linear program does
not have an optimal solution at a corner point?

Infeasibility: There is no feasible solution
to the linear program, e.g.,
Min x1
s.t. x1  -1
x1  0
 There is no real value that is simultaneously
less than –1 and greater than 0
75
Infeasible LP’s
76
Pathological Cases

Unboundedness: The linear program is
feasible but the optimal value is not finite,
e.g.,
max x1 + x2
s.t.
x1  3
x2  4
x1, x2  0
x2
x2  4
(0,4)
x1  3
(0,0)
(3,0)
x1+x2
x1
77
Unbounded LP’s
78
Pathological Cases

No corner points: The feasible region has
no corner points
min x1
s.t. x1  0
x2 is free
Any solution with
x1 = 0 is optimal!
This case can never
happen for LPs in a
standard form
x2
(0,0)
x1
79
Pathological Cases

Are there other possibilities when a linear
program may not have an optimal solution
at a corner point?

NO!
Any linear program falls under one of the
four cases: (i) infeasible, (ii) unbounded,
(iii) no corner point, (iv) has an optimal
solution at a corner point
 There can be multiple optimal solutions to a
linear program

80
Multiple Optima
x2
(0,6)
What if the objective function of the
Wyndor problem was 3x1 + 2x2 instead of
3x1 + 5x2?
(2,6)
(3,9/2)
(4,3)
The optimal value of 18 is
achieved by all the solutions
on the line segment joining
(2,6) and (4,3)!
3x1 + 2x2 = 18
(4,0)
(0,0)
3x1 + 5x2 = 20
3x1 + 2x2 = 12
x1
81
Multiple Optima
82
Summary

A linear program always satisfies one of the
four cases

Pathological cases




It is infeasible
It is unbounded
It has no corner points but it has optimal solutions
Normal case

It has an optimal solution at one of the corner point
feasible solution (among possibly many others)
83
Summary

We shall always transform a linear program
into one of the standard forms before
solving it

We don’t need to worry about the third
pathological possibility
We only need to worry about infeasibility
and unboundedness
 When our LP (in standard form) is not
infeasible or unbounded, there is a corner
point feasible solution which is an optimal
solution

84
Introduction to Excel Solver
85
Loading Solver







Standard with every version of Excel
Insert MS Office or Excel master CD
Click on “Add/Delete” components
Open “Add-In” tools
Click on “Solver” or “add all”
Click “OK”
Solver should now appear in the “Tools” menu
86
56:171 Operations Research
Ch. 4 Solving Linear Programs:
The Simplex Method
87
Outline
The Simplex Method
 Simplex Method for Standard Form
 Theory of the Simplex Method
 Simplex Method for other LP problems
 Suggested Reading: 4.1, 4.2, 4.4, 4.5, 4.6

88
Review of Wyndor Glass
Example
Product Mix LP. Wyndor makes doors and windows. They
have three plants. A batch of doors requires 1 hour at Plant 1
plus 3 hours at Plant 3. A batch of windows requires 2 hours at
Plant 2 plus 2 hours at Plant 3. Plant 1 is available for 4 hours
per week, Plant 2 for 12 hours per week, and Plant 3 for 18
hours per week. The profit per door batch is $3000 and the
profit per window batch is $5000. What should Wyndor
manufacture to maximize profits?
Max Z = 3x1 + 5x2
s.t.
4
+ 2x2  12
3x1 + 2x2  18
x1, x2  0
1x1
profits (in thousands of $)
Plant 1
Plant 2
Plant 3
non-negativity
89
Standard & Augmented
Forms
Standard Form with Nonnegative RHS
Max Z = 3x1 + 5x2
s.t.
4
+ 2x2  12
3x1 + 2x2  18
x1, x2  0
1x1
Augmented Form
Max Z = 3x1 + 5x2
s.t.
1x1
+ s1
+ 2x2
+ s2
3x1 + 2x2
+s3
x1 , x2 , s1 , s2 , s3  0
=4
= 12
= 18
90
Geometric Representation of
Simplex Method
Max Z = 3x1 + 5x2
s.t.
+ s1
+ 2x2
+ s2
3x1 + 2x2
x1 , x2 , s 1 , s 2 , s 3  0
x2
(0,9)
(0,6)
1x1
+ s3
=4
= 12
= 18
(2,6)
(4,3)
(0,0)
(4, 0)
(6, 0)
x1
91
Geometric Representation of
Simplex Method
Max Z = 3x1 + 5x2
s.t.
+ s1
+ 2x2
+ s2
3x1 + 2x2
x1 , x2 , s 1 , s 2 , s 3  0
x2
(0,9)
(0,6)
1x1
(2,6)
(4,3)
(0,0)
+ s3
=4
= 12
= 18
(4, 0)
(6, 0)
x1
x1 = 0
x2 = 0
s1 = 4
s2 = 12
s3 = 18
Z =0
92
Geometric Representation of
Simplex Method
Max Z = 3x1 + 5x2
s.t.
(0,9)
?
(0,0)
+ s1
+ 2x2
+ s2
3x1 + 2x2
x1 , x2 , s 1 , s 2 , s 3  0
x2
(0,6)
1x1
+ s3
=4
= 12
= 18
(2,6)
(4,3)
(4, 0)
(6, 0)
x1
x1 = 0
x2 = 6
s1 = 4
s2 = 0
s3 = 6
Z = 30
93
Geometric Representation of
Simplex Method
Max Z = 3x1 + 5x2
s.t.
+ s1
+ 2x2
+ s2
3x1 + 2x2
x1 , x2 , s 1 , s 2 , s 3  0
x2
(0,9)
(0,6)
1x1
(2,6)
(4,3)
(0,0)
+ s3
=4
= 12
= 18
(4, 0)
(6, 0)
x1
x1 = 2
x2 = 6
s1 = 2
s2 = 0
s3 = 0
Z = 36
94
Algebraic Representation
Max Z = 3x1 + 5x2
s.t.
1x1
+ s1
+ 2x2
+ s2
3x1 + 2x2
x1 , x2 , s 1 , s 2 , s 3  0
+ s3
=4
= 12
= 18
3 equations in 5 unknowns
 Multiple solutions
 Guided search to move to optimal
solution
 “Simplex Method”

95
Tabular Representation
Max Z = 3x1 + 5x2
s.t.
1x1
+ s1
+ 2x2
+ s2
3x1 + 2x2
x1 , x2 , s 1 , s 2 , s 3  0
+ s3
=4
= 12
= 18
Initial Simplex Method Tableau
Z
x1
-3
x2
-5
s1
0
s2
0
s3
0
RHS
0
s1
s2
s3
1
0
3
0
2
2
1
0
0
0
1
0
0
0
1
4
12
18
96
Simplex Method (Tabular
Form)






Z
x1
-3
x2
-5
s1
0
s2
0
s3
0
RHS
0
s1
s2
s3
1
0
3
0
2
2
1
0
0
0
1
0
0
0
1
4
12
18
s1, s2 and s3 in this tableau represent basic variables
x1 and x2 are non-basic variables
Basic solutions are obtained by setting the non-basic
variables to zero and solve the basic variables
Basic solutions represent corner points
Systematically change basic solution to improve
objective function …
… while maintaining feasibility!
97
Basic Variables in Simplex
Tableaux
x1
x2
s1
s2
s3
RHS
Z
-3
-5
0
0
0
0
s1
1
0
1
0
0
4
s2
0
2
0
1
0
12
s3
3
2
0
0
1
18
Any column corresponding to a basic
variable in Simplex tableaux should have
one and only one nonzero element, which is
equal to “1”
 The collection of the basic variables is
called the basis

98
Each Iteration of Simplex
Method



Optimality Test: the current basic solution is optimal if
and only if every coefficient in row 0 is nonnegative (0)
Determine the entering basic variable by selecting the
variable with the “most negative” coefficient in row 0.
Determine the leaving basic variable by applying the
minimum ratio test:




Pick each coefficient in pivot column that is >0
Divide each of them into the RHS
Identify the row with smallest ratio
Solve the new basic solution by using elementary row
operations (multiply a row by a constant; or add/subtract
a multiple of the pivot row to/from another row)
99
More Simplex Examples
Max Z = 2x1 - x2 + x3
s.t.
3x1 + x2
x1 - x2
x1 + x2
x1 ,
+ x3  60
+ 2x3  10
- x3  20
x2 , x3  0
x1
x2
x3
s1
s2
s3
RHS
Z
-2
1
-1
0
0
0
0
s1
3
1
1
1
0
0
60
s2
1
-1
2
0
1
0
10
s3
1
1
-1
0
0
1
20
100
More Simplex Examples
Max Z = 60x1 + 30x2 + 20x3
s.t.
8x1 + 6x2 +
x3  48
4x1 + 2x2 + 1.5x3  20
2x1 + 1.5x2+0.5x3  8
x2
5
x1 , x2 , x3  0
x1
x2
x3
s1
s2
s3
s4
RHS
Z
-60
-30
-20
0
0
0
0
0
s1
8
6
1
1
0
0
0
48
s2
4
2
1.5
0
1
0
0
20
s3
2
1.5
0.5
0
0
1
0
8
s4
0
1
0
0
0
0
1
5
101
Theory of the Simplex
Method
For any LP with feasible solutions and a bounded
feasible region:




If there is exactly one optimal solution, then it
must be a corner-point feasible (CPF) solution
If there are multiple optimal solutions, then at
least two must be adjacent CPF solutions
There are a finite number of CPF solutions
A CPF solution is optimal if there are no other
adjacent CPF solutions that are better
102
Corner Point Feasible
Solutions
X
Interior Point Solution
• Feasible? Yes
• Optimal? Never
X
103
Alternate Optima
104
Finite Number of CPF
Solutions
An upper bound of the number of CPF solutions:
 # variables# constraints   n  m  (m  n)!



# variables
m!n!

  n 
Example: m=50 constraints, n=100 decision variables
(m  n)! (50  100 )!
40

 2.01 10
m!n!
50!100!
Greater than the
number of atoms in
Universe!
105
Adjacent CPF Solutions
x2
X
larger Z
smaller Z
x1
106
Solving Other Types of
Linear Programs
107
Finding a Feasible Solution?
Constraints in  Form &
Minimization Problems
Equality Constraints
X
Not Feasible!
108
Equality Constraints
Max Z = 2x1 + 3x2
s.t.
Note:
x1 = x2 = 0
is not feasible
1x1 + 2x2  4
x1 + x2 = 3
x1 , x2  0
3
How to achieve
feasibility?
2
(2,1)
X
3
4
109
Big M Method
Max Z = 2x1 + 3x2
s.t.
1x1 + 2x2  4
x1 + x2 = 3
x1 , x2  0
Max Z = 2x1 + 3x2
s.t.
Add
artificial
variable a1
Strategy: Start with artificial
variables, then remove
artificial variables from the
basic variables using penalty
M
- M a1
M>0 is a VERY
big number
1x1 + 2x2 + s1
=4
x1 + x2
+ a1 = 3 Note:
x1 , x2 , s1, a1  0
x1 = x2 = 0
is now feasible
110
Big M Simplex Tableaux
Max Z = 2x1 + 3x2
s.t.
– M a1 = 0
1x1 + 2x2 + s1
=4
x1 + x2
+ a1 = 3
x1 , x2 , s1, a1  0
An extra
Nonzero
Element; need
To remove!
x1
x2
s1
a1
RHS
Z
-2
-3
0
M
0
s1
1
2
1
0
4
a1
1
1
0
1
3
111
Big M Simplex Tableau
x1
x2
s1
Z –(M+2) –(M+3) 0
s1 1
2
1
a1 1
1
0
a1
0
0
1
RHS
–3M
4
3
Initial
Solution
x1 = 0
x2 = 0
s1 = 4
a1 = 3
112
 Constraints
Max Z = 2x1 + 5x2 + 3x3
s.t.
1x1 – 2x2 + x3  20
2x1 + 4x2 + x3 = 50
x1 , x2 , x3  0
Subtract surplus variable
1x1 – 2x2 + x3 – s1 = 20
to create equality
Add artificial variable
for equality…
artificial variables?
1x1 – 2x2 + x3 – s1 + a1 = 20
big M method
113
 Big M
Max Z = 2x1 + 5x2 + 3x3
s.t.
– Ma1 – Ma2
1x1 – 2x2 + x3 – s1 + a1
= 20
2x1 + 4x2 + x3 +
+ a2 = 50
x1 , x2 , x3 , s1 , a1 , a2  0
Tableau with revised row 0:
x1
x2
x3
s1
Z -(3M+2) -(2M+5) -(2M+3) 1M
a1
1
-2
1
-1
a2
2
4
1
0
a1
0
1
0
a2
0
0
1
RHS
-70M
20
50
114
Variables Allowed to be
Negative
xj allowed to be any value (+ or –)
Substitute
xj = xj+ – xj –
xj+ , xj –  0
115
Negative RHS’s
0.4x1 – 0.3x2  – 10
Is exactly equivalent to
Multiply by –1
– 0.4x1 + 0.3x2  10
Change sign of the inequality and use
corresponding methods for the new constraint
116
Minimization Problems
Min Z = 0.4x1 + 0.3x2
Is exactly equivalent to
Multiply by –1
Max -Z = – 0.4x1 – 0.3x2
117
Summary of All Types of LP
Situations
Solutions
Initial Basic
Variables
 constraint
Slack variable
Slack variable
Equality constraint
Artificial variable + Big M
Method
Artificial variable
 constraint
Surplus variables +
Artificial variable
artificial variables + Big M (NOT surplus var.)
Variables allowed
to be negative
Change of variables
Negative RHS
Multiply both sides by -1
Minimization
problems
Multiply objective function
by -1
Combination of cases: use Big M method last
118
LP Solution Problems
Unbounded Solutions
 No Feasible Solutions

119
Unbounded Solutions
Z
s1
s3





x1
-3
x2
-5
s1
0
s3
0
RHS
0
1
3
0
-2
1
0
0
1
4
6
No coefficient in pivot column is positive
No leaving basic variable
Can bring in unlimited x2
Z increases without limit!
LP is “unbounded”
120
unbounded
Unbounded Solutions
x2
(0,0)
(4, 0)
x1
121
Example
Breadco Bakeries bake two kinds of bread:
french and sourdough. Each loaf of french
bread can be sold for 36cents, and each loaf of
sourdough bread for 30cents. A loaf of french
bread requires 1 yeast packet and 6 oz of flour;
sourdough requires 1 yeast packet and 5 oz of
flour. At present Breadco has 5 yeast packets
and 10 oz of flour. Additional yeast packets
can be purchased at 3 cents each, and
additional flour at 4 cents/oz. Formulate and
solve an LP that can be used to maximize
Breadco’s profits.
122
No Feasible Solutions
Min Z = 2x1 + 3x2
s.t.
½x1 + ¼x2  4
x1 + 3x2  36
x1 + x2 = 10
x1 , x2  0
An LP is infeasible if an artificial variable
is greater than zero in a final solution
x1
x2
s1
s2
a1
a2
RHS
-Z
2M-1
0
0
M
0
4M-3
-30-6M
s1
1/4
0
1
0
0
-1/4
3/2
a1
-2
0
0
-1
1
-3
6
x2
1
1
0
0
0
1
10
123
Introduction to Interior Point
Solution Approach



x2


(0,9)
(0,6)

(2,6)
Starts at inner feasible point
Moves through interior of feasible
region
Always improves objective function
Longer computer time per iteration
Less iterations for large problems
Faster than Simplex method for huge
problems
(4,3)
(0,0)
(4, 0)
(6, 0)
x1
124
Sensitivity and Duality Analysis
in LP
125
Outline
Binding vs. Slack Constraints
 LP Sensitivity

Ranges of optimality
 Shadow prices

LP Duality
 Suggested Readings: 4.7, parts of 6.1, 6.2,
6.6

126
Binding vs. Slack Constraints
127
Binding vs. Slack Constraints
Slack
Constraint
Binding
Constraints
128
Ranges of Optimality
Objective function (OF) coefficients
 Right-hand sides (RHS)

129
Review of Wyndor Glass
Example
Product Mix LP. Wyndor makes doors and windows. They
have three plants. A batch of doors requires 1 hour at Plant 1
plus 3 hours at Plant 3. A batch of windows requires 2 hours at
Plant 2 plus 2 hours at Plant 3. Plant 1 is available for 4 hours
per week, Plant 2 for 12 hours per week, and Plant 3 for 18
hours per week. The profit per door batch is $3000 and the
profit per window batch is $5000. What should Wyndor
manufacture to maximize profits?
Max Z = 3x1 + 5x2
s.t.
4
+ 2x2  12
3x1 + 2x2  18
x1, x2  0
1x1
profits (in thousands of $)
Plant 1
Plant 2
Plant 3
non-negativity
130
Wyndor Glass Example
How much can c1 or c2 change
without changing the optimal
solution?
Max Z = c1x1 + c2x2
s.t.
4
+ 2x2  12
3x1 + 2x2  18
x1, x2  0
1x1
profits (in thousands of $)
Plant 1
Plant 2
Plant 3
non-negativity
131
Allowable Range to Stay
Optimal

Allowable range to stay optimal is the
range of values for cj over which the
current optimal solution remains optimal,
assuming no change in the other
coefficients.
132
Graphical Sensitivity Analysis
x2
(0,9)
(0,6)
(2,6)
(4,3)
(0,0)
(4, 0)
(6, 0)
x1
Allowable range to stay optimal for c1: 0  c1  7.5
133
Graphical Sensitivity Analysis
x2
(0,9)
(0,6)
(2,6)
(4,3)
(0,0)
(4, 0)
(6, 0)
x1
134
Ranges of Optimality
Objective function coefficients
 Right-hand sides

135
Wyndor Glass Example
How much can the RHS’s
change so that the current optimal
CPF solution is still feasible?
Max Z = 3x1 + 5x2
s.t.
 b1
+ 2x2  b2
3x1 + 2x2  b3
x1, x2  0
1x1
profits (in thousands of $)
Plant 1
Plant 2
Plant 3
non-negativity
136
Allowable Range to Stay
Feasible

Allowable range to stay feasible is the
range of values for bi over which the
current optimal CPF solution remains
feasible, assuming no change in the other
right-hand sides.
137
Graphical Sensitivity Analysis
Max Z = 3x1 + 5x2
s.t.
x2
(0,9)
(0,6)
(0,0)
(4, 0)
(6, 0)
4
+ 2x2  12
3x1 + 2x2  18
x1, x2  0
1x1
x1
138
Graphical Sensitivity Analysis
x2
(0,9)
(0,6)
(0,0)
(4, 0)
(6, 0)
x1
139
Graphical Sensitivity Analysis
x2
(0,9)
(0,6)
(0,0)
X
(4, 0)
(6, 0)
x1
Allowable range to stay feasible for b1 is 2  b1
140
Graphical Sensitivity Analysis
Max Z = 3x1 + 5x2
s.t.
4
+ 2x2  12
3x1 + 2x2  18
x1, x2  0
1x1
x2
(0,9)
(0,6)
(0,0)
(4, 0)
(6, 0)
x1
141
Graphical Sensitivity Analysis
x2
(0,9)
X
(0,0)
(0,6)
(4, 0)
(6, 0)
x1
142
Graphical Sensitivity Analysis
Max Z = 3x1 + 5x2
s.t.
x2
(0,9)
(0,6)
(0,0)
X
(4, 0)
(6, 0)
4
+ 2x2  12
3x1 + 2x2  b3
x1, x2  0
1x1
x1
Allowable range to stay feasible for b3 is 12  b3 24
143
Shadow Prices
144
Wyndor Glass Example
How much is additional
RHS resource worth to us?
Max Z = 3x1 + 5x2
s.t.
4
+ 2x2  12
3x1 + 2x2  18
x1, x2  0
1x1
profits (in thousands of $)
Plant 1
Plant 2
Plant 3
non-negativity
145
Graphical Solution
Max Z = 3x1 + 5x2 (Z*=36)
x2
s.t.
(0,9)
(0,6)
(2,6)
4
+ 2x2  12
3x1 + 2x2  18
x1, x2  0
1x1
(4,3)
(0,0)
(4, 0)
(6, 0)
x1
146
Resource Shadow Price
The shadow price of resource i (bi) is the
rate at which Z could be improved by
slightly increasing the amount of this
resource
 Shadow price = 0 for any slack constraint.

147
Algebraic Solution

The simplex method identifies the shadow price by the
coefficients of the slack variables in row 0 of the final
simplex tableau.
Final Simplex Tableau
Shadow price of
b 1, b 2, b 3
Z
s1
x1
0
0
x2
0
0
s1
0
1
s2
3/2
1/3
s3
1
-1/3
RHS
36
2
x2
x1
0
1
1
0
0
0
1/2
-1/3
0
1/3
6
2
148
LP Duality
149
Dual LP Example
Primal Problem
Max Z = 3x1 + 5x2
s.t.
1x1 + 0x2  4
0x1 + 2x2  12
3x1 + 2x2  18
x1 , x2  0
x1* = 2
x2* = 6
Z* = 36
Dual Problem
Min W = 4y1 + 12y2 + 18y3
s.t.
1y1 + 0y2 + 3y3  3
0y1 + 2y2 + 2y3  5
y1 , y2, y3  0
y1* = 0
y2* = 3/2
y3* = 1
W* = 36
150
Dual Interpretation
Maximize
Profits
Minimize Total
Implicit Price
Max Z = 3x1 + 5x2
s.t.
1x1 + 0x2  4
0x1 + 2x2  12
3x1 + 2x2  18
x1 , x2  0
Resources
Quantities
Owner of resources
Min W = 4y1 + 12y2 + 18y3
s.t.
1y1 + 0y2 + 3y3  3
0y1 + 2y2 + 2y3  5
y1 , y2, y3  0
Value
Shadow
Prices
Buyer of resources
151
Weak Duality
If x is a feasible solution for the primal
problem and y a feasible solution for the
dual, then Z  W
 That is, the value of the dual is an upper
bound on the primal problem

152
Strong Duality
If Z* is the optimal value for the primal
problem and W* the optimal value for the
dual, then Z* = W*
 That is, the optimal value of the primal and
the optimal value of the dual are equal

153
Example
Consider the following problem
Max Z=2x1+7x2+4x3
s.t.
x1 + 2x2 + x3  10
3x1 + 3x2 + 2x3  10
and
x10, x2 0, x3 0.
(a) Construct the dual problem for this primal
problem.
(b) Use the dual problem to demonstrate that the
optimal value of Z for the primal problem cannot
exceed 25.
154
Example
Prove that for any linear programming
problem in standard form if the primal
problem has an unbounded feasible
region that permits increasing Z
indefinitely, then the dual problem has no
feasible solutions.
155
Example
Consider the following LP
Max Z=-x1+5x2
s.t.
x1 + 2x2  0.5
-x1 + 3x2  0.5
and
x10, x2 0.
What is the missed value at the following Row 0 of the
optimal tableau?
Z
x1
0
x2
0
s1
0.4
s2
1.4
RHS
?
156
Why do we care?

Dual problem may be easier to solve








Number of constraints affects computation far more than
number of variables for simplex method
Primal has 1000 constraints, 100 variables
Dual has 100 constraints, 1000 variables
Dual is easier to solve
Evaluate a primal solution using dual feasibility
Sensitivity Analysis
Weak duality often used in solving integer
programs (branch and bound)
Economic interpretation and insight
157
Chapter 8
Transportation and Assignment
Problems
158
Previous LP Applications
Product Mix
 Investment
 Diet/Nutrition

159
Two Additional Important LP
Applications
 Transportation
Problems
Shipping Planning
 Production Scheduling
 Water Distribution

 Assignment
Problems
 Suggested readings: 8.1&8.3
160
Transportation Problem
161
Transportation Problem
Example
High Plains Electronics manufactures MP3 players at
three separate overseas plants. It ships finished
products to three US distribution centers. Shipping
costs (per unit) are shown below, as are the
production capacities of each plant and the demand
from each distribution center (units per week):
Supply = Demand
Plant
Singapore
Spain
Argentina
Demand
Distribution Center
LA
Boston
Denver
4
10
6
8
16
6
14
18
10
200
300
200
Capacity
100
300
300
700
162
Transportation Problem
Defined
Minimize total “shipping” cost (or
maximize profit) of products from source to
destinations
 Supply equals demand
 Distribution quantities often have integer
values
 Integrality assured: all basic solutions
(including optimal solution) have integer
value when supply/demand are integer
 Linear costs assumed

163
General LP Form
m
n
min/max  cij xij
Optimize some objective
i 1 j 1
s.t.
n
x
j 1
ij
m
x
i 1
ij
 si
Supply constraints
 dj
Demand constraints
xij  0
Non-negativity constraints
164
General LP Form
Objective function
Supply constraints
Demand constraints
165
Prohibited Routes
What if some routes are infeasible or
prohibited?
 Create allocation cost that is so large that it
will be quickly forced leaving from the
basis – “Big M”
 Example …

166
Supply  Demand
What if supply does not equal demand?
 Feasible solutions exist only if total supply
equals total demand

m
n
s  d
i 1
i
j 1
j
Create “dummy” source or destination
Example …
167
Check Processing Example
(Dummy Destination)
A bank has two sites at which checks are processed. Site 1 can
process 10,000 checks per day, and site 2 can process 6000 checks
per day. The bank processes 3 types of checks: vendor, salary, and
personal. The processing cost per check depends on the site (see
table). Each day, 5000 checks of each type must be processed.
Formulate a transportation problem to minimize the daily cost of
processing checks.
Site
Vendor
checks
Salary
checks
Personal
checks
1
5 cents
4 cents
2 cents
2
3 cents
4 cents
5 cents
168
Production Scheduling
Example
The NORTHERN AIRPLANE COMPANY builds commercial
airplanes. The production of the jet engines must be scheduled
for the next 4 months. The unit storage cost is 0.015 million
dollars per month. The demand, supply capacity, and
production costs are listed in the table. The production
manager wants a schedule for the number of engines to be
produced in each of the 4 months to minimize the total
production and storage costs.
Month
Scheduled
Installation
Maximum
Production
Unit Cost of Unit Cost of
Production
Storage
1
10
25
1.08
0.015
2
15
35
1.11
0.015
3
25
30
1.10
0.015
4
20
10
1.13
169
Water Distribution Example
(Dummy Source)
Two reservoirs are available to supply the water needs of three cities.
Each reservoir can supply 50 million gallons of water per day. Each
city would like to receive 40 million gallons per day. For each million
gallons per day of unmet demand, there is a penalty. At city 1, the
penalty is $20; at city 2, the penalty is $22; and at city 3, the penalty
is $23. The cost of transporting 1 million gallons of water from each
reservoir to each city are shown in the table. Formulate a
transportation problem that can be used to minimize the sum of
shortage and transport costs.
City 1
City 2
City 3
Reservoir 1
$7
$8
$10
Reservoir 2
$9
$7
$8
170


Transportation Problem
Solutions
Transportation problem is a special type of linear
programming problem. So it can be solved by
simplex method
A streamlined procedure (the transportation
simplex method) is available to achieve
tremendous computational savings by exploiting
the special structure of transportation problems.
171
Assignment Problem
172
Assignment Problem Example
MACHINECO has 4 machines and 4 jobs to be completed. Each
machine must be assigned to complete one job. The time required to
set up each machine for completing each job is shown in the table.
MACHINECO wants to minimize the total setup time needed to
complete the 4 jobs.
Job 1
Job 2
Job 3
Job 4
Machine 1
14
5
8
7
Machine 2
2
12
6
5
Machine 3
7
8
3
9
Machine 4
2
4
6
10
173
General LP Form
n
n
min  cij xij
Optimize some objective
i 1 j 1
s.t.
n
x
j 1
ij
n
x
i 1
ij
1
Resource constraints
1
Task constraints
0
xij  
1
If i is not assigned to j
If i is assigned to j
174
Assignment Problem








The assignment problem is a special type of
transportation problem
Assign discrete resources (people, machines,
vehicles, …) to tasks (jobs, routes, …)
Number of resources and tasks are equal
Each resource is assigned to one task
Each task is assigned to one resource
Binary solutions assured
Linear costs assumed
Highly degenerate (many zero basic variables)
175
Assignment Problem
Extensions

# resources unequal to #tasks


Use dummy resources or tasks
Prohibited assignments

Use large allocation cost – “Big M”
176
Another Assignment Problem
Example
A company has decided to assign 3 new machines to 4 locations. The
estimated cost in dollars per hour of materials handling involving each
of the machines is given in the table for the respective locations.
Location 2 is not suitable for machine 2.
Location 1
Location 2
Location 3
Location 4
Machine 1
13
16
12
11
Machine 2
15
--
13
20
Machine 3
5
7
10
6
177
Assignment Solutions

Could try complete enumeration



n! assignments possible
n =10 means 10!=3.6 million possible assignments
Linear Programming solution



Small problem can be solved by the general simplex
method
Applying transportation simplex method is a relatively
fast way
Specialized algorithms available, which are in
preference to the transportation simplex method
178
Ch. 12
Integer Programming
179
Outline

Integer Programs




General Integer
Models
0-1 (Binary) Models
Mixed Integer Models
IP Examples

IP Solution
Techniques



Branch and Bound
Application articles
Suggested Readings:
11.1-11.7
180
Integer Programming
An integer programming problem (IP) is
an LP in which some or all of the variables
are required to be integers.
 IP is generally much more difficult to solve
than LP due to

Exponential growth of number of solutions
 Property of simplex method is invalid for IP

181
Max Z = 33x1 + 12x2
s.t.
–x1 + 2x2  4
5x1 + 2x2  16
2x1 – x2  4
x1, x2  0 and integer
Optimalx2IP Solution
x1 = 2
x32 = 3
ZIP* = 102
2
Rounded LP Solution
x1 =
12
x2 = 1
ZRLP* = 78
1
An IP Example
Optimal LP Solution
x1 = 8/3
x2 = 4/3
ZLP* = 104
Feasible
and optimal
Not feasible,
superoptimal
Feasible,
Not optimal
x1
2
182
Bounds on IP’s
For a max (min) IP
 The LP obtained by omitting all integer
constraints on variables is called the LP
relaxation of the IP
 The optimal value of the LP relaxation is an
upper-bound (lower-bound) to the IP
 If all variables in the optimal solution of LP
relaxation are integers, then the optimal
solutions of LP and IP are the same
183
General IP Example
Pawtucket University is planning to buy new copier
machines for its library. Two different models are
considered: Model A and B. Model A can handle
20,000 copies a day, and costs $6,000. Model B can
handle 10,000 copies a day, but costs only $4,000. At
least six copiers are needed. And at least one of them
is model A. The copiers need to be able to handle a
capacity of at least 75,000 copies with minimum cost.
184
Binary IP Problems
General IP problems important,
but not compelling
 Integer Programming of greater importance
is to model “yes-no” decisions
 “BIP” models

185
Binary IP Example
The CALIFORNIA MFG. COMPANY is considering expansion by building
a new factory in either Los Angeles or San Francisco, or both cities. It
also is considering buiding at most one new warehouse, but the choice
of location is restricted to a city where a new factory is being built.
The net present value and the capital required for each alternative is
in the table. The total capital available is $10 million. The objective is
to maximize the total net present value.
Net Present
Value
Capital
Required
Factory in LA
$9 million
$6 million
Factory in SF
$5 million
$3 million
Warehouse in LA
$6 million
$5 million
Warehouse in SF
$4 million
$2 million
186
Mixed Integer Programs
“MIP”
 Both integer and continuous variables

max z = 3x1 + 2x2
s.t. x1 +x2 6
x1, x2  0, x1 integer
187
Other Integer Programming Examples
188
Set-Covering Problem
There are six cities (cities 1-6) in Kilroy County. The county must
determine where to build fire stations. The county wants to build the
minimum number of fire stations needed to ensure that at least one
fire station is within 15 minutes of each city. The times required to
drive between the cities are shown in the table. Formulate a BIP to
tell Kilroy how many fire stations should be built and where they
should be located.
1
2
3
4
5
6
1
0
10
20
30
30
20
2
10
0
25
35
20
10
3
20
25
0
15
30
20
4
30
35
15
0
15
25
5
30
20
30
15
0
14
6
20
10
20
25
14
0
189
Either-Or Constraints*
Dorian Auto is considering manufacturing three types of autos:
compact, midsize, and large. The resources required for, and the profits
yielded by, each type of car are shown in the table. At present, 6000
tons of steel and 60,000 hours of labor are available. For production of
a type of car to be economically feasible, at least 1000 cars of that type
must be produced. Formulate an IP to maximize Dorian’s profit.
Compact
Midsize
Large
Steel required (tons)
1.5
3
5
Labor Required (hrs)
30
25
40
Profit (k$)
2
3
4
190
Summary of IP Formulation

Mutually exclusive alternatives



Contingent decisions


At most one of x1, x2, …, xn can be equal to 1: x1+x2+…xn1
Exactly one of x1, x2, …, xn must be equal to 1: x1+x2+…xn=1
x1 can be equal to 1 only if x2 is equal to 1: x1x2
Set-Covering Problem

For member i of set 1, let Si be the acceptable members of set 2 used
to “cover” member i
x
jSi

j
1
Either-or constraint*

Use auxiliary binary variable and big M
191
Solving Integer Programs
192
BIP Branch and Bound
Example
The CALIFORNIA MFG. COMPANY is considering expansion by buiding
a new factory in either Los Angeles or San Francisco, or both cities. It
also is considering buiding at most one new warehouse, but the choice
of location is restricted to a city where a new factory is being built.
The net present value and the capital required for each alternative is
in the table. The total capital available is $10 million. The objective is
to maximize the total net present value.
Net Present
Value
Capital
Required
Factory in LA
$9 million
$6 million
Factory in SF
$5 million
$3 million
Warehouse in LA
$6 million
$5 million
Warehouse in SF
$4 million
$2 million
193
B&B Solution
Solution tree:
x1 = 0
1
All , B=16,
X=(5/6,1,0,1)
x1 =1
Z*= -
3
B=16,
X=(1, 0.8, 0, 0.8)
2
B=9, X=(0,1,0,1)
Z*= 9
Fathomed
x2 = 0
x2 =1
4
B=13,
X=(1, 0, 0.8, 0)
5
B=16,
X=(1,1,0,0.5)
Fathomed
x3 = 0
x3 = 1
6
B=16
X=(1,1,0, 0.5)
x4 = 0
8
X=(1,1,0,0)
Fathomed
Z*= 14
7
Infeasible
Fathomed
x4 = 1
9
Infeasible
Fathomed
194
Summary of BIP Branch and
Bound (Maximization Prob.)


For minimization problem: first convert to a maximization
problem by multiplying -1 to the objective fn (remember
to change the sign back for the optimal solution)
Initialization



Set initial incumbent (Z*) = -
Apply bounding, fathoming, and optimality test to the whole
problem
Iteration




Branching: branch remaining subproblems created most recently
(break ties by selecting larger bound ) by fixing next variable at 0
and 1. Choose the first one in the natural ordering to be the
branching variable.
Bounding: the optimal Z value of LP relaxation gives the bound.
Rounding down bound if all obj. fn. coefficients are integers.
Fathoming: Apply the three fathoming tests.
Optimality test: Stop when there are no remaining subproblems;
the current incumbent is optimal.
195
Summary of Fathoming Tests
Test 1: Its boundZ*
 Test 2: Its LP relaxation has no feasible
solution
 Test 3: The optimal solution for its LP
relaxation is integer. If this solution is
better than the incumbent, it becomes the
new incumbent, and test 1 is reapplied to
all unfathomed subproblems.

196
Another BIP Branch&Bound
Example
Max Z = 4x1 +9x2+6x3
s.t.
5x1+8x2+6x3  12
xi =0 or 1 (i=1,2,3)
197
Review of Mixed Integer
Programs
“MIP”
 Both integer and continuous variables

max z = 3x1 + 2x2
s.t. x1 +x2 6
x1, x2  0, x1 integer
198
MIP Branch-And-Bound

Changes from BIP branch-and-bound:




Select branching variable only from the integerrestricted variables that have a noninteger value in the
optimal solution for the current LP relaxation
Branching based on
xj[xj*] and xj[xj*]+1
xj—branching variable, [xj*]=greatest integer xj*
Never rounding down Z for MIP (due to possible noninteger decision variables)
For fathoming test 3, only check if integer-restricted
variables are integer
199
Summary of MIP Branch and
Bound (Maximization Prob.)

Initialization



Set Z* = -
Apply bounding, fathoming, and optimality test to the whole
problem
Iteration




Branching: branch remaining subproblems created most recently
(break ties by selecting larger bound ). Choose the first one as the
branching variable among the integer-restricted variables with
noninteger value in the optimal solution. Branching based on
xj[xj*] and xj[xj*]+1.
Bounding: the optimal Z value of LP relaxation gives the bound.
Fathoming: Apply the three fathoming tests.
Optimality test: Stop when there are no remaining subproblems;
the current incumbent is optimal.
200
Summary of Fathoming Tests
Test 1: Its boundZ*
 Test 2: Its LP relaxation has no feasible
solution
 Test 3: The optimal solution for its LP
relaxation has integer values for the
integer-restricted variables. If this solution
is better than the incumbent, it becomes the
new incumbent, and test 1 is reapplied to
all unfathomed subproblems.

201
MIP Branch & Bound
Example
From H&L p. 518:
Max Z = 4x1 - 2x2 + 7x3 - x4
s.t.
x1
+5x3
 10
x1 + x2 -x3
1
6x1 -5x2
0
-x1
+2x3 -2x4  3
and
xi 0, (i=1,2,3,4)
xi is an integer, for i=1,2,3
202
Suggested Reading

Grandine, “Assigning Season Tickets Fairly,”
Interfaces 28:4, Jul/Aug 1998
How to assign pooled season baseball tickets
fairly among pool participants
 Allows personal restrictions and preferences
 Results considered more “fair” than intuitive or
simplistic assignment methods

203
Suggested Reading

Bertsimas, Darnell, Soucy, “Portfolio Construction
through Mixed Integer Programming.., Interfaces
29:1, 1999.


Construct investment portfolios to meet financial goals
Outcomes
1.
2.
3.
4.
5.
Keep existing clients
Reduced stock names by 40-60%
Reduced annual trading costs by $4 million
Improved trading process
Better market analysis
204
Ch. 12
Nonlinear Programming
205
Outline

Nonlinear Programming
Basic concepts
 Unconstrained optimization
 Constrained optimization
 Nonlinear examples


Suggested Reading: 12.1, 12.2, 12.4
206
A Standard Form of NLP
Problem
max f(x1, …, xn)
s. t.
gi(x1, …, xn)  bi for i=1, 2, …, m.
x1, …, xn  0
where f and/or gi are given nonlinear functions.
207
NLP Example (1)
It costs a company c dollars per unit to
manufacture a product. If the company
charges p dollars per unit for the product,
customers demand D(p) units. To maximize
profits, what price should the firm charge?
208
NLP Example (2)
If K units of capital and L units of labor are
used, a company can produce KL units of
manufactured good. Capital can be
purchased at $4/unit and labor can be
purchased at $1/unit. A total of $8 is
available to purchase capital and labor. How
can the firm maximize the quantity of the
good that can be manufactured?
209
NLP Example (3)
Truckco is trying to determine where they should locate a single
warehouse. The positions in the x-y plane (in miles) of their four
customers and the number of shipments made annually to each
customer are given in the table. Truckco wants to locate the
warehouse to minimize the total distance trucks must travel
annually from the warehouse to the four customers.
Customer
X
Y
# of Shipments
1
5
10
200
2
10
5
150
3
0
12
200
4
12
0
300
210
Terminology
Nonlinear
 Convex and concave functions
 Convex set
 Global optimum
 Local optima

211
Example of Nonlinear
Functions
Functions with terms that are more
complicated than a single variable
multiplied by a coefficient
 f(x) = 10x2
 f(x,y) = 8xy
 f(x) = exp(x)
 f(x,y) = (ln(xy))3/2
 f(x,y,z) = 10x + 20y + 30z
Linear!

212
Concave and Convex
Functions
d2 f
 0, for all x
2
dx
f(x)
Concave
Any point on a
line segment
connecting any
two points on f(x)
is  f(x)
d2 f
 0, for all x
2
dx
Any point on a
line segment
connecting any
two points on f(x)
is  f(x)
Convex
x
Sum of concave (convex) functions are concave (convex)
213
Concave or Convex Function?
f(x)
Concave
Neither
Convex
Both !
x
214
Convex Set

Convex set: for each pair of point in the set, the
entire line segment joining these two points is
also in the set.
Convex
Convex
Nonconvex
Nonconvex
Nonconvex
215
Global Optimum

Point that is maximum (minimum) over the
entire range of a function
Global Optimum
216
Local Optimum

Point that is maximum (minimum) over a
segment of the range of a function
Local optimum
Global optimum
217
Local or Global Optimum ?
Local
Minimum
Local
Maximum
218
Local Optimum
If f’(x0)=0 and f”(x0)<0, then x0 is a local
maximum. If f’(x0)=0 and f”(x0)>0, then
x0 is a local minimum.
219
Examples
Show f(x)=x1/2 is a concave function for
x>0.
 Is f(x)=x3-x2 convex or concave?
 Find the local maxima and local minima
of f(x)=-x+x2+x3

220
Gradient Search Methods
Looking for (global) optimum
 Dependent on starting point!


Excel uses gradient search
Global
Optimum
Local
Optimum
221
Optimality Results
For NLP problem with no constraints, if the objective
function is concave (convex)
local maximum (minimum)  global maximum (minimum)
 For NLP problem with constraints, if the objective
function is concave (convex) and the feasible region is a
convex set
local maximum (minimum)  global maximum (minimum)
 The feasible region for any LP problem is a convex set
 Under standard form, if gi(x1, …, xn) are convex
functions, for all i, the feasible region for a NLP is a
convex set.

222
Optimality Examples
max f(x) = 12x - 3x4 - 2x6
max f(x,y) = 5x - x2 + 8y - 2y2
s. t.
3x + 2y  6
x2+y2  4
x  0, y  0
223
Ch. 9
Network Optimization Models
224
Topics Covered
Math Programming






Intro to LP
Solving LP’s
LP Sensitivity
LP Applications
Integer Programming
Nonlinear programming
Other Models



Network Theory
Queuing Theory
Markov Chains
225
Outline
Terminology
 Shortest-path problem
 Minimum spanning tree problem
 Maximum flow problem
 Minimum cost flow problem
 Suggested readings: 9.1—9.6

226
Terminology
arc (link)
nodes
directed arc
connected network
unconnected
cycle (directed)
spanning tree:
• connected network
• no cycles
227
Shortest Path Problem
Undirected and connected network
 Find the shortest (distance, cost, time) route
from one node (origin) to another node
(destination) given a network with known
arc penalties (distance, cost, time)
 Solve using shortest-path algorithm

228
Shortest Path Example
The road system of SEERVADA PARK is shown below.
Location O is the entrance. Other letters are ranger
stations. The numbers are distances in miles. Station T
has a scenic wonder. Determine which route from the
entrance to station T has the smallest total distance.
A
2
5
Entrance O
7
2
5
4
B
1
4
C
3
4
T
D
1
7
E
229
Shortest Path Algorithm
1
A
2
2
5
O
5
4
B
1
4
2
4
2
7
2
5
O
B
1
4
C
A
2
5
O
7
2
4
B
1
4
C
5
5
4
D
3
4
T
1
7
E
T
Unsolved nodes
D
3
4
4
7
B
7
1
E
4
1
4
E
4
3
2
5
O
7
1
2
T
D
A
T
D
3
C
5
5
4
1
C
A
5
4
B
4
E
7
2
5
O
7
1
2
A
T
D
3
C
3
7
1
E
7
Solved nodes
230
Min Spanning Tree Problem
Undirected and connected network
 Minimize (cost, distance) of connecting all
nodes in the network (find a spanning tree
with minimal distance)
 Solve using Minimum Spanning Tree
Algorithm

231
Min Spanning Tree Example
In SEERVADA PARK, telephone lines must be installed
under the roads. Lines will be installed under JUST
enough roads to provide connection between EVERY
pair of stations. Where should the lines be laid with a
MINIMUM total number of miles of line installed?
A
2
5
O
7
2
5
4
B
1
4
C
3
4
T
D
1
7
E
232
Spanning Tree
A
2
2
5
O
7
4
B
1
4
5
D
3
C
1
7
A
T
2
5
O
D
3
4
7
1
E
A
2
2
5
B
1
4
T
Not a spanning tree: cycles
A
O
4
B
C
4
Not a spanning tree: unconnected
5
1
4
E
7
2
7
5
4
C
1
E
4
Spanning tree: distance=24, not
optimal


2
2
5
O
D
3
T
7
B
1
4
C
7
5
4
T
D
3
1
7
E
4
Spanning tree: distance=20, not
optimal
any spanning tree has (#nodes – 1) links
Find the spanning tree with minimal distance
233
Minimum Spanning Tree
Algorithm
Starting from O:
1
A
2
5
O
7
2
2
4
C
4
B
1
6 5
4 3
3
4
T
D
1 5
7
E
Minimal distance = 14
Algorithm:
 Start from any node, connect it to the nearest node
 Repeat connecting to the nearest unconnected node
until all nodes are connected
234
Maximum Flow Problem
Directed and connected network
 Find the maximum flow between a
“source” and “sink” (e.g., supply and
demand) through a network of branches
with known flow capacities
 Augmenting Path Algorithm

235
Maximum Flow Example
In SEERVADA PARK, trams are used to transport
sightseers from the entrance to station T. The number at
the base of each arrow gives the upper limit on the
number of tram trips. Determine how to route the various
tram trips to maximize the number of trips.
Scenic
Wonder
(sink)
T
A 3
1
Entrance 5 7
(source) O
4
2
C
4
B
4
9
D
5
1
E 6
236
Terminologies

Residual Network: a network showing remaining arc
remaining arc
remaining arc
capacities
capacity from OB capacity from BO
O 7


0
B
Add a flow of 5
from O to B:
O 2
5 B
Augmenting Path: a directed path from the source to the sink
in the residual network such that every arc on this path has
strictly positive remaining arc capacity.
Residual capacity of augmenting path: the minimum of the
remaining arc capacities on the augmenting path.
237
Augmenting Path Algorithm
Remove the arrows:
3
A
0
1
5
O
0 04
B 5
2
7
4
0
0 C
4
0
A
3
1
2
2
O
5 04
B 0
2
2
4
0
0 C



4
0 T
0
0
0
0 1
0
0
0 5
O
3
D
E 1
5 04
B 0
2
2
4
0
E 6
0
5 1
0
D
9
3
1
0 A
0 C
6
3 T
5
4
1
0
0
0
5 1
0
E 1
3
0
A
4
0
1
O
5 13
B 0
2
2
4
0
0 C
4
D
9
0 T
5
1
3
0
5 1
0
D
5
4 T
5
E 1
Identify an augmenting path
Add the flow of the residual capacity in this path.
Repeat until no augmenting path
238
4
Augmenting Path Algorithm
(cont.)
0
4 A
0
1
O
7 11
B 0
2
0
4
0
0 C
6
4
3
3
0
5 1
0
D
3
O
7 11
B 0
2
1 C
E 1
2
0
2 C



2
3
3
1
5 0
2
7 11
B 0
2
0
0
0
4 A
0
O
0
3
0
1
6 T
5 1
5
0
4 A
D
E 0
2
7 T
6 1
O
3
3
3
1
5 0
D
2
E 1
1
0
A
4
7
0
7 10
B 1
2
0
1
0
3 C
1
7 T
5
4
3
1
4 0
3
D
1
8 T
6
E 0
Identify an augmenting path
Add the flow of the residual capacity in this path.
Repeat until no augmenting path
239
Another Max Flow Example
An airline company must determine how many
connecting flights daily can be arranged from Juneau to
Dallas. Connecting flights must stop in Seattle and then
stop in Los Angeles or Denver. The number on each arc
is the limit of the number of daily flights between pairs
cities. Help the company to maximize the number of
connecting flights daily from Juneau to Dallas.
L
2
J
3
S
3
1
2
Da
De
240
Minimum Cost Flow Problem




A directed and connected network with at least one
supply node and at least one demand node.
A capacity (uij) is assigned to arc ij
The cost per unit flow through arc ij is cij
The net flow (bi) at node i is:




>0 for supply node
<0 for demand node
=0 for other nodes
The objective is to minimize the total cost of sending
available supply to satisfy given demand
241
Minimum Cost Flow Problem

Generalized network problem


Transportation, assignment, max flow, shortest
path, …
Formulated as linear program

Wide utility, flexible, general
Integer solution guaranteed
 Solved using Network Simplex Method



We will not review in this class
General LP formulation…
242
Min Cost Flow Example—
Distribution Network
Products are produced at two factories, A and B, and
shipped to two warehouses, D and E. C is a distribution
center.
c—cost per unit flow
b—net flow
u—arc capacity
[50]
A
2
10
B
[40]
[-30]
D
9
4
2
3
C
3
1
80
E
[-60]
243
Min Cost Flow Example—
Traffic Assignment
Each hour, an average of 900 cars enter the network at
node 1 and seek to travel to node 6. The numbers above
each arc are Max # of cars per hour (Time to traverse the
arc). Formulate an LP that minimizes the total time
required for all cars to travel from node 1 to node 6.
800(10)
1
2
600(30)
4
300(10)
100(70)
400(60)
600(30) 6
600(50)
3
400(60)
600(30)
5
244
Special Cases

Transportation problem:




Assignment problem:





Supply node: source
Demand node: destination
uij=
Same formulation as transportation problem, plus:
# supply nodes = # demand nodes
bi=1 for each supply node
bi=-1 for each demand node
Shortest-path problem:




Supply node: origin (supply =1)
Demand node: destination (demand =1)
Replace an undirected link with a pair of directed arcs in opposite
directions
uij=
245
A Network Model for Project
Management
246
Project Management
When to use:





Management complex projects
Many parallel tasks
Deadlines and milestones must be met
Difficult to know “what to do first”
Difficult to know when project is in trouble
247
Examples






Building a new airport
Designing a new computer product
Launching an advertising campaign
Construction projects of all types
Maintenance projects
Curriculum reviews
248
Critical Path Method (CPM)

Critical Path Method (CPM)
 Developed by DuPont (1950’s)
 Plan and control maintenance of chemical
plants
 Credited with reducing length of maintenance
shutdown by 40%

Suggested Readings: 22.1—22.3 (among
additional chapters on the CD-ROM)
249
Critical Path Method (cont.)





Graphical method of portraying relationship of
project activities
An activity is any discrete part or task of a project
which takes resources and time to complete
Activities exhibit precedence relations (some
must be completed before others can start)
Activities with their precedence relations form a
project network
Critical Path Method finds the longest path
through the resulting project network
250
Precedence Relations
Activity
Immediate Predecessor
Duration (days)
A
B
C
D
(None)
A
A
B, C
4
3
5
2
251
Simple Project Network
Activity “A”
proceeds “B”
B
“Activity on Node”
representation
D
A
C
Represent precedence
relations as “arcs”
Project Network
252
Symbol for Activities
Earliest
Start
Time
Latest
Start
Time
Activity
Name
ES
EF
LS
LF
Earliest
Finish
Time
Latest
Finish
Time
Activity
Duration
253
Finding the Critical Path
B
3
A
D
4
2
C
5
254
Finding the Critical Path
Start at
time t=0
0
B
3
A
D
4
2
C
5
255
Finding the Critical Path
B
0+4=4
0
A
4
3
D
4
2
C
5
256
Finding the Critical Path
4
B
3
0 A
D
4
4
2
4
C
5
257
Finding the Critical Path
4
B
7
3
0 A 4
D
4
2
4
C
9
5
258
Finding the Critical Path
4 B
3
7
0 A 4
?
4
4 C
D
2
9
5
259
Finding the Critical Path
4 B
3
7
0 A 4
9
4
D
2
4 C
9
5
260
Finding the Critical Path
The earliest the project
B
4 can7 complete is t =11
3
A
0
4
9
4
D
11
2
4 C 9
5
261
Finding the Critical Path
B
4
7
3
0 A 4
D 11
9
4
2 11
C
4
9
5
262
Finding the Critical Path
0 A 4
4 B 7
6 3 9
4
4 C 9
4 5 9
D 11
9
2 11
9
263
Finding the Critical Path
0 A 4
4
?
4 B 7
3
6 9
D 11
9
9 2 11
4 C 9
4 5 9
264
Finding the Critical Path
0 A 4
4 B 7
6 3 9
D 11
9
9 2 11
0 4 4
4 C 9
4 5 9
265
Finding the Critical Path
A
0
4
Define
Activity Slack:
B
4
7
S = LS-ES = LF-EF
6 3 9
D 11
9
9 2 11
0 4 4
4 C 9
4 5 9
266
Finding the Critical Path
B
4
7
S=0-0=4-4=0
0 A 4
6 3 9
S=9-7=2
S=11-11=0
D 11
9
9 2 11
0 4 4
C
4
9
4 5 9
S=9-9=0
267
Finding the Critical Path
S=0
A
0
4
Critical Path: Path with
activity slacks
4 B zero
7 S=2
S=0
6 3 9
D 11
9
9 2 11
0 4 4
4 C 9
4 5 9 S=0
268
Summary of CPM
Terminology






Critical Path: the chain of activities along which the delay
of any activity will delay the project
Earliest Start Time (ES): the earliest that an activity could
possibly start, given precedence relations
Latest Start Time (LS): the latest that an activity could
possibly start without delaying the project
Earliest Finish Time (EF): the earliest that an activity
could possibly finish
Latest Finish Time (LF): the latest that an activity could
possibly finish without delaying the project
Activity Slack: the amount of “play” in the timing of the
activity; slack = LST-EST = LFT-EFT
269
Example
Widgetco is about to introduce a new product (product 3).
One unit of product 3 is produced by assembling 1 unit of
product 1 and 1 unit of product 2. The production has the
following activities:
Activity
Predecessors
A. Train workers
none
B. Purchase raw material none
C. Produce product 1
A, B
D. Produce product 2
A, B
E. Test product 2
D
F. Assemble
products 1 & 2
C, E
Time
6 days
9
8
7
10
12
270