Transcript Motivating the dual problem

EE/Econ 458
Duality
J. McCalley
1
Our example (and HW) problem
max F  3 x1  5 x 2
F
s.t.
 x3
x1
 x4
2 x2
3 x1
 2 x2
 x5

4

12

18
 3 x1
 5 x2
 x3
x1
 x4
2 x2
3 x1
 2 x2
 x5

0

4

12

18
x1  0 , x 2  0 , x 3  0 , x 4  0 , x 5  0
After one iteration, we obtained the following Tableau:
Basic
Eq.
variable #
0
F
1
x3
2
x2
3
x5
F
1
0
0
0
x1
-3
1
0
3
Coefficients of
x2
x3
0
0
0
1
1
0
0
0
x4
2.5
0
0.5
-1
x5
0
0
0
1
Right
side
30
4
6
6
Add 5 ×
pivot row
Add -2 ×
pivot row
Optimality test indicates we need to do another iteration
(since coefficient of x1is negative in above tableau).
2
Our example (and HW) problem
In performing the second iteration, the only choice for the
entering variable is x1 (coefficients of all other variables in
the tableau are non-negative).
To determine the leaving variable, we identify the constraints
that most limit the increase in x1, as shown below.
Basic
Eq.
variable #
0
F
1
x3
2
x2
3
x5
F
1
0
0
0
x1
-3
1
0
3
Coefficients of
x2
x3
0
0
0
1
1
0
0
0
x4
2.5
0
0.5
-1
x5
0
0
0
1
Right
side
30
4
6
6
4
 4
1
6
 2
3
And so the constraint identifying the leaving variable is the
last one, since its ratio is smallest (2<4).
And so what is the leaving variable?
3
Our example (and HW) problem
The leaving variable is x5, since it is the variable in the last
constraint which gets pushed to 0 as x1 increases.
Basic
Eq.
variable #
0
F
1
x3
2
x2
3
x1
Pivot element
F
1
0
0
0
x1
-3
1
0
3
Coefficients of
x2
x3
0
0
0
1
1
0
0
0
x4
2.5
0
0.5
-1
x5
0
0
0
1
Right
side
30
4
6
6
Pivot row
Pivot column
Make the pivot element “1” by dividing the pivot row by 3.
Basic
Eq.
variable #
0
F
1
x3
2
x2
3
x1
F
1
0
0
0
x1
-3
1
0
1
Coefficients of
x2
x3
0
0
0
1
1
0
0
0
x4
2.5
0
0.5
-0.333
x5
0
0
0
0.333
Right
side
30
4
6
2
This is equivalent to dividing both sides of the following by 3.
3x1+0x2+0x3-x4+x5=6  (1/3)*[3x1+0x2+0x3-x4+x5] =6*(1/3)
 x1+0x2+0x3-0.3333x4+0.3333x5=2
Now what is the next step?
4
Our example (and HW) problem
Use the last equation to eliminate the -3 in the top equation.
B asic
E q.
variable #
F
x3
x2
x1
0
1
2
3
F
x1
1
0
0
0
-3
1
0
1
C oefficients of
x2
x3
0
0
1
0
0
1
0
0
x4
x5
R ight
side
2.5
0
0.5
-0.333
0
0
0
0.333
30
4
6
2
This is equivalent to multiplying the bottom equation by 3 and
adding it to the top equation:
3*[x1+0x2+0x3-0.3333x4+0.3333x5] =2*3
-3x1+0x2+0x3 -x4 +x5 =6
+{F+3x1+0x2+0x3+2.5x4+0x5=30}
-----------------------------------------F+0x1+0x2+0x3+1.5x4+x5=36
Basic
Eq.
variable #
0
F
1
x3
2
x2
3
x1
F
1
0
0
0
x1
0
1
0
1
Coefficients of
x2
x3
0
0
0
1
1
0
0
0
x4
x5
1.5
1
0
0
0.5
0
-0.333 0.333
Right
side
36
4
6
2
5
Our example (and HW) problem
Use the last equation to eliminate the 1 from the 2nd equation:
B asic
E q.
variable #
F
x3
x2
x1
0
1
2
3
F
x1
1
0
0
0
-3
1
0
1
C oefficients of
x2
x3
0
0
1
0
0
1
0
0
x4
x5
R ight
side
2.5
0
0.5
-0.333
0
0
0
0.333
30
4
6
2
This is equivalent to multiplying the bottom equation by -1 and
adding it to the 2nd equation:
-1*[x1+0x2+0x3-0.3333x4+0.3333x5] =2*-1
-x1+0x2+0x3 +0.3333x4 -0.3333x5 =-2
+{x1+0x2+1x3 +0x4
+0x5
=4}
-----------------------------------------------------0x1+0x2+1x3+0.3333x4-0.3333x5=2
Basic
Eq.
variable #
0
F
1
x3
2
x2
3
x1
F
1
0
0
0
x1
0
0
0
1
Coefficients of
x2
x3
0
0
0
1
1
0
0
0
x4
x5
1.5
1
0.333 -0.333
0.5
0
-0.333 0.333
Right
side
36
2
6
2
6
Introduction to duality
Assume the problem we have been working on is actually a
resource allocation problem: Problem (1)
max F  3 x1  5 x 2
s.t. x1
4
2 x 2  12
Resources
3 x1  2 x 2  18
x1  0 , x 2  0
Question: How to gain the most, in terms of the value of our optimized
objective function, by increasing one resource or another?
Problem (1)
Problem(2)
max F  3 x1  5 x 2
max F  3 x1  5 x 2
s.t. x1
4
s.t. x1
5
2 x 2  12
2 x 2  12
3 x1  2 x 2  18
3 x1  2 x 2  18
x1  0 , x 2  0
CPLEX yields F1*=36
x1  0 , x 2  0
CPLEX yields F2*=36
(increases by 0)
7
Introduction to duality
Problem (1)
Problem(3)
max F  3 x1  5 x 2
max F  3 x1  5 x 2
4
s.t. x1
4
s.t. x1
2 x 2  12
2 x 2  13
3 x1  2 x 2  18
3 x1  2 x 2  18
x1  0 , x 2  0
x1  0 , x 2  0
CPLEX yields
F1*=36
F3*=37.5
CPLEX yields
(increases by 1.5)
Problem (4)
max F  3 x1  5 x 2
max F  3 x1  5 x 2
4
s.t. x1
 b1
F *
 b2
F *
0
 1 .5
1
4
2 x 2  12
2 x 2  12
3 x1  2 x 2  18
3 x1  2 x 2  19
x1  0 , x 2  0
x1  0 , x 2  0
CPLEX yields F1*=36
F *
 b3
Problem (1)
s.t. x1
Summary
CPLEX yields F4*=37
(increases by 1.0)
8
Introduction to duality
F *
 b1
F *
 b2
F *
 b3
0
 1 .5
1
Assume F* represents my optimal profits. Assume
also that I have a little extra money to spend. Then
this information tells me my optimal objective will
improve most if I spend that money to increase a
unit of resource 2 (b2), less if I spend it to increase a
unit of resource 3 (b3), and it will do me no good at
all to spend that money to increase resource 1 (b1) .
(Important note: ΔF*/Δbi is how the optimal value of F changes with bi.)
Go back and inspect the Tableau for the solution to Problem 1.
Basic
Eq.
variable #
0
F
1
x3
2
x2
3
x1
F
1
0
0
0
x1
0
0
0
1
Coefficients of
x2
x3
0
0
0
1
1
0
0
0
x4
x5
1.5
1
0.333 -0.333
0.5
0
-0.333 0.333
Right
side
36
2
6
2
The coefficients of the slack variables in this final tableau are exactly
the same as the right-hand-sides of the above: 0, 1.5, and 1 !
9
Introduction to duality
This always happens for linear programs. Here is a general rule:
F *
 b1
F *
 b2
F *
 b3
0
 1 .5
1
The coefficients of the slack variables in the objective
function expression of the final tableau give the
improvement in the objective for a unit increase in
the right-hand-sides of the corresponding constraints.
Basic
Eq.
variable #
0
F
1
x3
2
x2
3
x1
F
1
0
0
0
x1
0
0
0
1
Coefficients of
x2
x3
0
0
0
1
1
0
0
0
x4
x5
1.5
1
0.333 -0.333
0.5
0
-0.333 0.333
Right
side
36
2
6
2
x3 is SV for constr 1.
x4 is SV for constr 2.
X5 is SV for constr 3.
What if your objective function was profits per hr, measured in $/hr?
And what if one of your equations expressed a constraint on MW
output for a certain generator? Units would be $/MW-hr.
Where have we seen this before?
10
Introduction to duality
F *
 b1
F *
 b2
F *
 b3
0
 1 .5
1
These slack variable coefficients are called dual variables,
dual prices, or shadow prices (implicit rather than explicit
values associated with obtaining additional resources). We
denote them as λi corresponding to the ith constraint.
Basic
Eq.
variable #
0
F
1
x3
2
x2
3
x1
F
1
0
0
0
x1
0
0
0
1
Coefficients of
x2
x3
0
0
0
1
1
0
0
0
x4
x5
1.5
1
0.333 -0.333
0.5
0
-0.333 0.333
Right
side
36
2
6
2
11
Introduction to duality
But we need to be a little careful…
F *
From the left and below, we see that λ3 =1, which we
0
 b1
now understand to mean that if we increase b3 by 1
F *
unit, our objective function will increase by 1, from
 1 .5
 b2
36 to 37, and this, as we have learned, is the case.
F *
 b3
1
Basic
Eq.
variable #
0
F
1
x3
2
x2
3
x1
F
1
0
0
0
x1
0
0
0
1
Coefficients of
x2
x3
0
0
0
1
1
0
0
0
x4
x5
1.5
1
0.333 -0.333
0.5
0
-0.333 0.333
Right
side
36
2
6
2
But what if we increase b3 by 6, to 24? Will we see an increase in F*
by 6 as well, to 42? Will increasing b3 to 26 see F* increase to 44?
Problem (1)
Problem (5)
Problem (6)
max F  3 x1  5 x 2
max F  3 x1  5 x 2
max F  3 x1  5 x 2
s.t. x1
4
s.t. x1
4
2 x 2  12
2 x 2  12
3 x1  2 x 2  18
3 x1  2 x 2  24
x1  0 , x 2  0
x1  0 , x 2  0
CPLEX yields F *=36 CPLEX yields F *=42
s.t. x1
4
2 x 2  12
3 x1  2 x 2  26
x1  0 , x 2  0
CPLEX yields F *=42
12
Introduction to duality
There are two observations to be made here:
F *
 b1
F *
 b2
F *
 b3
1. Changing some bi do not influence F*.
0
 1 .5
1
Only a subset of resources limit the objective.
You may have labor hours, trucks, & pipe fittings as
resources, each of which are individually constrained.
You have a large truck fleet and a whole warehouse of
pipe fittings, but you don’t have enough labor. And so
you increase labor until you hit the limit on pipe fitting
inventory. You then increase pipe fitting inventory, but
soon you hit the limit on trucks.
2. ΔF*/Δbi is not accurate if Δbi gets too large.
A better definition of the dual variables follows:
i 
F *
 bi
This says that the λi = ΔF*/Δbi only for incrementally small Δbi. In
an LP, λi = ΔF*/Δbi is no longer true when Δbi becomes so large
that its corresponding constraint is no longer binding.
13
Motivating the dual problem
Consider our now very familiar example problem (call it P):
max F  3 x1  5 x 2
s.t.
P
x1
4
2 x 2  12
3 x1  2 x 2  18
x1  0 , x 2  0
Let’s express linear combinations of multiples of the constraints,
where the constraint multipliers are denoted λi on the ith constraint.
We assume that λi≥0 to avoid having to change the inequality.
max F  3 x1  5 x 2
s.t.

( x1
 4 ) 1
(2 x 2  12 )  2
 ( 3 x1  2 x 2  18 )  3
( x1 1  3 x1  3 )  ( 2 x 2  2  2 x 2  3 )  4 1  12  2  18  3
max F  3 x1  5 x 2
s.t.
( x1

 4 ) 1
(2 x 2  12 )  2
 ( 3 x1  2 x 2  18 )  3
( 1  3  3 ) x1  ( 2  2  2  3 ) x 2  4 1  12  2  18  3
The composite inequality
14
Motivating the dual problem
Composite inequality: ( 1  3 3 ) x1  ( 2  2  2  3 ) x 2
 4 1  12  2  18  3
The left-hand-side of the composite inequality, being a linear
combination of our original inequalities, must hold at any feasible
solution (x1,x2), and in particular, at the optimal solution. That is, it is a
necessary condition for satisfying the inequalities from which it came.
It is not, however, a sufficient condition (see notes for example) unless
we constrain how we choose the λi.
The objective function.
F  3 x1  5 x 2
The composite inequality.
( 1  3  3 ) x1  ( 2  2  2  3 ) x 2  4 1  12  2  18  3
Let’s develop criteria for selecting λ1, λ2, λ3.
Concept 1: Make sure our choices of λ1, λ2, λ3 are such that each
coefficient of xi in the composite inequality is at least as great as
the corresponding coefficient in the objective function expression:
1  3  3  3
Any solution (x1,x2) results in a value F less than or
equal to the left-hand-side of the composite inequality.
2 2  23  5
15
Motivating the dual problem
Composite inequality: ( 1  3 3 ) x1  ( 2  2  2  3 ) x 2
 4 1  12  2  18  3
Concept 2: Because F is less than or equal to the left-hand side of the
composite inequality (by concept 1), and because the left-hand-side of
the composite inequality is less than or equal to the right hand side of
the composite inequality, we can also say that any solution (x1,x2) results
in a value F which is less than or equal to the right-hand-side of the
composite inequality.
Concept 3: Concept 2 implies the right-hand-side of the composite
inequality is an upper bound on the value that F may take. This is true
for any value of F, even the maximum value F*.
Δ
0
F*
Right-hand-side of
composite inequality
16
Motivating the dual problem
Composite inequality: ( 1  3 3 ) x1  ( 2  2  2  3 ) x 2
 4 1  12  2  18  3
Concept 4: Now choose λ1, λ2, λ3 to minimize the right-hand-side of the
composite inequality, subject to the constraints we imposed in choosing
the λi ‘s. 1  3  3  3
2 2  23  5
This creates a least upper bound to F*, i.e., it pushes right-hand-side of
the composite inequality as far left as possible, while guaranteeing righthand-side remains ≥ F* (due to enforcement of above constraints).
Δ
0
R ig h t-h an d -sid e o f
co m p o site in eq u ality
F*
Δ
0
F*
Δ
0
F*
17
Motivating the dual problem
Composite inequality: ( 1  3 3 ) x1  ( 2  2  2  3 ) x 2
 4 1  12  2  18  3
Concept 5: Given that the right-hand-side of the composite inequality is
an upper bound to F*, then finding its minimum, subject to constraints,
implies Δ=0. So we have a new problem to solve:
Problem D
min G  4  1  12  2  18  3
subject to
1  3  3  3
22  23  5
1  0 ,  2  0 ,  3  0
What does concept 5 tell us about the above problem, if we solve it?
The value of the obtained objective function, at the optimum, will be
the same as the value of the original objective function at its optimum,
i.e., F*=G*.
18
Motivating the dual problem
In other words, solving problem D is equivalent to solving problem P.
Problem
P
Problem
max F  3 x1  5 x 2
s.t.
x1
4
2 x 2  12
3 x1  2 x 2  18
x1  0 , x 2  0

  

P rimal P roblem
D
min G  4 1  12  2  18  3
subject to
1  3  3  3
22  23  5
1  0 ,  2  0 ,  3  0
        
Dual P roblem
Problem P is called the primal problem. Problem D is called the dual
problem. They are precisely equivalent. To show this, let’s use CPLEX to
solve Problem D.
19
Motivating the dual problem
The CPLEX code to do it is below (note I use y instead of λ in code):
Problem
D
min G  4 1  12  2  18  3
subject to
1  3  3  3
22  23  5
1  0 ,  2  0 ,  3  0
        
minimize
The solution gives
1 y1 + 12 y2 + 18 y3
G*=36
subject to
λ 1=0
1 y1 + 3 y3 >= 3
λ 2=1.5
2 y2 + 2 y3 >=5
λ 3=1
y1 >= 0
y2 >= 0 They are:
y3 >= 0
1. Coefficients of the SVs λ 4
end
Dual P roblem
Using CPLEX, following solution
of the above dual problem, I
typed the following command:
display solution dual –
CPLEX gave:
Constraint Name Dual Price
c1 What are these? 2.000000
c2
6.000000
, λ 5 of
the objective function
expression in last tableau of
dual problem (not the values of
the SVs). They are sensitivities.
2. Values of the DVs at the optimal
solution to the dual of this
problem – “dual to the dual”
which is the primal! So these
20
are the values x1 and x2.
Motivating the dual problem
The CPLEX code to do it is below (note I use y instead of λ in code):
Problem
D
min G  4 1  12  2  18  3
The solution gives
subject to
G*=36
1  3  3  3
λ 1=0
λ 2=1.5
22  23  5
λ 3=1
1  0 ,  2  0 ,  3  0
        
Problem D1
min G  4  1  12  2  18  3
subject to
1  3  3  3
22  23  4
1  0 ,  2  0 ,  3  0
The solution gives
G*=30
λ 1=0
λ 2=1
λ 3=1
Dual P roblem
Using CPLEX, following solution
of the above dual problem, I
typed the following command:
display solution dual –
CPLEX gave:
Constraint Name Dual Price
c1
2.000000
c2
6.000000
c1, c2 are righthand-sides of the
two constraints in
the dual
G *
 c1
G *
c2
2
6
21
Motivating the dual problem
There is a circular relationship here, which can be stated as:
The dual of the dual to a primal is the primal.
If you called Problem D our primal problem, and took its dual, you
would get our original primal problem P back, as illustrated below.
Problem D
Problem
min G  4  1  12  2  18  3
max F  3 x1  5 x 2
subject to
s.t.
P
4
x1
2 x 2  12
1  3  3  3
3 x1  2 x 2  18
22  23  5
1  0 ,  2  0 ,  3  0
        
x1  0 , x 2  0

  

Dual P roblem
Primal Problem
Problem
P
Problem
max F  3 x1  5 x 2
s.t.
x1
4
2 x 2  12
3 x1  2 x 2  18
x1  0 , x 2  0

  

P rimal P roblem
D
min G  4 1  12  2  18  3
subject to
1  3  3  3
22  23  5
1  0 ,  2  0 ,  3  0
        
Dual P roblem
22
Primal-dual relationships
1. Number of decision variables and constraints:
• # of dual decision variables = # of primal constraints.
• # of dual constraints = # of primal decision variables.
2. Coefficients of decision variables in dual objective
are right-hand-sides of primal constraints.
Problem
Problem
P
max F  3 x1  5 x 2
s.t.
x1
4
2 x 2  12
3 x1  2 x 2  18
x1  0 , x 2  0

  

P rimal P roblem
D
min G  4 1  12  2  18  3
subject to
1  3  3  3
 2 
2
 23  5
1  0 ,  2  0 ,  3  0
        
Dual P roblem
Problem
Problem
P
max F  3 x1  5 x 2
3. Coefficients of decision
variables in primal objective
are right-hand-sides of dual
constraints.
s.t.
x1
4
2 x 2  12
3 x1  2 x 2  18
x1  0 , x 2  0

  

P rimal P roblem
D
min G  4 1  12  2  18  3
subject to
1  3  3  3
 2 
2
 23  5
1  0 ,  2  0 ,  3  0
        
Dual P roblem
23
Primal-dual relationships
4. Coefficients of one variable across multiple primal constraints are
coefficients of multiple variables in one dual constraint.
Problem
Problem D
P
min G  4 1  12  2  18  3
max F  3 x1  5 x 2
s.t.
4
x1
2 x 2  12
3 x1  2 x 2  18
subject to

x1  0 , x 2  0

  

1
 3 3  3
2 2  23  5
1  0 ,  2  0 ,  3  0
        
Dual Problem
P rimal P roblem
Problem
Problem D
P
min G  4 1  12  2  18  3
max F  3 x1  5 x 2
s.t.
x1
4
2 x 2  12
3 x1  2 x 2  18
x1  0 , x 2  0

  

P rimal P roblem
subject to

1
 3 3  3
2 2  23  5
1  0 ,  2  0 ,  3  0
        
Dual Problem
24
Primal-dual relationships
5. Coefficients of one variable across multiple dual constraints are
coefficients of multiple variables in one primal constraint.
P roblem P
P ro b lem D
m ax F  3 x1  5 x 2
m in G  4 1  1 2  2  1 8  3
4
s.t. x1
su b ject to
2 x 2  12
3 x1  2 x 2  18
1

 3 3  3
2 2  2 3  5
1  0 ,  2  0 ,  3  0
x1  0 , x 2  0

  

        
D u a l P ro b lem
P rim al P ro b lem
P roblem P
P roblem D
P roblem P
P roblem D
m ax F  3 x1  5 x 2
m in G  4 1  1 2  2  1 8  3
m ax F  3 x1  5 x 2
m in G  4 1  1 2  2  1 8  3
subject to
s.t. x1
s.t. x1
4
2 x 2  12
3 x1  2 x 2  18
x1  0 , x 2  0

  

P rim al P roblem
1

4
 3 3  3
2 x 2  12
2 2  2 3  5
3 x1  2 x 2  18
1  0 ,  2  0 ,  3  0
        
D u a l P ro b lem
x1  0 , x 2  0

  

P rim al P roblem
subject to
1

 3 3  3
2 2  2 3  5
1  0 ,  2  0 ,  3  0
        
D u a l P ro b lem
6. If primal objective is maximization, dual objective is minimization
7. If primal constraints are ≤, dual constraints are ≥.
25
Obtaining the dual
We can immediately write down the dual given the primal.
Example: Let’s return to the example we used to illustrate use of
CPLEX in the notes called “Intro_CPLEX.”
max F  5 x1  4 x 2  3 x 3
Subject to:
2 x1  3 x 2  x 3  5
4 x 1  x 2  2 x 3  11
3 x1  4 x 2  2 x 3  8
x1  0 , x 2  0 , x 3  0
By inspection:
min G  5 1  11  2  8  3
Subject to:
2 1  4  2  3  3  5
3 1   2  4  3  4
1  2  2  2  3  3
1  0 ,  2  0 ,  3  0
Let’s check it using CPLEX….
(if we got it right above, they should have the same
objective function value at the optimal solution).
26
Obtaining the dual
Primal
maximize
5 x1 + 4 x2 + 3 x3
subject to
2 x1 + 3 x2 + x3 <= 5
4 x1 + x2 + 2x3 <= 11
3 x1 + 4 x2 + 2 x3 <= 8
x1 >= 0
x2 >= 0
x3 >= 0
end
The solution is
(x1,x2,x3)=(2,0,1), F*=13.
The primal SV coefficients,
i.e., the values of the dual
variables are:
(λ1, λ2, λ3)=(1,0,1)
Dual
minimize
5 y1 + 11 y2 + 8 y3
subject to
2 y1 + 4 y2 + 3 y3 >= 5
3 y1 + 1 y2 + 4 y3 >= 4
1 y1 + 2 y2 + 2 y3 >= 3
y1 >= 0
y2 >= 0
y3 >= 0
end
The solution is
(λ1, λ2, λ3)=(1,0,1), G*=13.
The dual SV coefficients,
i.e., the values of the dual
variables are:
(x1,x2,x3)=(2,0,1).
27
Viewing primal-dual relationship
Generalize the primal-dual problems:
max F  c1 x1  c 2 x 2  ...  c n x n
min G  b1  1  b 2  2  ...  b m  m
s .t .
s .t .
a11 x1  a12 x 2  ...  a1 n x n  b1
a 11  1  a 21  2  ...  a m 1  m  c 1
a 21 x1  a 22 x 2  ...  a 2 n x n  b 2
a 12  1  a 22  2  ...  a m 2  m  c 2






a m 1 x1  a m 2 x 2  ...  a m n x n  b m
a 1 n  1  a 2 n  2  ...  a m n  m  c n
x1 , x 2 ,..., x n  0
 1 ,  2 ,...,  n  0
28
Viewing primal-dual relationship
Primal Problem
λ1
x1
x2
…
xn
a11
a12
…
a1n
≤b1
a21
a22
…
a2n
≤b2
⁞
⁞
⁞
⁞
am1
am2
…
amn
c1
c2
≤bm
≤
≤
≤
λ2
λm
Coefficients of
Right side
Dual Problem

Right side
Coefficients for dual objective
function
Coefficients of
…
cn
Coefficients for
primal objective function
29
Viewing primal-dual relationship
Primal Problem
x1
x2
x3
2
3
1
≤5
4
1
2
≤11
3
4
2
≤8
λ2
5
≤
≤
≤
λ3
Coefficients of
Right side
Dual Problem
λ1

Right side
Coefficients for dual objective
function
Coefficients of
4
3
Coefficients for
primal objective function
30
The duality theorem
Duality theorem: If the primal problem has an optimal solution
x*, then the dual
problem has an optimal solution λ* such that

G (  *)  F ( x *)
But what if the primal does not have an optimal solution?
Then what happens in the dual?
To answer this, we must first consider what are the alternatives
for finding an optimal solution to the primal? There are two:
1.
The primal is unbounded.
2.
The primal is infeasible.
31
Unbounded vs. infeasibility
Infeasible
max F  3 x1  5 x 2 CPLEX> read ex.lp
s.t.
x1
4
2 x 2  12
3 x1  2 x 2  18
x2  7
x1  0 , x 2  0
Unbounded
max F  3 x1  5 x 2 CPLEX> read ex1.lp
Problem 'ex.lp' read.
s.t.
Read time = 0.00 sec.

CPLEX> primopt
Bound infeasibility column 'x2'.
Presolve time = 0.00 sec.
Presolve - Unbounded or infeasible.
Solution time = 0.00 sec.
CPLEX>
x1
 4
4 x1
 12
3 x1
 18
x1  0 , x 2  0
Problem 'ex1.lp' read.
Read time = 0.00 sec.
CPLEX> primopt
Dual infeasible due to empty column
'x2'.
Presolve time = 0.00 sec.
Presolve - Unbounded or infeasible.
Solution time = 0.00 sec.
CPLEX>
32
An unbounded primal
Let’s inspect the unbounded problem.
max F  3 x1  5 x 2
s.t.
x1
 4
4 x1
 12
3 x1

 18
x1  0 , x 2  0
Recall that the objective function for the dual establishes an
upper bound for the objective function of the primal, i.e.,. G (  ) 
F ( x)
for any sets of feasible solutions λ and x. If F(x) is unbounded,
then the only possibility for G(λ) is that it must be infeasible.
33
An unbounded primal
Write down the dual.
max F  3 x1  5 x 2
s.t.
x1
 4

min G  4 1  12  2  18  3
This constraint
4 x1
 12
cannot be satisfied,
1  4  2  3  3  3
3 x1
 18
therefore this
0 1  0  2  0  3  5
x1  0 , x 2  0
problem is
1  0 ,  2  0 ,  3  0
infeasible.
Likewise, we can show that if the dual is unbounded, the primal
must be infeasible.
s .t .
However, it is not necessarily true that if the primal (or dual) is
infeasible, then the dual (or primal) is unbounded. It is possible
for an infeasible primal to have an infeasible dual and vice-versa,
that is, both the primal and the dual may be both be infeasible.
Finally, if both primal and dual have feasible solutions, both have
34
optimal solutions.
Primal-Dual relations

DUAL
PRIMAL
Optimal
Infeasible
Unbounded
Optimal
Possible
Impossible
Impossible
Infeasible
Impossible
Possible
Possible
Unbounded
Impossible
Possible
Impossible
maximize
5 x1 + 4 x2 + 3 x3
subject to
2 x1 + 3 x2 + x3 <= 5
4 x1 + x2 + 2x3 <= 11
3 x1 + 4 x2 + 2 x3 <= 8
x1 >= 0
x2 >= 0
x3 >= 0
end
A comment on using CPLEX

CPLEX will name the slack variables
c1, c2, …, cm, cm+1,…, cm+n
where there are m constraints and n decision
variables. Therefore the first m slack variables
(c1, c2, …, cm) correspond to the explicit
inequality constraints, and the last n slack
variables (cm+1,…, cm+n) correspond to the
nonnegativity constraints on the decision
variables.
CPLEX> primopt
…
Primal simplex - Optimal:
Objective = 1.3000000000e+01
…
CPLEX> display solution variables Variable Name
Solution Value
x1
2.000000
x3
1.000000
All other variables in the range 1-3 are 0.
CPLEX> display solution slacks Constraint Name
Slack Value
slack c2
1.000000
slack c4
-2.000000
slack c6
-1.000000
All other slacks in the range 1-6 are 0.
CPLEX> display solution dual Constraint Name
Dual Price
c1
1.000000
c3
1.000000
All other dual prices in the range 1-6 are 0.
CPLEX> quit
maximize
5 x1 + 4 x2 + 3 x3
subject to
2 x1 + 3 x2 + x3 <= 5
4 x1 + x2 + 2x3 <= 11
3 x1 + 4 x2 + 2 x3 <= 8
x1 >= 0
x2 >= 0
x3 >= 0
end
A comment on using CPLEX

The values of the slack variables at the solution
were c1=0, c2=1, c3=0, c4=-2, c5=0, c6=-1.
Note that CPLEX does not print the values of
slack variables that are zero.
CPLEX> primopt
…
Primal simplex - Optimal:
Objective = 1.3000000000e+01
…
CPLEX> display solution variables Variable Name
Solution Value
x1
2.000000
x3
1.000000
All other variables in the range 1-3 are 0.
CPLEX> display solution slacks Constraint Name
Slack Value
slack c2
1.000000
slack c4
-2.000000
slack c6
-1.000000
All other slacks in the range 1-6 are 0.
CPLEX> display solution dual Constraint Name
Dual Price
c1
1.000000
c3
1.000000
All other dual prices in the range 1-6 are 0.
CPLEX> quit
maximize
5 x1 + 4 x2 + 3 x3
subject to
2 x1 + 3 x2 + x3 <= 5
4 x1 + x2 + 2x3 <= 11
3 x1 + 4 x2 + 2 x3 <= 8
x1 >= 0
x2 >= 0
x3 >= 0
end
A comment on using CPLEX

The fact that c1=0 and c3=0 indicates that the
first and third constraints are binding. That
c2=1 indicates the left-hand side of the second
constraint is less than the right-hand-side by 1.
Let’s check:
Constraint 1: 2 x1  3 x 2  x 3  5
2 ( 2 )  3( 0 )  1  5
Constraint 2:
4 x 1  x 2  2 x 3  11
4 ( 2 )  0  2 (1)  10
Constraint 3:
3 x1  4 x 2  2 x 3  8
3 ( 2 )  4 ( 0 )  2 (1)  8
CPLEX> primopt
…
Primal simplex - Optimal:
Objective = 1.3000000000e+01
…
CPLEX> display solution variables Variable Name
Solution Value
x1
2.000000
x3
1.000000
All other variables in the range 1-3 are 0.
CPLEX> display solution slacks Constraint Name
Slack Value
slack c2
1.000000
slack c4
-2.000000
slack c6
-1.000000
All other slacks in the range 1-6 are 0.
CPLEX> display solution dual Constraint Name
Dual Price
c1
1.000000
c3
1.000000
All other dual prices in the range 1-6 are 0.
CPLEX> quit
maximize
5 x1 + 4 x2 + 3 x3
subject to
2 x1 + 3 x2 + x3 <= 5
4 x1 + x2 + 2x3 <= 11
3 x1 + 4 x2 + 2 x3 <= 8
x1 >= 0
x2 >= 0
x3 >= 0
end
A comment on using CPLEX

The fact that c5=0 indicates that the second
inequality constraint is binding, i.e., x 2  0
which is consistent with the fact that x2=0.
CPLEX> primopt
…
Primal simplex - Optimal:
Objective = 1.3000000000e+01
…
CPLEX> display solution variables Variable Name
Solution Value
x1
2.000000
x3
1.000000
All other variables in the range 1-3 are 0.
CPLEX> display solution slacks Constraint Name
Slack Value
slack c2
1.000000
slack c4
-2.000000
slack c6
-1.000000
All other slacks in the range 1-6 are 0.
CPLEX> display solution dual Constraint Name
Dual Price
c1
1.000000
c3
1.000000
All other dual prices in the range 1-6 are 0.
CPLEX> quit
maximize
5 x1 + 4 x2 + 3 x3
subject to
2 x1 + 3 x2 + x3 <= 5
4 x1 + x2 + 2x3 <= 11
3 x1 + 4 x2 + 2 x3 <= 8
x1 >= 0
x2 >= 0
x3 >= 0
end
A comment on using CPLEX

The facts that c4=-2, c6=-1 is interesting
because these slacks are negative. This is a
result of the fact that the corresponding
constraints are actually “greater than or less
to” constraints instead of “less than or equal to
constraints.” The way they are treated in CPLEX
is as follows:
x1  0  x1  c 4  0
x3  0  x3  c 6  0
so that when x1=2,, c4=-2, and
when x3=1, , c6=-1..
CPLEX> primopt
…
Primal simplex - Optimal:
Objective = 1.3000000000e+01
…
CPLEX> display solution variables Variable Name
Solution Value
x1
2.000000
x3
1.000000
All other variables in the range 1-3 are 0.
CPLEX> display solution slacks Constraint Name
Slack Value
slack c2
1.000000
slack c4
-2.000000
slack c6
-1.000000
All other slacks in the range 1-6 are 0.
CPLEX> display solution dual Constraint Name
Dual Price
c1
1.000000
c3
1.000000
All other dual prices in the range 1-6 are 0.
CPLEX> quit