Transcript Linear Programming (Optimization)

Chap 2. The Geometry of LP
 In the text, polyhedron is defined as P = { x  Rn : Ax  b }. So some
of our earlier results should be taken with modifications.
 Thm 2.1:
(a) The intersections of convex sets is convex.
(b) Every polyhedron is a convex set.
(c) Convex combination of a finite number of elements of a convex
set also belongs to that set.
(recall that S closed for convex combination of 2 points.
 S closed for convex combination of a finite number of pts)
(d) Convex hull of a finite number of vectors (polytope) is convex.
Linear Programming 2011
1
Pf) (a) Let x, y  iI Si , Si convex  x, y  Si ,  i
 x + (1-)y  Si  i since Si convex
 x + (1-)y  iI Si ,   Si convex.
(b) Halfspace { x : a’x  b } is convex.
P =  halfspaces  From (a), P is convex
( or we may directly show A( x + (1-)y )  b. )
(c) Use induction. True for k = 2 by definition.
Suppose statement holds for k. Suppose k+1  1.
Then i=1k+1 ixi = k+1xk+1 + (1-k+1) ( i=1k ( i / (1-k+1)) xi )
i / (1-k+1)  0 and sum up to 1, hence i = 1k ( i / (1-k+1)) xi  S.  i
k+1  xi  S.
=1
i
(d) Let S be the convex hull of vectors x1, … , xk and y, z  S
 y = i=1k ixi , z = i=1k ixi for some i , i
y + (1-)z =   ixi + (1-)  ixi = i=1k (i +(1-)i) xi
i +(1-)i  0 and sum up to 1  convex comb. of xi
 y + (1-)z  S.

Linear Programming 2011
2
Extreme points, vertices, and b.f.s’s
 Def: (a) Extreme point ( as we defined earlier)
(b) x  P is a vertex if  c  Rn such that c’x < c’y  y  P and y  x.
( x is unique optimal solution of min c’x, x  P )
(c) Consider polyhedron P and x*  Rn. Then x* is a basic solution if
all equality constraints are active at x* and  n linearly independent
active constraints among the constraints active at x*.
( basic feasible solution if x* is basic solution and x*  P )
 Note: Earlier, we defined the extreme point same as in the text.
Vertex as 0-dimensional face ( dim (P) + rank (A=, b=) = n ) which is
the same as basic feasible solution in the text.
We defined basic solution (and b.f.s) only for the standard LP. ( xB = B1b, x = 0 )
N
Definition (b) is new. It gives an equivalent characterization of
extreme point. (b) can be extended to characterize a face F of P.
Linear Programming 2011
3
x3
x2
A
P
C
E
D
x1
B
 Fig. 2.6: P = { (x1, x2, x3): x1+x2+x3 = 1, x1, x2, x3  0}
Three constraints active at A, B, C, D. Only two constraints active at
E. Note that D is not a basic solution since it does not satisfy the
equality constraint. However, if P is denoted as P = { (x1, x2, x3):
x1+x2+x3  1, x1+x2+x3  1, x1, x2, x3  0}, D is a basic solution by the
definition in the text, i.e. whether a solution is basic depends on the
representation of P.
Linear Programming 2011
4
A
E
D
P
F
B
C
 Fig. 2.7: A, B, C, D, E, F are all basic solutions. C, D, E, F are
basic feasible solutions.
Linear Programming 2011
5
 Comparison of definitions in the notes and the text
Notes
Text
Extreme
point
Geometric definition
Geometric definition
Vertex
0-dimensional face
Existence of c vector which makes
x* as the unique optimal solution
for the LP
Basic
solution,
b.f.s.
Defined for standard
form. Set n-m
variables at 0 and
solve the remaining
system. b.f.s. if
nonnegative.
Defined for general polyhedron.
Satisfy equality constraints and n
linearly independent constraints
are active. ( 0-dimensional face if
feasible)
Linear Programming 2011
6
 Thm 2.3: x*  P, then x* vertex, extreme point, and b.f.s. are
equivalent statements.
Pf) We follow the definitions given in the text. We already showed in
the notes that extreme point and 0-dimensional face ( AIx* = bI, AI :
rank n, b.f.s. in the text) are equivalent.
To show all are equivalent, take the following steps:
x* vertex (1)  x* extreme point (2)  x* b.f.s. (3)  x* vertex
(1) x* vertex  x* extreme point
Suppose x* is vertex, i.e.  c  Rn such that x* is unique min of min
c’x, x  P.
If y, z  P, y, z  x*, then c’x* < c’y and c’x* < c’z.
Hence c’x* < c’( y + (1-)z ), 0    1  y + (1-)z  x*
Hence x* cannot be expressed as convex combination of two other
points in P
 x* extreme point.
Linear Programming 2011
7
(continued)
(2) x* extreme point  x* b.f.s.
Suppose x* is not a b.f.s.. Let I = { i : ai’x* = bi }
Since x* is not a b.f.s., the number of linearly independent vectors ai in I
< n.
Hence  nonzero d  Rn such that ai’d = 0,  i  I.
Consider y = x* + d, z = x* - d. But, y, z  P for sufficiently small
positive , and x* = (y+z)/2, which implies x* is not an extreme point.
(3) x* b.f.s.  x* vertex
Let x* be a b.f.s. and let I = { i : ai’x* = bi }
Let c = iI ai . Then c’x* = iI ai’x*= iI bi
 x  P, we have c’x = iI ai’x  iI bi = c’x*, hence x* optimal.
For uniqueness, equality holds  ai’x = bi , i  I.
Since x* is b.f.s., it is the unique solution of ai’x = bi , i  I
Hence x* is a vertex.

Linear Programming 2011
8
 Note: Whether x* is a basic solution depends on the representation of
P. However, x* is b.f.s. iff x* extreme point and x* being extreme point
is independent of the representation of P. Hence the property of being
a b.f.s. is also independent of the representation of P.
 Cor 2.1: For polyhedron P  , there can be finite number of basic or
basic feasible solutions.
 Def: Two distinct basic solutions are said to be adjacent if we can find
n-1 linearly independent constraints that are active at both of them. ( In
Fig 2.7, D and E are adjacent to B; A and C are adjacent to D.)
If two adjacent basic solutions are also feasible, then the line segment
that joins them is called an edge of the feasible set ( one dimensional
face).
Linear Programming 2011
9
2.3 Polyhedra in standard form
 Thm 2.4: P = { x : Ax = b, x  0 }, A: m  n, full row rank.
Then x is a basic solution  x satisfies Ax = b and  indices B(1),
… , B(m) such that AB(1), … , AB(m) are linearly independent and xi = 0,
i  B(1), … , B(m).
Pf) see text.
( To find a basic solution, choose m linearly independent columns
AB(1), … , AB(m). Set xi = 0 for all i  B(1), … , B(m), then solve Ax = b
for xB(1), … , xB(m). )
 Def: basic variable, nonbasic variable, basis, basic columns, basis
matrix B. (see text)
( BxB = b  xB = B-1b )
Linear Programming 2011
10
 Def: For standard form problems, we say that two bases are adjacent if
they share all but one basic column.
 Note: A basis uniquely determines a basic solution.
Hence if have two different basic solutions  have different basis.
But two different bases may correspond to the same basic solution. (e.g.
when b = 0 )
Similarly, two adjacent basic solutions  two adjacent bases
Two adjacent bases with different basic solutions  two adjacent
basic solutions.
However, two adjacent bases only not necessarily imply two adjacent
basic solutions. The two solutions may be the same solution.
Linear Programming 2011
11
 Check that full row rank assumption on A results in no loss of
generality.
 Thm 2.5: P = { x : Ax = b, x  0 }  , A : m  n, rank is k < m.
Q = { x : AIx = bI, x  0 }, I = { i1, … , ik } with linearly indep. rows.
Then P = Q.
Pf) Suppose first k rows of A are linearly independent.
P  Q is clear. Show Q  P.
Every row ai’ of A can be expressed as ai’ = j=1k ijaj’.
Hence, for x  P, bi = ai’x = j=1k ij aj’x = j=1k ij bj , i = 1, … , m
i.e. bi is also linear combination of bj , j  I.
Suppose y  Q, then  i = 1, … , m,
ai’y = j=1k ij aj’y = j=1k ij bj = bi
Hence, y  P  Q  P

Linear Programming 2011
12
2.4 Degeneracy
 Def 2.10: A basic solution x  Rn is said to be degenerate if more
than n of the constraints are active at x.
 Def 2.11: P = { x  Rn: Ax = b, x  0 }, A: m  n, full row rank.
Then x is a degenerate basic solution if more than n – m of the
components of x are 0 ( i.e. some basic variables have 0 value)
 For standard LP, if we have more than n – m variables at 0 for a basic
feasible solution x*, it means that more than n – m of the
nonnegativity constraints are active at x* in addition to the m
constraints in Ax = b.
The solution can be identified by defining n-m nonbasic variables
( value = 0). Hence, depending on the choice of nonbasic variables,
we have different bases, but the solution is the same.
Linear Programming 2011
13
A
D
C
P
B
E
 Fig 2.9: A and C are degenerate basic feasible solutions. B and E
are nondegenerate. D is a degenerate basic solution.
Linear Programming 2011
14
x3=0
A
x4=0
B
x5=0
P
x2=0
x1=0
x6=0
 Fig 2.11: (n-m)-dimensional illustration of degeneracy. Here, n=6,
m=4. A is nondegenerate and basic variables are x1, x2, x3, x6. B
is degenerate. We can choose x1, x6 as the nonbasic variables.
Other possibilities are to choose x1, x5, or to choose x5, x6.
Linear Programming 2011
15
 Degeneracy is not purely geometric property, it may depend on
representation of the polyhedrom
ex) P = { x : Ax = b, x  0 }, A: m  n
P’ = { x : Ax  b, -Ax  -b, x  0 }
We know that P = P’, but representation is different.
Suppose x* is a nondegenerate basic feasible solution of P.
Then exactly n – m of the variables xi* are equal to 0.
For P’, at the basic feasible solution x*, we have n – m variables set
to 0 and additional 2m constraints are satisfied with equality. Hence,
we have n + m active constraints and x* is degenerate.
Linear Programming 2011
16
2.5 Existence of extreme points
 Def 2.12: Polyhedron P  Rn contains a line if  a vector x  P and a
nonzero d  Rn such that x + d  P for all   R.
Note that if d is a line in P, then A(x + d)  b for all   R
 Ad = 0
Hence d is a vector in the lineality space S. ( in P = S+K+Q )
 Thm 2.6: P = { x  Rn: ai’x  bi, i = 1, … , m }  , then the following
are equivalent.
(a) P has at least one extreme point.
(b) P does not contain a line.
(c)  n vectors out of a1, … , am, which are linearly independent.
Pf) see proof in the text.
Linear Programming 2011
17
 Note that the conditions given in Thm 2.6 means that the lineality
space S = {0}
 Cor 2.2: Every nonempty bounded polyhedron (polytope) and every
nonempty polyhedron in standard form has at least one basic
feasible solution (extreme point).
Linear Programming 2011
18
2.6 Optimality of extreme points
 Thm 2.7: Consider the LP of minimizing c’x over a polyhedron P.
Suppose P has at least one extreme point and there exists an optimal
solution.
Then there exists an optimal solution which is an extreme point of P.
Pf) see text.
 Thm 2.8: Consider the LP of minimizing c’x over a polyhedron P.
Suppose P has at least one extreme point.
Then, either the optimal cost is - , or there exists an extreme point
which is optimal.
Linear Programming 2011
19
(continued)
Idea of proof in the text)
Consider any x  P. Let I = { i : ai’x = bi }
Then we move to y = x + d, where ai’d = 0, i  I and c’d  0.
Then either the optimal cost is -  ( if the half line d is in P and c’d
< 0 ) or we meet a new inequality which becomes active ( cost does
not increase).
By repeating the process, we eventually arrive at an extreme point
which has value not inferior to x.
Therefore, for any x in P, there exists an extreme point y such that c’y
 c’x. Then we choose the extreme point which gives the smallest
objective value with respect to c.
Linear Programming 2011
20
 ( alternative proof of Thm 2.8)
P = S + K + Q. Pointedness of P implies S = {0}.
Hence x  P  x = i qidi + j rjuj , where di’s are extreme rays of K
and uj’s are extreme points of P and qi  0, rj  0, j rj = 1.
Suppose  i such that c’di < 0, then LP is unbounded.
( For xP, x + di  P for   0. Then c’( x+di )  -  as   )
Otherwise, c’di  0 for all i , take u* such that c’u* = minj c’uj
Then  x  P,
c’x = i qi(c’di) + j rj(c’uj)  j rj(c’uj)  (c’u*) j rj = c’u*.
Hence LP is either unbounded or  an extreme point of P which is an
optimal solution.
Proof here shows that the existence of an extreme ray di of the pointed
recession cone Ax  0 ( if have min problem and polyhedron is Ax  b)
such that c’di < 0 is the necessary and sufficient condition for
unboundedness of the LP.
( If P has at least one extreme point, then LP unbounded
  an extreme ray di in recession cone K such that c’di < 0) 
Linear Programming 2011
21