Transcript Анализ эффективности некоторых экстрем

SYMMETRIC DUALITY IN OPTIMIZATION
AND IT’S APPLICATIONS
Valery I. Zorkaltsev,
Professor,
Head of Laboratory,
Energy Systems Institute
Siberian Branch of the Russian Academy of Sciences
E-mail: [email protected]
International conference
“Optimization and applications"
Montenegrio
2009 г.
1
Definition of symmetric duality
For wide class of optimization problems they
use special constructions called Dual
optimization problems:
L  L* ,
where
L  primal optimization problem;
L*  dual optimization problem;
  transition rule (often polysemantic).
2
For dual problem one can specify problem
dual to it
*
**
L L .
Symmetric duality is event, when dual problem
to dual problem coincides with primal problem
L**  L.
3
Applications of dual problems:
• to prove optimality of obtained solutions;
• for justification of optimization algorithms;
• in solution interpretation;
• for making optimization algorithms;
• for researching and solving many complicated
problems of operation research, including Nash
equilibrium finding.
4
Lecture plan
1. Theory of symmetric duality in optimization:
– Lagrangian multipliers;
– Theorems of alternative systems of linear
inequalities;
– Legendre-Fenchel conjugate functions and their
extensions.
2. Application of symmetric duality
optimization algorithms and regularization
in
5
Lecture plan (continuation)
3. Application in models:
– load-flow models (electric circuits, hydraulic
circuits, nonlinear transportation problems);
– models of thermodynamic equilibrium and
geometric programming;
– economic equilibrium models.
6
1. Lagrange multipliers of constraints
Primal problem:
f 0 ( x)  min
xX
f i ( x)  0, i  1, . . . , n.
(1)
Lagrange problem:
n
f 0 ( x)    i f i ( x)  min
i 1
xX
where  i  Lagrange multipliers, which satisfy
conditions (1).
Modified Lagrange problem (Sh. Churkveidze):
1n
f 0 ( x)   ( f i ( x)   i ) 2  min
xX
2 i 1
7
2. Theory of alternative systems of
linear inequalities
Any system of linear inequalities can be
confronted with an alternative system of linear
inequalities S by formal rules
S  S*
so that proposition is right: One and only one
system of two is consistent: S or S*.
Moreover, backward transformation takes place:
S  S and S **  S.
*
**
That is alternative systems S
symmetric.
and
S*
are
8
Three examples of theorems of
alternative systems of linear inequalities
It is assigned: А – m n matrix , b – vector in
m
n
m
u

R
x

R
R . Sought vectors –
,
.
Remark. System of linear equations can be
considered as special case of system of linear
inequalities. The converse proposition is not
correct.
9
1. Fredgolm’s theorem (about alternative
systems of linear inequalities)
Either there is x  R
n
Ax  b,
or there is u  R
m
AT u  0, bT u  1.
10
2. Farkas’ theorem
Either the following system possesses a solution
Ax  b, x  0,
or the following system is solvable
AT u  0, bT u  0.
11
3. Gail’s theorem
n
x

R
Either there is
such, that
Ax  b,
or there is vector u  R m such, that
A u  0, u  0, b u  1.
T
T
12
Applications of alternative systems
of linear inequalities theory
1. Identification of system of linear
inequalities incompatibility
–
If a vector from the solution set of an alternative
system S* will be obtained during the process of
searching the solution of system S, then absence
of the solution of initial system S will be proved.
We have practical and effective (as computation
has shown) method for identification of problem
constraints inconsistency.
13
2. For determination of redundant constraints,
exclusion of which doesn’t change the solution
set, including situations in algorithms
– Gomory or Kelly cuts;
– Fourier-Chernikov convolutions for description of
systems of linear inequalities solutions.
3. For identification of solutions of systems of
linear inequalities with minimal set of active
constraints – relative to interior points of
systems of linear inequalities solution set.
14
4. For creation of new algorithms for solving
systems of linear and on the basis of this
nonlinear inequalities («Alternative approach»,
which is developed by U. Evtushenko, A.
Golicov).
5. All theory of linear optimization duality is
contained in theorems of alternative systems of
linear inequalities. Duality of linear optimization
is the basis for wide class of nonlinear problems.
15
Search of solutions and identification of inconsistence
of system of double-sided linear inequalities
• The more restricted class of problems is considered the more
interesting results about characteristics of this class of problems
can be obtained
• Initial system: find x Rn satisfying the following
conditions
(S )
Ax  b, x  x  x
• Alternative system of one inequality: find u  Rm, such that
 (u)  0,
where
(S * )
 (u )  b u  x ( A u )  x ( AT u )  .
T
T
T

T


Here for y  R m vectors ( y ) , ( y ) have components:
( y)i  max{0, yi },
( y)i  min{0, yi }, i  1, ..., m.
16
Comparison of variants of interior point
methods for problems of permissible
regimes of electric power systems
Algorithm
Number of iterations for problems
inconsistent
consistent
6*7
40*80
2*7
19*19
201*201
A
1
10
7
23
116
B
1
15
6
24
107
C
1
1
16
13
28
D
1
1
5
5
8
E
1
4
26
24
88
A, B – primal algorithms
C, D – dual algorithms
Е – primal-dual algorithm
17
Mutually dual problems of linear programming
(P)
c Τ x  min, x  X ,
(P*)
bΤ u  max, u U ,
X   x  R n : Ax  b, x  0 ,
U  u  R m : g (u )  c  AT u  0 .
Let X , U be sets of optimal solutions of problems (P), (P*).
Let’s introduce sets of recession directions for this problems :
U *  v  Rm: AT v  0, bT v  0 ,




X *  s  Rn : As  0, s  0, cT s  0 .
*
According to Farkas and Geil theorems pairs of sets X , U and
U , X * are alternative.
Symmetric duality takes place for LP problems:
( P*)*  P.
18
Theorem of duality for LP
*
P
Four events are possible for problems (Р), ( ) :
1. If X   , U   then X   , U   ,
*
*
X   , U  .
2. If X   , U   . Тогда X   , U   ,
*
*
X   , U  .
3. If X   , U   . Тогда X   , U   ,
X*   , U*   .
19
Theorem of duality for LP
(continuation)
4. If X   , U  . Then X   , U   ,
X*   , U*  .
For any x  X , u  U
x j g j u   0, j  1,..., n.
There is
x  X , u  U such that
x j  g j u   0, j  1,..., n.
In this and only this case x  ri X, u  ri U .
20
Equivalent representations of LP problem
in the form of optimization problems
1. Primal problem
Τ
c x  min, x  X .
2. Dual problem
bΤ u  max, u  U .
3. Self-dual problem
c Τ x  bΤ u  min, x  X , u U .
4. Symmetric problem (problem of complementan
rity)
 x j g j (u )  min, x  X , u U .
j 1
21
Reresentation of a linear
programming problem as a system
of linear inequalities
cΤ x  bΤ u  0,
Ax  b, x  0,
AT u  c.
It allows to consider problems of linear programming as
a special case of systems of linear inequalities.
22
3. Conjugate functions
F ( x), ( y ) for x, y  R n .
1. Functions F ( x), ( y) is Legendre conjugate
of each other if
1
 f ,
where
  ( x), f  F ( x).
That is
( f ( x))  f (( x))  x, x  R n .
23
3. Conjugate functions
2. Functions F ( x), ( y) is Fenchel conjugate of
each other, if
( y )  max { xT y  F ( x) },
and
x
F ( x)  max{ xT y  ( y ) }.
y
24
3. Generalization of conjugate
functions of Legendre-Fenchel
3. Functions
other, if
F ( x), ( y ) is conjugate of each
( y )  max{ x T Ry  F ( x) },
x
F ( x)  max{ xT Ry  ( y) }.
y
where R  symmetric positive defined matrix.
Following functions are mutually inverse
1
1
  R  ( x), f  R F ( x).
Following inequality is held
F ( x)   ( y )  x Ry
T
x, y  R .
n
25
Symmetric duality
1. Primal problem (S)
c Τ x  F ( x)  min,
Ax  b, x  0.
(S)
(1)
2. Dual problem (S*)
( y )  bΤ u  min,
(S*)
T
(2)
A u  y  c.
Note: problems (S) and (S*) have different
structure of variables; dual to dual problem
coincide with primal problem S **  S .
26
Symmetric duality (continuation)
3. Self-dual problem: subject to (1), (2)
c x  b u  F ( x)  ( y)  min .
Τ
Τ
(SD )
4. Symmetric problem: subject to (1), (2)
F ( x)  ( y)  x y  min.
T
(SS )
27
Symmetric duality. Equivalent
system of equalities and inequalities
Ax  b, x  0,
T
A u  y  c,
cΤ x  bΤ u  F ( x)  ( y )  0.
(1) 

( 2)  ( L )
(3) 
Constraint (3) can be substituted with еquality
T
F ( x)  ( y)  x y  0.
(4)
28
Examples
I. Symmetric duality for problems of
quadratic programming
1 T
1 T 1
F ( x)  x Qx , ( y )  y Q y,
2
2
where Q  positive definite matrix
xT Qx  0, x  0.
Mutually dual problems
F ( x)  cT x  min, Ax  b, x  0;
bT u  ( y)  max, AT u  y  c.
29
Examples
II. Especially important case of
separable functions
n
n
j 1
j 1
F ( x)   F j ( x j ), ( y )   ( y j ).
One form of writing the equivalent system of
equalities and inequalities
Ax  b, x  0,
y  A u  c  ,
T

y  f ( x), x  ( y).
30
Theorem
(for separable F ( x), ( y))
Let fj be continious increasing functions, then
(S) is a problem of minimization of strictly
convex function with linear constraints, (S*),
(SD) are problems of minimization of convex
function with linear constraints.
If, at the same time, f j (0)  0, f j ()   and a
system Ax=b, x  0 is consistent, then problems
(S), (S*), (SD), (SS), (L) have coincident and
unique (relatively to vectors x, y) solutions.
Vector u is unique if rank A=m.
31
Applications, separable case
1. Regularization of linear
problems: having small   0
programming
1 n 2
F ( x)    x ,  ( y )   y j .
j 1
 j 1
n
2
j
Primal problem: regularization by Tihonov
n
c x    ( xi ) 2  min, Ax  b, x  0.
T
i 1
Dual problem: search of pseudosolution of dual
problem of linear programming
1 T
 2
b u  ( A u  c)  min .

T
32
Applications
(with c  0 and self-conjugated
functions
of
n
2
the kind F ( x)   ( x)   ( xi ) )
i 1
2. «Alternative» way of searching for normal
solutions for system of inequalities with n
variables
Ax  b, x  0,
x  min .
This is equivalent to problem with m variables
A u 
T
 2
 b u  min, u  R ,
T
m
Such approach is preferable when m  n.
33
Applications
3. Load-flow models (nonlinear transportation
problems, electric circuits, hydraulic circuits
including heat, water and gas delivery problems)
i  1, . . . , m  indices of nodes,
j  1, . . . , n  indices of arcs,
A  incedence matrix,
b  vector of volumes of delivery in system
and out of system,
34
Load-flow models
c  vector of pressure gains (or electro-
motive forces, or conveyance tariffs) on
arcs,
x  vector of flows on arcs,
u  vector of pressures (tensions, prices) in
nodes,
y  vector of pressure losses (tension losses,
price rises) on arcs.
35
Load-flow models
Ax  b  flow balance in nodes (first
Kirchhoff law),
T

y  ( A u  c)  balances of pressures on arcs,
y  f ( x), x  ( y)  interrelations of pressure
loss and flow on arcs.
For example,
1
y j  rj x j , x j  y j  Ohm law,
rj
1/ 2
 1

2
y j   j (x j ) , x j  
y j   Darcy law.


j


36
Results for hydraulic circuits obtained
using the theory of symmetric duality
1. Conditions for existence and uniqueness of
classical load-flow model solution are clarified.
2. Possibilities for choosing the form of
mathematical models representation are
expanded.
3. Foundations for constructing and theoretical
justification of algorithms for solving loadflow problems are obtained.
37
Results for hydraulic circuits obtained
using the theory of symmetric duality
4. Theoretical research is held (including
clarification of conditions for existence and
uniqueness), algorithms for solving nonclassical load-flow problems are developed,
where some components of vectors x, y, u, b
and c may be fixed аnd other components of
these vectors should be found.
38
Transport model with piecewise
defined nonlinear costs
• Model is applied in analysis of operation of
natural gas and oil delivery systems to find and
eliminate bottlenecks in proper time.
• Let x j be flow through the arc j,
s j − costs coefficient for the arc j, s j  0,
Fj − nonlinear function.
For each arc costs function will be
~
G j ( x j )  s j x j  Fj ( x j ),
39
where

~
0, x j  x j  x j ,
Fj ( x j )  

 F j ( x j  x j ), x j  x j .
for x j  0, x j  x j .
costs
Gj (x j )
Normal
regime
xj
Extremal
regime
xj
flow
40
Primal optimization problem
n

j 1
~
(Fj ( x j )  s j x j ) 
 h (b  b )  min
i
iI cons
i
i
(1)
Ax  b  0,
x j  x j , j  1,...,n,
( 2)
(3)
bi  bi  0, i  I src ,
( 4)
(5)
0  bi  bi , i  I cons ,
• bi – volume of delivery into the net at source node (if
i  I src ) or out of the net at consumer node (if i  I cons ),
• hi – penalty for incomplete delivery in node i,
• Isrc – set of numbers of source nodes,
• Icons – set of numbers of consumer nodes.
41
Economic interpretation
tariff on arc j 
~
f j (x j )  s j
yj  sj
~
 j (yj )  xj yj
sj
0
xj
xj
Fig. 1. Plot of marginal costs
~
Fj ( x j )  s j x j
 flow through 


 arc j

( y j  s j ) x j – revenue from transportation on arc j
Fj ( x j )  s j x j – transportation costs on arc j
~
 j ( y j )  x j y j – profit (surplus) of transport company on arc j
42
Calculation experiments results
• Results of calculations for method of interior points on
number of example networks
Table 1. Results of computations
(Number of
nodes,
number of
arcs)
Amount of
iterations of
interior points
method
Time of
computation,
sec
Achieved
accuracy of
equality
constraints
Achieved
accuracy of
optimality
conditions
(25, 30)
23
0.400
1.1053*10-7
0.00190019
(50, 67)
39
0.631
1.61968*10-10
0.00702529
(75, 109)
48
2.254
2.33416*10-8
0.00256343
(100, 116)
68
6.850
1.48463*10-7
0.00640007
(200, 240)
93
77.919
1.41919*10-5
0.00021684
43
Diagrams for computation results
Results of calculations is shown on two diagrams
Time of computation, sec
Number of
iterations
Amount of
variables
Amount of
variables
44
Problem of finding bottlenecks in natural gas
delivery network in order to obtain system
reliability
• Two examples were computed for real networks:
– Aggregated network for natural gas delivery system
(21 nodes, 28 arcs)
– Detailed network for the same system
(337 nodes, 589 arcs)
• Two aims of computation for each example:
– 1) to determine nodes with low supply and arcs with
utilized capacity when only normal regime is allowed
– 2) to determine abilities to increase supply of nodes with
low supply and find arcs switched to extremal regime
when extremal regime is allowed
45
Aggregated network. Only normal regime is allowed
Linear load-flow
n
s x
j
j 1
j

 h (b  b )  min,
i
i
i
iI cons
Ax  b, x  x  x,
bi  bi  0, i  I src ,
0  bi  bi , i  I cons
Aggregated network. Extremal regime is allowed
Nonlinear load-flow
n

j 1
~
(Fj ( x j )  s j x j ) 
 h (b  b )  min
i
i
i
iI cons
Ax  b, x  x,
bi  bi  0, i  I src ,
0  bi  bi , i  I cons
46
Detailed network for natural gas delivery system
Number of nodes: 337
Number of arcs: 589
Amount of iterations of interior points method: 82
Time of calculation: 649.359 sec
47
Final word
• I’d like to give thank to people who helped me
make this report:
– Perjabinsky Sergey,
– Medvezhonkov Dmitry.
• Thank you for your attention!
48