Transcript 5. Nonlinear Functions of Several Variables

5. Nonlinear Functions of
Several Variables
Jieun Yu ([email protected])
Kyungsun Kwon ([email protected])
Kyunghwi Kim ([email protected])
Sungjin Kim ([email protected])
Computer Networks Research Lab.
Dept. of Computer Science and Engineering
Korea University
Computer Networks Research Lab.
Korea University
Contents
Introduction
Newton’s Method for Nonlinear Systems
Fixed-Point Iteration for Nonlinear Systems
Minimum of a Nonlinear Function of Several
Variables
 MATLAB’s Methods
 Case Study




2
Introduction
Chapter 1
Chapter 2
Chapter 3,4
Chapter 5
3
Introduction
 A zero of a nonlinear function of a single variable
 Nonlinear functions of several variables
 The latter problem is much more difficult
 Newton’s method is applicable to this problem as
well as that of a single variable
4
5.1 Newton’s Method for
Nonlinear Systems
Computer Networks Research Lab.
Korea University
Newton’s Method
 Tangent line function
f ( x)  f ( x 0 )  ( x  x 0) f ' ( x 0)
 Iteration form
f ( x0 )
x1  x0 
f ' ( x0 )
 Tangent plane function
f ( x , y )  f ( x 0 , y 0 )  ( x  x 0 ) fx ( x 0 , y 0 )
 ( y  y 0 ) fy ( x 0 , y 0 )
g ( x, y )  g ( x 0, y 0)  ( x  x 0) gx ( x 0, y 0)
 ( y  y 0) gy ( x 0, y 0)
 Iteration using r, s
x  x0  r
y  y0  s
6
Newton’s Method (cont’)
 Step1
r fx ( x 0 , y 0 )  s fy ( x 0 , y 0 )   f ( x 0 , y 0 )
r gx ( x 0, y 0)  s gy ( x 0, y 0)   g ( x 0, y 0)
 Step2
x1  x 0  r
y1  y 0  s
7
5.1.1 Matrix-Vextor Notation
 Step1
 fx( x 0, y 0) fy ( x 0, y 0)  r   f ( x 0, y 0)
 gx( x 0, y 0) gy ( x 0, y 0)  s    g ( x 0, y 0) 

  

J ( xold ) y   F ( xold ), ( where y  xnew  xold )
 Step2
 x1   x 0   r 
 y 1   y 0    s 
     
xnew  xold  y
8
Ex5.1 Intersection of a Circle and a
Parabola
f ( x, y )  x 2  y 2  1
( Circle )
g ( x, y )  x  y
( Parabola )
2
9
Ex5.1 Intersection of a Circle and a
Parabola
 Step1
– Partial derivative
 2 x 2 y   r    f ( x 0, y 0 ) 
2 x  1  s    g ( x 0, y 0) 

  

– Initial estimate (x0,y0) = (1/2, 1/2)
1 1  r  1 / 2
1  1  s   1 / 4

   
r  3 / 8, s  1 / 8
10
Ex5.1 Intersection of a Circle and a
Parabola
 Step2
– The second approximate solution (x1,y1)
 x1   x 0   r 
 y 1   y 0    s 
     
1 / 2 3 / 8




1
/
2
1
/
8

 

 7 / 8


5
/
8


11
Ex5.1 Intersection of a Circle and a
Parabola
• Initial estimate and 2 iterations of Newton’s method
iterati
on
x
y
||
0
0.5
0.5
1
0.875
0.625
2
0.79067
0.61806 0.084611
0.39528
The true solution is
(x, y) = (0.78615, 0.61803)
12
5.1.2 MATLAB Function for Newton’s
Method for Nonlinear Systems
JF ( x) y   F ( x)
y   JF ( x) \ F ( x)
x _ new  x _ old  y '
\ operation -> 100p
13
Example 5.2
a System of Three Equations
f1 ( x1 , x2 , x3 )  x12  x22  x32  1
f 2 ( x1 , x2 , x3 )  x12 
 0,
x32 1 / 4  0,
f 3 ( x1 , x2 , x3 )  x12  x22  4 x3
0
 x12  x22  x32  1 


F ( x)   x12 
x32 1 / 4 
x2  x2  4x

2
3
 1

2 x1 2 x2 2 x3 


J ( x)  2 x1 0 2 x3 
2 x 2 x 4 
2
 1

14
Example 5.2
a System of Three Equations
15
Example 5.3
Intersection of a Circle and an Ellipse
3x 2  4 y 2  3
y
0
x2  y2  3 / 2  0
1
0.5
x
0
-0.5
-1
-1
-0.5
0
0.5
1
6 x 8 y 
J ( x, y )  

2
x
2
y


x 0  [0.5 0.5]
16
Example 5.3
Intersection of a Circle and an Ellipse
17
Example 5.4
Positioning a Robot Arm
• the end of the first link(x1,y1)
x1  d 1 cos( ), y1  d 1 sin(  )
10
• the end of the second link(x2,y2)
(x2,y2)
8
x 2  x1  d 2 cos(   ),
y 2  y1  d 1 sin(    )
6
β
(10,4)
4
(x1,y1)
 We need to solve
2
0
α
0
2
4
6
8
10
p1  d 1 cos( )  d 2 cos(   ),
p 2  d 1 sin(  )  d 2 sin(    )
18
Example 5.4
Positioning a Robot Arm
 Find the angles so that the
arm will move to the
point(10,4)
– The length of link
(d1,d2) = (5, 6)
– initial angles
(α , β) = (0.7,0.7)
– The system of equations
5 cos( )  6 cos(   )  10  0,
6 sin(  )  6 sin(    )  4  0
19
5.2 Fixed-Point Iteration For
Nonlinear Systems
Computer Networks Research Lab.
Korea University
5.2 Fixed-Point Iteration For Nonlinear Systems
 Ex 5.5 Fixed-Point Iteration for a System of two Nonlinear Function
– Consider the problem of finding a zero of the system
f1 ( x1 , x2 )  x13  10 x1  x2  5  0,
f 2 ( x1 , x2 )  x1  x23  10 x2  1  0
– Coverting x1  g1 ( x1 , x2 ), x2  g 2 ( x1 , x2 )
form
x1  0.1x13  0.1x2  5,
x2  0.1x1  0.1x23  0.1
– The graph of the equations
f1  0 and f 2  0
21
5.2 Fixed-Point Iteration For Nonlinear Systems
x2
f1  0
1
0.8
0.6
0.4
f2  0
0.2
0
x1
0
0.2
0.4
0.6
0.8
1
x
Iteration
x1
x2
0
0.60000
0.60000
1
0.53840
0.18160
0.42291
2
0.50255
0.15444
0.044975
3
0.50275
0.15062
0.0038204
4
0.50235
0.15062
0.00039661
5
0.50238
0.15058
4.9381e-05
22
5.2 Fixed-Point Iteration For Nonlinear Systems
23
5.2 Fixed-Point Iteration For Nonlinear Systems
 Using MATLAB
function G = ex5_5(x)
G = [ (-0.1*x(1)^3 + 0.1*x(2) + 0.5)
(0.1*x(1) + 0.1*x(2)^3 + 0.1)];
% Actually using function : Fixed_pt_sys
>> x=Fixed_pt_sys(@ex5_5, [0.6 0.6], 0.00001, 5)
24
5.2 Fixed-Point Iteration For Nonlinear Systems
 Result
25
5.2 Fixed-Point Iteration For Nonlinear Systems
 Ex 5.5 Fixed-Point Iteration for a System of Three Nonlinear Function
f1 ( x1 , x2 , x3 )  x12  50 x1 
f 2 ( x1 , x2 , x3 )  x12
x22 
x32  200  0,
 20 x2 
x32  50  0,
f 3 ( x1 , x2 , x3 )   x12
 x22  40 x3  75  0
– Coverting this system to a fixed point iteration form
x  g (x )
50 x1   x12  x22  x32  200,
20 x2   x12
40 x3  x12  x22
 x32  50,
 75
26
5.2 Fixed-Point Iteration For Nonlinear Systems
x1  g1 ( x1 , x2 , x3 )  0.02 x12  0.02 x22  0.02 x32  4,
x2  g 2 ( x1 , x2 , x3 )  0.05 x12
 0.05 x32  2.5 ,
x3  g 3 ( x1 , x2 , x3 )  0.025 x12  0.025 x22
 1.875
– Using the preceding MATLAB function and a starting estimate of (2, 2, 2)
Iteration
x1
x2
x3
x
0
1
2
3
4
5
6
7
8
9
10
2.0000
3.7600
3.5729
3.6502
3.6272
3.6346
3.6323
3.6330
3.6328
3.6328
3.6328
2.0000
2.1000
1.6528
1.7621
1.7232
1.7350
1.7312
1.7324
1.7320
1.7321
1.7321
2.0000
-1.6750
-1.4113
-1.4876
-1.4643
-1.4719
-1.4695
-1.4702
-1.4700
-1.4701
-1.4701
4.0759
0.5518
0.15403
0.050903
0.0115899
0.0050696
0.0016031
0.00050866
0.00016117
5.1098e-05
27
5.2 Fixed-Point Iteration For Nonlinear Systems
 Using MATLAB
function G = ex5_6(x)
G = [ (-0.02*x(1)^2 - 0.02*x(2)^2 - 0.02*x(3)^2 + 4)
(-0.05*x(1)^2 - 0.05*x(3)^2 + 2.5)
(0.025*x(1)^2 + 0.025*x(2)^2 - 1.875)];
% Actually using function : Fixed_pt_sys
>> x=Fixed_pt_sys(@ex5_6, [2 2 2], 0.00001, 10)
28
5.2 Fixed-Point Iteration For Nonlinear Systems
 Result
29
5.2 Fixed-Point Iteration For Nonlinear Systems
 If g(x) maps D into D, then g has a fixed point in D. In other words, if g(x)
is in D whenever x is in D, then there is some point p in D such that p=g(p)
 If
G( p)   1
sequence of approximations to the fixed point
x ( k 1)  g ( x ( k ) ),
intial point x ( 0 ) is sufficiently close to the fixed point p
If there is a constant K<1 such that for every x in D
g i ( x) K

xi
n
For each i= 1,….,n and each j=1,…,n,
A bound on the error at the mth step is given by
x
( m)
p

K m (1)

x  x (0)
1 K

30
5.2 Fixed-Point Iteration For Nonlinear Systems
 Example 5.5
x1  0.1x13  0.1x2  5  g1 ( x1 , x2 ),
x2  0.1x1  0.1x23  0.1  g 2 ( x1 , x2 )
First check to make sure that g(x) maps the retangle 0  x1  1, 0  x2  1
That is, for 0  x1, x2  1 we have 0  g1 , g2  1
In fact 0.4  g1  0.6 and 0.1  g 2  0.3
g1 ( x)
g1 ( x)
g 2 ( x)
g 2 ( x)
 0.3 x12 ,
 0.1,
 0.1,
 0.3 x22
x1
x2
x1
x2
Which are all less than 0.5 (for 0  x1, x2  1 ), as is required for the corollary
31
5.3 Minimum of a Nonlinear
Function of Several Variables
Computer Networks Research Lab.
Korea University
Example 5.7
 f ( x, y)  x 2  y 2  1
 we define
and
g ( x, y)  x 2  y
,
h( x, y)  ( x 2  y 2  1)2  ( x 2  y)2
 The minimum value of h(x,y) is 0, which occurs when f(x,y)=0
and g(x,y)=0.
 The derivatives for the gradient are
hx ( x, y )  2( x 2  y 2  1)2 x  2( x 2  y )2 x,
hy ( x, y )  2( x 2  y 2  1)2 y  2( x 2  y )( 1).
33
Table 5.8
 Result of ffmin
Table 5.8 Approximate zeros at each step of iteration
Step
x
y
Change
0
0.5
0.5
1
0.875
0.625
-0.26831
2
0.74512
0.61133
-0.035987
3
0.79251
0.61902
-0.0079939
4
0.78446
0.6178
-0.00019414
5
0.78657
0.6181
-1.3631e-05
34
5.3.1 MATLAB Function
for Minimization by Gradient Descent
35
5.3.1 MATLAB Function
for Minimization by Gradient Descent
function h = ex_min(x)
h = (x(1)^2 + x(2)^2 - 1)^2 + (x(1)^2 - x(2) )^2;
function dh = ex_min_g(x)
dh = [ -(4*(x(1)^2 + x(2)^2 - 1)*x(1) + 4*(x(1)^2 - x(2))*x(1))
-(4*(x(1)^2 + x(2)^2 - 1)*x(2) - 2*(x(1)^2 - x(2)))]';
xmin = ffmin(@ex_min, @ex_min_g, [0.5 0.5], 0, 5)
0 0.5000 0.5000
1.0000
0.8750
0.6250 -0.2683
2.0000
0.7451
0.6113 -0.0360
3.0000
0.7925
0.6190 -0.0080
4.0000
0.7845
0.6178 -0.0002
5.0000
0.7866
0.6181 -0.0000
36
Example 5.8
 Given three points in the plane, we wish to find
the location of the point P=(x,y) sp that the sum of
the squares of the distances from P to the three
given points,
( x1, y1 ), ( x2 , y2 ), and ( x3 , y3 )
is as small as possible.
f ( x, y )  ( x  x1 ) 2  ( y  y1 ) 2  ( x  x2 ) 2  ( y  y2 ) 2  ( x  x3 ) 2  ( y  y3 ) 2 .
f x ( x, y )  2( x  x1 )  2( x  x2 )  2( x  x3 )  6 x  2 x1  2 x2  2 x3 ,
f y ( x, y )  2( y  y1 )  2( y  y2 )  2( y  y3 )  6 y  2 y1  2 y2  2 y3 .
37
Figure 5.7
38
5.4 MATLAB’s Methods
Computer Networks Research Lab.
Korea University
MATLAB’s METHODS
 FMINS
– finds the minimum of a scalar function of several
variables, starting at an initial estimate
– Note
• The fmins function was replaced by fminsearch in Release
11 (MATLAB 5.3).
• In Release 12 (MATLAB 6.0), fmins displays a warning
message and calls fminsearch
– Syntax
• x = fmins('fun',x0)
• x = fminsearch(fun,x0)
– starts at the point x0 and finds a local minimum x of the
function described in fun.
– x0 can be a scalar, vector, or matrix.
40
MATLAB’s METHODS
 Examples
–
A classic test example for multidimensional minimization is the Rosenbrock
banana function
–
The traditional starting point is (-1.2,1). The M-file banana.m defines the function.
–
The minimum is at (1,1) and has the value 0.
a
41
5.5 Nonlinear system:
Case Study
Computer Networks Research Lab.
Korea University
Nonlinear system:
Case Study
 The analytical model to compute the 802.11 DCF
throughput
43
Nonlinear system:
Case Study
 The stationary probability τ
– The station transmits a packet in a generic slot time
 The conditional collision probability p
aa
44
Nonlinear system:
Case Study
 Two Equations represent a nonlinear system in the
two unknowns τ and p
W = Cwmin
m = Maximum backoff stage
N = The number of stations
45
Nonlinear system:
Case Study
 Solve the nonlinear problem of two variables
– Assumptions
• W = 32
•m=3
• N = 3, 10, 50
– Newton’s Method and fminsearch(fun,x0)
 Problem1 by Newton’s Method (W=32, m = 3, N =3)
64τ p^4 + 34pτ - 33τ - 4p +2 =0
(1)
p = 1-(1- τ)^2
(2)
*Equations (1) and (2) is nonlinear system
46
Nonlinear system:
Case Study
• function f = newton_case1(x)
f = [ (64*x(2)*x(1)^4 + 34*x(1)*x(2) -33*x(2) -4*x(1) +2)
(x(2)^2 - 2*x(2) + x(1))
];
• function df = newton_case1_j(x)
df = [ (256*x(2)*x(1)^3 + 34*x(2) - 4)
1
(64*x(1)^4 + 34*x(1)-33)
( 2*x(2) -2)
];
τ = 0.4165
p = 0.6595
aa
47
Nonlinear system:
Case Study
 Problem1 by fminsearch(fun,x0) (W=32, m = 3, N
=3)
– function h = fminsearch_case1(x)
h = (64*x(2)*x(1)^4 + 34*x(1)*x(2) -33*x(2) -4*x(1) +2)^2 +
((1-x(2))^2 -1 +x(1))^2;
48
Nonlinear system:
Case Study

Problem2 by Newton’s Method (W=32, m = 3, N =10)
64τ p^4 + 34pτ - 33τ - 4p +2 =0
p = 1-(1- τ)^9
(1)
(2)
*Equations (1) and (2) is nonlinear system
τ = 0.1259
p = 0.7021
–
–
–
dd
49
Nonlinear system:
Case Study
 Problem2 by fminsearch(fun,x0) (W=32, m = 3, N
=10)
– function h = fminsearch_case1(x)
h = (64*x(2)*x(1)^4 + 34*x(1)*x(2) -33*x(2) -4*x(1) +2)^2 +
((1-x(2))^2 -1 +x(1))^9;
50
Nonlinear system:
Case Study

Problem3 by Newton’s Method (W=32, m = 3, N =50)
64τ p^4 + 34pτ - 33τ - 4p +2 =0
p = 1-(1- τ)^49
(1)
(2)
*Equations (1) and (2) is nonlinear system
τ = 0.0427
p = 0.8823
–
–
–
dd
51
Nonlinear system:
Case Study
 Problem2 by fminsearch(fun,x0) (W=32, m = 3, N
=50)
– function h = fminsearch_case1(x)
h = (64*x(2)*x(1)^4 + 34*x(1)*x(2) -33*x(2) -4*x(1) +2)^2 +
((1-x(2))^2 -1 +x(1))^49;
52