Solutions for Nonlinear Equations

Lecture 8 Alessandra Nardi Thanks to Prof. Newton, Prof. Sangiovanni, Prof. White, Jaime Peraire, Deepak Ramaswamy, Michal Rewienski, and Karen Veroy

Outline

• Nonlinear problems • Iterative Methods • Newton’s Method – Derivation of Newton – Quadratic Convergence – Examples – Convergence Testing • Multidimensonal Newton Method – Basic Algorithm – Quadratic convergence – Application to circuits

I 1

Nonlinear Problems - Example

1

V d V t

I r I d

I d



s

0

Need to Solve

I r



I d



I

1  0 1

R e

1 

e

1

I s

(

e V t

 1 ) 

I

1  0

g

(

e

1 ) 

I

1 0

Nonlinear Equations

• Given

g(V)=I

• It can be expressed as:

f(V)=g(V)-I

 Solve

g(V)=I

equivalent to solve

f(V)=0

Hard to find analytical solution for f(x)=0 Solve iteratively

Nonlinear Equations

– Iterative Methods • Start from an initial value

x 0

• Generate a sequence of iterate

x n-1 , x n , x n+1

which hopefully converges to the solution

x*

• Iterates are generated according to an iteration function

F: x n+1 =F(x n )

Ask • When does it converge to correct solution ?

• What is the convergence rate ?

Newton-Raphson (NR) Method

Consists of linearizing the system.

Want to solve f(x)=0  Replace f(x) with its linearized version and solve.

f

(

x

) 

f

(

x

* ) 

df dx

(

x

* )(

x



x

* )

Taylor Ser ies f

(

x k

 1 )  

x k

 1 

x f k

(

x k

)    

df dx df

(

x k

)(

x k

 1

dx

(

x k

)    1

f

(

x k



x k

) )

Iteration function

Note: at each step need to evaluate

f

and

f’

Newton-Raphson Method

– Graphical View

Newton-Raphson Method

– Algorithm Define iteration Do

k = 0 x k

 1  to

x k

….

  

df dx

(

x k

)    1

f

(

x k

) until convergence • How about convergence?

• An iteration

{x (k) }

is said to converge with order

q

exists a vector norm such that for each

k



N

: if there

q x k

 1 

x

ˆ 

x k



Newton-Raphson Method

– Convergence 0 

f x

* 

k

) 

df

(

x k

)(

x

* 

x k

)  2

d f dx dx

2 some

x

 [ *

k

, * ] 

x k

) 2 But 0  Mean Value theorem truncates Taylor series

k

) 

df dx

(

x k

)(

x k

 1 

x k

) by Newton definition

Newton-Raphson Method

– Convergence Subtracting

df dx

(

x k

)(

x k

 1 

x

* )  2

d f k



x

Dividing through (

x k

 1 

x

* )  [

df dx

(

x k

)]  1 2

d f

2

d x

Let [

df dx

(

x k

)]  1 2

d f

2

d x



K k

then

x k

 1 

x

* 

k K x k



x

* 2 Convergence is quadratic

k



x

* 2 )

Newton-Raphson Method

– Convergence Local Convergence Theorem

If

a

)

df dx bounded aw ay from ze ro

 2

b

)

d dx

2

f bounded



K is bounded

Then Newton’s method converges given a sufficiently close initial guess (and convergence is quadratic)

Newton-Raphson Method

– Convergence Example 1 

x

2  1  0,

fin d x

(

x

*  1 )

df dx

(

x k

)  2

x k

2 2

k k k

 1 

x k

)

k

 1 

x

*   

k

* 2    1

x k

)     2

k

 1 

x

* )  1 2

x k

(

x k



x

* ) 2 Convergence is quadratic

Newton-Raphson Method

– Convergence Example 2 

x

2  0,

x

*  0

df

(

x k

)

dx

 2

k

 2

x k k

 1 (

x k

 0) 2 

df dx

 1 Note : not bounded away from zero

x k

 1 1 2 

x k

  for

x k



x

*  0

k

 1 

x

* )  1 2 (

x k



x

* ) Convergence is linear

Newton-Raphson Method

– Convergence Example 1, 2

Newton-Raphson Method

– Convergence

x

0 = Initial Guess,

k

 0 Repeat {    

x x k

 1 

x k

  

k

1 } Until ?

x k

 1 

x k



threshold

? 

threshold

?

Newton-Raphson Method

– Convergence Convergence Checks Need a "delta-x" check to avoid false convergence f(x)

x k

 1 

x k

 

x a

 

x r x k

 1

x k

 1

x k

 

f a x

* X

Newton-Raphson Method

– Convergence Convergence Checks Also need an "   " check to avoid false convergence f(x)  

f a x

*

x k

 1

x k

X

x k

 1 

x k

 

x a

 

x r x k

 1

Newton-Raphson Method

– Convergence demo2

Newton-Raphson Method

– Convergence Local Convergence Convergence Depends on a Good Initial Guess f(x)

x

1

x

1 X

x

2

x

0

x

0

Newton-Raphson Method

– Convergence Local Convergence Convergence Depends on a Good Initial Guess

Nonlinear Problems – Multidimensional Example -

v

1

b i

+ 1

v

1

i

2 +

v

2

b

Nonlinear Resistors

i

  

v

2

i

3 +

v

3

b

Nodal Analysis At Node 1:

i

1  1   2    0

g v

1 

v

2   0 At Node 2:

i

3    2 0    3

g v

1 

v

2   0 Two coupled nonlinear equations in two unknowns

Multidimensional Newton Method

Problem: Find

x

*  N

x

* such tha t and

F

: N  N    0

F

(

x

) 

F

(

x

* ) 

J

(

x

* )(

x



x

* )

Taylor Ser ies J

 (

x

) 

x k

 1        

F

1 (  

x

1 

F N

( 

x

1

x

)

x

) 

x k

    

F

1 ( 

x N



F N

 ( 

x N x

)

x

)       

Jacobian M atrix J

(

x k

)  1

F

(

x k

)

Iteration function

Multidimensional Newton Method

Computational Aspects

Iteration

:

x k

 1 

x k



J

(

x k

)  1

F

(

x k

) Do not compute

J

(

x k

)  1 (it is not sparse).

Instead solve :

J

(

x k

) (

x k

 1 

x k

)  

F

(

x k

) Each iteration requires: 1. Evaluation of

F(x k )

2. Computation of

J(x k )

3. Solution of a linear system of algebraic equations whose coefficient matrix is

J(x k )

and whose RHS is

-F(x k )

Multidimensional Newton Method

Algorithm

x

0 = Initial Guess,

k

 0 Repeat { Compute

F x

Solve

J F J x x k

 1 

x k

  

k

1 } Until

x k

 1 

x k

, for

x k

 1 o h

Multidimensional Newton Method

Convergence Local Convergence Theorem

If

a

)

J F

 1

b

)

J F

  

J F

  Inverse is bounded 

x



y

 Derivative is Lipschitz Cont 

Then Newton’s method converges given a sufficiently close initial guess (and convergence is quadratic)

Application of NR to Circuit Equations Companion Network • Applying NR to the system of equations we find that at iteration k+1: – all the coefficients of KCL, KVL and of BCE of the linear elements remain unchanged with respect to iteration k – Nonlinear elements are represented by a linearization of BCE around iteration k  This system of equations can be interpreted as the STA of a linear circuit ( companion network ) whose elements are specified by the linearized BCE.

Application of NR to Circuit Equations Companion Network • General procedure: the NR method applied to a nonlinear circuit whose eqns are formulated in the STA form produces at each iteration the STA eqns of a linear resistive circuit obtained by linearizing the BCE of the nonlinear elements and leaving all the other BCE unmodified • After the linear circuit is produced, there is no need to stick to STA, but other methods (such as MNA) may be used to assemble the circuit eqns

Application of NR to Circuit Equations Companion Network – MNA templates

Note: G

0



G 0 =G 0

and I

d

depend on the iteration count k (k) and I

d =I d (k)

Application of NR to Circuit Equations Companion Network – MNA templates

Modeling a MOSFET

(MOS Level 1, linear regime)

d

Modeling a MOSFET

(MOS Level 1, linear regime)

DC Analysis Flow Diagram

For each state variable in the system

Implications

• Device model equations must be continuous with continuous derivatives and derivative calculation must be accurate derivative of function (not all models do this - Poor diode models and breakdown models don’t be sure models are decent - beware of user-supplied models) • Watch out for floating nodes (If a node becomes disconnected, then J(x) is singular) • Give good initial guess for x (0) • Most model computations produce errors in function values and derivatives. Want to have convergence criteria || x (k+1) - x (k) || <  such that  > than model errors.

Summary

• Nonlinear problems • Iterative Methods • Newton’s Method – Derivation of Newton – Quadratic Convergence – Examples – Convergence Testing • Multidimensonal Newton Method – Basic Algorithm – Quadratic convergence – Application to circuits