Transcript No Slide Title

Multistep Methods
Lecture 11
Alessandra Nardi
Thanks to Prof. Jacob White, Deepak Ramaswamy, Michal
Rewienski, and Karen Veroy
Last lecture review
• Transient Analysis of dynamical circuits
– i.e., circuits containing C and/or L
• Examples
• Solution of ODEs (IVP)
– Forward Euler (FE), Backward Euler (BE) and
Trapezoidal Rule (TR)
– Multistep methods
– Convergence
• Consistency
• Stability
Outline
• Convergence for one-step methods
– Consistency for FE
– Stability for FE
• Convergence for multistep methods
– Consistency (Exactness Constraints)
• Selecting coefficients
– Stability
• Region of Absolute Stability
• Dahlquist’s Stability Barriers
Multistep Methods – Common Algorithms
TR, BE, FE are one-step methods

l j
l j
ˆ
ˆ

x


t

f
x
, u  tl  j 
Multistep Equation:  j
 j
k
k
j 0
j 0
Forward-Euler Approximation:
FE Discrete Equation:
Multistep Coefficients:
BE Discrete Equation:
Multistep Coefficients:

x  tl   x  tl 1   t f  x  tl 1  , u tl 1  
xˆ l  xˆ l 1  t f  xˆ l 1 , u  tl 1  
k  1, 0  1, 1  1, 0  0, 1  1
xˆ l  xˆ l 1  t f  xˆ l , u  tl  
k  1, 0  1, 1  1, 0  1, 1  0
t
f  xˆ l , u  tl    f  xˆ l 1 , u  tl 1  
2
1
1
Multistep Coefficients: k  1,  0  1, 1  1,  0  , 1 
2
2
Trap Discrete Equation: xˆ l  xˆ l 1 


Multistep Methods – Convergence Analysis
Convergence Definition
Definition: A finite-difference method for solving
initial value problems on [0,T] is said to be
convergent if given any A and any initial condition
max
xˆ  x  l t   0 as t  0
l
 T
l0, 
 t 
xˆ computed with t
t
l
xˆ computed with
2
l
xexact
Multistep Methods – Convergence Analysis
Order-p Convergence
Definition: A multi-step method for solving initial
value problems on [0,T] is said to be order p
convergent if given any A and any initial condition
max
xˆ  x  l t   C  t 
l
 T
l0, 
 t 
p
for all t less than a given t0
Forward- and Backward-Euler are order 1 convergent
Trapezoidal Rule is order 2 convergent
Multistep Methods – Convergence Analysis
Two types of error
The Local Truncation Error (LTE) of an integratio n method at tl 1
is the difference between th e computed value xˆ l 1 and the exact
value of the solution x(tl 1 ), assuming no previous error has been made.
The Global Truncation Error (GTE) of an integratio n method at tl 1
is the difference between th e computed value xˆ l 1 and the exact
value of the solution x(tl 1 ), assuming that only the initial
condition is known exactly.
Multistep Methods – Convergence Analysis
Two conditions for Convergence
• For convergence we need to look at max error
over the whole time interval [0,T]
– We look at GTE
• Not enough to look at LTE, in fact:
– As I take smaller and smaller timesteps t, I would
like my solution to approach exact solution better
and better over the whole time interval, even
though I have to add up LTE from more timesteps.
Multistep Methods – Convergence Analysis
Two conditions for Convergence
1) Local Condition: One step errors are small
(consistency)
Typically verified using Taylor Series
2) Global Condition: The single step errors do not
grow too quickly (stability)
All one-step methods are stable in this sense.
One-step Methods – Convergence Analysis
Consistency definition
Definition: A one-step method for solving initial
value problems on an interval [0,T] is said to be
consistent if for any A and any initial condition
xˆ  x  t 
1
t
 0 as t  0
One-step Methods – Convergence Analysis
Consistency for Forward Euler
Forward-Euler definition
xˆ1  x  0  tAx  0
  0, t 
Expanding in t about zero yields
dt
2
dt 2
x( t )  x  0   t

dx  0   t  d 2 x  
2
dt
Noting that
x (0)  Ax(0) and subtracting
d
2
dt 2
xˆ1  x  t  
 t  d 2 x  
2
Proves the theorem if
derivatives of x are
bounded
One-step Methods – Convergence Analysis
Convergence Analysis for Forward Euler
Forward-Euler definition
xˆ l 1  xˆ l  tAxˆ l
Expanding in t about l t yields
x   l  1 t   x  l t   tAx  l t   el
where el is the "one-step" error bounded by
el  C  t  , where C  0.5max [0,T ]
2
dt 2
d 2 x  
One-step Methods – Convergence Analysis
Convergence Analysis for Forward Euler
Subtracting the previous slide equations
xˆ l 1  x   l  1 t    I  tA  xˆ l  x  l t    el
Define the "Global" error E l  xˆ l  x  l t 
E l 1   I  tA E l  el
Taking norms and using the bound on el
 1  t A
E
l
 C  t 
E l 1   I  tA E l  C  t 
2
2
One-step Methods – Convergence Analysis
A helpful bound on difference equations
A lemma bounding difference equation solutions
u l 1  1    u l  b, u 0  0,   0

Then
ul 
b
e l
If
To prove, first write ul as a power series and sum
u l   1   
j 0
l 1
j
1  1   
b
b
1  1   
l
One-step Methods – Convergence Analysis
A helpful bound on difference equations
To finish, note (1   )  e  (1   )l  e l
1  1   


ul 
b
b
b
1     1 e l
1  1   
l
l
Mapping the global error equation to the lemma
b

 
E l 1


  1  t A  E l  C  t 
2


One-step Methods – Convergence Analysis
Back to Convergence Analysis for Forward Euler
Applying the lemma and cancelling terms
b

 


t A
E l   1  t A  E l 1  C  t  


C

t
2
2
e


l t A
Finally noting that l t  T ,
max l0,L E l  e
AT
A
t
C
One-step Methods – Convergence Analysis
Observations about Convergence Analysis for FE
max l0,L E l  e
AT
A
t
C
• Forward-Euler is order 1 convergent
• The bound grows exponentially with time interval
• C is related to the solution second derivative
• The bound grows exponentially fast with norm(A).
Multistep Methods
Definition and Observations

l j
l j
ˆ
ˆ

x


t

f
x
, u  tl  j 
Multistep Equation:  j
 j
k
k
j 0
j 0

1) If 0  0 the multistep method is implicit
2) A k  step multistep method uses k previous x ' s and f ' s
3) A normalization is needed, 0  1 is common
4) A k -step method has 2k  1 free coefficients
How does one pick good coefficients?
Want the highest accuracy
Multistep Methods
Simplified Problem for Analysis
d
Scalar ODE:
v  t    v(t ), v  0   v0

dt
Why such a simple Test Problem?
• Nonlinear Analysis has many unrevealing subtleties
• Scalar equivalent to vector for multistep methods.
multistep
d
x  t   Ax(t ) discretization
dt
Let Ey(t )  x(t )
k
l j
l j
ˆ
ˆ

x


t

Ax
 j
 j
k
Decoupled
Equations
l j
ˆ

y
 j
j 0
j 0
l j
j
 yˆ
j 0
k
k
k
 t   j E 1 AEyˆ l  j
j 0
1
 t   j 
j 0

k
j 0

 yˆ l  j

n 
Multistep Methods
Simplified Problem for Analysis
Scalar ODE:
d
v  t    v(t ), v  0   v0
dt
k
k

l j
l j
ˆ
ˆ

v


t


v
Scalar Multistep formula:  j
 j
j 0
j 0
Must Consider ALL  
Im   
Decaying
Solutions
O
s
c
i
l
l
a
t
i
o
n
s
Growing
Solutions
Re   
Multistep Methods – Convergence Analysis
Global Error Equation
Multistep formula:
Exact solution Almost
satisfies Multistep Formula:
k
k
j 0
j 0
l j
l j
ˆ
ˆ

v


t


v
0
 j
 j
d
l

v
t


t

v
t

e



j  l j 
j
l j 
dt
j 0
j 0
k
Global Error: E l  v  tl   vˆl
k
Local Truncation
Error (LTE)
Subtracting yields difference equation for global error
l
l 1
l k
l




t

E





t

E






t

E

e
 0
 1
 k
0
1
k
Multistep Methods – Making LTE small
Exactness Constraints
Multistep methods are designed so that they are exact for a
polynomial of order p. These methods are said to be of order p.
d
p 1
Suppose v  t   t  v  t   pt
dt
p
k
Then
   k  j  t 
v t 
j 0
j
k j
p
k
 t   j p   k  j  t 
j 0
d
v  tk  j 
dt
p 1

Multistep Methods – Making LTE small
Exactness Constraints
k
k
Then  j   k  j  t   t   j p   k  j  t 
j 0
p
p
j 0
k
 t    j  k  j 
 j 0
p
k
   j p k  j 
k
 k
p
p 1 
  j   k  j      j p  k  j    0
j 0
 j 0

If
p 1
j 0

p 1 
k

e


then ek  0 for v(t )  t p
As any smooth v(t) has a locally accurate Taylor series in t:
if
k
 k
p
p 1 
  j  k  j     j p  k  j    0 for all p  p0
j 0
 j 0

k
 k
 l
d
p0 1
Then   j v  tl  j     j v  tl  j    e  C  t 
dt
j 0
 j 0

Multistep Methods – Making LTE small
Exactness Constraints – k=2 Example
k
 k
p
p 1 
Exactness Constraints:   j  k  j     j p  k  j    0
j 0
 j 0

For k=2, yields a 5x6 system of equations for Coefficients
1

p=1 2

p=2  4

p=3 8

p=4 
16
p=0
1 1
1 0
1 0
0
1
4
0
1
2
1 0 12 3
1 0 32 4
 0 
0    0 

1

1   0 
 2 
0     0 
  0   
0
0

 1 
0    0 
  2 
Multistep Methods – Making LTE small
Exactness Constraints – k=2 Example
Exactness
Constraints for
k=2
1
2

4

8
16
 0 
0 0    0 
1
1 1   0 
 2 

2 0    0 
  0   
1 0 12 3 0 
0
 1   
1 0 32 4 0    0 
  2 
1 1
1 0
1 0
0
1
4
Forward-Euler 0  1, 1  1,  2  0, 0  0, 1  1,  2  0,
2
FE satisfies p  0 and p  1 but not p  2  LTE  C  t 
Backward-Euler 0  1, 1  1,  2  0, 0  1, 1  0,  2  0,
2
BE satisfies p  0 and p  1 but not p  2  LTE  C  t 
Trap Rule 0  1, 1  1,  2  0, 0  0.5, 1  0.5,  2  0,
3
Trap satisfies p  0,1,or 2 but not p  3  LTE  C  t 
Multistep Methods – Making LTE small
Exactness Constraints
k=2 Example, generating Methods
First introduce a normalization, for example 0  1
1
1

1

1
1
0 0  1   1 
1 1  2   2 
2 0    0    4 
  

0 12 3 0   1   8 
0 32 4 0    2   16 
1
0
0
0
1
4
Solve for the 2-step method with lowest LTE
0  1, 1  0,  2  1, 0  1/ 3, 1  4 / 3,  2  1/ 3
Satisfies all five exactness constraints LTE  C  t 
5
Solve for the 2-step explicit method with lowest LTE
0  1, 1  4,  2  5, 0  0, 1  4,  2  2
Can only satisfy four exactness constraints LTE  C  t 
4
Multistep Methods – Making LTE small
0
10
d
v (t )  v (t )
d
FE
-5
L 10
T
E
Trap
Best Explicit Method
has highest one-step
accurate
-10
10
Beste
-15
10
-4
10
-3
10
-2
10
Timestep
-1
10
0
10
Multistep Methods – Making LTE small
0
10
M
d
a -2 d v(t )  v(t )
10
x
E -4
r 10
r
-6
10
o
r
t  0,1
FE
Where’s BESTE?
Trap
-8
10 -4
10
-3
10
-2
10
Timestep
-1
10
0
10
Multistep Methods – Making LTE small
M
a
x
E
r
r
o
r
worrysome
200
10
Best Explicit Method has
lowest one-step error but
global errror increases
as timestep decreases
d
v (t )  v (t )
d
100
10
Beste
0
10
FE
Trap
-100
10
-4
10
-3
10
-2
10
Timestep
-1
10
0
10
Multistep Methods – Stability
Difference Equation
Why did the “best” 2-step explicit method
fail to Converge?
Multistep Method Difference Equation
0  t 0  El  1  t 1  El 1 
v  l t   vˆ
l
  k  t  k  E l k  el
LTE
Global Error
We made the LTE so small, how come the Global
error is so large?
An Aside on Solving Difference Equations
Consider a general kth order difference equation
a0 x  a1 x
l
l 1

 ak x
l k
u
l
Which must have k initial conditions
1
x  x0 , x  x1 ,
0
,x
k
 xk
As is clear when the equation is in update form
1
x    a1 x 0 
a0
1
 ak x  k 1  u1 
Most important difference equation result
l
x can be related to u by x   h u
l
j 0
l j
j
An Aside on Solving Difference Equations
If a0 z  a1z
k
k 1

 ak  0 has distinct roots
1,  2 , , k
Then x   h u where h    j  j 
l
k
l j
l
j
l
j 0
l
j 1
To understand how h is derived, first a simple case
Suppose x = x  u and x  0
1
0
1
1
2
1
2
1
2
x = x  u =u , x   x  u   u  u
l 1
l
l
x = 
l
j 0
l
l j
u
0
j
An Aside on Solving Difference Equations
Three important observations
If  i <1 for all i, then x  C max j u
where C does not depend on l
l
j
If  i >1 for any i, then there exists
j
l
a bounded u such that x  
If  i  1 for all i, and if  i =1,  i is distinct
then x  Cl max j u
l
j
Multistep Methods – Stability
Difference Equation
Multistep Method Difference Equation
0  t 0  E  1  t 1  E
l
l 1

  k  t  k  E
l k
e
Definition: A multistep method is stable if and only if
T
l
max  T  E  C max  T  el
for any el
l0, 
l0, 

t

t


 t 
Theorem: A multistep method is stable if and only if
k
k 1
The roots of 0 z  1z   k  0 are either
Less than one in magnitude or equal to one and distinct
l
Multistep Methods – Stability
Stability Theorem Proof
Given the Multistep Method Difference Equation
0  t 0  E  1  t 1  E
l
l 1

  k  t  k  E
l k
k j




t

z
 0 are either
If the roots of   j
j
k
j 0
• less than one in magnitude
• equal to one in magnitude but distinct
Then from the aside on difference equations
E  Cl max l e
l
l
From which stability easily follows.
e
l
Multistep Methods – Stability
Stability Theorem Proof
k
roots of  j z
k j
0
j 0
-1
Im
As t  0, roots
move inward to
match  polynomial
1
Re
roots of   j  t  j  z k  j  0 for a nonzero t
k
j 0
Multistep Methods – Stability
A more formal approach
 
k
j 0
l j
l j
ˆ
ˆ






t

x

0




q
x
 0 (1 )
 j j
j
j
k
j 0
• Def: A method is stable if all the solutions of
the associated difference equation obtained
from (1) setting q=0 remain bounded if l
• The region of absolute stability of a method is
the set of q such that all the solutions of (1)
remain bounded if l
• Note that a method is stable if its region of
absolute stability contains the origin (q=0)
Multistep Methods – Stability
A more formal approach
It can be shown that the region of absolute stability of a method
 
k
is the set of q such that all the roots of
j 0
j
  j q z k  j  0 are such
that zi  1,i  1,...,k p 1 where k p 1 is the number of distinct roots,
and that the roots of unit modulus are of multiplici ty 1.
Def: A method is A-stable if the region of absolute
stability contains the entire left hand plane (in the  space)
Im(z)
Im()
Re(z)
Re()
-1
-1
1
1
Multistep Methods – Stability
A more formal approach
• Each method is associated with two
polynomials  and :
  : associated with function past values
 : associated with derivative past values
• Stability: roots of  must stay in |z|1 and be
simple on |z|=1
• Absolute stability: roots of (q must stay in
|z|1 and be simple on |z|=1 when Re(q)<0.
Multistep Methods – Stability
Dahlquist’s First Stability Barrier
For a stable, explicit k-step multistep method, the
maximum number of exactness constraints that can be
satisfied is less than or equal to k (note there are 2k
coefficients). For implicit methods, the number of
constraints that can be satisfied is either k+2 if k is even
or k+1 if k is odd.
Multistep Methods – Convergence Analysis
Conditions for convergence – Consistency & Stability
1) Local Condition: One step errors are small (consistency)
Exactness Constraints up to p0 (p0 must be > 0)
 max
 T
l0, 
 t 
e
 C1  t 
l
p0 1
for t  t0
2) Global Condition: One step errors grow slowly (stability)
k
roots of  j z k  j  0 must be inside the unit circle
j 0
 max
T
E  C2 max  T  el
l0, 
t
 t 
l
 T
l0, 
 t 
Convergence Result: max
E  CT  t 
l
 T
l0, 
 t 
p0
Summary
• Convergence for one-step methods
– Consistency for FE
– Stability for FE
• Convergence for multistep methods
– Consistency (Exactness Constraints)
• Selecting coefficients
– Stability
• Region of Absolute Stability
• Dahlquist’s Stability Barriers