Transcript Chaos Theory and Predictability

Anthony R. Lupo
Department of Soil, Environmental, and Atmospheric Sciences
302 E ABNR Building
University of Missouri
Columbia, MO 65211

Some popular images…………..
Lo ren z (1 96 3 ) at tracto r
25
1 2 .5
0
1 2 .5
25
50
25
0
50
3 7 .5
25
25
1 2 .5
50
zz y y  xx
0

Any attempt at weather “forecasting is
immoral and damaging to the character of a
meteorologist” - Max Margules (1904) (1856
– 1920)
Margules work forms the
foundation of modern
Energetics analysis.




“Chaotic” or non-linear dynamics  Is perhaps
one of the most important “discovery” or way of
relating to and/or describing natural systems in
the 20th century!
“Caoz” Chaos and order are opposites in the
Greek language - like good versus evil.
Important in the sense that we’ll describe the
behavior of “non-linear” systems!

Physical systems can be classified as:
Deterministic  laws of motion are known and
orderly (future can be directly determined from
past)

Stochastic / random  no laws of motion, we can
only use probability to predict the location of
parcels, we cannot predict future states of the
system without statistics. Only give
probabilities!


Chaotic systems  We know the laws of
motion, but these systems exhibit “random”
behavior, due to non-linear mechanisms.
Their behavior may be irregular, and may be
described statistically.
E. Lorenz and B. Saltzman  Chaos is “order
without periodicity”.

Classifying linear systems

If I have a linear set of equations represented as:


x (t )  exp( t )b (1)
And ‘b’ is the vector to be determined. We’ll assume the
solutions are non-trivial.

Q: What does that mean again for b?

A: b is not 0!


Eigenvalues are a special set of scalars associated
with a linear system of equations (i.e., a matrix
equation) that are sometimes also known as
characteristic roots (source: Mathworld) ()
Eigenvectors are a special set of vectors
associated with a linear system of equations (i.e.,
a matrix equation) that are sometimes also
known as characteristic vectors (vector ‘b’)

Thus we can easily solve this problem since we can
substitute this into the equations (1) from before and we


get:
ab  b
 or 

a  I b  0
Solve, and so, now the general solution is:

xt   c1 exp1t b1  c2 exp2t b2

Values of ‘c’ are constants of course. The vectors b1 and
b2 are called “eigenvectors” of the eigenvalues 1 and 2.

Particular Solution:

One Dimensional Non-linear dynamics
 We
will examine this because it provides a nice
basis for learning the topic and then applying to
higher dimensional systems.

 However, this can provide useful analysis of atmospheric
systems as well (time series analysis). Bengtssen (1985) Tellus –
Blocking. Federov et al. (2003) BAMS for El Nino. Mokhov et al.
(1998, 2000, 2004). Mokhov et al. (2004) for El Nino via SSTs
(see also Mokhov and Smirnov, 2006), but also for temperatures
in the stratosphere. Lupo et al (2006) temperature and precip
records. Lupo and Kunz (2005), and Hussain et al. (2007) height
fields, blocking.

First order dynamic system:
dx
 x  f  x 
dt


(Leibnitz notation is “x –dot”?)
If x is a real function, then the first derivative
will represent a(n) (imaginary) “flow” or “velocity”
along the x – axis. Thus, we will plot x versus “x
– dot”

Draw:

Then, the sign of “f(x)” determines the sign of the one –
dimensional phase velocity.

Flow to the right (left): f(x) > 0 (f(x) < 0)


Two Dimensional Non-linear dynamics
Note here that each equation has an ‘x’ and a ‘y’ in it.
Thus, the first deriviatives of x and y, depend on x
and y. This is an example of non-linearity. What if in
the first equation ‘Ax’ was a constant? What kind of
function would we have?
x  Ax  By
y  Cx  Dy

Solutions to this are trajectories moving in the (x,y)
phase plane.


Coupled set: If the set of equations above are
functions of x and y, or f(x,y).
Uncoupled set: If the set of equations above
are functions of x and y separately.


Definitions
Bifurcation point: In a dynamical system, a
bifurcation is a period doubling, quadrupling,
etc., that accompanies the onset of chaos. It
represents the sudden appearance of a
qualitatively different solution for a nonlinear
system as some parameter is varied.

Example: “pitchfork” bifurcation (subcritical)
dx
3
 x  rx  x
dt

Solution has three roots, x=0, x2 = r

The devil is in the details?............

An attractor is a set of states (points in the
phase space), invariant under the dynamics,
towards which neighboring states in a given
basin of attraction asymptotically approach in
the course of dynamic evolution. An attractor
is defined as the smallest unit which cannot
be itself decomposed into two or more
attractors with distinct basins of attraction
How we see it…….
Mathematics looks at Equation of Motion (NS)
is space such that:


Closed or compact space such that 
boundaries are closed and that within the
space divergence = 0
Complete set  div = 0 and all the
“interesting” sequences of vectors in space,
the support space solutions are zero.

Ok, let’s look at a simple harmonic oscillator
(pendulum):
mx  kx  0


Where m = mass and k = Hooke’s constant.
When we divide through by mass, we get a
Sturm – Liouville type equation.

One way to solve this is to make the problem
“self adjoint” or to set up a couplet of first
order equations like so let:
x  y
y   x
 where 
2
k
 
m
2

Then divide these two equations by each other to
get:
dy
2 x
 
dx
y
 which 
y   x  Const
2

2
2
What kind of figure is this?

A set of ellipses in the phase space.


Here it is convenient that the origin is the center!
At the center, the “flow” is still, and since the first
derivative of x is positive, we consider the “flow” to be
anticyclonic (NH) “clockwise” around the origin. The
eigenvalues are:
1, 2  i

Now as the flow does not approach or repel from the
center, we can classify this as “neutrally stable”.
Thus, the system behaves well close to certain “fixed
points”, which are at least neutrally stable.

System is forever predictable in a dynamic sense, and well
behaved.

we could move to an area where the behavior changes, a
bifurcation point which is called a “separatrix”.

Beyond this, system is unpredictable, or less so, and can
only use statistical methods. It’s unstable!



Hopf’s Bifurcation:
Hopf (1942) demonstrated that systems of non-linear
differential equations (of higher order that 2) can
have peculiar behavior.
These type of systems can change behavior from one
type of behavior (e.g., stable spiral to a stable limit
cycle), this type is a supercritical Hopf bifurcation.


Hopf’s Bifurcation:
Subcritical Hopf Bifurcations have a very different
behavior and these we will explore in connection
with Lorenz’s equations, which describe the
atmosphere’s behavior in a simplistic way. With
this type of behavior, the trajectories can “jump”
to another attractor which may be a fixed point,
limit cycle, or a “strange” attractor (chaotic
attractor – occurs in 3 – D only!)

Example of an elliptic equation in
meteorology:
 2
z a 
 2 
    fz a 2    f
p 
p p

 
z a z ag 
  Vh  z a  



p
t  R 2  
 
Q
f
    Vh  T  






p  ˆ Vh
p
c
 ˆ
p








k




k



F

  p


 


Taken from Lupo et al. 2001 (MWR)

Ok, let’s modify the equation above:

d is now the “damping constant”, so let’s “damp“
(“add energy to”) this expression d > 0 (d< 0).


mx  dmx  kx  0
Then the oscillator loses (gains) energy and the
determinant of the quadratic solution is also less
(greater) than zero! So trajectories spin toward (away
from) the center. This is a(n) “(un)stable spiral”.

Example: forced Pendulum (J. Hansen)

Another Example: behavior of a temperature
series for Des Moines, IA (taken from Birk et.
al. 2010)

Another Example: behavior of 500 hPa
heights in the N. Hemi. (taken from Lupo et.
al. 2007, Izvestia)


Sensitive Dependence on Initial Conditions (SDIC – not a
federal program  ).
Start with the simple system :
xn1  k  x

A iterative-type equation used often to demonstrate population
dynamics:
2
n 1
n
n
x

2
n
 r k  x  x

Experiment with k = 0.5, 1.0, 1.5, 1.6, 1.7, 2.0




For each, use the following xn and graph side-byside to compare the behavior of the system.
Xn = -0.5, Xn = -0.50001
Try to find: “period 2” attractor or attracting point:
behavior1  behavior2  behavior 1  behavior2,
and a “period 4” attractor.
Period 2 behaves like the large-scale flow?


Examine the initial conditions. One can be
taken to be a “measurement” and the other, a
“deviation” or “error”, whether it’s “generated”
or real. It’s a point in the ball-park of the
original.
Asside: Heisenberg’s Uncertainty Principle 
All measurements are subject to a certain
level of uncertainty.

X = 0.5
x
i
X = 0.50001
2
2
1
1
y
i
0
0
1
1
2
2
0
50
100
i
150
200
0
50
100
i
150
200

What’s the diff?
4
2
x
i
y
i
0
2
4
0
50
100
i
150
200




The differences that emerge illustrate the concept of Sensitive
Dependence on Initial Conditions (SDIC). This is an important
concept in Dynamic systems. This is also the concept behind
ENSEMBLE FORECASTING!
Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NCEP: The
generation of perturbations. Bull. Amer. Meteor. Soc., 74, 2317
– 2330.
Toth, Z., and E. Kalnay, 1997: Ensemble forecasting at NCEP and
the breeding method. Mon. Wea. Rev., 125, 3297 – 3319.
Tracton, M.S., and E. Kalnay, 1993: Ensemble forecasting at the
National Meteorological Center: Practical Aspects. Wea. and
Forecasting, 8, 379 – 39



The basic laws of geophysical fluid dynamics describe
fluid motions, they are a highly non-linear set of
differentials and/or differential equations.
e.g.,





dV V 
1

 V  V   p  2 V  V
dt
t

Given the proper set of initial and/or boundary
conditions, perfect resolution, infinite computer
power, and precise measurements, all future states of
the atmosphere can be predicted forever!


Given that this is not the case, these equations
have an infinite set of solutions, thus anything in
the phase space is possible.
In spite of this, the same “solutions” appear time
and time again!



Note: We will define Degrees of Freedom  here this will
mean the number of coordinates in the phase space.
Advances in this area have involved taking expressions
with an infinite number of degrees of freedom and
replacing them with expressions of finite degrees of
freedom.
For the equation of motion, whether we talk about math or
meteorology, we usually examine the N-S equations in 2D sense. Mathematically, this is one of the Million dollar
problems to solve in 3-d (no “uniqueness” of solutions!).

“Chaotic” Systems:

1. A system that displays SDIC.

2. Possesses “Fractal” dimensionality

Fractal geometry – “self similar”

Norwegian Model
L. Lemon



Fractals:
Fractal geometry was developed by Benoit Mandelbrot
(1983) in his book the Fractal Geometry of Nature.
Fractal comes from “Fractus” – broken and irregular.
Fractals are precisely a defining characteristic of the
strange attractor and distinguishes these from
familiar attractors.



3. Dissipative system:
Lyapunov Exponents - defined as the average rates of
exponential divergence or convergence of nearby
trajectories.
They are also in a very real sense, they provide a
quantitative measure of SDIC. Let’s introduce the concept
using the simplest type of differential equation.

Simple differential equation:
x  x
 or 
x  x  0

with the solution as:
xt   x0e  x0e
rt
 t
 co e
 t

Thus, after some large time interval “t”, the distance
e(t) between two points initially separated by e(0) is:
e t   e 0e


 t
Thus, the SIGN of the exponent  here is of crucial
importance!!!!
A positive value for – infers that trajectories separate
at an exponential rate, while a negative value implies
convergence as t  infinity!

Well, we can use our simple differential equation to get the
value of the exponent as:
1  e t  
  ln 
t e 0 


So, in the general case of our differential equation, we can
think of a (particular) solution as a point on the phase
space, and the neighboring points as encompassing an ndimensional ball of radius e(0)!
With an increase in time, the ball will become an ellipsoid
in non-uniform flow, and will continue to “deform” as time
approaches infinity.


There must be, by definition, as many
Lyapunov exponents as there are dimensions
in the phase space.
Again, positive values represent divergence,
while negative values indicate convergence of
trajectories, which represent the exponential
approach to the initial state of the Attractor!

There must also, by definition, be one
exponent equal to zero (which means the
solution is unity) or corresponds to the
direction along the trajectory, or the change
in relative divergence/convergence is not
exponential.
1
limt 0   3  fdVol   i  0
T Vol
i 1.. n

Now, for a dissipative system, all the
trajectories must add up to be negative!

Lorenz (1960), Tellus:
 2
2
ˆ
   k      
t

First Low Order Model (LOM) in meteorology,
derived using “Galerkin” methods, which
approximate solutions using finite series.
(e.g. Haltiner and Williams, 1980).


Lorenz (1960)
Tellus



Lorenz (1963), J. Atmos. Sci., 20, 130 - 142
Investigated Rayleigh – Bernard (RB)
convection, a classical problem in physics.
We need to scale the primitive equations (use
Boussinesq approx), then use Galerkin
Techniques again.


Lorenz (1963)
solution:
x    y  x 
y   xz  rx  y
z  xy  bz
 where 
  Pr
Ra
r
Racr
4
b
1  a 2 


Lorenz (1963) – then using the initial
conditions:  = 10.0 , b = 8/3, r = 28.0, and
a non-dimensional time step of 0.0005.
Then using 50 lines of FORTRAN code, and
the “leapfrog” method, we can produce:

Lo ren z (1 96 3 ) at tracto r
The “Butterfly”
25
1 2 .5
0
1 2 .5
25
50
25
0
50
3 7 .5
25
25
1 2 .5
50
zz y y  xx
0


We cannot solve Lorenz’s (1963) LOM unless
we examine steady state conditions; that is
dx/dt, dy/dt, and dz/dt all equal zero.
The “trivial solution” x = y = z = 0, is the
state of no convection.

But, if we solve the equations, we get some
interesting roots; (0 < r < 1).
1  b
2  
3  
 1
2
 1
   1
 
   r  1
2
 2 
  1
 
   r  1
2
 2 
2

But when r > 1, we get convection and
chaotic motions:
x  y  br  1, z  r  1
x  y   br  1, z  r  1


Predictability:
SDIC in the flow exists in set A if there exists
error > 0, such that for any x  A and any

neighborhood U of x, there exists y U and t
> 0. such that | g t x  g t y | e (error )


In ‘plane’ English: there will always be SDIC in a system
(it’s intrinsic to many systems). Possible outcomes are
larger than the error in specifying correct state!
SDIC means that trajectories are “unpredictable”, even if
the dynamics of a system are well-known
(deterministic).


Thus, if you wish to compute trajectories of
X in a system displaying SDIC, after some
time  t, you will accumulate error in the
prediction regardless of increases in
computing power!
There is always resolution and measurement
error to contend with as well. This will further
muddy the waters.
Singular Values and Vectors
•
•
Is the factor by which initial error will grow for
infinitesimal errors over a finite time at a particular
location (singular vectors, as the name implies, give
the direction).
Can be numerically estimated using linear theory.
Singular values/vectors are dependent upon the
choice of norm; they are critically state dependent.


Thus, after some large time interval “t”, the
distance e(t) between two points initially
separated by e(0) is (from slide 48 and 49):
e t   e 0e
 t

Thus, if the “error” doubles, or the ratio
between one trajectory and another:
e (t )
t
e S 2
e (0)

and the time to accomplish this is:
t
ln( 2)



This is the basis for stating that the
predictability of various phenomena is about
the size of it’s growth period. For
extratropical cyclones this is approximately
0.5 – 3 days.
For the planetary scale, the time period is
roughly 10 – 14 days (evolution of large-scale
troughs and ridges).


This is why we say that 10 – 14 days is the of
time is the limit of dynamic weather
prediction.
In atmospheric science, we know that this is
the time period for the evolution of Rossby –
inertia waves, which are the result of the very
size and rotation rate of the planet earth! (f =
2sinf)


Now, the question is, if we know exactly the
initial state (is it possible to know this?) of the
atmosphere at some time t, can we make
perfect forecasts?
This question is central to the contention that
the atmosphere contains a certain amount of
inherent unpredictability.

Laplace argued that given the entire and
precise state of the universe at any one
instant, the entire cosmos could be predicted
forever and uniquely, by Newton’s Laws of
motion. He was a firm believer in
determinism.



But, can we know the exact initial state? Let’s
revisit Heisenberg!
Exact solutions do exist, so in theory we can
find them.
What we can never do –even in principle - is
specify the exact initial conditions!


Measurement error and predictability:
If we solve for t (as we did earlier for errordoubling) :
ln x(t )   ln x(0) 
t
h

Where h is the sum of the positive Lyapunov
exponents.

Suppose our uncertainty is at a level of 10-5,
then:
ln x(t )   5 ln10
t1 
h


Now, let’s improve the accuracy by 5 orders
of magnitude, or 10-10:
ln x(t )   10 ln10
t2 
h

Then, we should be able to infer that:
t 2  2t1

Or, this increase in precision only doubles the
“forecast” time. Thus, input error, will swell
very quickly! Should we be pessimistic? 


Not a great “return” on investment! Pessimistic
about our prospects on forecasting? From a selfish
standpoint, no because this demonstrates that we
cannot turn over weather forecasting to computers.
From a scientific standpoint, no as well, because
we just need to realize that forecasting beyond a
certain limit at a certain scale is inevitable. As long
as we realize the limitations, we can make good
forecasts.


One beneficial issue has been stimulated for
operational meteorology by Chaos Theory,
and that is ‘how do we express “uncertainty”
in forecasts’?
Example:

The End!

Overtime!

Fractal Dimension:

We’re used to integer whole numbers for
dimensionality, but the Fractal can have a
dimensionality that is not a whole number.
For example, the Koch Snowflake (1904)
dimension is 1.26.

What? How can you have 1.25 dimensions?
But the snowflake fills up space more
efficiently than a smooth curve or line (1-D)
and is less efficient than an area (2-d). So a
dimension between one and two captures this
concept.

Example: (Sierpinski Gasket, 1915)

Has a Fractal (Hausdorf) dimension of 1.59

Hausdorf dimension:

d = ln(N(e)) / [ln(L) – ln(e)]


N(e) = is the smallest number of “cubes”
(Euclidian shapes) needed to cover the space.
Here it is: 3n or makes 3 copies of itself
with each iteration.


The denominator is: ln( L / e) where L = 1
(full space) and e is copy scale factor ((1/2)n
length of full space with each iteration).
So we get:
d = n ln(3) / n ln (2) = 1.59

Questions?

Comments?

Criticisms?

[email protected]