Slide 1

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 2

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 3

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 4

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 5

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 6

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 7

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 8

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 9

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 10

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 11

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 12

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 13

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 14

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 15

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 16

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 17

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 18

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 19

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 20

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 21

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 22

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 23

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 24

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU


Slide 25

The law of series
SoMaChi Symposium, November, 2007

Tomasz Downarowicz
Institute of Mathematics and Computer Science
Wroclaw University of Technology
Wybrzeze Wyspiańskiego 27
50-370 Wroclaw, Poland

What is the law of series?
In the common sense, a series is noted
when a random event considered
extremely rare happens several times in a
relatively short period of time. The name
"law" suggests that such series are
observed often enough to indicate an
unexplained physical force or statistical
rule provoking them.

Examples
In 1891 an Englishman Charles Wells, during one night,
twelve times broke the bank at one of Monte Carlo casinos
winning a million francs. At one stage he won 23 times out of
30 successive spins of the wheel. Wells returned to Monte
Carlo in November of that year and won again. During this
session he made another million francs in three days,
including successful bets on the number five for five
consecutive turns.

For 250 years English bell ringers have tried to answer the
question whether it is possible to ring all permutations of
seven bells following a certain "method" called common
bob Stedman triples. On January 22, 1995 a team at Saint
John's Church in London, has finally succeeded. Within few
days it was revealed that two other teams, working
independently and not knowing about each-other, had also
solved this century-old bell-ringing (and in fact
mathematical) mystery.
In Lower Silesia, Poland, two major floods
occurred in 1997 and 1998, after many
years of silence.

Clusters
In many random processes in reality we observe
this phenomenon, often called clustering. This
applies for example to climate anomalies, some
other natural cataclysms, power shortages,
certain types of occurrences in the stock market,
etc.
Understanding whether clusters indeed appear
(or are they just a random fluctuation), and why,
could be an important step toward a having a
better control over these processes.

Paul Kammerer
An Austrian biologist dr. Paul
Kammerer (1880-1926) was
the first scientist to study the
law of series (law of seriality,
in some translations). His
book Das Gesetz der Serie
contains many examples
from his and his nears' lives.

Kammerer’s ”series”
Kammerer observed (or received reports from
his relatives and friends) of ”series” of
encounters of the same number (in various
situations), meeting people with the same last
name, and the like. He spent hours sitting in a
park and noting ”series” of people wearing
glasses. Or he simply observed the times when
clients enter a shop and ”discovered” that the
average number per minute actually occurs
very rarely, yielding to periods of absence or of
high occupancy.

Synchronicity
The law of series is a particular
case of the theory of
synchronocity postulated by a
Swiss professor of philosophy
Karl Gustav Jung (1875-1961 )

and a Nobel prize winner in
physics, Austrian Wolfgang
Pauli (1900-1958)
Wolfgang Pauli

The opposition
Other scientists believe that synchronicity and
law of series are just manifestations of ordinary
random fluctuations.
American mathematician,
Warren Weaver (1894-1978)
(collaborator of Calude Shannon)
argues that such events have
positive probability, hence MUST
occur from time to time.

Attracting and repelling
In order to give the law of series a definite meaning we
define attracting as a deviation of a signal process
toward clustering stronger than in the Poisson process .
Similarly, repelling is defined as clustering weaker than
in the Poisson process.

There is no doubt that the postulates of Pauli and Jung
concern so defined attracting. As to Kammerer,
apparently less familiar with probability theory, it seems
that in most of his experiments he merely "discovered"
the ordinary clustering of the Poisson process.

• In many real processes attracting is
perfectly understandable as a result of
strong physical dependence. A good
example here are series of ill-fallings due
to a contagious disease.
• The dispute on the law of series clearly
concerns only events for which there are
no obvious attracting mechanisms, and we
expect them to appear completely
independently, governed by pure chance.

Our voice in the debate
Jointly with Yves Lacroix
we have obtained two
results in ergodic theory
which support Pauli and Jung.
Roughly speaking, we have proved that with
regard to "elementary" events (cylinder sets of
small probability):
1. In any process of positive entropy attracting is
possible while repelling is not;
2. In the majority of processes many elementary
events occur with very strong attracting.

Rigorous definitions
Because attracting and repelling must not
depend on the intensity  of the signals we
change the time unit so that  = 1
We call this normalization. In the normalized
Poisson process, the waiting time for the first
signal has exponential distribution:
FP(t) = 1– e–t (t  0).
We call such behavior unbiased.

• We will say that a normalized stationary signal
process reveals attracting with intensity  from a
distance t, if the distribution function F of the
waiting time for the first signal satisfies the
inequality
F(t) < 1– e–t –  
• Generally, we will say that a process reveals
attracting only, if at each t holds,
F(t)  1– e–t ,
without the two functions being equal (i.e., strict
inequality holds at some point). Analogously,
inverted inequalities (with +) define repelling.

Interpretation
In an interval of time of length t the expected number of
signals is t, i.e., t. The value F(t) is the probability, that
in such interval there will be at least one signal. The ratio
t/F(t) hence represents the conditional expectation of the
number of signals in these time intervals of length t in
which at least one signal is observed. If F(t) < 1– e–t, this
conditional expectation is larger than in the Poisson
process. In other words, if we observe the process for
time t, there are two possibilities: either we detect
nothing, or, once the first signal occurs, we expect a
larger global number of observed signals than if we were
dealing with the Poisson process. The first signal
"attracts" further repetitions.

Ergodic theory setup
• We will consider a dynamical system consisting
of a probability space (X,Σ,μ) and a measurepreserving transformation T: X→X. If P is a finite
measurable partition of X, it generates a process
(X,Σ,μ,P) the elements of which are sequences
over the alphabet P and the transformation is the
shift T(xn) = (xn+1).
• A cylinder set B of length n is a set
{x: x[0,n-1] = B} (BєPn)

The main theorems
• I. Any stationary ergodic finite-states process with
positive entropy has the following property: For every 
>0 the joint measure of all cylinders of length n,
revealing repelling with intensity  (from any distance t)
converges to zero as n tends to infinity.
• II. In every non-periodic dynamical
system there exists a residual set
of partitions P with the following
property: There exists a subset of
natural numbers of upper density 1,
such that every P-cylinder of a length from this set
reveals attracting with intensity close to 1.

A comment on the theorem
• It concerns exclusively small sets, i.e., rare events.
It does not apply to the observations of single
numbers in a roulette, umbrellas or glasses on
passing pedestrians (one of Kammerer's favorite
experiment subjects). More adequate is the event
of breaking the roulette bank.
• It concerns cylinder sets, that is exact repetitions of
a specific configuration of large events. Breaking a
roulette bank is a union of several cylinders rather
than one cylinder.
• The best example is the occurrence of a specific
long configuration in a computer program or in a
genetic code.

Sketch of the proof of Theorem 1
The proof consists of one technical trick and two major
observations. The trick is to consider the repetitions of
a concatenation BA whose left part B is much longer
than the right part A. For a moment we can think that
A is the last letter of the considered block. Then we
observe the process of repetitions of the block B and
the process of symbols directly following these
repetitions. On the figure below, a realization of such
"induced" process is the sequence ...
A-1A0A1A2...

We prove that for a typical long block B, such induced
process is almost independent.

Crush course on entropy
• If P = {C1,C2,…,Cr} is a finite partition of X into sets of
measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or
static) entropy equals H(μ,P) = -Σ pi logpi.
• Given two finite partitions P and Q their join is defined as
PvQ = {C∩D: CєP, DєQ}.
• Conditional entropy:
H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q)
Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and
only if the partitions are independent.
• Partitions P and Q are almost independent if
H(μ,P|Q) > H(μ,P) - .

Entropy of a process
The Kolmogorov-Sinai or dynamical entropy of a
process (X,Σ,μ,P) is defined as
h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),
where P-n = PvT(P)v…vTn-1(P).
Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only
for an independent process (the partitions Tn(P)
are independent). The process is called
- independent if
h(μ,T,P) > H(μ,P) - 

Proof of -independence of the
induced process

This implies that except for cylinders B of joint measure
at most  the induced process is -independent

• Much more difficult is to prove that this process is also
 -independent of the positioning of the repetitions of B.
• The second key observation is much easier. Assume for
simplicity, that the independencies are strict. Then it is
not hard to prove that the strongest repelling for the
block BA occurs, when B is distributed with the maximal
possible repelling, i.e., periodically.
But with such distribution of the B's, and with the
assumed independence, the occurrences of BA are the
same as in an independent process with discrete time
(with unit equal to the period of the B's).

The proof of the second result (that in a
typical process cylinders of certain lengths
reveal strong attracting) is completely
different and does not use advanced
ergodic theory. Instead we construct
explicitly a dense family of partitions which
satisfy an open condition.

Conclusion
Suppose we observe a process which we believe is
independent (for example tossing a coin). In reality such
process is never perfectly independent; the generating
partition is always slightly perturbed from independent.
Then we know that such perturbation may only result in
attracting, never in repelling. Moreover, typically it will
lead to strong attracting for cylinders of certain large
lengths. For example, if we are tossing a 0-1 coin and
we are interested in seeing a specific sequence (e.g. the
first 1000 digits of the binary expansion of π) then it is
very likely that we will observe this sequence appearing
in strong clusters (relatively to its probability).
But remember! This implies that the waiting time for the
first occurrence usually is much longer than the inverse
of the probability!

THANK YOU