Transcript Slides

MaxEnt 2006
CNRS, Paris, France, July 8-13, 2006
A new bound for discrete distributions
based on Maximum Entropy
†
Gzyl H., ‡Novi Inverardi P.L. and ‡Tagliani A.
†
USB and IESA - Caracas (Venezuela)
‡
Department of Computer and Managements Sciences
University of Trento (Italy)
Aim of the paper
MaxEnt 2006
Our aim is to compare some classical bounds for nonnegative integer-valued
random variables for estimating the survival probability
and a new tighter bound obtained through the Maximum Entropy method
constrained by fractional moments given by
We will show the superiority of this last bound with respect to the others.
A new bound for discrete distributions based on Maximum Entropy
2
MaxEnt 2006
Some classical distribution bounds
Most used candidates as an upper bound of
are:
• the Chernoff’s bound (Chernoff (1952))
• the moment bound (Philips and Nelson (1995))
• the factorial moment bound
These three bounds come from the Markov Inequality and involve only
integer moments or moment generating function ( mgf ).
A new bound for discrete distributions based on Maximum Entropy
3
MaxEnt 2006
Akhiezer’s bound
Given two distribution F and G sharing the first 2Q moments, the following
distribution bound is well known in literature (Akhiezer (1965)),
where
,
and
is the Hankel matrix.
For any real value of x, Q(x) represents the maximum mass can be
concentrated at the point x under the condition that 2Q moments are met;
being Q(x) the reciprocal of a polynomial of degree 2Q in x, the bound goes
to zero at the rate x -2Q as x goes to infinity: this behavior gives relatively
sharp tail information but no much on the central part of the distribution.
A new bound for discrete distributions based on Maximum Entropy
4
Improving Akhiezer’s bound
MaxEnt 2006
The main question is
how to choose
the approximant distribution sharing the same 2Q moments of F(x) which allows
the bound improvement.
A key role in achieving this improvement is played by the MaxEnt technique.
constrained by integer moments
MaxEnt
constrained by fractional moments
A new bound for discrete distributions based on Maximum Entropy
5
Improving Akhiezer’s bound: integer moments
MaxEnt 2006
If
 X is a discrete r.v. with pmf

with assigned
is the MaxEnt M-approximant of P based on
the first M integer moments
then combining
and
we prove that the Akhiezer’s bound can be replaced by the following uniform
bound
This bound is tighter than Akhiezer’s bound because
.
Further, H[P] is unknown but, it can be estimated through Aitken D2- method.
A new bound for discrete distributions based on Maximum Entropy
6
Improving Akhiezer’s bound: fractional moments
MaxEnt 2006
If
• X is a non negative discrete r.v. with pmf
•
a sequence of M+1 fractional moments
with
•
chosen according to
is the MaxEnt M-approximant of P, based on
the previous M fractional moments
then the chain of inequalities, similarly in the case of integer moments, gives
Numerical evidence proves that
, even for small M, then the
Akhiezer’s bound can be replaced by the following new uniform and computable
bound
A new bound for discrete distributions based on Maximum Entropy
7
MaxEnt 2006
Improving Akhiezer’s bound: a new bound
The proposed bound is sharper than Akhiezer’s bound in the central part of the
distribution and vice versa, the Akhiezer’s bound is sharper than it in the tails.
Combining Akhiezer’s and our MaxEnt bound, we have a sharper upper
bounds, valid for X ≥ 0 and M which guarantees
:
where
is obtained from
through a convergence
accelerating process, so that
may be assumed. Here the
maximum value allowed of Q stems from the number of given moments or
from numerical stability requirements.
A new bound for discrete distributions based on Maximum Entropy
8
How to calculate fractional moments?
MaxEnt 2006
As seen, the fractional moments play the role of building blocks of the
proposed procedure.
But, how to calculate them?
Several scenarios will be analyzed, depending on the available information on
the distribution of
. This latter is assumed given by a finite or infinite
sequence of moments and/or by the mgf.
Here we present three cases:
a) both
b)
c)
and
are known;
assigned and existing mgf ;
known, R finite or infinite
A new bound for discrete distributions based on Maximum Entropy
9
Case a): both
and
are known
MaxEnt 2006
A new bound for discrete distributions based on Maximum Entropy
10
Case b)
assigned and existing mgf
MaxEnt 2006
A new bound for discrete distributions based on Maximum Entropy
11
Case c)
known, R finite or
MaxEnt 2006
infinite
A new bound for discrete distributions based on Maximum Entropy
12
MaxEnt 2006
A numerical example (1)
A new bound for discrete distributions based on Maximum Entropy
13
MaxEnt 2006
A numerical example (1)
A new bound for discrete distributions based on Maximum Entropy
14
MaxEnt 2006
A numerical example (1)
A new bound for discrete distributions based on Maximum Entropy
15
MaxEnt 2006
A numerical example (2)
A new bound for discrete distributions based on Maximum Entropy
16
MaxEnt 2006
A numerical example (2)
A new bound for discrete distributions based on Maximum Entropy
17
THANK YOU
for your attention!
A new bound for discrete distributions based on Maximum Entropy
18
Density reconstruction via ME technique
MaxEnt 2006
A new bound for discrete distributions based on Maximum Entropy
19
Density reconstruction via ME technique
MaxEnt 2006
But it is not the unique choice!
A new bound for discrete distributions based on Maximum Entropy
20
An uniform bound
MaxEnt 2006
A new bound for discrete distributions based on Maximum Entropy
21
Aitken
D2-method
MaxEnt 2006
A new bound for discrete distributions based on Maximum Entropy
22
How to choose fractional moments?
MaxEnt 2006
A new bound for discrete distributions based on Maximum Entropy
23
Fractional moments
MaxEnt 2006
A new bound for discrete distributions based on Maximum Entropy
24
Comparing and combining bounds
MaxEnt 2006
A new bound for discrete distributions based on Maximum Entropy
25