Transcript slides - Caroline Uhler

ML
ML estimation
estimation in
in Gaussian
Gaussian
graphical
graphical models
models
Caroline Uhler
Department of Statistics
UC Berkeley
Convex Algebraic Geometry Seminar
March 17, 2010
Outline
Outline
Background and setup:
- Gaussian graphical models
- Maximum likelihood estimation
Existence of the maximum likelihood estimate (MLE)
“Cone” problem
“Probability” problem
ML degree of a graph
Example:
Gaussian
Gaussian graphical
graphical models
models
-
undirected graph with
-
covariance matrix
-
concentration matrix with
Gaussian graphical model:
Data:
-
i.i.d samples from
-
sample covariance matrix
-
sufficient statistics
Example
Example K
K2,3
2,3
5
4
3
1
2
The corresponding Gaussian graphical model consists of multivariate
Gaussians with concentration matrix of the form
Given a sample covariance matrix
the sufficient statistics are
Maximum
Maximum likelihood
likelihood estimation
estimation
Log-likelihood function:
Theorem
Theorem(regular
(regularexponential
exponentialfamilies):
families):
In
InaaGaussian
Gaussiangraphical
graphicalmodels
modelsthe
theMLEs
MLEs
i.e.
i.e.
Then
Then
isisPD-completable.
PD-completable.
isisuniquely
uniquelydetermined
determinedby
by
and
and
exist
existififand
andonly
onlyifif
Cones
Cones
Cone of concentration matrices:
Cone of sufficient statistics:
where
respectively
is the convex dual to
Cones
Cones and
and statistical
statistical theory
theory
Theorem
Theorem(exponential
(exponentialfamilies):
families):
The
Themap
map
isisaahomeomorphism
homeomorphismbetween
between
and
and
The
takes
Theinverse
inversemap
map
takesthe
thesufficient
sufficientstatistics
statisticsto
tothe
theMLE
MLEof
of
the
isisthe
theconcentration
concentrationmatrix.
matrix.Here,
Here,
theunique
uniquemaximizer
maximizerof
ofthe
the
determinant
determinantover
over
Existence
Existence of
of MLE:
MLE: 22 Problems
Problems
Given a graph
?
Under what conditions on
(i.e. describe
?
does the MLE exist?
)
Under what conditions on
does the MLE exist?
Problem
Problem 1:
1: Example
Example K
K2,3
2,3
Example:
Example:
5
4
3
1
2
Problem
Problem 1:
1: Example
Example K
K2,3
2,3
Existence
Existence of
of MLE
MLE for
for the
the m-cycle
m-cycle
m
1
2
3
Theorem
Theorem(Barrett,
(Barrett,Johnson
Johnson&&Tarazaga,
Tarazaga,1993):
1993):
The
TheMLE
MLEexists
existsfor
for
odd,
odd,
ififand
andonly
onlyififfor
foreach
each
with
with
Existence
Existence of
of MLE
MLE for
for K
K2,m
2,m
Theorem
Theorem(U.):
(U.):
The
TheMLE
MLEexists
existsfor
for
ififand
andonly
onlyifif
Existence
Existence of
of MLE:
MLE: 22 Problems
Problems
Given a graph
?
Under what conditions on
?
Under what conditions on
And with what probability?
does the MLE exist?
does the MLE exist?
Probability
Probability of
of existence
existence
Reminder:
MLE exists
Note:
Note: Existence
Existenceof
ofthe
theMLE
MLEisis invariant
invariantunder
under
a)
a)Rescaling:
Rescaling:
where
where
isisdiagonal.
diagonal.
b)
b)Orthogonal
Orthogonaltransformation:
transformation:
where
where
Assume:
have length 1 and
isisorthogonal.
orthogonal.
Existence
Existence of
of MLE
MLE for
for the
the m-cycle
m-cycle
m
1
2
3
Theorem
Theorem(Buhl,
(Buhl,1993):
1993):
The
TheMLE
MLEexists
existson
onthe
them-cycle
m-cyclewith
withprobability
probability11for
for
not
notexist
existfor
for
For
let
For
let
The
TheMLE
MLEexists
existsififand
andonly
onlyifif
((
and
anddoes
does
).).
are
arenot
notgraph
graphconsecutive.
consecutive.
Existence
Existence of
of MLE
MLE for
for K
K2,m
2,m
Theorem
Theorem(U.):
(U.):
The
TheMLE
MLEexists
existson
on
exist
existfor
for
with
withprobability
probability11for
for
For
let
For
let
The
TheMLE
MLEexists
existsififand
andonly
onlyifif
lie
lieoutside
outside
and
and
and
anddoes
doesnot
not
((
lie
liebetween
between and
and
This
Thishappens
happenswith
withprob.
prob.
).).
or
or
ML-degree
ML-degree of
of aa graph
graph
The maximum likelihood degree of a statistical model is the
number of complex solutions to the likelihood equations for generic
data.
Generic data: The number of solutions is a constant for all data,
except possibly for a lower-dimensional subset of the data space.
ML-degree
ML-degreeAof
of aa graph
graph
Theorem
Theorem(Sturmfels
(Sturmfels&&U.,
U.,2010):
2010):
chordal
chordalififand
andonly
onlyifif
Conjecture
Conjecture(Drton,
(Drton,Sullivant
Sullivant&&Sturmfels,
Sturmfels,2009):
2009):
The
TheML-degree
ML-degreeof
ofan
anm-cycle
m-cycle
Theorem
Theorem(U.):
(U.):
isisgiven
givenby
by
Click to add title
The
TheML-degree
ML-degreeof
ofthe
thebipartite
bipartitegraph
graph
isisgiven
givenby
by
Proof
Proof
Let
generic. Without loss of generality
ML-degree is the number of complex solutions to
diagonal
diagonal
degree 2m+1
Barrett, Johnson & Loewy (1996), The real positive definite completion
problem: cycle completability, Mem. Amer. Math. Soc. 584.
Buhl (1993), On the existence of maximum likelihood estimators for
graphical Gaussian models, Scan. J. of Stat. 20.
Sturmfels & U.: Multivariate Gaussians, semidefinite matrix
completion, and convex algebraic geometry (to appear in AISM)
U.: Maximum likelihood estimation in Gaussian graphical models (in
progress)
k
n
a
Th
!
u
yo