Transcript Bayesian Belief Propagation

Bayesian Belief Propagation
Reading Group
Overview
Problem Background


Bayesian Modelling
Markov Random Fields
Examine use of Bayesian Belief Propagation
(BBP) in three low level vision applications.



Contour Motion Estimation
Dense Depth Estimation
Unwrapping Phase Images
Convergence Issues
Conclusions
Problem Background
A problem of probabilistic inference
Estimate unknown variables given
observed data.
For low level vision:

Estimate unknown scene properties (e.g.
depth) from image properties (e.g. Intensity
gradients)
Bayesian models in low level vision
A statistical description of an estimation problem.

Given data d, we want to estimate unknown parameters u
Two components


Prior Model p(u) – Captures know information about unknown
data and is independent of observed data. Distribution of
probable solutions.
Sensor Model p(d|u) – Describes relationship between sensed
measurements d and unknown hidden data u.
Combine using Bayes’ Rule to give the posterior
Markov Random Fields
Image Data Nodes (d)
Pairwise Markov Random Field:
Model commonly used to represent images
Hidden Scene Nodes (u)
Sensor model
Prior model
ui
Neighborhood Ni
p(ui | u)  p(ui | {uk }), uk  Ni
Contour Motion Estimation
Yair Weiss
Contour Motion Estimation
Estimate the motion of contour using only local
information.
Less computationally intensive method than
optical flow.
Application example: object tracking.
Difficult due to the aperture problem.
Contour Motion Estimation
Actual
Ideal
Aperture Problem
Contour Motion Estimation
ui  xi , y i ,1
ui+1
T
ui
i+2
Brightness Constant Constraint Equation
i+1
ui-1
Prior Model: ui+1 = ui + n
dI I
I
I
 where
x  n ~yN(0,sp) 0
dt x
y
t
i
 I i I i I i 
di  
,
,


x

y

t


i-1
i-2
where Ii
di-2
ui-2
di-1
ui-1
di
ui
di+1 di+2
ui+1 ui+2
p(ui
T
= I(xi,yi,t)

T
| d i )i  iexp  
d u 0


d u 
T
i
2s
2
i
2
i




 (ui 1  ui ) 2 

p(ui 1 | ui )  exp  
2


2
s
p


1D Belief Propagation
di-1
di-2
1
ui 
s
2
i
di 
1
s i2
ui-2
1
s 
 2
i

i 
1
1
i
Iterate
s  until message values converge

 2
i
1
s  s 
 2
di+2
di+1
di
i
 2
 i ( i , s i )
i
ui-1
i ( i , s i )
ui+2
ui+1
ui
1
i 

s i21
d i 1 
1
s 

2
i1
i 1
1
22 
1
 (ui  di ) 2 (ui  si i)1 s
( ii12 ui ) 2 
p(ui | dp)(ubi i|(d
ui)) 
 exp
p(d i| ui ) p2(ui |ui 1 ) p (u2 i |ui 1 )  2 
2s i
2(s i )
2(s i ) 1 

s 
 2
i
 1
1
s  2 
 s i 1 s 
i 1

2
p


2 

 
Results
Contour motion estimation [Weiss]
Faster and more accurate solutions over pre-existing methods such
as relaxation.
Results after iteration n are optimal given all data within distance of
n nodes.
Due to the nature of the problem, all velocity components should
and do converge to the same value.
Interesting to try algorithm on problems where this is not the case
Multiple motions within the same contour
Rotating contours (requires a new prior model)
Only one dimensional problems tackled but extensions to 2D are
discussed.
Also use of algorithm to solve Direction Of Figure (DOF) problem
using convexity (not discussed)
Dense Depth Estimation
Richard Szeliski
Depth Estimation
Assume smooth variation in disparity
i
Depth Zi
Disparity ui = 1 / Zi
Define prior using Gibbs Distribution:
Ep(u) is an energy functional:
E p (u) 

 E p (u) 

p(u)  exp  
T 


1
1 T
2
2




u
(
x

1
,
y
)

u
(
x
,
y
)

u
(
x
,
y

1
)

u
(
x
,
y
)

u A pu

2 ( x, y )
2
Depth Estimation
Disparity:
s i2
di 
1
Zi
related to
correlation metric
di
i

d  d1 , d 2 , d 3 ,  d n 2
Image T=0
Image T=1
Image T=t
Image T=t+1
Image T=t+2
Image t=t+3

T

u  u1 , u2 , u3 , un 2

T
d  Hu  R, where R ~ Ν (0,Σ)
p(d | u)  exp  Es (u) 
s 1 0
0 s
Es(u) is an energy functional:
2

Where H is a measurement
matrix
and
Σ
1
T
 0
Es (u)  Hu  d  Σ 1 Hu  d 
2

 0 0





 s n 2 

0

0
0

Depth Estimation
Posterior:
p(u | d)  p(d | u) p(u)  exp  E(u)
E(u) is the overall energy: E (u)  E p (u) / T  Es (u)
1
E (u)  uT Au  uT b  c
2
where
A  A p  HT Σ1H
and
b  HT Σ 1d
Energy function E(u) minimized when u=A-1b
Matrix A-1 is large and expensive to compute
Gauss-Seidel Relaxation
Minimize energy locally for each node ui keeping all other
nodes fixed.

Leads to update rule:


u  a  bi   aiju j 
jN i



i


1
ii
This is also the estimated mean of the marginal probability
distribution p(ui|d) given by Gibbs Sampling.
For the 1-D example given by Weiss:
1

i
u 
s i2
di 
1
s p2
1
s i2
ui 1 

2
s p2
1
s p2
1
ui 1
ui 
s
2
i
di 
1
s i2
1
s 
 2
i

1
i 
s 

1
1
 2
i
s  s 
 2
i
 2
i
i
Results
Dense depth estimation [Szeliski]
Dense (per pixel) depth estimation from a sequence of images with
known camera motion.
Adapted Kalman Filter: estimates of depth from time t-1 are used to
improve estimates at time t.
Uses multi-resolution technique (image pyramid) to improve
convergence times.
Uses Gibbs Sampling to sample the posterior.

Stochastic Gauss-Seidel relaxation
Not guaranteed to converge.
Problem can be reformulated to use message passing.
Does not account for loops in the network, only recently has belief
propagation in networks with loops been fully understood [Yedidia et
al]
Unwrapping Phase Images
Brendan Frey et al
Unwrapping Phase Images
Wrapped phase images are produced by
devices such as MRI and radar.
Unwrapping involves finding shift values
between each point.
Unwrapping is simple in one dimension


One path through data
Use local gradient to estimate shift.
For 2D images, the problem is more difficult
(NP-hard)


Many paths through the data
Shifts along all paths must be consistent
Zero-Curl Constraint
Data Point
(x,y+1)
a(x,y+1) (x+1,y+1)
Shift node
b(x,y)
b(x+1,y)
Constraint Node
(x,y)
a(x,y)
(x+1,y)
a( x, y)  b( x  1, y)  a( x, y  1)  b( x, y)  0
Sensor Data
Estimating relative shift (variables a and b) values [-1,0 or 1] between
each data point.

Use local image gradient as sensor input
 I (1,0)  I (0,0) 
 I (0,1)  I (0,0) 







Sensor nodes: d   I ( x  1, y)  I ( x, y)
 I ( x, y  1)  I ( x, y)





 I (n  1, n)  I (n, n) 


Gaussian sensor model:
 a (0,0) 
 b(0,0) 


  


Hidden shift nodes: u  a ( x, y )
 b ( x, y ) 





 b ( n, n ) 


Estimate from wrapped image
 1

p(d | u)  exp   (u  d)T Σ 1 (u  d) 
 2

Belief
1.0
Belief Propagation
Data Point
m5
Shift node
-1
m5
m4
m1
m3
m2
Constraint Node
1
1
1
  

(m4i m5i )
2
(m
a ( x, 
y ) exp
i )  p(a(((dkx*,y)l
i2i| d/)2s
m4i b

j )m
T j m2 k m3l
1
i
)
m
5i
m 4 m4i5
j  1k  1l  1
0
1
a(x,y)
Results
Unwrapping phase images [Frey et al.]
Initialize message to uniform distribution and iterate to
convergence.
Estimates a solution to an NP-Hard problem in O(n) time
in the number of the nodes.
Reduction in reconstruction error over relaxation
methods.
Does not account for loops in the network, messages
could cycle leading to incorrect belief estimates.
Not guaranteed to converge.
Convergence
Convergence only guaranteed when network is
a tree structure and all data is available.
In networks with loops, messages can cycle
resulting in incorrect belief estimates.
Multi-resolution methods such as image
pyramids can be used to speed up convergence
times (and improve results).
Conclusion
BBP used to infer marginal posterior distribution of hidden
information from observable data.
Efficient message passing system is linear in the number of nodes
as opposed to exponential.
Propagate local information globally to achieve more reliable
estimates.
Useful for low level vision applications



Contour Motion Estimation [Weiss]
Dense Depth Estimation [Szeliski]
Unwrapping Phase Images [Frey et al]
Improved results over standard relaxation algorithms.
Can be used in conjunction with multi-resolution framework to
improve convergence times.
Need to account for loops to prevent cycling of messages [Yedidia et
al].