Transcript Bayesian modeling in the context of robust cue integration David C. Knill

Bayesian modeling in the context
of robust cue integration
David C. Knill
Center for Visual Science
University of Rochester
The Bayesian approach as a framework
for psychophysics
David C. Knill
Center for Visual Science
University of Rochester
Some properties of a useful
psychophysical framework
• Support building predictive models of perceptual
performance.
• Support bridging statements between models and
descriptions of behavior.
• Explain “why” perception / sensorimotor control
works the way it does.
• Help guide psychophysical research
– Suggests new and interesting theoretical questions.
– Supports scaling down perceptual / sensorimotor
problems to bring them into the lab.
– Scales up naturally
World
model
World
model
Noise
Sensory
processing
Generative model
Sensory
Features
Information
World
model
Noise
Sensory
processing
Generative model
Sensory
Features
p(S | I)
Bayesian
Computations
World
model
Noise
Sensory
processing
Generative model
Estimate
Sensory
Features
p(S | I)
*
Task
model
Bayesian
Computations
World
model
Noise
Sensory
processing
Generative model
Estimate
Sensory
Features
p(S | I)
*
Task
model
Bayesian
Computations
Ideal Observer
World
model
Noise
Sensory
processing
Generative model
Estimate
Sensory
Features
p(S | I)
*
Task
model
Bayesian
Computations
Ideal Observer
Estimate
Sensory
Features
Human Observer
Estimate
Sensory
Features
p(S | I)
*
Task
model
Bayesian
Computations
Rational Observer
World
model
Generative model
Estimate
Sensory
Features
p(S | I)
*
Task
model
Bayesian
Computations
Rational Observer
Ideal observer models
Rational observer models
The domain of
Bayesian models
Description
of sensorimotor / perceptual
behavior
Cue integration:
Estimating slant from monocular
and binocular cues
Linear process model
Texture data
(It)
Stereo data
(Is)
Slant from
texture
Slant from
stereo
St
wt
Ss
ws
+
Sst
Action /
Decision
Normative (ideal observer) model
Stereo likelihood
Normative (ideal observer) model
Texture likelihood
Normative (ideal observer) model
Joint likelihood
Humans weight sensory cues
“optimally”
• Discrimination thresholds in single cue
conditions predict weights measured in
multi-cue experiments.
– Ernst and Banks, 2002; Knill and Saunders,
2003; Alais and Burr (2004); etc., etc., etc.
Texture information
Binocular information
Least Reliable
Equally reliable
Most Reliable
What are cue weights?
What are cue weights?
• Summary descriptions of perceptual
performance.
What are cue weights?
• Summary descriptions of perceptual
performance.
• Summary descriptions of the information
available for a task.
What are cue weights?
• Summary descriptions of perceptual
performance.
• Summary descriptions of the information
available for a task.
• Support logical links between behavior and
rational / normative models of performance.
Robust non-linear cue integration
Robust non-linear cue integration
• Classical question
– How does the brain interpret multiple sensory
cues when they have large “conflicts”
Robust non-linear cue integration
• Classical question
– How does the brain interpret multiple sensory
cues when they have large “conflicts”
• For visual depth cues
– Re-conceptualize the problem
• what normally gives rise to what we call large cue
“conflicts?”
Answer
• Most depth cues are informative because of
statistical regularities in the world
Answer
• Most depth cues are informative because of
statistical regularities in the world
– Examples
• Texture - isotropy, homogeneity
• Figure shape - isotropy, symmetry
• Motion - rigidity
Answer
• Most depth cues are informative because of
statistical regularities in the world
– Examples
• Texture - isotropy, homogeneity
• Figure shape - isotropy, symmetry
• Motion - rigidity
• Constraints don’t always apply
– True prior model = mixture of constraints
Answer
• Most depth cues are informative because of
statistical regularities in the world
– Examples
• Texture - isotropy, homogeneity
• Figure shape - isotropy, symmetry
• Motion - rigidity
• Constraints don’t always apply
– True prior model = mixture of constraints
• Large “conflicts” arise when strong
constraint does not hold.
Compression cue = slant suggested by circle interpretation
Likelihood function - p(aspect ratio slant)
p(circle)
+
p(ellipse)
Figure shape cue
Disparity cue
Combined cues
Predictions of mixture model
All circles
Circles + narrow
range of ellipses
Circles + broad
range of ellipses
Stereoscopic slant = 35o
Model fits
Stereoscopic slant = 35o
Model fits
Stereoscopic slant = 55o
Conclusions
• Real cue integration problems are highly “nonlinear”
• The linear-gaussian Bayesian model scales up
easily - e.g. add a mixture component to the prior
on shape.
• The linear-gaussian process model does not scale
up.
• Bayesian formulation suggests new questions
– How does rational observer model compare to a
normative model.
– How do observers adapt their prior models across
different environments?
Bayesian learning model
Disparities
Slant
Estimate
slant and
shape
Retinal shape
Prior distribution of
figure shapes
Figure Shape
Behavior of the learning model
Results of matching experiment
Thank you