Transcript Recognition by Probabilistic Hypothesis Construction.

Recognition by Probabilistic
Hypothesis Construction
P. Moreels, M. Maire, P. Perona
California Institute of Technology
Background
Rich features
Probabilistic
constellations,
categories
Efficient
matching
• Fischler & Elschlager, 1973
• v.d. Malsburg et al. ‘93
• Burl et al. ‘96
• Weber et al. ‘00
• Fergus et al. ‘03
• Huttenlocher & Ullman, 1990
• Lowe ‘99, ‘04
Rich features,
probabilistic,
fast learning,
efficient matching
Outline
Objective: Individual object recognition
• D.Lowe, constellation model.
• Hypothesis and score.
• Scheduling of matches.
• Experiments: compare with D.Lowe.
Lowe’s recognition system
Models
…
Test image
Lowe’99,’04
Constellation model
Burl’96, Weber’00, Fergus’03
Pros and Cons
Lowe’s recognition system
+
-
Constellation model
•
Many parts  redundancy
• Principled
detection/recognition
•
Learn from 1 image
• Learn parameters from data
•
Fast
•
Manual tuning of parameters
•
Rigid planar objects
•
Sensitive to clutter
• Model clutter, occlusion,
distortions
•
High number of parameters (O(n2))
• 5-7 parts per model
• many training examples needed
• learning expensive
How to adapt the constellation
model to our needs ?
Reducing degrees of freedom
1. Common reference frame ([Lowe’99],[Huttenlocher’90])
model m
position of
model m
2. Share parameters ([Schmid’97])
3. Use prior information learned on foreground and background ([FeiFei’03])
Parameters and priors
Foreground
Constellation model
Clutter
Gaussian
Gaussian part
Gaussian
Gaussian background
shape pdf
appearance pdf
relative scale pdf
appearance pdf
log(scale)
Prob. of detection
0.8
0.8
0.9
0.75
Foreground
Sharing parameters
Clutter
Gaussian conditional
Gaussian part
Gaussian
Gaussian background
shape pdf
appearance pdf
relative scale pdf
appearance pdf
log(scale)
Prob. of detection
Based on [Fergus’03][Burl’98]
0.2
0.8
0.2
0.2
Hypotheses – features assignments
New scene (test image)
= models
from database
...
Interpretation
...
Hypotheses – model position
New scene (test image)
Models from
database
1
2
3
Θ = affine transformation
Score of a hypothesis
observed features
geometry + appearance
Hypothesis:
model + position + assignments
database of models
(Bayes rule)
Consistency
constant
Hypothesis probability
Score of a hypothesis
- Consistency between observations and hypothesis
‘null’ assignments
foreground features
geometry
appearance
geometry
- Probability of number of clutter detections
- Probability of detecting the indicated model features
- Prior on the pose of the given model
appearance
Efficient matching process
Scheduling – inspired from A*
scene features, no assignment done
empty hypothesis
‘null’ assignment
…
1 assignment
…
2 assignments
P P P P
can be compared
Score
P P
P P
 explore most promising
branches first
Pearl’84,Grimson’87
…
P P
P P
perfect completion
(admissible heuristic,
used as a guide for the search)
Increase computational efficiency:
- at each node, searches only a fixed number
of sub-branches
- forces termination
Recognition: the first match
No clue regarding geometry
 first match based on appearance
Initialization of
hypotheses queue
PPPPP
PPPPP
….
New scene
models
from
database
….
PPPPP
Scheduling – promising branches first
Updated hypotheses
queue
PPPP
models
from
database
….
New scene
PPP
….
?
PPP
Experiments
Toys database – models
153 model images
Toys database – test images (scenes)
- 90 test images
- multiple objects or different view of model
Kitchen database – models
100 model images
Kitchen database – test images
- 80 test images
- 0-9 models / test image
Lowe’s method
Our system
Examples
Test image
Test image
Identified model
Lowe’s model implemented using [Lowe’97,’99,’01,’03]
Identified model
Performance evaluation
a. Object found, correct pose  Detection
Test image hand-labeled
before the experiments
b. Object found, incorrect pose  False alarm
c. Wrong object found  False alarm
d. Object not found  Non detection
Models (database)
Scenes (test images)
Results – Toys images
- 153 model images
- 90 test images
- 0-5 models / test image
- 80% recognition with false
alarms / test set = 0.2
- Lower false alarm rate than
Lowe’s system.
Results – Kitchen images
- 100 training images
- 80 test images
- 0-9 models / test image
- 254 objects to be detected
- Achieves 77% recognition rate with 0 false alarms
Conclusions
• Unified treatment
• Best of both worlds
• Probabilistic interpretation of Lowe [‘99,’04].
• Extension of [Burl,Weber,Fergus ‘96-’03] to many-features,
many-models, one-shot learning.
• Higher performance
• Comparison with Lowe [‘99,’04].
• Future work: categories