Lecture01_Tracking I..
Download
Report
Transcript Lecture01_Tracking I..
Introduction To
Tracking
Mario Haddad
What is Tracking?
• Estimating pose (state)
• Possible from a variety of measured
sensors
–
–
–
–
–
–
Electrical
Mechanical
Inertial
Optical
Acoustic
Magnetic
2
DYNAMIC SCENE ANALYSIS
The input to the dynamic scene analysis is a sequence
of image frames 𝐹 𝑥, 𝑦, 𝑡 taken from the changing
world.
x, y are spatial coordinates.
Frames are usually captured at fixed time intervals.
𝑡 represents 𝑡 𝑡ℎ frame in the sequence.
Typical Applications
Motion detection. Often from a static camera.
Object localization.
Three-dimensional shape from motion.
Object tracking.
Example Application
Object Tracking Definition
Object tracking is the problem of determining
(estimating) the positions and other relevant
information of moving objects in image sequences.
Difficulties In Reliable Object
Tracking
Rapid appearance changes caused by
image noise,
illumination changes,
non-rigid motion,
...
Non-stable background
Interaction between multiple objects
...
Difficulties In Reliable Object
Tracking
Robust Density Comparison for Visual Tracking (BMVC 2009)
Difficulties In Reliable Object
Tracking
Motion Estimation
Block Matching Method
For a given region in one frame, find the corresponding
region in the next frame by finding the maximum
correlation score (or other block matching criteria) in a
search region
Block Matching Method
Block Matching Method
Optical Flow Motion Field
(a)
(b)
Visible Motion and True
Motion
OPTIC FLOW - apparent motion of the same (similar)
intensity patterns
Generally, optical flow corresponds to the motion field,
but not always:
Local Features for Tracking
If strong derivatives are observed in two
orthogonal directions then we can hope
that this point is more likely to be unique.
Many trackable features are called
corners.
Harris Corner Detection !
Aperture Problem
The Aperture Problem
Different motions – classified as similar
source: Ran Eshel
The Aperture Problem
Similar motions – classified as different
source: Ran Eshel
Tracking Methods
Mean-Shift
The mean-shift algorithm is an efficient
approach to tracking objects whose
appearance is defined by histograms.
(not limited to only color)
Motivation
Motivation – to track non-rigid objects, (like
a walking person), it is hard to specify
an explicit 2D parametric motion model.
Appearances of non-rigid objects can
sometimes be modeled with color
distributions
Mean Shift Theory
Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical billiard balls
Stolen from: www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/mean_shift/mean_shift.ppt
Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical billiard balls
Stolen from: www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/mean_shift/mean_shift.ppt
Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical billiard balls
Stolen from: www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/mean_shift/mean_shift.ppt
Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical billiard balls
Stolen from: www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/mean_shift/mean_shift.ppt
Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical billiard balls
Stolen from: www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/mean_shift/mean_shift.ppt
Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical billiard balls
Intuitive Description
Region of
interest
Center of
mass
Objective : Find the densest region
Distribution of identical billiard balls
Stolen from: www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/mean_shift/mean_shift.ppt
Mean Shift Vector
Given:
Data points and approximate location of the mean of this
data:
Task:
Estimate the exact location of the mean of the data by
determining the shift vector from the initial mean.
Mean Shift Vector
A Quick PDF Definition
A probability density function (pdf),
is a function that describes the
relative likelihood for this random
variable to take on a given value.
Mean-Shift Object Tracking
Target Representation
Choose a
reference
target model
Represent the
model by its
PDF in the
feature space
Choose a
feature space
0.35
Quantized
Color Space
Probability
0.3
0.25
0.2
0.15
0.1
0.05
0
1
2
3
.
color
Stolen from: www.cs.wustl.edu/~pless/559/lectures/lecture22_tracking.ppt
.
.
m
Mean-Shift Object Tracking
Target Model
Target Candidate
(centered at 0)
(centered at y)
0.35
0.3
0.3
0.25
0.25
Probability
Probability
PDF Representation
0.2
0.15
0.1
0.2
0.15
0.1
0.05
0.05
0
0
1
2
3
.
.
.
m
1
2
color
q qu u 1..m
3
.
.
.
m
color
m
q
u 1
u
1
Similarity f y f q , p y
Function:
Stolen from: www.cs.wustl.edu/~pless/559/lectures/lecture22_tracking.ppt
p y pu y u 1..m
m
p
u 1
Q is the target histogram,
P is the object histogram
(depends on location y)
u
1
Mean-Shift Object Tracking
Target Localization Algorithm
Start from
the position
of the model
in the current
frame
q
Search in the
model’s
neighborhood
in next frame
p y
Stolen from: www.cs.wustl.edu/~pless/559/lectures/lecture22_tracking.ppt
Find best
candidate by
maximizing a
similarity func.
f p y , q
Mean Shift
Mean-Shift in tracking task:
track the motion of a cluster of interesting
features.
1. choose the feature distribution to represent
an object (e.g., color + texture),
2. start the mean-shift window over the feature
distribution generated by the object
3. finally compute the chosen feature
distribution over the next video frame.
Mean Shift
Starting from the current window location, the
mean-shift algorithm will find the new peak or
mode of the feature distribution, which
(presumably) is centered over the object that
produced the color and texture in the first
place.
In this way, the mean-shift window tracks
the movement of the object frame by frame.
Examples
Examples
Other Mean Shift
Applications
Edge Preserving Smoothing
Segmentation
Contour Detection
Kalman Filter
Rudolf Emil Kalman
•
•
•
•
•
Born in 1930 in Hungary
BS and MS from MIT
PhD 1957 from Columbia
Filter developed in 1960-61
Now retired
Kalman Filter
• Noisy data in
hopefully less noisy data out
• The Kalman filter operates recursively on streams
of noisy input data to produce a statistically
optimal estimate of the underlying system state.
Motivation
Kalman Filter Applications
Tracking objects (e.g., missiles, faces, heads, hands)
Navigation
Many computer vision applications
– Stabilizing depth measurements
– Feature tracking
– Cluster tracking
– Fusing data from radar, laser scanner and
stereo-cameras for depth and velocity measurements
– Many more
Intuition
Robot
Odometer
GPS
Sand
Previous
state
We may encounter:
Wheel spin
GPS inaccuracy
Odometer
GPS
Kalman Filter
Not perfectly sure. Why ?
•
A𝐬𝐬𝐮𝐦𝐞 𝑲𝒌 = 𝟎. 𝟓 , what would we get?
Kalman Filter
Kalman filter finds the most optimum averaging factor for
each consequent state.
“somehow” remembers a little bit about the past
states.
Kalman Filter
State Prediction:
Measurement Prediction:
𝑥𝑘 - state prediction
𝑢𝑘 - control signal (Most of the time there is no control signal)
𝑤𝑘 - process noise
A,B,H - define the physics of interest ( acceleration, position, speed… )
𝑧𝑘 - measurement prediction
𝑣𝑘 - measurement noise
Kalman Filter
• Two groups of the equations for the Kalman filter:
o Time update equations (Prediction)
o Measurement update equations. (Correction)
• The time update equations are responsible for projecting forward (in time)
the current state and error covariance estimates to obtain the a priori
estimates for the next time step.
• The measurement update equations are responsible for the feedback—i.e. for
incorporating a new measurement into the a priori estimate to obtain an
improved a posteriori estimate.
Brace Yourselves..
Kalman Filter
Predict
1.
Predict the state ahead:
Update
1.
xˆt xˆt Kt zt Hxˆt
xˆt Axˆt 1
2.
2.
Predict the error covariance
ahead:
t At 1 AT Q
Update the state estimate:
Update the error
covariance:
t I Kt H t
where Kalman gain Kt is:
K t t H T H t H T R
55
1