Transcript Multi-Group Tracking with Adaptive Appearance Model

Multi-Group Tracking with
Adaptive Target Model
PhD thesis proposal
Loris Bazzani, PhD student (XXIV cycle)
University of Verona
Department of Computer Science
Objectives
Compare existing Multi-Target Tracking
methods, studying the sampling technique
Propose a new tracking method:
Group Tracking
Multi-
Model robustly and adaptively the target
Integrate target model with Multi-Group
tracking
2
Outline
Introduction
Multi-Target Tracking
State of the art
Multi-Group Tracking
Open issues
Target Modeling
Proposed ideas
Conclusions
3
Introduction (1)
Tracking: spatial and temporal localization of a mobile object
in an environment monitored by sensor(s)
Multi-target (MTT): keeping the identity of different targets
Reliable: insensible to noise and occlusions
Application to
Automated
Surveillance
Introduction (2)
Multi-Group Tracking (MGT):
Spatial and temporal localization of groups of objects
Motivations:
Humans prefer to stay in group rather than alone
High-level representation of the relations among the targets
MGT is simpler than MTT in a crowded scenario
MGT can help MTT when occlusions occur
5
Introduction (3)
Multi-Group Tracking (MGT): why is MGT a hard task?
6
Introduction (4)
Target Model:
A general and representative example that
summarizes any possible changing of the target
intrinsic variations: pose variation
and shape deformation
extrinsic variations: illumination
changes, camera movement, and
occlusions
Not considering the above variation causes the
failure of the tracking [Ross08]
Fit with the re-identification problem
7
Multi-Target Tracking (1)
Abstract Formulation [Arulampalam02]
State Space Approach for modeling
discrete-time dynamic systems
State: abstract nature of the target
Measurement: “visible” dimensions of the
state space
Filtering
We observe the real world events as a state by the
measurement process
Objective: estimating the state of the system at each instant
given the measurements
8
Multi-Target Tracking (2)
Data Association [Bar-Shalom87]
The observer has at his disposal a huge amount of
measurements
Finding the correct correspondences between
measurements and states of the system
9
Multi-Target Tracking (3)
10
Multi-Target Tracking (4)
Particle Filter (PF) [Isard01]
11
Multi-Target Tracking (5)
State Space Conformation
[Isard01]
(+) Efficient sampling
(-) No interaction modeling
[MacCormick00]
(+) Implicit interaction modeling
(-) Curse of dimensionality
[Lanz06]
(+) Efficient sampling
(+) Implicit interaction modeling
Multi-Target Tracking (6)
- HJS vs. MHT [Bazzani09] -
HJS pros:
1) One track is kept for each target
2) Partial occlusions are handled;
3) Deal with non-linearity of people
motion.
MHT cons:
1) Multiple tracks cause proliferation
in the number of tracks
2) Occlusions generate new tracks
3) Not robust to non-linear people
motion
Multi-Target Tracking (7)
Open issues of PF-based MTT (and MGT):
Sampling Method
Dynamic Model
Linear-Gaussian model
Observation Model
State Estimation
Maximum-A-Posteriori or Weighted Mean
14
Multi-Target Tracking (8.1)
Sampling
Sequential importance sampling/re-sampling [Arulampalam02]:
classical PF + degeneracy problem avoiding by re-sampling
Regularized PF [Arulampalam02]: resamples applying a Kernel
to the particles
MCMC [Andrieu03]: defines a Markov chain over the state
space, such that the stationary distribution of the chain is equal
to the sought posterior
Reversible-Jump MCMC [Khan05]: switches between variable
dimensional state spaces
Rao-Blackwellizing PF [Schindler05]: analytically computes a
portion of the distribution other the state space
Multi-Target Tracking (8.2)
Observation model
Likelihood: compare an observation z given a hypothesis of
state of the system x
Usually defined in the Gibbs form:
metric
where d is a
x and z MUST be represented in the same feature space
ISSUES:
feature space (for x and z) and metric definitions
occlusion handling
17
Multi-Target Tracking (9)
- Proposed Research Occlusion Handling
Study and implement RJ-MCMC particle filter
Propose a set of jumps in order to cope with
tracking a variable number of objects
Propose an observation model in RJ-MCMC
framework
18
Multi-Group Tracking (1)
(-) Social interactions cannot be caught
defines
a group as
the moving regions,
by the foreground
analysis
extracted
from a foreground
(-) The inter-group
dynamic analysis
yields a loss
of appearance informations
infers
MGT from
the tracks
(-) Directthe
dependence
from MOT
estimation
carried
out by the
MOT
(e.g.
(-) The MOT
estimation
is not
reliable
tracks
clustering) occur
when occlusions
uses
the out
foreground
(+) Cancel
the aboveinformation
problems and
MOT
to detect
groups,
buttask
then tracks
(-) Model
creation
is a hard
them as different entities
19
Multi-Group Tracking (2)
Foreground-based MGT
Tracking at three levels of abstraction [McKenna00] :
Regions: connected component that have been tracked for T
frames
People: one or more regions grouped together
Groups: one or more people grouped together, if they share
a region
(+) Simplicity
(-) heuristic FG analysis
20
Multi-Group Tracking (3)
MGT from MOT
MCMC PF for group tracking [Pang07]:
Track groups as ensemble of targets analyzing
: group variable
Treated as Bayesian estimation problem
Group structure model:
captures the relations among objects
22
Multi-Group Tracking (4)
- Proposed Research Problems:
Definition of “Group”: an entity containing targets with similar
characteristics (e.g. motion, interactions, ...)
Deterministic/formal definition as an ensemble of objects
Add non-deterministic component into the tracking method
Intra-group occlusions: if we know that the objects hasn’t left the
group, we infer that it is still into the group
Inter-group occlusions: tracking of groups
23
Multi-Group Tracking (5)
- Proposed Research -
24
Multi-Group Tracking (6)
- Proposed Research Use a MOT method
Create a MGT method (track only groups)
Definition of group dynamics -> sociological studies
Definition of a group observation model
Define a collaborative probabilistic framework in order to
share MGT and MOT informations
25
Target Modeling (1)
Train the model using the appearance
data available before tracking begins
Adapt the model to account for its
changes in appearance, using an online learning method
Open Issues:
Representation of the target: feature space
Leaning technique
27
Target Modeling (2)
Feature
Color Histogram: which is the best color space for
tracking? [Sebastian08]
(-) No spatial information
Color correlogram [Huang99], spatiogram [Birchfield05],
multi-resolution histogram [Hadjidemetriou01]
(+) Add the spatial information
(-) Increase the computational burden
28
Target Modeling (3)
Feature
Covariance descriptors [Porikli06]
Spatial and appearance attributes
(+) Natural way of fusing multiple features
(-) Computationally expensive -> integral images
29
Target Modeling (4)
Fixed target models
Ensemble of localized features [Gray08]
Define a feature space and let machine learning approach
find the best representation
AdaBoost extracts
the object representation: the most discriminative set of
features, and
the similarity measures: the most discriminative set of
likelihood ratio test
Used for re-identification problem
30
Target Modeling (5)
Adaptive target models
Incremental learning of Covariance-based descriptors
[Porikli06]
Principal Component Analysis (PCA) incremental
learning [Ross08]
Convex combination of models using a learning rate
Feature-based model
Parametric model
31
Target Modeling (6)
Patch-based (local) updating [Kwon09]
Evolve photometric and geometric appearance
Local Patches can be added, deleted or moved
to different position
Examining the patches by landscape analysis
Bad patches are modified on-line: background patches and
patches in regions with high density of patch are deleted
Good patches are moved
Appearance model is updated with a convex combination
32
Target Modeling (7)
- Proposed Research Part-based multiple-features:
Invariances:
view point, partial
occlusions
Maximally Stable Color Regions
Global
illumination, view point,
deformations, partial
occlusions
HS(V) histograms
Local
Recurrent high-structured patches
Temporal updating:
Delete the non-stable features
Cover the Variability of the stable features
33
partial occlusions,
illumination
Conclusions
Compare existing Multi-Target Tracking
methods, studying the sampling technique
Propose a new tracking method:
Group Tracking
Multi-
Model robustly and adaptively the target
Integrate target model with Multi-Group
tracking, using HJS and RJ-MCMC
34
The beginning
Now, we “just” put the proposed ideas into practice
Thanks for attention
Questions?
35
References
[Ross08] D.A. Ross, J. Lim, R.S. Lin, and M.H. Yang. Incremental learning for robust visual
tracking. International Journal of Computer Vision, 77(1):125–141, 2008.
[Arulampalam02] M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp. A tutorial on particle
filters for online nonlinear/non-gaussian bayesian tracking. IEEE Trans. on Signal Processing,
50(2):174–188, 2002.
[Bar-Shalom87] Y. Bar-Shalom. Tracking and data association. Academic Press Professional, Inc.,
San Diego, CA, USA, 1987.
[Isard01] M. Isard and J. MacCormick. Bramble: A bayesian multipleblob tracker. In IEEE Int. Conf.
on Computer Vision, 2001.
[MacCormick00] John MacCormick and Andrew Blake. A probabilistic exclusion principle for
tracking multiple objects. Int. J. Comput. Vision, 39(1):57–71, 2000.
[Lanz06] O. Lanz. Approximate bayesian multibody tracking. IEEE Trans. on Pattern Analysis and
Machine Intelligence, 28(9):1436–1449, 2006.
[Andrieu03] Christophe Andrieu, Nando de Freitas, Arnaud Doucet, and Michael I. Jordan. An
introduction to mcmc for machine learning. Machine Learning, 50(1):5–43, January 2003.
36
References
[Khan05] Z. Khan, T. Balch, and F. Dellaert. Mcmc-based particle filtering for tracking a variable number of
interacting targets. IEEE Trans. on Pattern Analysis and Machine Intelligence, 27(11):1805–1819, 2005.
[Schindler05] G. Schindler and F. Dellaert. A Rao-Blackwellized Parts-Constellation Tracker. In ICCV
Workshop on Dynamical Vision; International Conference on Computer Vision, 2005. Springer
[Mckenna00] Stephen J. Mckenna, Sumer Jabri, Zoran Duric, Harry Wechsler, and Azriel Rosenfeld. Tracking
groups of people. Computer Vision and Image Understanding, 2000.
[Pang07] Sze Kim Pang, Jack Li, and Simon Godsill. Models and algorithms for detection and tracking of
coordinated groups. In Symposium of image and Signal Processing and Analisys, 2007.
[Sebastian08] Sebastian, P.; Yap Vooi Voon; Comley, R., "The effect of colour space on tracking robustness,"
Industrial Electronics and Applications, 2008. ICIEA 2008. 3rd IEEE Conference on , vol., no., pp.2512-2516,
3-5 June 2008
[Huang99] J. Huang, S. Ravi Kumar, M. Mitra, W.J. Zhu, and R. Zabih. Spatial color indexing and applications.
International Journal of Computer Vision, 35(3):245–268, 1999.
[Birchfield05] ST Birchfield and S. Rangarajan. Spatiograms versus histograms for region-based tracking. In
IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005.
CVPR 2005, volume 2, 2005.
37
References
[Hadjidemetriou01] E. Hadjidemetriou, MD Grossberg, and SK Nayar. Spatial information in multiresolution
histograms. In Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001
IEEE Computer Society Conference on, volume 1, 2001.
[Porikli06] F. Porikli, O. Tuzel, and P. Meer. Covariance tracking using model update based on lie algebra.
In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 1,
2006.
[Gray08] D. Gray and H. Tao. Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized
Features. In Proceedings of the 10th European Conference on Computer Vision: Part I, pages 262–275.
Springer, 2008.
[Kwon09] J.S. Kwon and K.M. Lee. Tracking of a non-rigid object via patch-based dynamic appearance
modeling and adaptive basin hopping monte carlo sampling. pages 1208–1215, 2009.
38