IBM Smart Surveillance System S3 Sales and Technical Training
Download
Report
Transcript IBM Smart Surveillance System S3 Sales and Technical Training
Emerging Topics in
Video Surveillance
Rogerio Feris
IBM TJ Watson Research Center
[email protected]
http://rogerioferis.com
1
Video Surveillance E6998 -007 Senior/Feris/Tian
Outline
Video Surveillance in Crowded Scenarios
Online Learning – Self-adaptation in Surveillance
Other Recent Topics
2
Video Surveillance E6998 -007 Senior/Feris/Tian
Simple Scenarios
Few Objects – Background Subtraction + Tracking + Highlevel Event/Alert Detection
Current systems work well
3
Video Surveillance E6998 -007 Senior/Feris/Tian
Crowded Scenarios
Many objects, occlusions, shadows, etc.
Object Segmentation,
Tracking and Event Analysis
in crowded scenarios:
Open Problem!
4
Video Surveillance E6998 -007 Senior/Feris/Tian
Parts-based Detectors
[Pedro et al, A discriminatively trained, multiscale, deformable part
model, CVPR’08]
Occlusion Handling
Root filter (low-res) + Parts filters (high-res)
5
Video Surveillance E6998 -007 Senior/Feris/Tian
Parts-based Detectors
Score of a window: score of root + score of parts
Score of Parts: Appearance + Geometry
Efficient localization of parts through Dynamic Programming
SVM Classification (Structured prediction)
6
Video Surveillance E6998 -007 Senior/Feris/Tian
Detecting Pedestrians in Crowds
[Leibe et al, Pedestrian Detection in Crowded Scenes, CVPR’05]
Combination of different models: bag of features,
segmentation, and chamfer matching
7
Video Surveillance E6998 -007 Senior/Feris/Tian
Tracking in Crowds
[Andriluka et al, People-tracking-by-detection and people-detection-bytracking, CVPR’08]
Extends [Leibe et al, CVPR’05] to temporal-domain and
person articulation (parts) estimation
Click for Video Demo
8
Video Surveillance E6998 -007 Senior/Feris/Tian
Crowd Segmentation
[Dong et al, Fast Crowd Segmentation Using Shape Indexing,
ICCV’07]
9
Video Surveillance E6998 -007 Senior/Feris/Tian
Crowd Analysis
[Ali & Shah, A Lagrangian Particle Dynamics Approach for Crowd Flow
Segmentation and Stability Analysis, CVPR’07]
10
Video Surveillance E6998 -007 Senior/Feris/Tian
Online Learning
11
Video Surveillance E6998 -007 Senior/Feris/Tian
Offline Adaboost Learning
Adaboost ensembles many weak classifiers into one
single strong classifier
Initialize sample weights
For each cycle:
Find a classifier/rectangle feature
that performs well on the weighted
samples
Increase weights of misclassified
examples
Return a weighted combination of
classifiers
12
Video Surveillance E6998 -007 Senior/Feris/Tian
Offline Adaboost Learning
Major Problems:
Large number of examples required to train a robust classifier
time consuming to label data
slow training (may take several days)
No Adaptation to particular surveillance scenarios
13
Video Surveillance E6998 -007 Senior/Feris/Tian
Learning from Small Sets
Choice of Features (Levi & Weiss, CVPR’04)
Co-Training (Levin & Viola, ICCV’2003)
Online Adaptation:
Online Boosting (Oza’01, Javed’05, Bischof’06, Pham’07)
14
Video Surveillance E6998 -007 Senior/Feris/Tian
Online Boosting [Oza,2001]
Train a generic strong classifier (set of weak classifiers, # of weak classifiers
fixed) on a small training set.
Online Process:
Given one single example with known label:
“Slide” the example over each weak classifier
When the weak classifier receives the example
update the weak classifier online
update the weight of the example and pass to the next weak
classifier
15
Video Surveillance E6998 -007 Senior/Feris/Tian
Online Boosting [Oza,2001]
16
Video Surveillance E6998 -007 Senior/Feris/Tian
Online Boosting [Oza,2001]
17
Video Surveillance E6998 -007 Senior/Feris/Tian
Online Boosting
Car and People Detection [Omar Javed, CVPR’05]
Train a generic strong classifier (set of weak classifiers, # of weak classifiers
fixed) on a small training set.
While running the classifier on unlabeled data, if an example is confidently
predicted by a subset of weak classifiers use it for online learning
“Co-training framework”
BGS used for efficiency, for using more expensive features, and for
balancing the number of positive and negative examples
18
Video Surveillance E6998 -007 Senior/Feris/Tian
Online Boosting
Car and People Detection [Omar Javed, CVPR’05]
19
Video Surveillance E6998 -007 Senior/Feris/Tian
More Recent Work
[Helmut & Hurst, Online Boosting and Vision, CVPR’06]
[Bo Wu & Nevatia, Improving Part-based Object Detection by
Unsupervised, Online Boosting, CVPR’07]
[Pham & Cham, Online Learning Asymmetric Boosted Classifiers for
Object Detection, CVPR’07]
[Huang et al.,Incremental Learning of Boosted Face Detector,
ICCV’07] – Boosting Adaptation
IEEE Online Learning for Classification Workshop (CVPR’08)
20
Video Surveillance E6998 -007 Senior/Feris/Tian
Other Recent Topics
21
Video Surveillance E6998 -007 Senior/Feris/Tian
High-Resolution Imagery
[Kopf et al, Capturing and Viewing Gigapixel Images, SIGGRAPH’07]
How can we make use of high-resolution in video analytics?
Much more info – e.g., in face reco: skin texture, iris, etc.
22
Video Surveillance E6998 -007 Senior/Feris/Tian
Next Generation Neural Networks
[Hinton, Reducing the dimensionality of data with neural networks,
Science 2006]
New algorithm for learning deep belief nets
State-of-the art results in MNIST digit dataset (better than SVMs)
Youtube talk at Google:
http://www.youtube.com/watch?v=AyzOUbkUf3M
Matlab Code: http://www.cs.toronto.edu/~hinton/
23
Video Surveillance E6998 -007 Senior/Feris/Tian
Learning with lots of data
How can we recognize thousands of products in a retail store for loss
prevention?
80 Million Tiny Images (http://www.cs.nyu.edu/~fergus/)
Surveillance with Moving Cameras
Cameras in vehicles, or even wearable cameras. New challenges:
object detection, etc.
[Leibe et al, Dynamic 3D Scene Analysis from a Moving Vehicle,
CVPR 2007]
24
Video Surveillance E6998 -007 Senior/Feris/Tian
Many more recent topics:
Check for papers in recent computer vision conferences (like CVPR,
ICCV, and ECCV) and also specialized workshops/conferences such as
AVSS and PETS
25
Video Surveillance E6998 -007 Senior/Feris/Tian