Transcript IBM Smart Surveillance System S3 Sales and Technical Training

Emerging Topics in
Video Surveillance
Rogerio Feris
IBM TJ Watson Research Center
[email protected]
http://rogerioferis.com
1
Video Surveillance E6998 -007 Senior/Feris/Tian
Outline
 Video Surveillance in Crowded Scenarios
 Online Learning – Self-adaptation in Surveillance
 Other Recent Topics
2
Video Surveillance E6998 -007 Senior/Feris/Tian
Simple Scenarios
 Few Objects – Background Subtraction + Tracking + Highlevel Event/Alert Detection
 Current systems work well
3
Video Surveillance E6998 -007 Senior/Feris/Tian
Crowded Scenarios
 Many objects, occlusions, shadows, etc.
Object Segmentation,
Tracking and Event Analysis
in crowded scenarios:
Open Problem!
4
Video Surveillance E6998 -007 Senior/Feris/Tian
Parts-based Detectors
[Pedro et al, A discriminatively trained, multiscale, deformable part
model, CVPR’08]
 Occlusion Handling
 Root filter (low-res) + Parts filters (high-res)
5
Video Surveillance E6998 -007 Senior/Feris/Tian
Parts-based Detectors
 Score of a window: score of root + score of parts
 Score of Parts: Appearance + Geometry
 Efficient localization of parts through Dynamic Programming
 SVM Classification (Structured prediction)
6
Video Surveillance E6998 -007 Senior/Feris/Tian
Detecting Pedestrians in Crowds
[Leibe et al, Pedestrian Detection in Crowded Scenes, CVPR’05]
 Combination of different models: bag of features,
segmentation, and chamfer matching
7
Video Surveillance E6998 -007 Senior/Feris/Tian
Tracking in Crowds
[Andriluka et al, People-tracking-by-detection and people-detection-bytracking, CVPR’08]
 Extends [Leibe et al, CVPR’05] to temporal-domain and
person articulation (parts) estimation
 Click for Video Demo
8
Video Surveillance E6998 -007 Senior/Feris/Tian
Crowd Segmentation
[Dong et al, Fast Crowd Segmentation Using Shape Indexing,
ICCV’07]
9
Video Surveillance E6998 -007 Senior/Feris/Tian
Crowd Analysis
[Ali & Shah, A Lagrangian Particle Dynamics Approach for Crowd Flow
Segmentation and Stability Analysis, CVPR’07]
10
Video Surveillance E6998 -007 Senior/Feris/Tian
Online Learning
11
Video Surveillance E6998 -007 Senior/Feris/Tian
Offline Adaboost Learning
 Adaboost ensembles many weak classifiers into one
single strong classifier
 Initialize sample weights
 For each cycle:
 Find a classifier/rectangle feature
that performs well on the weighted
samples
 Increase weights of misclassified
examples
 Return a weighted combination of
classifiers
12
Video Surveillance E6998 -007 Senior/Feris/Tian
Offline Adaboost Learning
Major Problems:
 Large number of examples required to train a robust classifier
 time consuming to label data
 slow training (may take several days)
 No Adaptation to particular surveillance scenarios
13
Video Surveillance E6998 -007 Senior/Feris/Tian
Learning from Small Sets
 Choice of Features (Levi & Weiss, CVPR’04)
 Co-Training (Levin & Viola, ICCV’2003)
Online Adaptation:
 Online Boosting (Oza’01, Javed’05, Bischof’06, Pham’07)
14
Video Surveillance E6998 -007 Senior/Feris/Tian
Online Boosting [Oza,2001]
 Train a generic strong classifier (set of weak classifiers, # of weak classifiers
fixed) on a small training set.
Online Process:
 Given one single example with known label:
 “Slide” the example over each weak classifier
 When the weak classifier receives the example
 update the weak classifier online
 update the weight of the example and pass to the next weak
classifier
15
Video Surveillance E6998 -007 Senior/Feris/Tian
Online Boosting [Oza,2001]
16
Video Surveillance E6998 -007 Senior/Feris/Tian
Online Boosting [Oza,2001]
17
Video Surveillance E6998 -007 Senior/Feris/Tian
Online Boosting
Car and People Detection [Omar Javed, CVPR’05]
 Train a generic strong classifier (set of weak classifiers, # of weak classifiers
fixed) on a small training set.
 While running the classifier on unlabeled data, if an example is confidently
predicted by a subset of weak classifiers  use it for online learning
 “Co-training framework”
 BGS used for efficiency, for using more expensive features, and for
balancing the number of positive and negative examples
18
Video Surveillance E6998 -007 Senior/Feris/Tian
Online Boosting
Car and People Detection [Omar Javed, CVPR’05]
19
Video Surveillance E6998 -007 Senior/Feris/Tian
More Recent Work
 [Helmut & Hurst, Online Boosting and Vision, CVPR’06]
 [Bo Wu & Nevatia, Improving Part-based Object Detection by
Unsupervised, Online Boosting, CVPR’07]
 [Pham & Cham, Online Learning Asymmetric Boosted Classifiers for
Object Detection, CVPR’07]
 [Huang et al.,Incremental Learning of Boosted Face Detector,
ICCV’07] – Boosting Adaptation
 IEEE Online Learning for Classification Workshop (CVPR’08)
20
Video Surveillance E6998 -007 Senior/Feris/Tian
Other Recent Topics
21
Video Surveillance E6998 -007 Senior/Feris/Tian
High-Resolution Imagery
[Kopf et al, Capturing and Viewing Gigapixel Images, SIGGRAPH’07]
 How can we make use of high-resolution in video analytics?
 Much more info – e.g., in face reco: skin texture, iris, etc.
22
Video Surveillance E6998 -007 Senior/Feris/Tian
Next Generation Neural Networks
[Hinton, Reducing the dimensionality of data with neural networks,
Science 2006]
 New algorithm for learning deep belief nets
 State-of-the art results in MNIST digit dataset (better than SVMs)
 Youtube talk at Google:
http://www.youtube.com/watch?v=AyzOUbkUf3M
 Matlab Code: http://www.cs.toronto.edu/~hinton/
23
Video Surveillance E6998 -007 Senior/Feris/Tian
Learning with lots of data
 How can we recognize thousands of products in a retail store for loss
prevention?
 80 Million Tiny Images (http://www.cs.nyu.edu/~fergus/)
Surveillance with Moving Cameras
 Cameras in vehicles, or even wearable cameras. New challenges:
object detection, etc.
 [Leibe et al, Dynamic 3D Scene Analysis from a Moving Vehicle,
CVPR 2007]
24
Video Surveillance E6998 -007 Senior/Feris/Tian
Many more recent topics:
 Check for papers in recent computer vision conferences (like CVPR,
ICCV, and ECCV) and also specialized workshops/conferences such as
AVSS and PETS
25
Video Surveillance E6998 -007 Senior/Feris/Tian