Transcript Related Works

ARTICULATED HUMAN
DETECTION
Student: Yao-Sheng Wang
Advisor: Prof. Sheng-Jyh Wang
Department of Electronics Engineering
National Chiao Tung University
Hsinchu, Taiwan
1
OUTLINE







Introduction
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
2
OUTLINE
 Introduction











Motivation
Challenge
Representative Works
Potential Problems
Target
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
3
MOTIVATION
 Why we care about human detection?
 We are human beings!
 Wide range of applications:
 Automotive safety
 Surveillance system
 Indoor care
 Crime alert
 Human-Computer Interface … etc.
4
OUTLINE
 Introduction











Motivation
Challenge
Representative Works
Potential Problems
Target
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
5
CHALLENGE
 What makes human detection so difficult?







Illumination condition
Cluttered background
Change of viewpoints
Occlusion
Wearing difference
Diversity of human
Pose variation
6
CHALLENGE
 What makes human detection so difficult?







Illumination condition
Cluttered background
Change of viewpoints
Occlusion
Wearing difference
Diversity of human
Pose variation
7
CHALLENGE
 What makes human detection so difficult?







Illumination condition
Cluttered background
Change of viewpoints
Occlusion
Wearing difference
Diversity of human
Pose variation
8
CHALLENGE
 What makes human detection so difficult?







Illumination condition
Cluttered background
Change of viewpoints
Occlusion
Wearing difference
Diversity of human
Pose variation
9
CHALLENGE
 Progress on “Machine Learning” technology
Handle more general and complicate cases.
 Definition:
 “Articulated Human Detection”.
10
OUTLINE
 Introduction











Motivation
Challenge
Representative Works
Potential Problems
Target
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
11
REPRESENTATIVE WORKS (I)
 Deformable Part Model
 Root filter (mask).
 Part filter (mask).
 Penalty function.
[P. Felzenszwalb, D. McAllester, and D. Ramanan. A discriminatively trained, multi-scale, deformable
12
part model. In CVPR, 2008.]
REPRESENTATIVE WORKS (II)
 Pose-let:
[Lubomir Bourdev, Jitendra Malik. Poselets: Body Part Detectors Trained Using 3D Human Pose
13
Annotations. In ICCV, 2009.].
OUTLINE
 Introduction











Motivation
Challenge
Representative Works
Potential Problems
Target
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
14
POTENTIAL PROBLEMS
 Problems:
 System complexity increased with the
complexity of human poses.
 More detectors needed.
 Exhaustive search.
 Sliding window method + Image pyramid.
 Both problems leads to unacceptable speed
for applications in real life.
15
OUTLINE
 Introduction











Motivation
Challenge
Representative Works
Potential Problems
Target
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
16
TARGET
 Target in the thesis:
 Propose a detection scheme with acceptable
detection speed in dealing with highly intraclass variation from the change of pose and
viewpoint.
17
OUTLINE







Introduction
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
18
RELATED WORKS
 Better features:
 Cheap to compute and capture crucial
information at the same time. Ex: HOG.
 Better classifiers:
 Linear classifiers.
 Ex: Adaboost, Linear-SVM and Random-forests.
 Better prior knowledge:
 Ex: Information about ground plane.
19
RELATED WORKS
 Cascades:
 Cascade the part filters to reduce the searching
regions.
[P. Felzenszwalb, R. Girshick, D. McAllester. Cascade Object Detection with
Deformable Part Models. In CVPR, 2010.]
20
RELATED WORKS
 Discard non-promising hypotheses.
 Class-dependent:
 Branch and bound. (CVPR, 2008)
 Class-independent:
 What is an object? (CVPR, 2010)
 Closure boundary, different appearance or salience.
 Segmentation as selective search. (ICCV, 2011)
21
RELATED WORKS
 Feature response approximation:
 Feature approximation in testing step.
 Feature approximation in training step.
[P. Dollár, S. Belongie, P. Perona. The fastest pedestrian detector in the west. In BMVC, 2010.]
[R. Benenson, M. Mathias, R. Timofte, and L. Van Gool. Pedestrian detection at
22
100 frames per second. In CVPR, 2012.]
OUTLINE







Introduction
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
23
IDEA
Recall the memory of the first problem:
 System complexity increased with the
complexity of human poses (include variation
of viewpoints).
 How can we break the relation between
the complexity of system and the one of
human poses?
 Choose stable features or body parts for
detection.
24
IDEA
 Better prior knowledge:
25
IDEA
 Recall the memory of the second problem:
 Exhaustive search.
 “Sliding Window” + “Image Pyramid”.
 How can we reduce the searching region?
 Detect the common feature among these parts.
 Use the cumulative characteristic of the feature
to handle the variation of scale.
26
IDEA
 Common feature
 Body parts consist of combination of two edge
segments.
 Cumulative characteristic
 Edge detector with fixed size + Combination.
27
COMPARISON
 The previous works focus on reducing the
searching regions.
 Specifically against “Exhaustive Search”.
 Our method starts from breaking the
relation between complexity of system and
that of poses. Then, use the common
feature and cumulative characteristic to cut
down the searching space.
28
OUTLINE







Introduction
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
29
SYSTEM BLOCK
 Bottom-up system:
30
SYSTEM BLOCK
 Bottom-up system:
31
FAST PART DETECTION
 Steps:
 Detection of edge candidates.
 Production of part candidates.
 Refinement of part candidates.
32
DETECTION OF PART CANDIDATES
 Detection and combination of segments (9 orientations).
33
PRODUCTION OF PART CANDIDATES
 Constraints on combination of edges.
 Orientation, length ratio and color symmetry.
Neighbor orientation
consideration
34
REFINEMENT OF PART CANDIDATES
 HOG feature + Random forest training
Feature = [Length Orientation HOG_features]
feature134
feature400
feature2
feature33
?
?
35
SYSTEM BLOCK
 Bottom-up system:
36
PART COMBINATION
 Problem:
 No information about the classes of the limbs
due to the low resolution of images or
variation from hand gestures or appearance of
shoes...etc.
 Need another step to refine the combinations.
 What information left?
 Head-shoulder or head-torso.
37
PART COMBINATION
 Any possibility for us to estimate the
position and orientation of head-torso based
on the architecture of current combinations?
38
PART COMBINATION
 Problem:
 How to select body parts belong to specific
human from lots of part candidates?
 Too much possibilities for exhaustive search.

8
20
𝐶
𝑝=1
= 263949
39
PART COMBINATION
 Clues for reducing the number of possible
combinations.
 Center distance, length ration or width ratio
between two parts.
 Combination with the number of parts more
than four.

4
20
𝐶
𝑝=1
= 6195
40
PART COMBINATION
 Conclusion for the clues mentioned in the
previous slide.
 Too complicate to combine the parts for the
whole body.
 Start from low-level combination of parts to
reveal the benefits of physical constraints.
 Break the problems into two levels.
 Low-level combination.
 High-level combination.
41
LOW-LEVEL COMBINATION
 How far can we reach for low-level
combination?
 4-parts combination = lower body.
42
LOW-LEVEL COMBINATION
 False alarm exists.
 Joints relative position + Random Forest
feature400
feature2 feature33
feature134
?
?
43
HIGH-LEVEL COMBINATION
 Combination between the arms, legs, lower
bodies and uncombined single parts from
the low-level combination step.
 Upper bound of the number of combination:

𝑊
𝑘=1
4
𝑁 𝑘 +𝑀𝑘
(𝐶
−𝐷𝑘 )
𝑝=1
44
SYSTEM BLOCK
 Bottom-up system:
45
COMBINATION REFINEMENT
 Pose prediction.
 Detection with DPM detector.
46
POSE PREDICTION
Feature:
Relative size ratio and positions between low level combinations and architecture of each
low-level combination.
Random Forest.
47
DETECTION WITH DPM DETECTOR
 Use DPM detector to cover the intra-class
variation.
 Model:
48
USAGE OF HEAD-SHOULDER INFORMATION
 Much stronger than information of limbs.
 Head-shoulder to head-torso.
 Start from head-torso to combine limbs back.
49
SYSTEM ILLUSTRATION
Edge Candidates
Part Candidates
Part Detector
Parts
Low Level Part Combine
Low Level
Combination
High Level
Combination
High Level Part Combine
Result of
Detection
50
OUTLINE







Introduction
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
51
OUTLINE







Introduction
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
52
OUTLINE







Introduction
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
53