Related Works
Download
Report
Transcript Related Works
ARTICULATED HUMAN
DETECTION
Student: Yao-Sheng Wang
Advisor: Prof. Sheng-Jyh Wang
Department of Electronics Engineering
National Chiao Tung University
Hsinchu, Taiwan
1
OUTLINE
Introduction
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
2
OUTLINE
Introduction
Motivation
Challenge
Representative Works
Potential Problems
Target
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
3
MOTIVATION
Why we care about human detection?
We are human beings!
Wide range of applications:
Automotive safety
Surveillance system
Indoor care
Crime alert
Human-Computer Interface … etc.
4
OUTLINE
Introduction
Motivation
Challenge
Representative Works
Potential Problems
Target
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
5
CHALLENGE
What makes human detection so difficult?
Illumination condition
Cluttered background
Change of viewpoints
Occlusion
Wearing difference
Diversity of human
Pose variation
6
CHALLENGE
What makes human detection so difficult?
Illumination condition
Cluttered background
Change of viewpoints
Occlusion
Wearing difference
Diversity of human
Pose variation
7
CHALLENGE
What makes human detection so difficult?
Illumination condition
Cluttered background
Change of viewpoints
Occlusion
Wearing difference
Diversity of human
Pose variation
8
CHALLENGE
What makes human detection so difficult?
Illumination condition
Cluttered background
Change of viewpoints
Occlusion
Wearing difference
Diversity of human
Pose variation
9
CHALLENGE
Progress on “Machine Learning” technology
Handle more general and complicate cases.
Definition:
“Articulated Human Detection”.
10
OUTLINE
Introduction
Motivation
Challenge
Representative Works
Potential Problems
Target
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
11
REPRESENTATIVE WORKS (I)
Deformable Part Model
Root filter (mask).
Part filter (mask).
Penalty function.
[P. Felzenszwalb, D. McAllester, and D. Ramanan. A discriminatively trained, multi-scale, deformable
12
part model. In CVPR, 2008.]
REPRESENTATIVE WORKS (II)
Pose-let:
[Lubomir Bourdev, Jitendra Malik. Poselets: Body Part Detectors Trained Using 3D Human Pose
13
Annotations. In ICCV, 2009.].
OUTLINE
Introduction
Motivation
Challenge
Representative Works
Potential Problems
Target
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
14
POTENTIAL PROBLEMS
Problems:
System complexity increased with the
complexity of human poses.
More detectors needed.
Exhaustive search.
Sliding window method + Image pyramid.
Both problems leads to unacceptable speed
for applications in real life.
15
OUTLINE
Introduction
Motivation
Challenge
Representative Works
Potential Problems
Target
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
16
TARGET
Target in the thesis:
Propose a detection scheme with acceptable
detection speed in dealing with highly intraclass variation from the change of pose and
viewpoint.
17
OUTLINE
Introduction
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
18
RELATED WORKS
Better features:
Cheap to compute and capture crucial
information at the same time. Ex: HOG.
Better classifiers:
Linear classifiers.
Ex: Adaboost, Linear-SVM and Random-forests.
Better prior knowledge:
Ex: Information about ground plane.
19
RELATED WORKS
Cascades:
Cascade the part filters to reduce the searching
regions.
[P. Felzenszwalb, R. Girshick, D. McAllester. Cascade Object Detection with
Deformable Part Models. In CVPR, 2010.]
20
RELATED WORKS
Discard non-promising hypotheses.
Class-dependent:
Branch and bound. (CVPR, 2008)
Class-independent:
What is an object? (CVPR, 2010)
Closure boundary, different appearance or salience.
Segmentation as selective search. (ICCV, 2011)
21
RELATED WORKS
Feature response approximation:
Feature approximation in testing step.
Feature approximation in training step.
[P. Dollár, S. Belongie, P. Perona. The fastest pedestrian detector in the west. In BMVC, 2010.]
[R. Benenson, M. Mathias, R. Timofte, and L. Van Gool. Pedestrian detection at
22
100 frames per second. In CVPR, 2012.]
OUTLINE
Introduction
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
23
IDEA
Recall the memory of the first problem:
System complexity increased with the
complexity of human poses (include variation
of viewpoints).
How can we break the relation between
the complexity of system and the one of
human poses?
Choose stable features or body parts for
detection.
24
IDEA
Better prior knowledge:
25
IDEA
Recall the memory of the second problem:
Exhaustive search.
“Sliding Window” + “Image Pyramid”.
How can we reduce the searching region?
Detect the common feature among these parts.
Use the cumulative characteristic of the feature
to handle the variation of scale.
26
IDEA
Common feature
Body parts consist of combination of two edge
segments.
Cumulative characteristic
Edge detector with fixed size + Combination.
27
COMPARISON
The previous works focus on reducing the
searching regions.
Specifically against “Exhaustive Search”.
Our method starts from breaking the
relation between complexity of system and
that of poses. Then, use the common
feature and cumulative characteristic to cut
down the searching space.
28
OUTLINE
Introduction
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
29
SYSTEM BLOCK
Bottom-up system:
30
SYSTEM BLOCK
Bottom-up system:
31
FAST PART DETECTION
Steps:
Detection of edge candidates.
Production of part candidates.
Refinement of part candidates.
32
DETECTION OF PART CANDIDATES
Detection and combination of segments (9 orientations).
33
PRODUCTION OF PART CANDIDATES
Constraints on combination of edges.
Orientation, length ratio and color symmetry.
Neighbor orientation
consideration
34
REFINEMENT OF PART CANDIDATES
HOG feature + Random forest training
Feature = [Length Orientation HOG_features]
feature134
feature400
feature2
feature33
?
?
35
SYSTEM BLOCK
Bottom-up system:
36
PART COMBINATION
Problem:
No information about the classes of the limbs
due to the low resolution of images or
variation from hand gestures or appearance of
shoes...etc.
Need another step to refine the combinations.
What information left?
Head-shoulder or head-torso.
37
PART COMBINATION
Any possibility for us to estimate the
position and orientation of head-torso based
on the architecture of current combinations?
38
PART COMBINATION
Problem:
How to select body parts belong to specific
human from lots of part candidates?
Too much possibilities for exhaustive search.
8
20
𝐶
𝑝=1
= 263949
39
PART COMBINATION
Clues for reducing the number of possible
combinations.
Center distance, length ration or width ratio
between two parts.
Combination with the number of parts more
than four.
4
20
𝐶
𝑝=1
= 6195
40
PART COMBINATION
Conclusion for the clues mentioned in the
previous slide.
Too complicate to combine the parts for the
whole body.
Start from low-level combination of parts to
reveal the benefits of physical constraints.
Break the problems into two levels.
Low-level combination.
High-level combination.
41
LOW-LEVEL COMBINATION
How far can we reach for low-level
combination?
4-parts combination = lower body.
42
LOW-LEVEL COMBINATION
False alarm exists.
Joints relative position + Random Forest
feature400
feature2 feature33
feature134
?
?
43
HIGH-LEVEL COMBINATION
Combination between the arms, legs, lower
bodies and uncombined single parts from
the low-level combination step.
Upper bound of the number of combination:
𝑊
𝑘=1
4
𝑁 𝑘 +𝑀𝑘
(𝐶
−𝐷𝑘 )
𝑝=1
44
SYSTEM BLOCK
Bottom-up system:
45
COMBINATION REFINEMENT
Pose prediction.
Detection with DPM detector.
46
POSE PREDICTION
Feature:
Relative size ratio and positions between low level combinations and architecture of each
low-level combination.
Random Forest.
47
DETECTION WITH DPM DETECTOR
Use DPM detector to cover the intra-class
variation.
Model:
48
USAGE OF HEAD-SHOULDER INFORMATION
Much stronger than information of limbs.
Head-shoulder to head-torso.
Start from head-torso to combine limbs back.
49
SYSTEM ILLUSTRATION
Edge Candidates
Part Candidates
Part Detector
Parts
Low Level Part Combine
Low Level
Combination
High Level
Combination
High Level Part Combine
Result of
Detection
50
OUTLINE
Introduction
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
51
OUTLINE
Introduction
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
52
OUTLINE
Introduction
Related Works
Idea
Proposed Method
Experimental Results
Conclusion
Reference
53