Transcript System Overview

Object Segmentation Based on
Multiple Features Fusion and Conditional Random Field
CASIA_IGIT
National Laboratory of Pattern Recognition(NLPR)
Institute of Automation, Chinese Academy of Sciences(CASIA)
Reporter：Kun Ding（丁昆）
2013.10.17
Outline
• System Overview
• System Characteristics
• Results and Conclusions
Outline
• System Overview
• System Characteristics
• Results and Conclusions
System Overview
• Object Segmentation Pipeline
Superpixel Segmentation
Feature Extraction
SVM Classification
GrabCut
Feature
Engineering
Input Image
Superpixels
Features
Stage 1 :
Superpixel Classification
Probabilistic Output
Final Results
Stage 2 :
Pixel-based CRF Smoothing
System Overview
• Superpixel Classification
• Superpixel Segmentation
• Graph-based image segmentation
• Feature Extraction:
• To be detailed in next section
• SVM Classification[1]
• RBF kernel with Probabilistic Output
System Overview
• Pixel-Based CRF Smoothing
• Fusing several kinds of information as data term
• Solving with GrabCut with only a few iterations
Binarize
SVM Probabilistic Output
First
Iteration
Second
Iteration
CRF Smoothing Output
Outline
• System Overview
• System Characteristics
• Results and Conclusions
System Characteristics
• Superpixel Segmentation -Efficient Graph-Based Image Segmentation[2]
• Fast, property of edge-preserving
• Speeding up the whole procedure
• Improving the separability between foreground and
background
Superpixels and their edge-preserving property
System Characteristics
• Feature Engineering –
Superpixel-Based Multiple Features Fusion
Gradient Dense SIFT[3][4] dictionary with Bag-of-Words description
Texture Multi-scale LBP histogram
Color and skin RGB histogram and HS histogram with skin detection
Geometrical
PCA
Position, direction and roundness
Saliency Color spatial distribution, multi-scale local and global contrast
Probability derived from AdaBoost, with
Results of Object Detection
manifold ranking[6] refinement
System Characteristics
• Feature Engineering –
Superpixel-Based Multiple Features Fusion
• Illustration of object detection
Object Detection result
Rectangle Density as Probability
Refined with Manifold Ranking
System Characteristics
• Pixel-Based CRF Smoothing – GrabCut[7]
• Modified data term
• Solving by maxflow iteratively
SVM Result
Object Detection
Result
GMM Result for Foreground
and Background
CRF Smoothing Output
Outline
• System Overview
• System Characteristics
• Results and Conclusions
Conclusion and Results Exhibition
• Results Exhibition
Conclusion and Results Exhibition
• Conclusion
• Superpixel classification
• Feature fusion works
• CRF smoothing improves the results of SVM
• Object parts sometimes lost
• Context information is inadequate
Selected References
[1] C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines,
2001. Software available at http: //www.csie.ntu.edu.tw/˜cjlin/libsvm.
[2] Felzenszwalb P F, Huttenlocher D P. Efficient graph-based image
segmentation[J]. International Journal of Computer Vision, 2004, 59(2):
167-181.
[3] Lowe D G. Distinctive image features from scale-invariant keypoints[J].
International journal of computer vision, 2004, 60(2): 91-110.
[4] Vedaldi A, Fulkerson B. VLFeat: An open and portable library of
computer vision algorithms[C]//Proceedings of the international
conference on Multimedia. ACM, 2010: 1469-1472.
Selected References
[5] Liu T, Yuan Z, Sun J, et al. Learning to detect a salient object[J].
Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2011,
33(2): 353-367.
[6] Chuan Yang, Lihe Zhang, Huchuan Lu, Minghsuan Yang, Saliency
Detection via Graph-Based Manifold Ranking, CVPR2013, P3166-3173
[7] Rother C, Kolmogorov V, Blake A. Grabcut: Interactive foreground
extraction using iterated graph cuts[C]//ACM Transactions on Graphics
(TOG). ACM, 2004, 23(3): 309-314.
Thank you very much!
Any questions?
CASIA_IGIT
Leader:
Ying Wang
（王颖）
Members: Kun Ding
（丁昆）
Huxiang Gu
（谷鹄翔）
Yongchao Gong
（宫永超）
E-mails:
{ywang, kding, hxgu, yongchao.gong}@nlpr.ia.ac.cn