Transcript Facial Feature Extraction

Yuri Vanzine C490/B657
Computer Vision
Facial Feature Extraction
Potentially Bigger Project
Future Goal: Application Component
Need: Provide Ability to import Avatar appearance from web cam
Application needs to perform the following:
•
•
•
•
•
Receive Frontal and Profile Images of User’s Head
Localize face, locate position of eyes, mouth to determine face
geometry
Use face geometry to locate secondary features, eg. nose, facial hair, ears.
Extract face geometry and features and transform them into quantative
feature vectors.
Apply features and geometry to 3-D model in a standard 3-D model format,
eg. 3DS, OpenNURBS, DXF, DWG, etc.
Part of the Project for C490/B657
Face Localization, Locating Eyes and Mouth
Process phases:
•
•
•
Face Localization
Mouth Location
Left and Right Eye Location
Face Localization
Existing Techniques:
•
•
•
•
Skin-tone techniques in various colorspaces
Template-based
Eigen-faces
Other
Technique used for Face Detection:
•
•
•
RGB, YCbCr, HSV Late Fusion
Several weak, fast classifiers instead of
‘expensive’ ones
Environment is controlled: sufficient
lighting, head position, subject is told
to place head a certain way, but our
classifier is rotation, scale and
translation invariant!
RGB, YCbCr, HSV Late Fusion
RGB Colorspace
Matlab Code:
for xc=1:x
for yc = 1:y
iR = (int)R(xc,yc);
iG = (int)G(xc,yc);
iB = B(xc,yc);
mxRGB = max(max(iR, iG), iB);
mnRGB = min(min(iR, iG), iB);
if iR>95 && iG>40 && iB>20 && mxRGB-mnRGB>15 && abs(iR-iG) > 15 && iR > iG &&
iR > iB
else
F(xc,yc,1)=0;
F(xc,yc,2)=0;
F(xc,yc,3)=0;
end
end
end
HSV Colorspace
Bounding planes, defining skin-color, determined in [3]:
Matlab Code:
for xc=1:x
for yc = 1:y
iH = double(H(xc,yc))*255;
iS = double(S(xc,yc))*255;
iV = double(V(xc,yc))*255;
if iH>=0 && iS>=15 && iS>=(0.75*iH+0.3*iV-30) || iS<=-iH-0.1*iV+110 && iH<=0.4*iV+75 && iS<=0.08 * (100-iV) *iH +0.6* iV
else
F(xc,yc,1)=0;
F(xc,yc,2)=0;
F(xc,yc,3)=0;
end
end
end
YCbCr Colorspace
“YCbCr simple” from [3] (could not get it to work):
Late Fusion of Colorspaces
RGB
Bounding Box
HSV
RGB and HSV in this implementation is luminance-dependent.
Frequent false positives are likely.
This implementation should be replaced by YCbCr from [4].
YCbCr
To be implemented…
More examples of Bounded Images
Mouth Location (YCbCr)
See [4] for further detail.
Mouth Location (continued)
Convex Face
Mouth Mask
Bounding Box
Eye Location (YCbCr)
Eye Map
Eye Location (YCbCr)
Complexity Map
5
6
5
7
150
6
7
150
6
Average of Differences
between pixels in the
8-neighborhood,
followed by dilation
using a disk of 5 pixels
Eye Location (YCbCr)
Eye Map and Complexity Map Combined
AND
AND
Notice problems
because bounding box
needs to be replaced
by Connected
Component Mask
Matlab Demo of Bounding Box, EyeMap and MouthMap
Current Problems and Future Work
• Problem: Current skin-tone clustering is not robust.
• Solution: YCbCr luma-independent detection needs to be implemented.
• Problem: Manual thresholding is implemented.
• Solution: Automatic thresholding is desired.
• Problem: Bounding Box instead of Skin-Tone Connected component
• Solution: Improve algorithm extracting just face-mask
YCbCr Colorspace (continued)
YCbCr Colorspace (continued)
This model alone is sufficient. Taken from [4] (to be implemented in the future).
YCbCr is luminance-dependent. Direct use of YCbCr results in false positives.
Therefore YCbCr must be made luma-independent.
YCbCr Colorspace (continued)
YCbCr Colorspace (continued)
References
1.
2.
3.
4.
A Survey on Pixel-Based Skin Color Detection Techniques, Vladimir Vezhnevets, Vassili Sazonov,
Alla Andreeva, http://www.graphicon.ru/2003/Proceedings/Technical/paper509.pdf
Sanjay Kr. Singh, D. S. Chauhan, Mayank Vatsa, Richa Singh, A Robust Skin Color Based Face
Detection Algorithm, http://www2.tku.edu.tw/~tkjse/6-4/6-4-6.pdf
Face Detection in Color Images using Wavelet Packet Analysis, C. Garcia, G. Zikos, G. Tziritas,
http://www.csd.uch.gr/~tziritas/papers/ICMCS-99.pdf
Face Detection in Color Images, Rein-Lien Hsu, Mohamed Abdel-Mottaleb, Anil K. Jain,
http://www.csd.uch.gr/~hy471/papers/FaceDetectionColor.pdf