Transcript Face Recognition CPSC 4600/5600 @ UTC/CSE Face Recognition Introduction  Face recognition algorithms  Comparison  Short summary 

Face Recognition
CPSC 4600/5600 @ UTC/CSE
Face Recognition
Introduction
 Face recognition algorithms
 Comparison
 Short summary

2
Face Recognition Algorithms

We will introduce
Eigenfaces
 Fisherfaces
 Elastic Bunch-Graph Matching

3
Eigenfaces
Developed in 1991 by M.Turk
 Based on Principal Component Analysis
(PCA)
 Relatively simple
 Fast
 Robust

4
Eigenfaces

PCA seeks directions that are efficient for
representing the data
not efficient
Class A
efficient
Class A
Class B
Class B
5
Eigenfaces

PCA maximizes the total scatter
scatter
Class A
Class B
6
Eigenfaces
PCA reduces the dimension of the data
 Speeds up the computational time

7
8
Eigenfaces, the algorithm

Assumptions
Square images with Width = Height = N
 M is the number of images in the database
 P is the number of persons in the database

9
Eigenfaces, the algorithm

The database
 a1 


a2 






a
2
 N 
 b1 


b
2






b
2
 N 
 c1 


c
2






c
2
 N 
 d1 


d
2






d
2
 N 
 e1 


e
2 





e
 N2 
 f1 


f
2






f
2
 N 
 g1 


g
2 





g
2
 N 
 h1 


h
2






h
2
 N 
10
Eigenfaces, the algorithm

We compute the average face
 a1  b1 

1  a2  b2 
m
M

 aN 2  bN 2 
 h1 

 h2 
,


 hN 2 
where M  8
11
Eigenfaces, the algorithm

Then subtract it from the training faces
 a1  m1 
 b1  m1 
 c1  m1 
 d1  m1 








a

m
b

m
c

m
d

m
2
2 
2
2 
2
2 
2
2 
am  
, bm  
, cm  
, dm  
,
















a

m
b

m
c

m
d

m
N2 
N2 
N2 
N2 
 N2
 N2
 N2
 N2
 e1  m1 


e

m
2
2 
em  
,




 eN 2  mN 2 
 f1  m1 
 g1  m1 
 h1  m1 






f

m
g

m
h

m
2
2 
2
2 
2
2 
fm  
, gm  
, hm  












 f N 2  mN 2 
 g N 2  mN 2 
 hN 2  mN 2 
12
Eigenfaces, the algorithm

Now we build the matrix which is N2 by M
A   am bm cm d m em f m g m hm 

The covariance matrix which is N2 by N2

Cov  AA
13
Eigenfaces, the algorithm

Find eigenvalues of the covariance matrix
The matrix is very large
 The computational effort is very big


We are interested in at most M eigenvalues

We can reduce the dimension of the matrix
14
Eigenfaces, the algorithm

Compute another matrix which is M by M

LA A

Find the M eigenvalues and eigenvectors


Eigenvectors of Cov and L are equivalent
Build matrix V from the eigenvectors of L
15
Eigenfaces, the algorithm

Eigenvectors of Cov are linear combination
of image space with the eigenvectors of L
U  AV
V is Matrix of
eigenvectors
A   am bm cm d m em f m g m hm 

Eigenvectors represent the variation in the
faces
16
Eigenfaces, the algorithm
A: collection of the
training faces
U: Face Space /
Eigen Space
17
Eigenfaces

Eigenface of original faces
18
Eigenfaces, the algorithm

Compute for each face its projection onto
the face space
 
 f ,
 
 U h 
1  U   am  ,  2  U  bm , 3  U   cm  ,  4  U  d m ,
5  U   em  , 6  U 

m
 7  U   g m  , 8

m
Compute the threshold

1
  max i   j
2
 for i, j  1.. M
19
Eigenfaces: Recognition Procedure

To recognize a face
 r1 
 
r2
 
 
 
 rN 2 

Subtract the average face from it
 r1  m1 


r

m
2
2 
rm  




 rN 2  mN 2 
20
Eigenfaces, the algorithm

Compute its projection onto the face space U
 U

 rm 
Compute the distance in the face space
between the face and all known faces

    i
2
i
2
for i  1.. M
21
Eigenfaces, the algorithm

Reconstruct the face from eigenfaces
s  U

Compute the distance between the face
and its reconstruction
  rm  s
2
2
22
Eigenfaces, the algorithm

Distinguish between
If    then it’s not a face; the distance
between the face and its reconstruction is
larger than threshold
 If    and min  i   
then it’s a new face
 If    and  i   , (i  1.. M ) then it’s a known
face because the distance in the face space
between the face and all known faces is larger
than threshold

23
Eigenfaces, the algorithm

Problems with eigenfaces

Different illumination
24
Eigenfaces, the algorithm

Problems with eigenfaces
Different head pose
 Different alignment
 Different facial expression

25
Fisherfaces
Developed in 1997 by P.Belhumeur et al.
 Based on Fisher’s Linear Discriminant
Analysis (LDA)
 Faster than eigenfaces, in some cases
 Has lower error rates
 Works well even if different illumination
 Works well even if different facial express.

26
Fisherfaces

LDA seeks directions that are efficient for
discrimination between the data
Class A
Class B
27
Fisherfaces
LDA maximizes the between-class scatter
 LDA minimizes the within-class scatter

Class A
Class B
28
Fisherfaces, the algorithm

Assumptions
Square images with Width=Height=N
 M is the number of images in the database
 P is the number of persons in the database

29
Fisherfaces, the algorithm

The database
 a1 


a2 






a
2
 N 
 b1 


b
2






b
2
 N 
 c1 


c
2






c
2
 N 
 d1 


d
2






d
2
 N 
 e1 


e
2 





e
 N2 
 f1 


f
2






f
2
 N 
 g1 


g
2 





g
2
 N 
 h1 


h
2






h
2
 N 
30
Fisherfaces, the algorithm

We compute the average of all faces
 a1  b1 

1  a2  b2 
m
M

 aN 2  bN 2 
 h1 

 h2 
,


 hN 2 
where M  8
31
Fisherfaces, the algorithm

Compute the average face of each person
 a1  b1 


a

b
1 2
2 
x 
,


2


 aN 2  bN 2 
 e1 

1  e2 
z
2

 eN 2 
 c1  d1 


c

d
1 2
2 
y 
,


2


 cN 2  d N 2 
f1 
 g1  h1 



f2 
g

h
1 2
2 
, w 


2



f N 2 
g

h
N2 
 N2
32
Fisherfaces, the algorithm

And subtract them from the training faces
 a1  x1 
 b1  x1 
 c1  y1 
 d1  y1 








a

x
b

x
c

y
d

y
2
2 
2
2 
2
2 
2
2 
am  
, bm  
, cm  
, dm  
,
















 a N 2  xN 2 
 bN 2  xN 2 
 cN 2  y N 2 
 d N 2  yN 2 
 e1  z1 


e

z
2
2 
em  
,




 eN 2  z N 2 
 f1  z1 
 g1  w1 
 h1  w1 






f

z
g

w
h

w
2
2 
2
2 
2
2 
fm  
, gm  
, hm  












 f N 2  zN 2 
 g N 2  wN 2 
 hN 2  wN 2 
33
Fisherfaces, the algorithm

We build scatter matrices S1, S2, S3, S4

 e


, S   g

h 
S1  am am  bm bm , S 2  cm cm  d m d m ,
S3



e

f
f
m m
m m
4

g
m m  hm

m
And the within-class scatter matrix SW
SW  S1  S2  S3  S4
34
Fisherfaces, the algorithm

The between-class scatter matrix
S B  2  x  m  x  m   2  y  m  y  m   2  z  m  z  m   2  w  m  w  m 




We are seeking the matrix W maximizing
J W  
W  S BW
W  SW W
35

Fisherfaces, the algorithm
If SW is nonsingular ( M  N 2 ):
1
 Columns of W are eigenvectors of SW SB
 We have to compute the inverse of SW
We have to multiply the matrices
 We have to compute the eigenvectors

36
Fisherfaces, the algorithm
If SW is nonsingular ( M  N 2 ):
 Simpler:
Columns of W are eigenvectors satisfying
SB wi  i SW wi
 The eigenvalues are roots of

SB  i SW  0

Get eigenvectors by solving
 SB  i SW  wi  0
37
Fisherfaces, the algorithm
If SW is singular ( M  N ):
 Apply PCA first
2
Will reduce the dimension of faces from N2 to M
 There are M M-dimensional vectors


Apply LDA as described
38
Fisherfaces, the algorithm

Project faces onto the LDA-space
xLDA  W  x ,
y LDA  W  y ,
z LDA  W  z , wLDA  W  w

To classify the face
Project it onto the LDA-space
 Run a nearest-neighbor classifier

39
Fisherfaces, the algorithm

Problems
Small databases
 The face to classify must be in the DB

40
PCA & Fisher’s Linear Discriminant
41
PCA & Fisher’s Linear Discriminant
42
Comparison

FERET database
best ID rate: eigenfaces 80.0%, fisherfaces 93.2%
43
Comparison

Eigenfaces
project faces onto a lower dimensional subspace
 no distinction between inter- and intra-class
variabilities
 optimal for representation but not for
discrimination

44
Comparison

Fisherfaces
find a sub-space which maximizes the ratio of
inter-class and intra-class variability
 same intra-class variability for all classes

45
Local Feature Analysis
-- Elastic Bunch-Graph Matching
46
Face Features

Facial recognition utilizes distinctive features of
the face – including: distinct micro elements like:



Mouth, Nose, Eye, Cheekbones, Chin, Lips, Forehead,
Ears
Upper outlines of the eye sockets, the areas
surrounding the cheekbones, the sides of the
mouth, and the location of the nose and eyes.
The distance between the eyes, the length of the
nose, and the angle of the jaw.
47
Face Features




Some technologies do not utilize areas of the face
located near the hairline, so they are somewhat resistant
to moderate changes in hairstyle.
When used in identification mode, facial recognition
technology generally returns candidate lists of close
matches as opposed to returning a single definitive
match as does fingerprint and iris-scan.
The file containing facial micro features is called a
"template."
Using templates, the software then compares that image
with another image and produces a score that measures
how similar the images are to each other.
48
Face Features

Typical sources of images for use in facial
recognition include video camera signals
and pre-existing photos such as those in
driver's license databases.
including:
Distance between the micro elements
 A reference feature
 Size of the micro element
 Amount of head radiated from the face
(unseen by human eye). Heat can be
measured using an infrared camera.

49
A face recognition based on local feature
analysis

A face is represented as a graph, whose nodes,
positioned in correspondence to the facial
fiducial points.




A fiducial point is a point or line on a scale used for
reference or comparison purposes.
A face recognition system uses an automatic
approach to localize the facial fiducial points.
It then determines the head pose and compares
the face with the gallery images.
This approach is invariant to rotation, light and
scale.
50
A template for the 34 fiducial points on a face image:
51
EBGM




Elastic Bunch-Graph Matching (EBGM) algorithm
locates landmarks on an image, such as the eyes, nose,
and mouth.
Gabor jets are extracted from each landmark and are
used to form a face graph for each image. A face graph
serves the same function as the projected vectors in the
PCA or LDA algorithm; they represent the image in a low
dimensional space.
After a face graph has been created for each test image,
the algorithm measures the similarity of the face graphs.
Paper:http://www.snl.salk.edu/~fellous/posters/Bu97post
er/BUPoster.pdf
52
Summary

Three algorithms have been introduced

Eigenfaces
the dimension of the data from N2 to M
 Verify if the image is a face at all
 Allow online training
 Fast recognition of faces
 Problems with illumination, head pose etc
 Reduce
53
Summary

Fisherfaces
Reduce dimension of the data from N2 to P-1
 Can outperform eigenfaces on a representative
DB
 Works also with various illuminations
 Can only classify a face which is “known” to DB


Elastic Bunch-Graph Matching
Reduce the dimension of the data from N2 to M
 Recognize face with different poses
 Recognize face with different expressions

54
References
[1] M. Turk, A. Pentland, “Face Recognition Using Eigenfaces”
[2] J. Ashbourn, Avanti, V. Bruce, A. Young, ”Face Recognition Based
on Symmetrization and Eigenfaces”
[3] http://www.markus-hofmann.de/eigen.html
[4] P. Belhumeur, J. Hespanha, D. Kriegman, “Eigenfaces vs
Fisherfaces: Recognition using Class Specific Linear Projection”
[5] R. Duda, P. Hart, D. Stork, “Pattern Classification”, ISBN 0-47105669-3, pp. 121-124
[6] F. Perronin, J.-L. Dugelay, “Deformable Face Mapping For Person
Identification”, ICIP 2003, Barcelona
[7] B. Moghaddam, C. Nastar, and A. Pentland. A bayesian similarity
measure for direct image matching. ICPR, B:350–358, 1996.
http://www.face-rec.org/interesting-papers/
55
Hands-on Lab of Face Biometrics

Present one of the following algorithms
Elastic Bunch-Graph Matching (EBGM)
algorithm
 Bayesian Intrapersonal/Extrapersonal
Classifier, or
 One from http://www.face-rec.org/interestingpapers/


Hands-on Lab of Face Biometrics
http://www.cs.colostate.edu/evalfacerec/
 User Guide

56