Face Recognition Under Varying Illumination

Erald VUÇINI Vienna University of Technology Muhittin GÖKMEN Istanbul Technical University Eduard GRÖLLER Vienna University of Technology

Face Recognition System

Image Capture Face Detection Feature Projection

Database

Face Identification Erald Vuçini - Vienna University of Technology 2

Face Recognition Approaches Principal Component Analysis (PCA) Linear Discriminant Analysis (LDA) Local Feature Analysis Active Appearance Model Hidden Markov Model Support Vector Machine …

Erald Vuçini - Vienna University of Technology 3

Face Recognition – Problems The variations of the same face due to illumination viewing direction are almost always larger variations due to changes in the face identity than image

Erald Vuçini - Vienna University of Technology 4

Handling Variable Illumination Extract illumination invariant features Transform variable illumination to canonical representation Model 2D illumination variations Utilize 3D face models whose shapes and albedos are obtained in advance

Erald Vuçini - Vienna University of Technology 5

Outline of proposed approach I.

Dimensionality Reduction - LDA better than PCA regarding illumination II.

Image Synthesis - Solve the Small Sample Size (SSS) problem III.

Reconstruction illumination – Restore frontal

Erald Vuçini - Vienna University of Technology 6

Dimensionality Reduction

I.

Dimensionality Reduction - LDA better than PCA regarding illumination

II.

Image Synthesis - Solve the Small Sample Size (SSS) problem III.

Reconstruction – Restore frontal illumination

Erald Vuçini - Vienna University of Technology 7

Principal Component Analysis (PCA) One of the most commonly used methods in Face Recognition Maximizes the scattering of all projected samples x 2 x 2 x 1 PCA x 1

Erald Vuçini - Vienna University of Technology 8

PCA under Varying Illumination PCA fails with variant illumination The scatter being maximized is due to Between-class scatter Within-class scatter Discard 3 most significant principal components to reduce lighting variation

Erald Vuçini - Vienna University of Technology 9

LDA Interpretation LDA LDA is a class specific method

Erald Vuçini - Vienna University of Technology 10

LDA Problems LDA maximizes the ratio of Between-class scatter and Within-class Scatter Within-class Scatter singularity problem Fisher LDA ( FLDA ) removes Null Space FLDA handles best the variation in lighting, having lower error rate than PCA

Erald Vuçini - Vienna University of Technology 11

Image Synthesis I.

Dimensionality Reduction - LDA better than PCA regarding illumination

II.

Image Synthesis - Solve the Small Sample Size (SSS) problem

III.

Reconstruction – Restore frontal illumination

Erald Vuçini - Vienna University of Technology 12

Image Synthesis - Motivation Face Recognition Systems Performance related with training database LDA require many samples per class In many systems only one image per person is provided Quotient Image makes possible the synthesis of the image space of a given input image

Erald Vuçini - Vienna University of Technology 13

Lambertian Objects - Faces

Albedo

Image

 

Surface Normal Light Source Direction

The image space lives in a 3D linear subspace Three images are sufficient for generating the image space of the object

Erald Vuçini - Vienna University of Technology 14

Quotient Image (Definitions) Ideal class of faces Same shape Different albedos 

i n T s

Synthesis Problem: Given 3N images of N faces of the same class, illuminated under 3 lighting conditions Synthesize image space of new input

Erald Vuçini - Vienna University of Technology 15

Quotient Image (Definitions) Given objects

y

image

Q

and

a

we define quotient by the ratio of their albedos Q is illumination invariant Image space of y can be generated with Quotient Q 3 images of a Generalization : Use bootstrap of 3N images

Erald Vuçini - Vienna University of Technology 16

Quotient Image - Examples Quotient Quotient

Erald Vuçini - Vienna University of Technology 17

Quotient Image – Image Space Synthesis

Erald Vuçini - Vienna University of Technology 18

10 person Bootstrap 5 person Bootstrap

Erald Vuçini - Vienna University of Technology

1 person Bootstrap

19

Reconstruction I.

Dimensionality Reduction - LDA better than PCA regarding illumination II.

Image Synthesis - Solve the Small Sample Size (SSS) problem

III.

Reconstruction illumination – Restore frontal Erald Vuçini - Vienna University of Technology 20

YaleB Testing Database Yaleb Database 450 images of 10 persons Divided in 4 subsets Subset1 up to 10 ˚ Subset2 up to 25˚ Subset3 up to 45˚ Subset4 up to 75˚ Normalized

Erald Vuçini - Vienna University of Technology 21

Histogram Equalization Histogram equalization(HE) done as preprocessing increases the recognition rate Adaptive HE(AHE) is used as a preprocessing step in the iterative face recognition approach Results with the YaleB Database (PCA used) No Preprocessing HE Recognition Rate(%) 43.4

74 AHE 81.5

Erald Vuçini - Vienna University of Technology 22

Illumination Restoration Approach A face image with arbitrary illumination is restored to having frontal illumination. It has the following advantages: No need to estimate face surface normals No need to estimate light source directions and albedos No need to perform image warping Face images will be visually natural looking

Erald Vuçini - Vienna University of Technology 23

Algorithm Outline Compute mean face image and eigenspace Compute initial restored images Create iteration by replacing B ro with blurred H io Continue iteration until stopping criteria satisfied

Restored Image

H

io



Input Image

I

ik

B

ro

B

ik

Blurred Reference Image Blurred Input Image Erald Vuçini - Vienna University of Technology 24

Iteration Steps

Erald Vuçini - Vienna University of Technology 25

Experimental Results (Subset 3) Restoration

Erald Vuçini - Vienna University of Technology 26

Experimental Results (Subset 4) Restoration

Erald Vuçini - Vienna University of Technology 27

Results with YaleB Database HE+PCA(%) Subset1 Subset2 Subset3 Subset4 Overall 100 97.5

66.4

44.2

74

HE+New(%) 100 100 92.8

91.7

95.6

AHE+PCA(%) AHE+New(%) Subset1 Subset2 Subset3 Subset4 Overall 100 100 83.57

50.0

81.6

100 100 100 95.0

98.7

Erald Vuçini - Vienna University of Technology 28

Thank you for the attention!

Proposed Method Dimension Reduction Image Synthesis Reconstruction

Erald Vuçini - Vienna University of Technology 29