Transcript Automatic Fingerprint Verification

Soft Biometrics at CUBS
Venu Govindaraju
CUBS, University at Buffalo
[email protected]
www.cubs.buffalo.edu
Background
 Traits of biometrics
 Universality
 Distinctiveness
 Permanence
 Collectability
 Acceptability
 Present perfect?
 No biometric is truly universal. It is estimated that 24% of the population have unusable fingerprints
 Each biometric has a lower bound for errors
(constraint of algorithm + individuality)
 Individual biometrics need to be augmented by other
biometrics (multi-modal) or traits (soft biometrics)
Soft Biometrics
Definition[1]
 Soft biometric traits are those characteristics that provide some
information about the individual but are not distinctive enough to
sufficiently differentiate any two individuals
[1]
Soft Biometrics
 Not very distinctive
 Can be used to augment
regular biometrics
 Not typically used during
verification/identification
 More intuitive than strong
biometrics
[1] A. K. Jain, S. Dass, K. Nandakumar, “Soft Biometrics for Personal Identification”, SPIE
Defense and Security Symposium 2003
Soft Biometrics : Examples
Other classification
 Continuous: Age, Height, Weight etc.
 Discrete: Gender, Eye Color, Ethnicity etc.
Motivation
 Heckathorn[3] have shown that a combination of
personal attributes can be used to identify the
individual reliably
 Binning and Indexing
 Hardening primary biometric
 Speech Recognition
 Can be used to tune individual biometrics
 Socially aware computing (call centers)?
Extracting Soft Biometric Traits
 Devices
 Color video
 Stereo images
 Challenges
 Controlled vs Uncontrolled environment
 Pose variations
 Illumination variation
 Complex backgrounds
 Feature selection and extraction
 Features used in traditional biometrics do not encode soft
biometric traits
 Decision systems (soft thresholds)
Problems in Representation
Fuzzy class boundaries
Purely statistical features
Soft Biometrics Research at CUBS
 Speech
 Gender Identification
 Accent Identification
 Face
 Face Catalog: Semantic Face Retrieval
 Gender Classification
 Skin
 Skin spectroscopy
Soft Biometric Traits in Speech
 Gender
 There exists a difference in the pitch period between genders
 This difference is fundamental in the discrimination between
males and females
 Accent[1]
 Temporal features: onset time, closure/voicing/word duration
 Prosodic/Intonation slope patterns
 Formant frequencies
 Age
 The average power measurement and speech rate are used
as indicators for measurement of agedness in a speaker
[1]A Study of Temporal Features and Frequency characteristics in American English Foreign Accent
L.M. Arslan, J.H.L. Hansen , Journal of the Acoustical society of America, July 1997
Uses of Soft Biometrics in Speech
Soft Biometrics for binning
Primary
Biometric
P(w|x1)
Soft Biometric(s)
Soft Biometrics for improving accuracy
P(w|x1y)
Loose Gender Classification

3 Methods
 Fast Fourier Transform
 Linear Predictive Analysis
 Cepstral Analysis

Data
 75 files
 Males -41, Females -34
Male Low Male Medium
132Hz
156Hz
(PITCH)
Results
Male High
Female Low
Female Medium
171Hz
205Hz
230Hz
Female High
287Hz
Definition of Accent (linguistics)
 An accent is the perceived peculiarities of
pronunciation and intonation of a speaker or
group of speakers
 A foreign accent is defined in a way that the
phonology of the spoken language is modified by
the phonology of another language, more familiar
to the speaker
 3 major language groups
 American
 Chinese
 Indian
Proposed Approach for Accent
First identify the accent markers
Determine the effect of gender and co-articulation
Initially develop a text dependent model
Accumulate evidence over time
Features:
 formants
 phoneme duration
 instantaneous (mel)cepstral slopes
 HMMs





Accent Markers

A look at various non-native pronunciations of English
 CHINESE





‘r’ read sometimes as ‘l’ or ‘w’
‘v’ read as ‘w’
‘th’ read as ‘d’
‘n’ and ‘l’ often confused
Often drop articles like ‘the’ and ‘a’
 INDIAN SUBCONTINENT




Use of the rhotic ‘r’
Use of rolling ‘l’
Fast speech tempo with choppy syllables
Rhythmic variation of pitch
Webster’s Revised Unabridged Dictionary
Definition of non-native pronunciations of English – wordIQ.com
American -
Indian -
F3
F3
MALES – PHONEME CONTAINING ‘L’
PLEASE
STELLA
F2
F3
F3
F2
SLABS
F2
PLASTIC
F2
American -
Indian -
F3
F3
MALES – PHONEMES CONTAINING ‘R’ AND ‘AA’
BRING
RED
F2
F3
F3
F2
FRESH
ASK
F2
F2
American -
Indian -
F3
F3
FEMALES – SEGMENTED PHONEMES ‘L’, ‘R’, ‘AA’
PLEASE
STELLA
F2
F3
F3
F2
RED
F2
ASK
F2
Soft Biometrics for Law Enforcement
Novel Forensic System
Law Enforcement Application: Face Catalog
User can select some facial feature to describe.
System will prompt the user after each query with the best feature
for the next query.
Related Work
 Identikit [1] composes faces by putting together
transparencies of facial features.
 Evofit [2], automate the process of identikits.
 Phanthomas [3] face composition using elastic
graph matching.
 CAFIIRIS [4] and Photobook [5] use PCA for face
composition and matching.
 But general description of users are semantic!
1.
2.
3.
4.
5.
V. Bruce, “Recognizing Faces”, Faces as Patterns, pp. 37-58, Lawrence Earlbaum Associates, 1988
Frowd, C.D., Hancock, P.J.B., & Carson, D. (2004). “EvoFIT: A Holistic, Evolutionary Facial Imaging
Technique for Creating Composites”, ACM TAP, Vol. 1 (1)
“Phantomas: Elaborate Face Recognition “.Product description: http://www.global-securitysolutions.com/FaceRecognition.htm
J. K. Wu, Y. H. Ang, P. C. Lam, S. K. Moorthy, A. D. Narasimhalu, ”Facial Image Retrieval,
Identification, and Inference System”
A. Pentland, R. Picard, S. Sclaroff, “Photobook: tools for content based manipulation of image
databases”, Proc. SPIE: Storage and Retrieval for Image and Video Databases II, vol. 2185
Face Catalog System Overview
Semantic Face Retrieval System
Input Image
Face
Image
Database
Face Detection
Lip Location and
parameterization
Meta
Database
Eye Location
Parameterization of
other Features
Query Sub-System
Prompting Sub-System
user
Sorted
Images
Enrollment Sub-System
 Face Detection.
 Lips and eye detection.
 Locate and parameterize other
features.
Query Sub-System
 Pruning images based on descriptions given?
 What if user makes a mistake in one of the
description.
 Ranking images based on their probability of being
the required person is a better idea.
 Bayesian learning can be used to update probability
of each face being the required one.
 Prompting users the feature with highest entropy at
each step.
Example Query
Query = []
Query = [Spectacles = Yes]
Query = [Spectacles = Yes
+ Mustache = Yes]
Query = [Spectacles = Yes
+ Mustache = Yes
+ Nose = Big]
Probabilities of Faces
Results
 Results of Enrollment Sub-system
Features
(Database of 150 images)
Spectacles
Number of False
Accepts
1
Number of False
Rejects
2
Mustache
2
4
Beard
4
0
Long Hair
2
8
Balding
1
0
 Results of Query
Average no. of
queries.
(25 users, 125 test cases)
Top 5
Top 10
Top 15
7.12
5.08
2.49
Gender Classification in Images
 Gender classification
 Identifying male or female from facial image
 Existing approaches
 Geometric feature based [1]-[2]
 Appearance feature based (raw data feature or PCA
+ classifier) [3]
 Approaches using other features, e.g., wrinkle and
skin color [4]
[1] A. Burton, V. Bruce and N. Dench, “What’s the difference between men and women? Evidence from facial
measurements,” Perception, vol. 22, pp.153-176, 1993.
[2]R. Brunelli and T. Poggio, “Hyperbf network for gender classification,” DARPA Image Understanding Workshop, pp.
311-314, 1992.
[3]B.A. Golomb, D.T. Lawrence, T.J. Sejnowski, “Sexnet: A Neural Network Identifies Sex from Human Faces,”
Advances in Neural Information Processing Systems3, R.P Lippmann, J.E. Moody, D.S. Touretzky, eds. Pp. 572-577,
1991.
[4] J. Hayashi, M. Yasumoto, H. Ito, H. Koshimizu, “Age and gender estimation based on wrinkle texture and color
of facial images,”, Proceedings of 16th International Conference on Pattern Recognition, vol. 1, pp. 405 - 408, 11-15
Aug. 2002
Gabor Feature based gender classification system
Raw
Image
Preprocessing
(Face detection,
normalization, etc.)
Feature Extractor
Using
Gabor Wavelet
SVM
Classifier
Decision
Facial image Normalization



Mapping feature points to fixed
positions
Feature points
 Centers of two pupils
 Tip of the nose
Normalized image
 64 by 64
 Convert from color to grayscale
by averaging RGB components
Gabor feature

Gabor filter and Gabor wavelet [B.S. Manjunath, et al, PAMI,
1996]
Gabor Filter:
Fourier Transform
of g(x, y):
 1  x2 y2 

 1 
 exp 
g ( x, y)  
 2   2jWx 
2
 2  

 2   x  y 

x y 

2


 u  1 / 2 x
v 2 
 1  (  W )

G (u , v)  exp 
 2   where 
2
 v  


 2  u
 v  1 / 2 y
g mn ( x, y )  a  m g ( x' , y ' ,W , n / K ,  x ,  y ), a  1, m , n integer,
Gabor Wavelet:
x'  a  m ( x  cos  y  sin  ), y '  a  m ( x  sin   y  cos )
 : n / K ; K : number of orientations. S : number of scales, then
0  m  S, 0  n  K.
Gabor feature (cont.)

Redundancy reduction [B.S. Manjunath, et al, PAMI, 1996]
 Let U l andU h denote the lowest and highest frequencies of interest
 a,  u ,  v are determined by


1
a  (U / U ) S 1
h
l


(a  1)U l
 u 
(a  1) 2 ln 2

1


2
2 2

2 ln 2 u  
(2 ln 2)  u  2
  
 U l 
 v  tan
 2 ln 2 

2
2
K
U
U


l
l





Gabor feature (cont.)


Characteristics of Gabor wavelet
 A powerful tool to capture changes of signals
 Selective on certain frequency and orientation by setting
parameters m, n
Gabor feature for gender classification
 Gabor WT at 4 scalses, 4 orientations (m = 0, .., 3; n = 0, …, 3)
 Each output image of Gabor WT (64 by 64) is divided into nonoverlapping blocks of the size 2m+2 by 2m+2 (m: the scale number).
 Average of magnitudes in each block as a feature
(magnitude  (real component) 2  (imaginary component) 2 )
 Total number of features 4 
 64 64 /2   1360
3
m 0
m 2 2
Gabor feature (cont.)
S  4, K  4
U l  0.08
U h  0.64
Classification


Features
 1360-dimensional training and testing vectors fed into SVM
classifier
Classifier
 SVM with Gaussian RBF kernel [6] (B. Moghaddam, et al, PAMI
2002)
 Adjust γ to minimize error rate
 1360 features from Gabor WT (in 4 scales, 4 orientations) of
64×64 input image
 Training and testing vectors (of 1360 dimensions) normalized into
unit vectors
Experimental Results

Dataset: AR face database
[A.M. Martinez and R. Benavente, “The AR face
database,” CVC Tech. Report #24, 1998]
 Overall: 3265 frontal facial images including 136
Caucasian people (768 by 576, color)
 Training: 2246 samples including 91 individuals
 Testing: 1019 samples including 45 individuals

Test #1
 393 regular samples. Accuracy: 96.2%

Test #2
 626 irregular samples (occluded by dark sun-glasses or
masks) Accuracy: 92.7%
Method
Accuracy of test #1
Accuracy of test #2
Gabor feature + SVM with
Gaussian RBF kernel
96.2%
92.7%
Raw data feature + SVM with
Gaussian RBF kernel
94.7%
89.8%
Skin Spectroscopy






Measures the composition of the skin using IR(Deep tissue biometric)
Based on spectroscopy
Fool proof against fake fingers (Can detect liveness)
Can be easily integrated into solid state devices
Immune to surface degradations
Currently implemented by only one Vendor (Lumidigm Inc)
Skin composition
Chromophores in skin
 Melanin
 Absorbs light at all wavelengths
 Absorbance decreases with increase in wavelength
 Hemoglobin
 Strongest absorption bands in 405 – 430 nm and
540 – 580 nm.
 Lowest absorption beyond 620 nm
 Can be used for liveness testing
 Collagen, Keratin, Carotene
Spectra of Melanin and Hemoglobin
Sample Skin Spectrum
Sample skin spectrum (contd.)
Sample skin spectrum (contd.)
Results so far
 Soft classification based on skin color
 Melanin index used as indicator of skin color
 Spectral difference noticed between different skin
locations on the same individual
Thank You
[email protected]