Transcript Advanced Topics in Telecommunications Technology Management

The Accuracy of a Fingerprint/Voice
Multimodal Verification System
Karine Pellerin
Department of National Defence
Director Distributed Computing
Engineering and Integration 3-5 (DDCEI 3-5)
October 18, 2004
Agenda
•
•
•
•
•
•
•
Research Problem
Current Research
Experiment and Methodology
Experimental Results
Conclusions
Future Research
Questions
[email protected]
18 October 2004
Slide 2
Research Problem
• Need for Stronger Authentication Techniques
– Widely used authentication techniques
– Biometric Systems
• Importance of Stronger Authentication Techniques
– “Our multi-billion dollar screening regime is defenseless against terrorists
with forged security badge” (U.S. House Aviation Subcommittee Chairman
May 2004)
– 9/11 commission recommends vastly expanding and accelerating the
deployment of biometrics to help detect and deter terrorists (IBIA Aug 04)
• Research Problem: Accuracy of Biometric Systems
• Research question
– Does multimodal biometric systems have a better accuracy than biometric
systems?
[email protected]
18 October 2004
Slide 3
Current Research
• Multimodal research
– Different Combinations of Identifiers
• Voice and Facial, Fingerprint and Facial
– Different fusion strategies
• Sensor and feature level fusion
• Confidence and abstract level
– Adaptive Approaches
• Weighting individual biometric trait
• User specific thresholds
• Environment
• Conclusion of current multimodal research
– Integration of multiple biometric identifiers results in
performance improvement
[email protected]
18 October 2004
Slide 4
Experiment and Methodology
• Objective
– Test whether or not the accuracy of a fingerprint/voice
multimodal obtained using a weighted summation approach
would surpassed the performance of its subsystems.
• Methodology
–
–
–
–
–
Evaluate the accuracy of the fingerprint subsystem
Evaluate the accuracy of the voice subsystem
Perform a weighted summation approach
Evaluate the accuracy of the multimodal system
Compare the different accuracy results
[email protected]
18 October 2004
Slide 5
Accuracy Performance
• False Match Rate (FMR)
– A sample is falsely declared to match a “non self” template (i.e. an impostor is
accepted in the system)
FMR 
Number of ImpostorAcceptance
X 100 %
Number of ImpostorClaims
• False Non Match Rate (FNMR)
– A sample is falsely declared not to match a template of the same measure from
the same user supplying the sample (i.e. a valid user is rejected)
FNMR 
•
Number of Legitimate User Rejections
X 100 %
Number of LegitimateClaims
The Detection error trade-off curve (DET Curve) is used to effectively and
objectively compare different imperfect diagnostic and pattern matching systems.
[email protected]
18 October 2004
Slide 6
Fingerprint Subsystem
•
•
•
Subset of the FVC2000 Fingerprint database (DB3 Set A)
– 560 Fingerprints
VeriFinger 4.1 SDK released in December 2002
320 genuine and 25 280 impostor attempt trials for a total of 25 600 attempts
Fingerprint Impressions
Feature
Extraction Module
Fingerprint
Templates
Matching Module
Matching
Score
Matching Score (Opinion)
regarding a True Claimant
[email protected]
18 October 2004
Decision Module
Accept/Reject Claim
Slide 7
Fingerprint Subsystem Results
• Using the software optimal matching threshold
• False Non-Match Rate
– From the 320 matching scores
• 27 matching errors
• Highest score was 797
• Lowest score was 0 (A total of 6 scores of 0 were obtained)
• FNMR 
27 Legitimate User Rejections
X 100 %  8.43%
320 Legitimate Claims
• False Match Rate
– From the 25 280 matching scores
•
•
•
•
No matching errors
Highest score was 22
Lowest score was 0 (A total of 13 957 scores of 0 were obtained)
98% of all scores were equal or below 10
• FMR 
0 Impostor Acceptances
X 100 %  0.00%
25 280 ImpostorClaims
[email protected]
18 October 2004
Slide 8
Fingerprint Subsystem DET Curve
Fingerprint Subsystem DET Curve
(Using Logarithm Axes)
100.00%
FNMR
10.00%
1.00%
0.10%
0.001%
0.010%
0.100%
1.000%
10.000%
100.000%
FMR
[email protected]
18 October 2004
Slide 9
Voice Subsystem
•
•
•
Subset of XM2VTSDB Audio database
– 640 Speech Sentences
VoiceSafeEvaluator from Patni Computer Systems
320 genuine and 25 280 impostor attempt trials for a total of 25 600 attempts.
Sentence Repetitions
Feature
Extraction Module
Speech
Templates
Matching Module
Matching
Score
Matching Score (Opinion)
regarding a True Claimant
[email protected]
18 October 2004
Decision Module
Accept/Reject Claim
Slide 10
Voice Subsystem Results
•
•
Using the software optimal matching threshold
False Non-Match Rate
– From the 320 matching scores
• 93 matching errors
• Highest score was 1064
• Lowest score was -18 450 (A total of 45 scores of 0 were obtained)
• FNMR 
•
93 LegitimateUser Rejections
X 100%  29.06%
320 LegitimateClaims
False Match Rate
– From the 25 280 matching scores
• 3 992 matching errors
• Highest score was 1104
• Lowest score was -23 940 (A total of 3 664 scores of 0 were obtained)
• FMR 
3992 ImpostorAcceptances
X 100%  15.79%
25 280 ImpostorClaims
[email protected]
18 October 2004
Slide 11
Voice Subsystem DET Curve
Speech DET Curve
(Using Logarithm Axes)
100.00%
FNMR
10.00%
1.00%
0.10%
0.001%
0.010%
0.100%
1.000%
10.000%
100.000%
FMR
[email protected]
18 October 2004
Slide 12
Multimodal System
Fingerprint Impression
Nj = Number or Classes (Nj= 2)
Class1: True Claimant
Class 2: Impostor
Templates
Ni= Number of experts (Ni=2)
Expert 1: Fingerprint
Expert 2: Speech
Feature
Extraction
Expert 1
Accept
Reject
Matching
Opinion of Expert 1
Regarding class j
Mapping
[0,1] interval
2
Fj   wi oij
Threshold
i 1
Opinion of Expert 2
Regarding class j
Speech Repetition
Feature
Extraction
Mapping
[0,1] interval
Opinion regarding Class j
Oij= opinion of the i-th expert on class j
Wi= corresponding weight in the[0,1]
interval of expert i.
Matching
Constraint
Expert 2
[email protected]
Templates
18 October 2004
NI
w
i 1
i
1
Slide 13
Multimodal System Strategy 1
VS Two Subsystems
Strategy 1 DET Curve and subsystems
(Using Logarithm Axes)
100.00%
Strategy 1
Fingerprint
Subsystem
FNMR
10.00%
Voice
Subsystem
1.00%
0.10%
0.001%
0.010%
0.100%
1.000%
10.000% 100.000%
FMR
[email protected]
18 October 2004
Slide 14
Conclusions
1.
The integration of multiple biometrics does not always
result in accuracy improvement
2.
The accuracy improvement of a multimodal system in
comparison to its subsystems is highly dependable on the
initial accuracy of its subsystems and the fusion strategy
used
[email protected]
18 October 2004
Slide 15
Future Research
• Finding the most effective way to fuse independent expert opinion and
improving system accuracy is a significant research challenge
• As accuracy is dependant on the environment, the people and the
application used, it would be informative to have more research done with
other large databases and different applications
• More research is needed on the use of adaptive approaches
• More research is needed to validate the use of cross-comparison when
calculating FMR
• More research is needed on how to measure active impostor attempts
• Research assumes that a biometric identifier from one individual is
statistically independent from another biometric identifier of the same
individual
• Very little research on biometric sensor interoperability
[email protected]
18 October 2004
Slide 16