Transcript Fingerprint Minutiae Interoperability Results of the MTIT

Testing and Certification of Biometric
Components and System in Europe
a report on the intermediate findings of the BioTesting
Europe Project
Maria Margarida Castro Neves
Fraunhofer IGD, Germany
[email protected]
Agenda
1. About the “BioTesting Europe” project
2. Identified EU needs for testing in biometrics
3. Issues & Gaps in testing capabilities
4. Improving EU capabilities for assuring performance
2
BioTesting Europe
Project details
– 9 month project: finishing by Dec 2007
– Supporting Activity under “Preparatory Actions for Security Research”
Partners:
– European Biometrics Forum (coordinator)
– National Physical Laboratory (UK)
– Fraunhofer IGD (Germany)
– EC/JRC Ispra (Italy)
Objectives
– Consult to determine EU’s needs for testing of biometrics (Inventory)
– Identify where improved testing capabilities required (Gap Analysis)
– Prepare work plan/roadmap of coordinated actions to further develop
biometrics testing and certification capabilities
– Define the ‘business case’ for testing
3
European Approach
This is why national governments / authorities should
support a European approach for testing certificates:
– The vendors would not survive to pay
for 27 national tests/certificates
– Not-testing (before installing) would
undermine the EU-wide
security policy for the border control process
– We need to provide a comparable security
at all border control points along the
EU perimeter
Vice-Versa recognition works (well) for
CC-certification. It should work also
for Biometric Performance certification!
4
Project scope
Stakeholders consulted
– Suppliers
• Vendors
• System Integrators
– Operators
• end customer
– Test organisations
• Independent 3rd party labs
• In-house test labs
• Certification authorities
– Academics
Applications considered
(Criteria: relevance and urgency)
• Passports
• AFIS
• Visas (VIS BMS)
• Identity documents
• Registered traveller
Potential Scope:
•
Systems
•
Sub-systems
•
Devices
•
Processes
•
Personnel (training & education)
5
Questions to be answered
What testing is needed?
Which components should be certified?
Who should perform these tests?
What standards are applicable?
What do we already have & what needs to be developed ?
What R&D is needed?
What are the costs and who will pay/invest?
Inventory based on 38 Questionnaires
6
Example: e-borders
What needs testing for e-Passports and border control e-Gates?
– Qualities of enrolment
• Procedures
• Operating environment
– Interoperability
– Efficiency at the border
• Throughput
• Accuracy
• Accessibility
– Usability
• Consistency of processes
7
Testing needed / Tests conducted
Needs to be tested
Who tests
Comments
Operators Suppliers Test-labs
Performance
ST
Component level tests
O
(Sub-)System level tests
Accuracy 1:1
OST
Accuracy 1:N (with large N)
OS
Need v.large databases
Failure to Enrol/Acquire
OT
Need representative population &
environment
Throughput
OST
Interoperability
Conformance
Data format (levels1&2)
Data format
(level 3 – semantic level)
API
T
ST
S?
E.g. MINEX, MTIT
Some test tools
No methodologies / reference
data
OS
8
To be tested
Biometric data quality
Software kit to assess data
quality
Sensor testing
Quality & Conformance
Sensor ruggedness
Production quality
Usability / Accessibility
Security
Anti-spoofing
Data protection
Safety
Personnel
Who tests Comments
ST
Standards being developed
?
Validation / Calibration needed?
S
ST
S?
O?
ST
T?
ST
O
E.g. Appendix F
traditional type of test
Are all sensors the same quality
as the tested/certified one
Not tested to any standard
Few products tested under CC
Similar to security audit
CE plus?
9
Observations
Testing is carried out by Suppliers, Operators, and Test Organisations
– Mostly by suppliers & operators
– Most current test needs are being addressed
• By ad-hoc means rather than using standard schema / references
3rd party tests & certification will be complementary
to suppliers’ and operators’ tests
– Suppliers will test during development & production
– Operators need to test on their own data
• “Helps us understand our system”
Standard tests & certification must meet real needs
– Certify against applicable levels of performance, test scenario, etc.
– Must be a return on investment in carrying out the tests
10
Observations / Gaps
Fragmented approach to testing
– Few common requirements identified
– Disconnect between component-level tests & system-level tests
• Component-level performance not predictive of system-level performance
No methodologies / standards for some key areas of testing
– Usability/Accessibility (of particular EU interest)
– Level-3 conformance to data format standards
• i.e. is the record an accurate representation of the characteristic
– …
Biometrics not a mature technology –
still many unknowns about performance
– E.g. long-term performance of face, fingerprint, iris
• Ageing of face compared to photo image over lifetime of passport
• Performance expectations fingerprinting children (age limits)
11
Observations / Gaps
Usability and Accessibility
– Diverse concepts for Human-Computer-Interface (HCI) among vendors, creating
confusion for data subjects
– Standardization of usability related issues is not progressed far: ISO 24779 (Icons
& Symbols) is in early Working Draft status
– R&D: How can we separate out usability impacts on biometric performance?
Need for test data
– Determining high accuracy requires a lot of data
– Data protection legislation often prevents sharing/saving data
– Release of any data may compromise its use in testing
– Possible Technical Solutions:
• Possibility to consider synthetic data?
• If the test data can not travel to the System-Under-Test could the system
travel to the data?
12
Organisational structures (under consideration)
Do we need a network of test organisations?
– European – International?
– Which existing institution can take the role of an accreditation body?
– Criteria for including a test laboratory in such a network?
• Which type of labs are accepted:
–
–
–
•
•
Governmental lab / Independent lab
Consultant / integrators lab
Industry lab
No closed group - transparent conditions needed
What are the criteria that a lab drops out of the network
14
Organisational structures (under consideration)
As
–
–
–
resources are limited - where should the focus of testing be?
Biometric Performance testing
Protocol testing (according to SC17.3 work)
Security testing along Common Criteria …
What role for “Qualified product lists” / “certification”?
– Some performance aspects better suited to certification than others
• Conformance to standard
• Interoperability
• FAR/FRR – too dependent on target population/environment
– Scope of certificate
• Application specific?
• Duration?
15
Conclusions
BioTesting project underway
– Project finishes soon, but comments/opinions welcomed
Testing of usability issues is becoming urgent to achieve
desired levels of performance & interoperability
Focus of test and certification seems certain to change as
industry matures
16
Further information
Contact points
– [email protected]
• +31 624 603809 (direct)
• +353 1 488 5810 (secretariat)
–
[email protected]
• +44 20 8943 7029
–
[email protected]
• +49 6151 155 536
[email protected]
• +49 6151 155 535
–
Website
– www.biotestingeurope.eu
17