Keystroke Biometric & Stylometry Systems TEAMS 2 & 4 THE MICHAEL L. GARGANO 9TH ANNUAL RESEARCH DAY PRESENTATION PRESENTERS EDYTA ZYCH & VINNIE MONACO May 6, 2011 Seidenberg.
Download ReportTranscript Keystroke Biometric & Stylometry Systems TEAMS 2 & 4 THE MICHAEL L. GARGANO 9TH ANNUAL RESEARCH DAY PRESENTATION PRESENTERS EDYTA ZYCH & VINNIE MONACO May 6, 2011 Seidenberg.
Keystroke Biometric & Stylometry Systems
T EAMS T HE 2 & 4 M ICHAEL L. G ARGANO R ESEARCH D AY 9 TH A NNUAL P RESENTATION P RESENTERS E DYTA Z YCH & V INNIE M ONACO May 6, 2011
Seidenberg School of Computer Science and Information Systems Pace University, Graduate Center White Plains, New York
A
GENDA
Team and Project Leader Introductions KBS & Stylometry Projects Overview Project Specifications & Deliverables System Components & Enhancements Results & Conclusions Future Work
P
ROJECT
S
TAKEHOLDERS
Keystroke Biometric
Team Members
Vinnie Monaco
Tyrone Allman Mino Lamrabat Mandar Manohar
Stylometry
Team Members
Edyta Zych
Omar Canales Vinnie Monaco Thomas Murphy
Customers / SMEs
Dr. Tappert John Stewart Robert Zack
Customers / SMEs
Dr. Tappert John Stewart
T
WO
P
ROJECTS
A
CT
A
S
O
NE
, T
WO
T
EAM
L
EADS
Person Manager
Facilitate Weekly Meeting Schedule Task Assignments Driving Everyday Activities Tech Training & Documentation
Technical Manager
Subject Matter Expert (SME) Technical Scope Design & Implementation of all System Enhancements Programming Tasks
O
VERVIEW
: K
EYSTROKE
B
IOMETRIC
S
YSTEM
Pace University has conducted over 8 years of research on Keystroke Biometrics The Keystroke Biometric System (KBS) can be used for both identifying and authenticating users from their typing rhythms Keystroke dynamics are the patterns of rhythm and timing created when a person types, including: Overall speed Variations of speed moving between specific keys Common errors The length of time that keys are depressed (duration)
This semester’s work focuses solely on the KBS as it relaters to an online test taking environment
O
VERVIEW
: S
TYLOMETRY
Stylometry is the study of the unique linguistic styles and writing behaviors of individuals in order to determine authorship Stylometry uses statistical pattern recognition, and artificial intelligence techniques Stylometry features typically used to analyze text include word frequencies and identifying patterns in common parts of speech
This semester’s work focuses on text input being used in conjunction with the keystroke analysis to improve authentication results including
Determining authorship in documents (Beneficial academically to assist with on-line test taking) Protecting against plagiarism through a third party
P
ROJECT
S
PECIFICATIONS
Work closely with our project customer to define the most appropriate Keystroke & Stylometry Features and add additional features to assist in validating/authenticating the identity of students taking an online exam Extract the selected Feature Set for Keystroke Biometric and Stylometry Analysis and run experiments to measure program performance utilizing the enhanced systems: Input System, Feature Extractor and Classifier Run experiments and tests on the data collected to support the identification of subject and online test-taker authorship
KBS Stylometry
P
ROJECT
D
ELIVERABLES Website Input System Feature Extractor Classifier Systems User Manuals & Documentation Input System Feature Extractor Presentation Technical Papers
O
VERVIEW OF
S
YSTEM
C
OMPONENTS
Input System
Captures keystroke and stylometry data in an online test format
Feature Extractor
Measures raw data to obtain a feature vector for each sample
Classifier
Uses feature vectors to test authentication
I
NPUT
S
YSTEM
E
NHANCEMENTS
Upgraded from a Java Applet to a standalone java program. Implemented a user management system to simulate an online test taking environment Change to test taking format, instead of free text or copying tasks Moved to a more general XML data format, to handle both keystroke and stylometry data More restrictions in place on how users interact with the system Disable cut/copy/paste ability Users must complete the test in full Capture and log keystrokes from every successful login attempt
F
EATURE
E
XTRACTION
E
NHANCEMENTS
Feature extraction implemented in the functional language Clojure Easy integration with Java front end Better data handling, filtering, and mapping capabilities New Normalization method tested Old formula New formula Improved outlier removal Integrated stylometry and keystroke features
B
ENCHMARK
R
ESULTS
: 18
SUBJECTS
, 180
SAMPLES Before After
N
ORMALIZATION
R
ESULTS ON
B
ENCHMARK
D
ATA Good Bad Still OK
A
NALYSIS
/ R
ESULTS
40 students, 10 samples each from 1 test Weak training Keystroke and Stylometry biometrics
A
NALYSIS
/ R
ESULTS
38 students, 20 samples from 2 tests Strong training Stylometry biometrics
FRR (%)
20
K
EYSTROKE
C
OMBINED
D
ATA
38 students, 20 samples each from 2 tests Weak training ~11% equal error rate 38 students, 20 samples each from 2 tests 2 samples combined yielding 10 samples each Weak training ~5% equal error rate
20 0 FRR (%) 100 0 FRR (%) 100
K
EYSTROKE VS
. S
TYLOMETRY
ROC C
URVE
38 students, 10 samples from 2 tests Weak training No equal error rate for stylometry
S
TYLOMETRY
C
OMBINED
D
ATA
40 students, 10 samples each from 1 test No equal error rate 30 students, 30 samples each from 3 tests 6 samples combined yielding 5 samples each ~30% equal error rate
40 60 0 FRR (%) 100 0 FRR (%) 100
100
S
TYLOMETRY
C
OMBINED
D
ATA
24 STUDENTS , 10 W EAK SAMPLES T RAINING C OMBINED Authenticating students ~32% equal error rate Authenticating test ~35% equal error rate
100 0 FRR (%) 100 0 FRR (%) 100
F
UTURE
W
ORK
Keystroke and Stylometry Biometrics
Improve stylometry authentication results by identifying important features Combined more samples to obtain stylometry features on longer text input Determine if samples may be authenticated to a test, as opposed to the individual
Data Collection
Modify the input system to eliminate some problems with giving an online test Authenticate with first/last name only Ability to traverse the questions in the test Integrate keystroke authentication with users logging into the system
Q
UESTIONS
THANK YOU!
T
EAMS
K
EYSTROKE
& S
2 & 4
B
IOMETRIC TYLOMETRY
S
YSTEMS
Tyrone Mino
Allman,
Omar
Canales
Lamrabat,
Mandar
Manohar
Vinnie
Monaco,
Thomas
Murphy
John
Stewart,
Dr. Charles
Tappert
Robert
Zack,
Edyta