Why Johnny Can’t Encrypt A Usability Evaluation of GPG 5.0
Download
Report
Transcript Why Johnny Can’t Encrypt A Usability Evaluation of GPG 5.0
Why Johnny Can’t Encrypt
A Usability Evaluation of
GPG 5.0
Presented by Yin Shi
Overview
Introduction
Understanding
the Problem
Cognitive Walkthrough
User Test
Conclusion
Introduction
Effective security requires a different usability
standard
Security mechanisms are effective only when
used correctly
– Matt Bishop claimed that configuration errors are
the cause of more than 90% of all computer
security failures
Making security usable will require the
development of domain-specific user interface
design principles and techniques
Choose PGP 5.0 for our case study
– Designed by general consumer software standards
– “Significantly improved graphical user interface
makes complex mathematical cryptograph
accessible for novice computer users.”
Understanding the Problem
Defining
Usability for Security
– Definition: Security software is usable if
the people who are expected to use it:
Are
reliably made aware of the security tasks
they need to perform
Are able to figure out how to successfully
perform those tasks
Don’t make dangerous errors
Are sufficiently comfortable with the interface to
continue using it
Understanding the Problem
Problematic Properties of Security
– Five inherent properties of security
The
The
The
The
The
unmotivated user property
abstraction property
lack of feedback property
barn door property
weakest link Property
A Usability Standard for PGP
– Need for privacy and authentication
– What needs to be done
– How to do it and avoid dangerous errors
Evaluation Methods
Two
Methods
– An informal cognitive walkthrough
– A user test performed in a laboratory
Cognitive Walkthrough
Visual Metaphors (keys)
– PGP’s user interface relies on graphical depictions of
keys and locks
– Improvements
An
extension of the metaphor to distinguish public
keys for encryption and private keys for decryption
Different icons for public and private keys
Cognitive Walkthrough
Visual
Metaphors (signatures)
– The icon of the blue quill pen is used to
indicate signing is problematic
– Quill pen icon will not help user understand
they need to use their private keys to
generate signatures
– Improvements
Keep
quill pen to represent signing, but modify it
to show a private key as the nib of the pen
Use some entirely different icon for signatures
Cognitive Walkthrough
Different
Key Types
– Originally, PGP used the RSA algorithm for
encryption and signing
– PGP 5.0 uses the Diffie-Hellman/DSS
algorithm
– PGP 5.0 can handle RSA keys, but other
version PGP can’t handle DSS keys
– Lack of forward compatibility
Recipients
with RSA keys can’t decrypt it
Recipients with RSA keys can’t verify signatures
– PGP 5.0 alerts its users to this compatibility
issues in two ways
Cognitive Walkthrough
Different Key Types
– Uses different icons to depict the different key types
– When user attempt to encrypt documents using mixed
key types, a warning message is showed
– Improvement
Double-clicking
on a key pops up a Key properties window
Cognitive Walkthrough
Metaphor of choosing people
Human icons obscure the key type information
Better to display multiple keys that person owns
Cognitive Walkthrough
Key Server
– Are publicly accessible databases
– PGP offers three key server operations under the Keys
pull-down menu
Cognitive Walkthrough
Problems
with the presentation of the
Key Server
– Users may not realize that it exists
No
representation of it in the top level of
PGPkeys display
– PGPkeys keeps no records of key server
access
– PGP’s key revocation operation does not
send the resulting revocation certificate to
the key server
Cognitive Walkthrough
Key
Management Policy
– Two ratings for each public key
Validity
– how sure the user is that the key is
safe to encrypt with
Trust – how much faith the user has in the key
– May not realize PGP can automatically sets
the validity rating of a key based on
whether it has been signed by a certain
number of sufficiently trusted keys.
Cognitive Walkthrough
Irreversible Actions
–
–
–
–
–
Accidentally deleting the private key
Accidentally publicizing a key
Accidentally revoking a key
Forgetting the passphrase
Failing to back up the key rings
Consistency
– encoding
Too Much Information
– PGPkeys application presents the user with too
much information to make sense of
Owner’s
name, validity, trust level, creation date, and
size
Nothing to help the user figure out which parts of the
display are the most important to pay attention to
User Test
Test
Design
Initial task is to send the secret message
to the team members in a signed and
encrypted email
Main steps
– Generate a key pair, get the public keys
– Make their own public key available to team
members
– Type the secret message into an emails
– Sign the email using private key, encrypt the
email using the team member’s public keys
User Test
One
of the member had an RSA key
Participant would encounter mixed key
types warning message
Each of the five campaign members was
represented by a dummy email account
and a key pair:
– These were accessible to the test monitor
through a network laptop
The
test monitor could send email to the
participant from the appropriate dummy
account
User Test (Results)
Avoiding
dangerous errors
– Three of them accidentally emailed the secret
without encryption
– One forgot her passphrase
Figuring
out how to encrypt with any key
– One couldn’t figure out how to encrypt at all
– A reconfiguration of PGP may required
– Another one kept sending unencrypted test
messages, and finally succeeded after being
prompted to use the PGP plug in buttons
User Test (Results)
Figuring
with
out the correct key to encrypt
– 11 participants figured out how to encrypt,
but failed to understand the public key model
– Another one so completely misunderstood
the model that he generated key pairs for
each team member rather than for himself
Decrypting
an email message
– Five participants received encrypted email
– One can’t figure how to decrypt it
– Two took a very hard time to figure it out
– Other two were able to decrypt without any
problem
User Test (Results)
Publishing
the public key
– Ten could make their public key available to
the team members
– Two never addressed key distribution
– Those ten, five sent their keys to key server
– Three emailed to the team members
– Other two did both
Getting
other people’s public keys
– Eight successfully got the team members’
public keys
– The others either never seemed aware they
need other people’s public key, or they did
know how to get it
User Test (Results)
Handing the mixed key types problem
– Only four managed to send encrypted email correctly
– One didn’t have mixed key types problem
– The other three received a reply email for
complaining that they couldn’t decrypt email
Signing an email message
Verifying a signature on an email message
Creating a backup revocation certificate
– Only three participants managed to successfully send
encrypted email and decrypt a reply
– In response to direct prompting for backup
One
didn’t send the key pair to the key server
One sent email to the campaign manager
One simply ignored the prompt
User Test (Results)
Deciding
whether to trust keys from the
key server
– Of the eight participants, only three
expressed some concern over if they should
trust the keys
– None of the three made use of the validity
and trust labeling provided by PGPKeys
Conclusion/Questions
PGP 5.0’s user interface does not come even
reasonably close to achieving our usability
standard
It does not make public key encryption of
electronic mail manageable for average
computer users
Public work on usability evaluation in a
security context would be extremely valuable
We expect to find better design strategies