Transcript Why Johnny Can’t Encrypt A Usability Evaluation of GPG 5.0

Why Johnny Can’t Encrypt
A Usability Evaluation of
GPG 5.0
Presented by Yin Shi
Overview
 Introduction
 Understanding
the Problem
 Cognitive Walkthrough
 User Test
 Conclusion
Introduction
Effective security requires a different usability
standard
 Security mechanisms are effective only when
used correctly

– Matt Bishop claimed that configuration errors are
the cause of more than 90% of all computer
security failures
Making security usable will require the
development of domain-specific user interface
design principles and techniques
 Choose PGP 5.0 for our case study

– Designed by general consumer software standards
– “Significantly improved graphical user interface
makes complex mathematical cryptograph
accessible for novice computer users.”
Understanding the Problem
 Defining
Usability for Security
– Definition: Security software is usable if
the people who are expected to use it:
 Are
reliably made aware of the security tasks
they need to perform
 Are able to figure out how to successfully
perform those tasks
 Don’t make dangerous errors
 Are sufficiently comfortable with the interface to
continue using it
Understanding the Problem

Problematic Properties of Security
– Five inherent properties of security
 The
 The
 The
 The
 The

unmotivated user property
abstraction property
lack of feedback property
barn door property
weakest link Property
A Usability Standard for PGP
– Need for privacy and authentication
– What needs to be done
– How to do it and avoid dangerous errors
Evaluation Methods
 Two
Methods
– An informal cognitive walkthrough
– A user test performed in a laboratory
Cognitive Walkthrough

Visual Metaphors (keys)
– PGP’s user interface relies on graphical depictions of
keys and locks
– Improvements
 An
extension of the metaphor to distinguish public
keys for encryption and private keys for decryption
 Different icons for public and private keys
Cognitive Walkthrough
 Visual
Metaphors (signatures)
– The icon of the blue quill pen is used to
indicate signing is problematic
– Quill pen icon will not help user understand
they need to use their private keys to
generate signatures
– Improvements
 Keep
quill pen to represent signing, but modify it
to show a private key as the nib of the pen
 Use some entirely different icon for signatures
Cognitive Walkthrough
 Different
Key Types
– Originally, PGP used the RSA algorithm for
encryption and signing
– PGP 5.0 uses the Diffie-Hellman/DSS
algorithm
– PGP 5.0 can handle RSA keys, but other
version PGP can’t handle DSS keys
– Lack of forward compatibility
 Recipients
with RSA keys can’t decrypt it
 Recipients with RSA keys can’t verify signatures
– PGP 5.0 alerts its users to this compatibility
issues in two ways
Cognitive Walkthrough

Different Key Types
– Uses different icons to depict the different key types
– When user attempt to encrypt documents using mixed
key types, a warning message is showed
– Improvement
 Double-clicking
on a key pops up a Key properties window
Cognitive Walkthrough



Metaphor of choosing people
Human icons obscure the key type information
Better to display multiple keys that person owns
Cognitive Walkthrough

Key Server
– Are publicly accessible databases
– PGP offers three key server operations under the Keys
pull-down menu
Cognitive Walkthrough
 Problems
with the presentation of the
Key Server
– Users may not realize that it exists
 No
representation of it in the top level of
PGPkeys display
– PGPkeys keeps no records of key server
access
– PGP’s key revocation operation does not
send the resulting revocation certificate to
the key server
Cognitive Walkthrough
 Key
Management Policy
– Two ratings for each public key
 Validity
– how sure the user is that the key is
safe to encrypt with
 Trust – how much faith the user has in the key
– May not realize PGP can automatically sets
the validity rating of a key based on
whether it has been signed by a certain
number of sufficiently trusted keys.
Cognitive Walkthrough

Irreversible Actions
–
–
–
–
–

Accidentally deleting the private key
Accidentally publicizing a key
Accidentally revoking a key
Forgetting the passphrase
Failing to back up the key rings
Consistency
– encoding

Too Much Information
– PGPkeys application presents the user with too
much information to make sense of
 Owner’s
name, validity, trust level, creation date, and
size
 Nothing to help the user figure out which parts of the
display are the most important to pay attention to
User Test
 Test
Design
 Initial task is to send the secret message
to the team members in a signed and
encrypted email
 Main steps
– Generate a key pair, get the public keys
– Make their own public key available to team
members
– Type the secret message into an emails
– Sign the email using private key, encrypt the
email using the team member’s public keys
User Test
 One
of the member had an RSA key
 Participant would encounter mixed key
types warning message
 Each of the five campaign members was
represented by a dummy email account
and a key pair:
– These were accessible to the test monitor
through a network laptop
 The
test monitor could send email to the
participant from the appropriate dummy
account
User Test (Results)
 Avoiding
dangerous errors
– Three of them accidentally emailed the secret
without encryption
– One forgot her passphrase
 Figuring
out how to encrypt with any key
– One couldn’t figure out how to encrypt at all
– A reconfiguration of PGP may required
– Another one kept sending unencrypted test
messages, and finally succeeded after being
prompted to use the PGP plug in buttons
User Test (Results)
 Figuring
with
out the correct key to encrypt
– 11 participants figured out how to encrypt,
but failed to understand the public key model
– Another one so completely misunderstood
the model that he generated key pairs for
each team member rather than for himself
 Decrypting
an email message
– Five participants received encrypted email
– One can’t figure how to decrypt it
– Two took a very hard time to figure it out
– Other two were able to decrypt without any
problem
User Test (Results)
 Publishing
the public key
– Ten could make their public key available to
the team members
– Two never addressed key distribution
– Those ten, five sent their keys to key server
– Three emailed to the team members
– Other two did both
 Getting
other people’s public keys
– Eight successfully got the team members’
public keys
– The others either never seemed aware they
need other people’s public key, or they did
know how to get it
User Test (Results)

Handing the mixed key types problem
– Only four managed to send encrypted email correctly
– One didn’t have mixed key types problem
– The other three received a reply email for
complaining that they couldn’t decrypt email
Signing an email message
 Verifying a signature on an email message
 Creating a backup revocation certificate

– Only three participants managed to successfully send
encrypted email and decrypt a reply
– In response to direct prompting for backup
 One
didn’t send the key pair to the key server
 One sent email to the campaign manager
 One simply ignored the prompt
User Test (Results)
 Deciding
whether to trust keys from the
key server
– Of the eight participants, only three
expressed some concern over if they should
trust the keys
– None of the three made use of the validity
and trust labeling provided by PGPKeys
Conclusion/Questions
PGP 5.0’s user interface does not come even
reasonably close to achieving our usability
standard
 It does not make public key encryption of
electronic mail manageable for average
computer users
 Public work on usability evaluation in a
security context would be extremely valuable
 We expect to find better design strategies
