Transcript Why do field studies? - Institute for Public Policy Research

Louise Ferguson
UPA Voting and Usability Project
Director, Digital Habitats Ltd
[email protected]
E-voting: Starting from People
4 November 2004
© 2004 Digital Habitats
Paper ballots
• Much improved design in recent years
• Still suffer from design problems
© 2004 Digital Habitats
London Mayoral election 2004
“There were no instructions on the paper at
all. It simply said ‘what is your first and
second choice for mayor?’. I said my first
choice was ‘x’ and my second was the same
guy, so I voted for him twice. But…you’re not
allowed to vote for the same person twice.”
[Channel 4 news presenter Jon Snow]
© 2004 Digital Habitats
© 2004 Digital Habitats
Abbreviations – polling card
Your Polling Station is:
LORDSHIP LANE T R A HALL
ENTR. REAR OF BEW COURT
[From Southwark polling card, 2004]
© 2004 Digital Habitats
Some design issues
paper size and format - print quality - text layout - use
of white space - fonts - font sizes - use of bold,
capitals, underline - use of graphics, icons,
pictograms - use of colour (paper, text, graphics),
colour coding and ‘colour wording’ - contrast information (parties, candidates): wording,
appearance, position - explanation: wording,
appearance, position - instructions: wording,
appearance, position - syntax - active/passive positives/negatives - punctuation - text justification valid mark(s) and relative position - multiple ballots
per paper - abbreviations and acronyms consistency: information, formatting, terminology
© 2004 Digital Habitats
E-voting
• Opens up new possibilities
Creates more possibilities for tripping up - greater
complexity
• Requirements of e-voting are unique…
…but the design issues are not
• Easy to design badly, difficult to design well
© 2004 Digital Habitats
Even more design issues – e-voting
height and pitch of screen - controls, input devices: type, appearance,
labels, location, grouping, ruggedness, force required - screen format
and size - luminance - use of flashing/flash rates - display adjustability system navigation - logon procedure - security procedure - location of
information in navigation - location of explanation in navigation location of instructions in navigation - group and candidate grouping
and demarcation - ‘below the fold’ and scalability; scrolling, paging alphabetical order/randomisation - selection mechanism - relationship
to candidate information - button size, colour, position, relative position
- button text - ‘live’ areas on touch screen; mapping to input - feedback
(visual, auditory), response time - ballot review options: location,
language, options - procedure for reporting undervoting to voter procedure for amending, restarting; reversing, clearing votes - skipping
ballots - casting a blank ballot - system checking on inputs, alerts accessing help - content and format of help – website addresses
© 2004 Digital Habitats
For voters with
•
•
•
•
•
•
•
•
•
•
•
No vision, limited vision
Colour-blindness
Reading problems
Learning difficulties
Limited English
No hearing, limited hearing
Limited movement, shaking
Poor co-ordination
Limited strength
Limited physical access
…vision deteriorates from around 40 years old
© 2004 Digital Habitats
And even more design issues for accessibility
location - selecting accessibility options - system
timing, pacing - clarity of language - audio content audio options - audio quality, volume - screen
responsiveness - force required for input device troublesome colours/colour combinations - colour
coding - size of buttons - button colours - feedback
redundancy – coercion
© 2004 Digital Habitats
Can my grandmother use it?
© 2004 Digital Habitats
…and what the law says
“One area that was prescribed in law was that the
pilots had to have a replication of the ballot paper on
the voting screen. In practice, this did not work very
effectively as it was based on a paper design
principle and it did not fit with the general design
concepts used on the web.”
[Electoral Commission. Ballot paper design: Report
and recommendations. June 2003]
© 2004 Digital Habitats
• Electronic voting districts <33%
• Electronic voting reported problems >55%
[MSNBC, 3 Nov 2004]
© 2004 Digital Habitats
SCOPE research report Polls Apart, 2003
“We do not believe any kiosks represent an
improvement to the traditional ‘pen and paper’
method.”
© 2004 Digital Habitats
© 2004 Digital Habitats
E-voting in the wild…
• Is e-voting a private process?
• Who controls the home computer? – from household registration to
household computer
• Workplace monitoring
• E-voting period – implications for campaigning
• Interaction between campaign websites/pop-ups etc and official
voting sites
• Independent monitoring/observation?
• Where is the research?
© 2004 Digital Habitats
Usability
"The extent to which a product can be used by
specified users to achieve specified goals with
effectiveness, efficiency and satisfaction in a specified
context of use“
[ISO standard on usability: ISO 9241,1998]
Usability is key to free and fair elections.
© 2004 Digital Habitats
SCOPE research report Polls Apart, 2003
“…across all the channels in all the pilots [it
was felt that] the usability could be improved.
It sounds obvious but making something as
simple as possible will make it more accessible
to more people.”
© 2004 Digital Habitats
Dimensions of usability - the 5 Es
•
•
•
•
•
Effective
Efficient
Engaging
Error Tolerant
Easy to Learn
[Quesenbery, 2001]
© 2004 Digital Habitats
Designing for
usability means
designing for and
with users
Thanks to
Gary Larson
© 2004 Digital Habitats
Designing for usability
• Early focus on users and tasks
– User research BEFORE system design
– Typical users, not ‘experts’ or developers
• Empirical measurement
– Establish testable behavioural specifications
– Study user behaviour through user testing
– This is not market research
• Iterative design
– Modify design and repeat as necessary
© 2004 Digital Habitats
Usability testing
• People invariably say one thing and do another
• Observing what people actually do, one-to-one with
the realistic materials/equipment
• With real citizens
– Not experts or developers
• Not the same as focus groups, or system/functional
testing
© 2004 Digital Habitats
Voting ecosystem
• Electoral officials and poll workers
• Candidates and their representatives
• Voters
•
•
•
•
•
Voter registration
Election information
Candidate information
Polling day
Counting; getting the results
© 2004 Digital Habitats
Accessibility now on the agenda
• Campaigns from disability rights organisations
– SCOPE, RNIB
• Disability Rights Commission
• Legislation
– e.g. Disability Discrimination Act 1995
© 2004 Digital Habitats
Usability and accessibility
• Plentiful research on interaction design, cognitive
psychology, human-computer interaction
• No UK voting usability standards
• Systems not developed in compliance with standards
• Lack of usability expertise in the voting design
process
© 2004 Digital Habitats
International problem
“In general, voting systems have not been measured
for usability nor have they been developed using a
user-centered design process.
“We do not know the degree to which voters cast
their vote NOT as they intended due to confusion
with the user interface.”
[Sharon Laskowski, Head of Usability,
US National Institute for Standards and Technology]
© 2004 Digital Habitats
Don’t rely on system vendors
“Voting system vendors have limited awareness of
the field of usability and have only limited awareness
of accessibility.”
[Bill Killam, 2004, after having reviewed some 30
current e-voting systems and interviewed vendors]
© 2004 Digital Habitats
US standards
• IEEE
– Standard P-1583 Section 5.3: Usability and Accessibility Standards
– Updated version in progress
• Elections Assistance Commission
– Technical Guidelines Development Committee
– Started work 2004
– Due to report formally spring 2005
• Improving the Usability and Accessibility of Voting
Systems and Products.
National Institute for Standards and Technology report to US
Congress, April 2004
© 2004 Digital Habitats
IEEE Standard (2003 version)
Examples:
• “The use of colour for coding shall be redundant with
another coding method.”
• “Do not use abbreviations or acronyms.”
© 2004 Digital Habitats
Recommendations
• Learn form the fields of Human-Computer Interaction
and Design Research
• Learn from what’s happening elsewhere
– Don’t try and reinvent the wheel
– Consider NIST’s 10 Recommendations to US Congress
• Introduce user-centred design and usability testing to
the e-voting process
• Develop performance-based usability standards and
conformance tests for voting systems
© 2004 Digital Habitats
Resources and contacts
• Blog: http://www.louiseferguson.com/cityofbits.htm
• Website: http://www.louiseferguson.com
Voting resources:
http://www.louiseferguson.com/resources/evoting.htm
• Email: [email protected]
© 2004 Digital Habitats