Transcript Usability Evaluation - Georgia Institute of Technology

Formative Evaluation
cs3724: HCI
ANALYZE
analysis of
stakeholders,
field studies
Problem scenarios
claims about
current
practice
Scenario-Based Design
DESIGN
metaphors,
information
technology,
HCI theory,
guidelines
Activity
scenarios
Information scenarios
iterative
analysis of
usability
claims and
re-design
Interaction scenarios
PROTOTYPE & EVALUATE
summative
evaluation
Usability specifications
formative
evaluation
Usability Engineering
Reqs Analysis
Design
Evaluate
Prototype
Usability Engineering
Reqs Analysis
Design
Evaluate
Develop
many iterations
Usability Engineering
Formative evaluation
Summative
evaluation
User Interface Metrics
• Ease of learning
• learning time, …
• Ease of use
• perf time, error rates…
• User satisfaction
• surveys…
Not “user friendly”
Claims Analysis
• + advantages
• - disadvantages
• E.g. file management:
+
+
Claims Analysis
• + advantages
• - disadvantages
• E.g. file management:
+ fast learning
+ spatial learning aid memory
- switching between devices
- long to locate a known file…
+ flexible, power
+ fast for experts
- Learning commands
- aesthetics lacking?...
Usability Evaluation
• Analytic Methods:
• Heuristic Evaluation
• Models: GOMS, Fitts Law
• Empirical Methods:
• Usability Testing
– Wizard of Oz
• Controlled Experiment
Nielsen’s 10 Heuristics
1. Visible status, feedback
2. User control, undo, exits
3. Familiar, speak user’s language
4. Consistent, standards
5. Recognition over recall
6. Efficient, expert shortcuts
7. Aesthetic, minimalist
8. Prevent errors
9. Error recovery
10. Help, task-based
Nielsen’s 10 Heuristics
1. Visible status, feedback
2. User control, undo, exits
3. Familiar, speak user’s language
4. Consistent, standards
5. Recognition over recall
6. Efficient, expert shortcuts
7. Aesthetic, minimalist
8. Prevent errors
9. Error recovery
10. Help, task-based
wysiwyg
wizards
Acrobat error msg
Word, PPT
web nav, phone menu
Word bold
phone book
HomeFinder
undo, back
IIS help doc
Speak the User’s Language
Help documentation
Context help,
help doc,
UI
Applying Heuristics
• Lets look at a website…
Early Usability Testing
• Formative: helps guide design
• Early in design process
• when architecture is finalized, then its too late!
•
•
•
•
A few users (3-5)
Identify usability problems, incidents
Qualitative feedback from users
Quantitative usability measures
Usability Test Procedure
•
•
•
•
Wizard of Oz: act out the system
Benchmark task set
No cheating! Don’t teach them how to use the UI
Repeat:
•
•
•
•
Give user a task
Ask user to “think aloud”
Observe, note mistakes and problems
Avoid interfering, hint only if completely stuck
• Interview
• Verbal feedback
• Questionnaire
• 30 min - 1 hour / user
Analyze Results
• Initial reaction:
• “stupid user!”, “that’s developer X’s fault!”, “this sucks”
• Mature reaction:
• “how can we redesign UI to solve that usability problem?”
• the user is always ‘right’
• Identify usability problems
• Learning issues: e.g. can’t figure out or didn’t notice feature
• Performance issues: e.g. arduous, tiring to solve tasks
• Subjective issues: e.g. annoying, ugly
• Problem severity: critical vs. minor
• Redesign
Pizza Ordering System
• Scenario:
•
•
•
•
Volunteers!
Design ideas
Wizard of Oz usability test
Redesign
In action: Pizza Ordering System
• Scenario: VT students, studying/busy/tired, poor,
wants discounts, starving, credit card / hokie card/
no cash, 2 in together, half-n-half, split cost
•
•
•
•
Volunteers!
Design ideas
Wizard of Oz usability test
Redesign
Pizza R Us
Drag your toppings:
Whole
Half-n-half
peperoni
mushrooms
pineapple
PRICE: $10.00
checkout
• Size of pizza:
– large
– Small
• First half:
• Toppings:
– Peperoni
– Mushrooms
– Pineapple
• Price: 0.00
• 2nd half:
• Toppings:
– Peperoni
– Mushrooms
– pineapple
• % pza –ppm –large -…