Transcript A Comparison of Heuristic Evaluation, Cognitive Walkthroughs and Usability Testing Alfred Kobsa

A Comparison of
Heuristic Evaluation, Cognitive Walkthroughs
and Usability Testing
Alfred Kobsa
University of California, Irvine
Heuristic Evaluation
Normally performed by a team of 3-5 experts
•
•
•
•
Original designers are do not see their own mistakes easily
Experts go through guidelines (heuristics)
Rate individually, form consensus
Can be performed on early designs, mockups and prototypes
Pros:
• Cheaper than usability experiments
• Reasonably effective:
Rule of thumb: 1 expert ca. 40% of errors, 2: 50%, 3: 60%, 5: 80%
Cons:
• HCI experts find mostly local problems (when domain expertise is also
present, more and more global problems will be found)
Cognitive Walkthrough
Performed by original designers or experts
(individuals or groups)
• Analyst(s) imagine performing a task and walk through design
documents
• Analysts try to determine whether users will be able figure out where to
go and how to do things in the design
• Can be performed based on early designs, mockups and prototypes
Pros:
• Cheaper than usability experiments
• Reveals global errors more readily than heuristic evaluation
Cons:
• Original designers do not see their own mistakes easily
• Experts may need long to understand the task at hand
Usability testing
Performed by original designers (or experts)
• with users who are representative for the target population
• planned in teams, carried out by (parallel) subteams of 1-4 people
Pros:
• Reveals more usability problems than other methods
• Finds more global problems
• Finds more unique problems (?)
Cons:
• Time-intensive: several weeks to a few months
• Does not detect local/minor problems very well
☛ perform heuristic evaluation and/or cognitive walkthrough first