Transcript No Slide Title

Testing your design
• Without users:
– Cognitive walkthrough
– Action analysis
– Heuristic analysis
• With users:
– Usability testing
Messages
• Cognitive walkthrough enables a designer
to evaluate an interface without users
– Forces a designer to see the interface from
the perspective of a user
• Low-investment technique to identify taskrelated usability issues early in the design
process
– No implementation or users required
– But can be performed on existing interfaces
Analyze
users
Analysis
Site visits
Contextual
inquiry
User studies,
function tests
Evaluate
Ship
Process
Define
goals
Analyze
Tasks
Develop
Implement
Early
design
Heuristic evaluations
cognitive walkthroughs
Design
on paper
user studies
Evaluate
Field studies, call center,
interaction logs
Evaluate
Evaluate
Late
design
Prototype
Cognitive Walkthrough
• Question assumptions about what the users will be
thinking
• Identify controls that are obvious to the design
engineer but may be hidden from the user's point
of view
• Note inadequate feedback
• Uncover shortcomings in the current specification
• Question assumption about users tasks
• Focus most clearly on a problems that users will
have when they first use an interface
• A tool for developing the interface, not for
validating it
Reference
Cognitive Theory
1. The user sets a goal to be accomplished with the
system (for example, "check spelling of this
document").
2. The user searches the interface for currently
available actions (menu items, buttons, commandline inputs, etc.).
3. The user selects the action that seems likely to
make progress toward the goal.
4. The user performs the selected action and
evaluates the system's feedback for evidence
that progress is being made toward the current
goal.
Cognitive Walkthrough
• Identify “big” problems before
implementation
– Invest a little now, save a lot later
• Enables more rapid iteration earlier in
design
– Can do several evaluations of trouble points
• Evaluations are only effective if your team
– Has the right skill set
– Wants to improve the design, not defend it
Walkthrough Basics
• Imagine how well a user could perform
tasks with your low-fidelity prototype
• Manipulate prototype as you go
– Evaluate choice-points in the interface
– Evaluate labels or options
– Evaluate likely user navigation errors
• Revise prototype and perform again
When to do the Walkthrough
• Have a low-fidelity prototype of the
interface
• Know who the users are
• Have task descriptions
• Have scenarios designed to complete the
task
– You have a “functional” paper prototype
• Viable once the scenario and paper
prototype are completed
Who should do a walkthrough, and
when?
• you can do your own, informal, “in your
head” walkthroughs to monitor the design
as you work
• with a group of people, including other
designers and users
You need 4 things.
1. User personas
– who the users will be and what kind of goals and
experience they'll bring to the job
2. A description or a low-fidelity prototype of the
user interface.
3. Task descriptions
4. A complete, written list of the actions needed to
complete the task with the interface
• And an evaluation team:
– Design team
– Design team and users together
– Design team and other skilled designers
What to do? Tell a story
• about why the user would select each
action, and critique the story to make sure
it's believable with these questions:
1. Will users be trying to produce whatever
effect the action has?
2. Will users see the control (button, menu,
switch, etc.) for the action?
3. Once users find the control, will they
recognize that it produces the effect they
want?
4. After the action is taken, will users
understand the feedback they get, so they can
go on to the next action with confidence?
More on Questions
• Some extra questions can help
– What happens if the user is wrong? Is there
feedback to correct the error?
– How would a user of <other interface> react
here?
• Questions help you see problems
– They are a focus, not a blindfold
Best Approach
• Work as a group
– don’t partition the task
• Be highly skeptical of the design
– remember the goal!
• Each gap between what and how is a
usability problem
Past Empirical Results
• Users will try label-guided actions first (menu items,
buttons, etc.) before they experiment with direct
manipulations of unlabeled objects (tools, double clicking,
moving of objects).
• A well-labeled action will be especially salient.
• Providing few actions in the search set can help to narrow
the search if labeling cannot be provided, or if criteria for a
"good" label are difficult to establish.
• Set effects may prevent users to try untypical actions.
• Users are reluctant to extend their search beyond the
readily available menus and controls.
• Frequently used interfaces techniques may bias users to
search for them rather than for less frequent techniques.
Walkthrough Pros
• Easy to learn
• Can perform early in the design process
• Questions assumptions about what a user
may be thinking
• Helps identify controls obvious to the
designer but not a user
• Helps identify difficulties with labels and
prompts
• Helps identify inadequate feedback
Walkthrough Cons
• Is diagnostic, not prescriptive
• Focuses mostly on novice users
• Designers must put themselves in the users
mind
• Focus specifically on task-related issues
• The interactions are slower and not real
• Does not provide quantitative results
• A useful tool in conjunction with others
Walkthrough Example
• I have a library book that needs to be
returned today, but I am likely to forget.
To help me remember, I want to set a
reminder on my laptop. The reminder
should display and beep at 5:00pm to
remind me to return the book to the
library.
• Let’s walkthrough this task on my Outlook
calendar and identify usability issues, if
any
Walkthrough Example
• Will a user try to produce the effect that the
action has?
• Will a user see the control for the action?
• Will a user see that the control produces the
desired effect?
• Will a user select a different control instead?
• Will a user understand the feedback to proceed
correctly?
[email protected]
Your email address is invalid
What do you do with the results of
the walkthrough?
• Fix things.
– make the controls more obvious
– use labels that users will recognize
– eliminate actions users don't think have to be
performed or prompt for them.
Grading for Cognitive Walkthrough
•
•
User personas – 30 pts
•
Did they observe or analyze actual users doing related activities
Grade _____ (15 pts max)
•
•
Are the personas complete with relevant (and observed) personal details and goals
Grade _____ (15 pts max)
•
•
Interface Description – 30 pts
•
•
Is there a description of a conceptual model?
Grade _____ (15 pts max)
•
•
A description or a low-fidelity prototype of the user interface. – 15 pts
Is the description detailed enough to evaluate cues, affordances, and feedback at each decision
point
Grade _____ (15 pts max)
•
•
Task descriptions – 40 pts
•
Are they concrete, detailed examples of all (or most) of the activities the system should
support?
Grade _____ (20pts max)
•
•
•
•
•
A complete, written list of the actions needed to complete the task with the interface – 25 pts.
Do you have a step by step story of how each persona would accomplish each task with the
proposed interface
Grade _____ (20pts max)
Formal action analysis
• GOMS approach (Goals Operators Methods
Selection rules)
–
users apply specific selection rules for choosing between
available methods in the pursuit of goals. Methods consist of
operators.
– The method breaks tasks in to smaller and smaller sub-goals
until they can be accomplished with methods and operators of
known durations. “keystroke-level analysis.”
– Can be used to predict time it takes a skilled user to complete
tasks.
• Time consuming and difficult to do.
• The analysis depends on the analysts.
• Focuses on detail and expert performance.
Back-of-the-Envelope Action
Analysis
• List the actions and think about them.
• List actions including mental actions
• This will highlight long and difficult
operations.
Heuristic Analysis
• Have several evaluators use the nine
heuristics to identify problems with the
interface, analyzing either a prototype or a
paper description of the design.
• Each evaluator should do the analysis alone.
• Then combine the problems identified by
the individual evaluators into a single list.
Heuristic Analysis Reference - Jakob Nielsen
Nielsen’s Ten Heuristics
Visibility of system status
The system should always keep users informed about what is going on,
through appropriate feedback within reasonable time.
Match between system and the real world
The system should speak the users' language, with words, phrases and
concepts familiar to the user, rather than system-oriented terms.
Follow real-world conventions, making information appear in a natural
and logical order.
User control and freedom
Users often choose system functions by mistake and will need a clearly
marked "emergency exit" to leave the unwanted state without having to
go through an extended dialogue. Support undo and redo.
Consistency and standards
Users should not have to wonder whether different words, situations, or
actions mean the same thing. Follow platform conventions.
Error prevention
Even better than good error messages is a careful design which prevents a
problem from occurring in the first place.
Nielsen’s Ten Heuristics
Recognition rather than recall
Make objects, actions, and options visible. The user should not have to
remember information from one part of the dialogue to another.
Instructions for use of the system should be visible or easily
retrievable whenever appropriate.
Flexibility and efficiency of use
Accelerators -- unseen by the novice user -- may often speed up the
interaction for the expert user such that the system can cater to both
inexperienced and experienced users. Allow users to tailor frequent
actions.
Aesthetic and minimalist design
Dialogues should not contain information which is irrelevant or rarely
needed. Every extra unit of information in a dialogue competes with the
relevant units of information and diminishes their relative visibility.
Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), precisely
indicate the problem, and constructively suggest a solution.
Help and documentation
Even though it is better if the system can be used without documentation,
it may be necessary to provide help and documentation. Any such
information should be easy to search, focused on the user's task, list
concrete steps to be carried out, and not be too large.
Evaluation Results
The Need for Multiple Evaluators
Each row represents one of the 19 evaluators and each column represents one of the 16 usability
problems. Each square shows whether the evaluator represented by the row found the usability
problem represented by the column: The square is black if this is the case and white if the
evaluator did not find the problem. The rows have been sorted in such a way that the most
successful evaluators are at the bottom and the least successful are at the top. The columns
have been sorted in such a way that the usability problems that are the easiest to find are to
the right and the usability problems that are the most difficult to find are to the left.