Introduction to Teambased Usability Testing “As companies design more for usability and understanding, they will discover a competitive edge, for these principles save.

Download Report

Transcript Introduction to Teambased Usability Testing “As companies design more for usability and understanding, they will discover a competitive edge, for these principles save.

Introduction to Teambased Usability Testing
“As companies design more for usability and
understanding, they will discover a competitive
edge, for these principles save customers time
and money while increasing morale.”
- Donald Norman
Objectives
Usability overview
 Setting objectives
 Preparing the test plan
 Conducting the test

Usability

Usability “means that the people who
use the product can do so quickly and
easily to accomplish their own tasks.”
- Redish and Dumas,
A Practical Guide to Usability Testing
Usability

Is not functionality
-

Is not beta testing
-

Functionality is how the system works
Beta testing is unfocused and at end of process
Is not market research
-
Market research is about customer demands
When to test

Design phase
-

surveys and focus groups
low fidelity prototype testing
high fidelity prototype testing
heuristic evaluation
Development phase
-
usability evaluation
Advantages of a usability lab
Lab is helpful for tightly controlling the
variables
 Lab helps capture low-level details

-

keyboard monitoring
videotaping for later analysis
eye movement tracking
Labs allow less intrusive observations
Disadvantages of a usability lab
Labs are intimidating to users
 Labs are rarely portable so they cannot
go into the users’ environment
 Sometimes low-level information is not
as important as making high-level
design decisions
 Labs are expensive

How long will the test take?

How much usability testing has already
occurred?
-
Have users been profiled?
Has a task analysis occurred?
Have usability objectives been set?
How complex is the product?
 How much of the product are you going
to test?

How long will the test take?
How many test participants do you need
to get the information you want?
 How much training or prior experience
does the test team have with the
product?
 How many different usability objectives
do you need to test?

Who should run the test?

The most successful approach to
usability testing is a cross-functional,
team-based approach
-
-
People bring varying expertise to the team
All groups benefit from experience with
users
Interpretation of challenges and design
decisions are more creative with teams
Who should run the test?
Team-based testing provides a great
way to get support involved early
 Developers learn to “hear” user
questions when they design
 Writers and trainers can help design
communication into the product
 Human factors experts and user
interface designers can provide helpful
principles about users and design

Who should run the test?
More people can be involved in planning
the test than in conducting the test
 Target 3-5 people for conducting the test
 Sometimes best if the developers are not
actively involved in running the test,
although they should observe if possible

How do you ensure success?
You need the buy-in of upper management
if the results are to be applied
 You need the cooperation of development
to ensure they don’t feel attacked
 You need the support of marketing, sales,
and product management for funding
 You need the support of your immediate
manager to spend time testing

How do you ensure success?
Pick a manageable portion of the
product to test
 Select a high-profile project
 Select an area that will produce
significant results
 Advertise your successes (become part
of the sales story)

Stages of a usability test
Setting objectives
 Creating the test plan
 Creating questionnaires
 Selecting test participants
 Training the team
 Conducting the pilot test
 Conducting the tests

Usability objectives
Created during user/task analysis and
product scoping
 Must be measurable
 Should indicate:

-
Type of user
Task to be performed
Specific performance criteria
Defining usability objectives

Five attributes of usability (Nielsen, 1994):
-
-
-
-
-
Learnability: system is easy to learn so users
can get started quickly
Efficiency: system should be easy to use,
resulting in high productivity
Memorability: system should be easy to
remember
Errors: system should have low error rate and
allow error recovery
Satisfaction: system should be pleasant to use
Examples
Novice PC users can change the ink
cartridge in their color inkjet printer in less
than 5 minutes using the user manual
 Users will be able to install and configure
the default application in less than 15
minutes
 Users should rate the product as either
“easy to use” or “very easy to use”
(4 or higher on a 5 point scale)

Considerations
A single usability objective may result in
multiple design decisions
 Emphasizes importance of having crossfunctional team
 Focus on objective that are most important
to the success of the product

Test plan
Explains what you are testing and how
you will conduct the test
 Do not skip this step

-
-

Important to have a plan that everyone can
agree to and understand
Helps communicate your intentions to
others in the organization
Do not overdo this step
Test plan

Purpose
-

Problem Statement/Test Objectives
-

Why are you conducting the test?
What usability objectives are you testing?
User Profile
-
Who are your users and their defining
characteristics?
Test plan

Method
-

Task List
-

Details how you will run the test
List of the tasks to be completed, projected
times for completion, required state of
system to test tasks
Test Environment/Equipment
-
List what you need to conduct the test
Test plan

Test Team Roles
-

Evaluation Measures
-

Define what each person will do
What data you will collect through the test
Conclusion
-
What will happen after the test - reports,
design meetings, presentations
Pre-test questionnaire
Ensure that participant meets user profile
 Gather additional detail about the
participants’ knowledge and experience

Post-task questionnaire
Asks questions about a task as soon as
it is completed
 Captures feedback when it is fresh
 Can help measure change in attitude
toward the task
 Keep it short

Post-test questionnaire
Measure satisfaction and reaction to the
product and the overall experience
 Ask participants to elaborate on a few
critical areas/issues

Training the team
Make sure everyone understands the
usability objectives and the testing
procedure
 Define everyone’s roles and discuss how
they should behave during the test

Conducting a pilot test
Conduct a practice test using the complete
team, all prepared materials, and the actual
facilities
 Make sure at least one person observes the
test to make recommendations
 Refine any of the materials or facilities as
necessary

Selecting test participants
Number of participants depends on the
number of user groups (differentiating
characteristics)
 Target 3-5 users from each user group
 Best if users come from the real user
population rather than internally

Observers
Not actively involved in conducting the
test, but can benefit from observing
 Must not comment or react to the
participant’s actions or comments
 Should be separated from the actual
room where the testing is going on
 Should be removed if they cannot avoid
feeling defensive or being intrusive

Data logger
Be familiar with any data logging forms
 Establish standard codes ahead of time
for recording events and comments
 Remain open to unforeseen directions
 Record user comments that summarize
or emphasize good and bad points
 Keep track of time

Facilitator
Main contact with the participants
 Conducts the briefing and debriefing
 Only person to talk to the participants
during the test

Briefing
Thank them for participating in the
product evaluation
 Assure users that they are not being
tested; instead, they are helping you
evaluate the product
 Explain to them what will happen during
the test; they will be especially
interested in any recording devices and
observers

Briefing
Ask them to sign a consent form and a
non-disclosure agreement
 Encourage them to think out loud so
you can record their actions and the
reasons behind their actions
 Make sure you explain what they should
do if they have questions or experience
problems

During the test
Be sensitive to excessive frustration
 Be ready to handle unplanned situations
 Remind participants to talk out loud
 Look and listen for the unexpected
 Avoid intervening unless necessary; try
using questions to redirect rather than
take over as the expert

Debriefing
Get users to elaborate on significant
events or comments
 Ask for reasons to specific behaviors
from the test
 Watch the video tape with them and ask
them to explain what was going on in
their mind

Summary

Build a cross-functional team to:
-

take advantage of different skills
get “buy in” from different areas
build awareness of usability testing benefits
When you test:
-
Set measurable objectives
Test with the right people
Be prepared for the test
Share your results
Questions?
For more information about this presentation
and other training seminars, visit
www.userfirst.net. This presentation is at
www.userfirst.net/teamusability.ppt.
Scott DeLoach
Founding Partner, User First Services, Inc.
Certified RoboHELP Instructor and Consultant
404.520.0003
[email protected]