GOMS - Georgia Institute of Technology
Download
Report
Transcript GOMS - Georgia Institute of Technology
(1) Action Analysis
(2) Automated Evaluation
CS 160, Spring 2002
James Landay
April 15, 2002
4/15/2002
1
Hall of Fame or Hall of Shame?
java.sun.com
4/15/2002
2
Hall of Fame
Good branding
java logo
value prop
Inverse pyramid
writing style
Fresh content
changing first
read
news in sidebar
4/15/2002
3
Hall of Fame or Hall of Shame?
Bryce 2
for building
3D models
4/15/2002
4
Hall of Shame!
Icons all look
similar
what do they
do????
How do you exit?
Note
nice visuals, but
must be usable
What if purely for
entertainment?
4/15/2002
5
(1) Action Analysis
(2) Automated Evaluation
CS 160, Spring 2002
James Landay
April 15, 2002
4/15/2002
6
Outline
Review
Action analysis
GOMS? What’s that?
The G, O, M, & S of GOMS
How to do the analysis
Announcements
Automated evaluation tools
4/15/2002
7
Review
What is a Serif & what is it for?
the curly feet at the top & bottom of fonts
give the eye a line to read by: A Serif Font
Sans Serif means “without Serif”
What are some “signals” used w/ text?
4/15/2002
type size
type weight
word spacing
line length
leading (line spacing)
type shifts
8
Action Analysis Predicts
Performance
Cognitive model
model some aspect of human understanding,
knowledge, intentions, or processing
two types
competence
• predict behavior sequences
performance
• predict performance, but limited to routine behavior
Action analysis uses performance model to
analyze goals & tasks
generally done hierarchically (similar to TA)
4/15/2002
9
GOMS – Most Popular Action
Analysis
Family of UI modeling techniques
based on Model Human Processor
GOMS stands for (?)
Goals
Operators
Methods
Selection rules
Input: detailed description of UI/task(s)
Output: qualitative & quantitative measures
4/15/2002
10
Quick Example
Goal (the big picture)
go from hotel to the airport
Methods (or subgoals)?
walk, take bus, take taxi, rent car, take train
Operators (or specific actions)
locate bus stop; wait for bus; get on the bus;...
Selection rules (choosing among methods)?
Example: Walking is cheaper, but tiring and slow
Example: Taking a bus is complicated abroad
4/15/2002
11
Goals
Something the user wants to achieve
Examples?
go to airport
delete File
create directory
Hierarchical structure
may require many subgoals
4/15/2002
12
Methods
Sequence of steps to accomplish a goal
goal decomposition
can include other goals
Assumes method is learned & routine
Examples
drag file to trash
retrieve from long-term memory command
4/15/2002
13
Operators
Specific actions (small scale or atomic)
Lowest level of analysis
can associate with times
Examples
Locate icon for item on screen
Move cursor to item
Hold mouse button down
Locate destination icon
User reads the dialog box
4/15/2002
14
Selection Rules
If > 1 method to accomplish a goal,
Selection rules pick method to use
Examples
IF <condition> THEN accomplish <GOAL>
IF <car has automatic transmission> THEN
<select drive>
IF <car has manual transmission> THEN
<find car with automatic transmission>
4/15/2002
15
GOMS Output
Execution time
add up times from operators
assumes experts (mastered the tasks)
error free behavior
very good rank ordering
absolute accuracy ~10-20%
Procedure learning time (NGOMSL only)
accurate for relative comparison only
doesn’t include time for learning domain knowledge
4/15/2002
16
GOMS Output
Ensure frequent goals achieved quickly
Making hierarchy is often the value
functionality coverage & consistency
does UI contain needed functions?
consistency: are similar tasks performed similarly?
operator sequence
in what order are individual operations done?
4/15/2002
17
How to do GOMS Analysis
Generate task description
pick high-level user Goal
write Method for accomplishing Goal - may invoke
subgoals
write Methods for subgoals
this is recursive
stops when Operators are reached
Evaluate description of task
Apply results to UI
Iterate!
4/15/2002
18
Comparative Example - DOS
Goal: Delete a File
Method for accomplishing goal of deleting a file
retrieve from Long term memory that command verb
is “del”
think of directory name & file name and make it the
first listed parameter
accomplish goal of entering & executing command
return with goal accomplished
4/15/2002
19
Comparative Example - Mac
Goal: Delete a File
Method for accomplishing goal of deleting
a file
find file icon
accomplish goal of dragging file to trash
Return with goal accomplished
4/15/2002
20
Comparative Example - DOS
Goal: Remove a directory
Method for accomplishing goal of removing a
directory ?????
4/15/2002
21
Comparative Example - DOS
Goal: Remove a directory
Method for accomplishing goal of removing a
directory
Accomplish goal of making sure directory is empty
Retrieve from long term memory that command verb
is ‘RMDIR’
Think of directory name and make it the first listed
parameter
Accomplish goal of entering and executing a
command
Return with goal accomplished
4/15/2002
22
Comparative Example - MAC
Goal: Remove a directory
Method for accomplishing goal of removing a
directory
????
4/15/2002
23
Comparative Example - MAC
Goal: Remove a directory
Method for accomplishing goal of removing a
directory
Find folder
Accomplish goal of dragging folder to trash
Return with goal accomplished
4/15/2002
24
Applications of GOMS
Compare different UI designs
Profiling (time)
Building a help system? Why?
modeling makes user tasks & goals explicit
can suggest questions users will ask & the
answers
4/15/2002
25
What GOMS can model
Task must be goal-directed
some activities are more goal-directed
creative activities may not be as goal-directed
Task must be a routine cognitive skill
as opposed to problem solving
good for things like machine operators
Serial & parallel tasks (CPM-GOMS)
4/15/2002
26
Real-world GOMS
Applications
Keystroke Level Model (KLM)
Mouse-based text editor
Mechanical CAD system
NGOMSL
TV control system
Nuclear power plant operator’s associate
CPM-GOMS
Telephone operator workstation
4/15/2002
27
Advantages of GOMS
Gives qualitative & quantitative measures
Model explains the results
Less work than user study – no users!
Easy to modify when UI is revised
Research: tools to aid modeling process
since it can still be tedious
4/15/2002
28
Disadvantages of GOMS
Not as easy as HE, guidelines, etc.
Takes lots of time, skill, & effort
Only works for goal-directed tasks
Assumes tasks performed by experts
without error
Does not address several UI issues,
readability, memorizability of icons,
commands
4/15/2002
29
Announcements
Make sure your web sites are up to date
I scanned last night and saw lots of material missing
PowerPoint slides, all assignments, mailto link for team!
We will start grading these soon
I will schedule team meetings for Fri. & Mon. to demo
your system (turn in write-up on Fri.)
I will email a midterm survey today – please fill out so we
can adjust
PocketPC teams come to my office after lecture to pickup
H/W – THANK YOU MICROSOFT!!!!!
Questions????
4/15/2002
30
Rapid Iterative Design is the Best
Practice for Creating Good UIs
We have seen how computer-based tools can improve
the Design (e.g., Denim) & Prototyping (e.g., VB) phases
Design
Prototyping
Evaluation
4/15/2002
31
Automated GOMS Tools
Can save, modify and re-use the model
Automation of goal hierarchy, method,
selection rule creation
4/15/2002
32
QGOMS tool
4/15/2002
33
CRITIQUE
Hudson et al (1999)
1. Prototype system
in this case with the SubArctic toolkit
2. Demonstrate a procedure (task)
record events
apply rules
3. Automatically generate KLMs
4. Semi-automatically generate classic
GOMS models
4/15/2002
34
Automated Web Evaluation
Motivation
Approaches
GOMS inspired models
remote usability testing
4/15/2002
35
Factors Driving Repeat Visits
Should Drive Evaluation
High quality content
Ease of use
Quick to download
75%
66%
58%
(Source: Forrester Research, 1/99)
4/15/2002
36
Max –
WebCriteria’s GOMS Model
Predicts how long information seeking tasks
would take on a particular web site
Automated procedure:
seed with start page and goal page
procedure
reads page
model predicts how long to find & click proper link
load time, scan time, and mouse movement time
repeat until find goal page
Claim time is directly related to usability
4/15/2002
37
Sample of Max’s Reports
4/15/2002
38
Advantages of Max-style Model
Inexpensive (no users needed)
Fast (robot runs & then computes model)
Can run on many sites & compare ->
benchmarks
4/15/2002
39
Disadvantages of Max-style Model
Focus on time (much of it download time)
only 3rd in important factors driving repeat visits
can’t tell you anything about your content
doesn’t say anything directly about usability problems
Robots aren’t humans
doesn’t make mistakes
remember, GOMS assumes expert behavior!
doesn’t account for understanding text
only tries the best path – users will use many
Major flaw is the lack of real users in the process
4/15/2002
40
Warning
I am a founder of the
following company –
watch for bias!
The Trouble With Current
Site Analysis Tools
Leave
4/15/2002
Unknowns
Who?
What?
Why?
Did they find it?
Satisfied?
42
NetRaker Provides User-centric
Remote Evaluation Using Key Metrics
NetRaker Index
short pop-up survey shown to 1 in n visitors
on-going tracking & evaluation data
Market Research & Usability Templates
surveys & task testing
invitation delivered through email, links, or pop-ups
4/15/2002
43
NetRaker Index:
On-going customer intelligence gathering
Small number of rotated questions increases response rate
4/15/2002
44
NetRaker Index:
On-going customer intelligence gathering
Small number of rotated questions increases response rate
4/15/2002
45
NetRaker Index:
On-going customer intelligence gathering
Small number of rotated questions increases response rate
4/15/2002
46
NetRaker Index:
On-going customer intelligence gathering
Small number of rotated questions increases response rate
4/15/2002
47
NetRaker Index:
On-going customer intelligence gathering
Increasing these indices (e.g., retention) moderately (5%) leads
to a large increase in revenue growth
4/15/2002
48
NetRaker Usability Research:
See how customers accomplish real tasks on site
4/15/2002
49
NetRaker Usability Research:
See how customers accomplish real tasks on site
4/15/2002
50
NetRaker Usability Research:
See how customers accomplish real tasks on site
4/15/2002
51
NetRaker Usability Research:
See how customers accomplish real tasks on site
4/15/2002
52
WebQuilt: Visual Analysis
Goals
link page elements to user actions
identify behavior/nav. patterns
highlight potential problems areas
Solution
interactive graph based on web content
nodes represent web pages
edges represent aggregate traffic between pages
designers can indicate expected paths
color code common usability interests
filtering to show only target particpants
use zooming for analyzing data at varying granularity
4/15/2002
53
4/15/2002
54
4/15/2002
55
4/15/2002
56
Advantages of NetRaker
Fast
can set up research in 3-4 hours
get results in 36 hours
More accurate
can run with large samples (50-200 users -> stat. sig.)
uses real people (customers) performing tasks
natural environment (home/work/machine)
Easy-to-use
templates make setting up easy
Can compare with competitors
indexed to national norms
4/15/2002
57
Disadvantages of NetRaker
Miss observational feedback
facial expressions
verbal feedback (critical incidents)
Need to involve human participants
costs some amount of money (typically $20$50/person)
People often do not like pop-ups
need to be careful when using them
4/15/2002
58
Summary
GOMS
provides info about important UI properties
doesn’t tell you everything you want to know about a UI
only gives performance for expert behavior
hard to create model, but still easier than user testing
changing later is much less work than initial generation
Automated usability
faster than traditional techniques
can involve more participants -> convincing data
easier to do comparisons across sites
tradeoff with losing observational data
4/15/2002
59
Next Time
Advanced User Testing
Appendix A from The Design of Sites
Gomoll paper
Statistica Ch1, and parts of Ch3
Lewis & Rieman Ch. 5
4/15/2002
60