Evaluating User Interfaces Walkthrough Analysis

Joseph A. Konstan [email protected]

CSci 5115

October 10

   Introduction to Evaluation Cognitive Walkthrough Other Evaluation Methods 2

CSci 5115

Interface Development Methodology

  Prototype and Iterate  keep iterating until it is good enough  evaluate along the way to assess What is Good? What is Good Enough?

 set usability goals  should relate to tasks 3

CSci 5115

Casual Iteration

  Find major usability problems  missing features  user confusion  poor interaction Try interface with specific tasks  first use designers, then move towards users  observe overall usage 4

CSci 5115

Casual Iteration

   Remember the goal  don’t defend the interface  don’t bias the tests towards the interface If possible, allow user exploration  may even lead to capturing new tasks Consider alternative ways to fix a problem 5

CSci 5115

Limits of Casual Iteration

   Does not indicate when to stop Financial trade-offs Justification of delay 6

CSci 5115

Usability Goals and Measures

  Concrete, quantitative measures of usability  learning time  use time for specific tasks and users  error rates  measures of user satisfaction Comparative usability goals  compare with prior versions or competitors 7

CSci 5115

Things to Watch

   Goals should be realistic  100% is never realistic Many goals go beyond the application UI  training, manuals Testing goals should help improve the UI  detail--not just good/bad 8

CSci 5115

Exercise: Setting Usability Goals

 In project groups, come up with 2 usability goals for your project  discuss the feasibility of testing these goals  what is needed for the test  when in the process can they be tested?

 how much effort, user preparation/training, etc.?

 what would you learn from the test?

9

CSci 5115

Interface Evaluation

 Goals of interface evaluation  find problems  find opportunity for improvement  determine if interface is “good enough” 10

CSci 5115

With or Without Users

   Users are expensive and inconsistent  usability studies require several users  some users provide great information, others little Users are users  cannot be simulated perfectly Best choice--Both 11

CSci 5115

Evaluation Without Users

  Quantitative Methods  GOMS/keystroke analysis  back-of-the-envelope action analysis Qualitative Methods  expert evaluation  cognitive walkthrough  heuristic evaluation 12

CSci 5115

Walkthrough Analysis

  Economical interface evaluation  low-fidelity prototype  development team  users optional Effective, if  goal is improvement, not defense  some team members skilled  proper motivation 13

CSci 5115

Cognitive Walkthrough

  Goals  imagine user’s experience  evaluate choice-points in the interface  detect confusing labels or options  detect likely user navigation errors Start with a complete TCUID scenario  never try to “wing it” on a walkthrough 14

CSci 5115

Tell a Believable Story

   How does the user accomplish the task Action-by-action Based on user knowledge and system interface 15

CSci 5115

Best Approach

   Work as a group  don’t partition the task Be highly skeptical  remember the goal!

Every gap is an interface problem 16

CSci 5115

Who Should Do the Walkthrough

   Designers, as an early check Team of designers & users  remember: goal is to find problems  avoid making it a show Skilled UI people may be valuable team members 17

How Far Along

CSci 5115   Basic requirements  description or prototype of interface  know who users are (and their experience)  a task description  a list of actions to complete the task (scenario)  DO NOT try to create the action list on the fly!

Viable once the scenario and interface sketch are completed 18

How to Proceed

CSci 5115  For each action in the sequence  tell the story of why the user will do it  ask critical questions  will the user be trying to produce the effect?

 will the user see the correct control?

 will the user see that the control produces the desired effect?

 will the user select a different control instead?

 will the user understand the feedback to proceed correctly?

19

CSci 5115

Walkthroughs are not Perfect

  They won’t find every problem  limited by nature  new users who know what task they need to accomplish  biased towards correct action sequence  limited in implementation  hard to shed the expertise of evaluators A useful tool in conjunction with others 20

CSci 5115

Exercise: Cognitive Walkthrough Analysis

     In non-project groups of 3-5 Users and Task to be announced Scenario developed jointly Perform walkthrough  identify problems  estimate error probabilities (25% intervals) Remember who your users are!

21

CSci 5115

GOMS/Keystroke Analysis

   Formal action analysis  accurately predict task completion time for skilled users Break task into tiny steps  keystroke, mouse movement, refocus gaze  retrieve item from long-term memory Look up average step times  tables from large experiments 22

CSci 5115

GOMS/Keystroke Analysis

  Primary utility: repetitive tasks  e.g., telephone operators  benefit: can be very accurate (within 20%)  may identify bottlenecks Difficulties  challenging to decompose accurately  long/laborious process  not useful with non-experts 23

CSci 5115

Back-of-the-Envelope Action Analysis

  Coarse-grain  list basic actions (select menu item)  each action is at least 2-3 seconds  what must be learned/remembered?

 what can be done easily?

 documentation/training?

Goal is to find major problems  Example: 1950’s 35mm camera 24

CSci 5115

Expert Evaluation

   Usability specialists are very valuable  double-specialists are even better An inexpensive way to get a lot of feedback Be sure the expert is qualified in your area 25

CSci 5115

Looking Ahead

  Next week: Heuristic Evaluation Walkthroughs Due  “raw” notes • • • notes from each step of walkthrough copy of prototype used, markups copy of scenarios used (note changes or fixes)  processed results • 1-2 pages of issues identified, solutions not needed 26