Transcript Design Processes Lecture 1
Processes of Design Fifth lecture:
Evaluation
24 October 2003 William Newman
Evaluation: What do we Mean?
In the dictionary:
Determining or estimating the value of…
• • • In interactive system design: Identifying faults Testing perfor mance QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.
Rating quality,eg from 1 to 10
Evaluation in the Inner Loop • • • Assessing design options The ATM design session D. Well, one of the things you could do is, presumably you could just whisk your card through, rather than have to put it in the slot and get it back, and that might be quicker than waiting for it to come out. And I don’t see why they don’t do that always, anyway. I suppose it’s so they can chew it up if they want to.
J. Yes.
Evaluating by simulating.
Alternatives to simulation • Evaluating with theory-based laws e.g. Fitts’ Law:
T pos
=
K
log 2 (
A
/
W
+ 1) • • Evaluating with an informal rule or heuristic The quest for accuracy and completeness.
Two ways to evaluate • • • • Empirical: conduct an experiment Analytical: evaluate on paper Why two ways?
Cost of simulation.
What do we simulate?
• • • • The technology The user The context The task • Why are these familiar to us?
Evaluating in terms of requirements • • • • The problem statement: Design
a camera-based text-capture system to enable students in libraries to copy text from paper to word processor in less time than by typing
.
A concise statement of requirements As we expand requirements, we refine our evaluations
Simulating the technology • • • • How to avoid building the complete and final system?
Working and non-working simulations What’s missing from non-working sim’s?
Wizard of Oz methods.
Xerox Star • • • • Designed in the late 1970s Technology performance problems Escalating costs Never fully evaluated
Simulating the User • • • You’re not them!
Knowing the user Xerox Star again
Simulating the Context • Taking account of: • • • Other systems (Airport control, Aegis) Other people (GPs) Physical environment
Simulating the Task • • • • Sets up the
process
of evaluation Easy? We
design
the task Not so easy? The user can redesign it (the computer game) and may never learn it.
The Limits of Evaluation