Transcript Interaction Devices Human Computer Interaction CIS 6930/4930 Interaction Performance ► 60s vs. Today  Performance ► Hz -> GHz  Memory ►k -> GB  Storage ►k -> TB  Input ► punch cards -> ► Keyboards,

Interaction Devices
Human Computer Interaction
CIS 6930/4930
Interaction Performance
►
60s vs. Today
 Performance
► Hz
-> GHz
 Memory
►k
-> GB
 Storage
►k
-> TB
 Input
► punch
cards ->
► Keyboards, Pens, tablets, mobile
phones, mice, cameras, web cams
 Output
► 10
character/sec ->
► Megapixel displays, HD capture and
display, color laser, surround sound,
force feedback, VR
►
Substantial bandwidth increase!
Interaction Performance
►
Future?
Gestural input
Two-handed input
3D/6D I/O
Others: voice, wearable, whole
body, eye trackers, data gloves,
haptics, force feedback
 Engineering research!
 Entire companies created
around one single technology




►
Current trend:
 Multimodal (using car
navigation via buttons or voice)
 Helps disabled (esp. those w/
different levels of disability)
Keyboard and Keypads
►
QWERTY keyboards been
around for a long time




(1870s – Christopher Sholes)
Cons: Not easy to learn
Pros: Familiarity
Stats:
► Beginners:
1 keystroke per
sec
► Average office worker: 5
keystrokes (50 wpm)
► Experts: 15 keystrokes per
sec (150 wpm)
►
Is it possible to do better?
Keyboard and Keypads
►
►
Look at the piano for possible
inspiration
Court reporter keyboards (one
keypress = multiple letters or a
word)
 300 wpm, requires extensive
training and use
►
How important is:
 Accuracy
 Training
►
Keyboard properties that matter
 Size
 Adjustability
►
Reduces RSI, better
performance and comfort
 Mobile phone keyboards,
blackberry devices, etc.
►
QWERTY
Keyboard Layouts
 Frequently used pairs far apart
 Fewer typewriter jams
 Electronic approaches don’t jam.. why
use it?
►
DVOARK (1920s)




►
150 wpm->200 wpm
Reducing errors
Takes about one week to switch
Stops most from trying
ABCDE – style
 Easier for non-typists
 Studies show no improvement vs.
QWERTY
►
Number pads
 What’s in the top row?
 Look at phones (slight faster), then look
at calculators, keypads
►
Those for disabled




Split keyboards
KeyBowl’s orbiTouch
Eyetrackers, mice
Dasher - 2d motion with word prediction
Keys
►
Current keyboards have
been extensively tested




►
►
Size
Shape
Required force
Spacing
Speed vs. error rates for
majority of users
Distinctive click gives audio
feedback
 Why membrane keyboards
are slow (Atari 400?)
► Environment
hazards might
necessitate
► Usually speed is not a factor
Keys Guidelines
►
►
►
►
►
Special keys should be denoted
State keys (such as caps, etc.)
should have easily noted states
Special curves or dots for home
keys for touch typists
Inverted T Cursor movement
keys are important (though
cross is easier for novices)
Auto-repeat feature
 Improves performance
 But only if repeat is
customizable (motor impaired,
young, old)
►
Two thinking points:
 Why are home keys fastest to
type?
 Why are certain keys larger?
(Enter, Shift, Space bar)
►
This is called Fitt’s Law
Keypads for small devices
►
►
►
►
►
►
PDAs, Cellphones, Game consoles
Fold out keyboards
Virtual keyboard
Cloth keyboards (ElekSen)
Haptic feedback?
Mobile phones
 Combine static keys with dynamic soft
keys
 Multi-tap a key to get to a character
 Study: Predictive techniques greatly
improve performance
 Ex. LetterWise = 20 wpm vs 15 wpm
multitap
►
Draw keyboard on screen and tap w/ pen
 Speed: 20 to 30 wpm (Sears ’93)
►
Handwriting recognition (still hard)
 Subset: Graffiti2 (uses unistrokes)
Pointing Devices
Direct manipulation needs some pointing device
► Factors:
►
 Size of device
 Accuracy
 Dimensionality
►
Interaction Tasks:
 Select – menu selection, from a list
 Position – 1D, 2D, 3D (ex. paint)
 Orientation – Control orientation or provide direct
3D orientation input
 Path – Multiple poses are recorded
►
ex. to draw a line
 Quantify – control widgets that affect variables
 Text – move text
►
►
Faster w/ less error than keyboard
Two types (Box 9.1)
 Direct control – device is on the screen surface
(touchscreen, stylus)
 Indirect control – mouse, trackball, joystick,
touchpad
Direct-control pointing
►
First device – lightpen
 Point to a place on screen and press a
button
 Pros:
►
►
Easy to understand and use
Very fast for some operations (e.g.
drawing)
 Cons:
►
►
►
►
Hand gets tired fast!
Hand and pen blocks view of screen
Fragile
Evolved into the touchscreen
 Pros: Very robust, no moving parts
 Cons: Depending on app, accuracy
could be an issue
►
1600x1600 res with acoustic wave
 Must be careful about software design
for selection (land-on strategy).
►
If you don’t show a cursor of where you
are selecting, users get confused
 User confidence is improved with a
good lift-off strategy
Direct-control pointing
► Primarily
for novice
users or large user
base
► Case study: Disney
World
► Need to consider those
who are: disabled,
illiterate, hard of
hearing, errors in
usage (two touch
points), etc.
Indirect-Control Pointing
►
Pros:
 Reduces hand-fatigue
 Reduces obscuration problems
►
Cons:
 Increases cognitive load
 Spatial ability comes more into play
►
Mouse
 Pros:
►
►
►
►
►
Familiarity
Wide availability
Low cost
Easy to use
Accurate
 Cons:
►
►
►
►
Time to grab mouse
Desk space
Encumbrance (wire), dirt
Long motions aren’t easy or obvious (pick up and replace)
 Consider, weight, size, style, # of buttons, force feedback
Indirect-Control Pointing
► Trackball
 Pros:
► Small
physical footprint
► Good for kiosks
►
Joystick
 Easy to use, lots of buttons
 Good for tracking (guide or
follow an on screen object)
 Does it map well to your
app?
►
Touchpoint
 Pressure-sensitive ‘nubbin’ on
laptops
 Keep fingers on the home
position
Indirect-Control Pointing
► Touchpad
 Laptop mouse device
 Lack of moving parts,
and low profile
 Accuracy, esp. those w/
motor disabilities
► Graphics




Tablet
Screen shot
comfort
good for cad, artists
Limited data entry
Comparing pointing devices
►
Direct pointing
 Study: Faster but less accurate than indirect (Haller ’84)
►
►
►
►
Lots of studies confirm mouse is best for most tasks for
speed and accuracy
Trackpoint < Trackballs & Touchpads < Mouse
Short distances – cursor keys are better
Disabled prefer joysticks and trackballs
 If force application is a problem, then touch sensitive is preferred
 Vision impaired have problems with most pointing devices
► Use multimodal approach or customizable
► Read Vanderheiden ’04 for a case study
►
►
cursors
Designers should smooth out trajectories
Large targets reduce time and frustration
Example
► Five
fastest places to click on for a righthanded user?
Example
► What
affects time?
Fitts’s Law
Paul Fitts (1954) developed a model of human hand
movement
► Used to predict time to point at an object
► What are the factors to determine the time to point to
an object?
►


►
Just from your own experience, is this function linear?


►
D – distance to target
W – size of target
No, since if Target A is D distance and Target B is 2D
distance, it doesn’t take twice as long
What about target size? Not linear there either
T = a + b log2(D/W + 1)
 T = mean time
 a = time to start/stop in seconds (empirically
measured per device)
 b = inherent speed of the device (empirically
measured per device)
 Ex. a = 300 ms, b = 200 ms/bit, D = 14 cm, W =
2 cm
►
Ans: 300 + 200 log2(14/2 + 1) = 900 ms
 Really a slope-intercept model
Fitts’s Law
►
T = a + b log2(D/W + 1)
 T = mean time
 a = time to start/stop in seconds (empirically measured per device)
 b = inherent speed of the device (empirically measured per device)
[time/bit or ms/bit]
 Ex. a = 300 ms, b = 200 ms/bit, D = 14 cm, W = 2 cm
► Ans:
300 + 200 log2(14/2 + 1) = 900 ms
 Question: If I wanted to half the pointing time (on average), how much do
I change the size?
►
►
Proven to provide good timings for most age groups
Newer versions taken into account





Direction (we are faster horizontally than vertically)
Device weight
Target shape
Arm position (resting or midair)
2D and 3D (Zhai ’96)
Examples
►T
= a + b log2(D/W + 1)
a=300, b= 200, X, W = 10
800
700
500
400
300
200
100
Distance (cm)
29
27
25
23
21
19
17
15
13
11
9
7
5
3
0
1
Time (ms)
600
Examples
►T
= a + b log2(D/W + 1)
a=300, b= 200, D=30, X
1400
1200
800
600
400
200
Distance (cm)
29
27
25
23
21
19
17
15
13
11
9
7
5
3
0
1
Time (ms)
1000
Examples
BLUE a=300, b=200, D=15, W=[1-30]
PURPLE a=300, b=200, D=[1-30], W=15
1200
800
600
400
200
Variable
29
27
25
23
21
19
17
15
13
11
9
7
5
3
0
1
Time (ms)
1000
Fitts’s Law
►T
= a + b log2(D/W + 1)
 T = mean time
 a = time to start/stop in seconds
(empirically measured per device)
 b = inherent speed of the device
(empirically measured per device)
[time/bit or ms/bit]
 First part is device characteristics
 Second part is target difficulty
Very Successfully Studied
►
Applies to
Feet, eye gaze, head mounted sights
Many types of input devices
Physical environments (underwater!)
User populations (even retarded and
drugged)
 Drag & Drop and Point & Click




►
Limitations
Dimensionality
Software accelerated pointer motion
Training
Trajectory Tasks (Accot-Zhai Steering
Law is a good predictor and joins Fitt’s
Law)
 Decision Making (Hick’s Law)




Very Successfully Studied
►
Results (what does it say about)





►
Buttons and widget size?
Edges?
Popup vs. pull-down menus
Pie vs. Linear menus
iPhone/web pages (real borders) vs.
monitor+mouse (virtual borders)
Interesting readings:
 http://particletree.com/features/visual
izing-fittss-law/
 http://www.asktog.com/columns/022
DesignedToGiveFitts.html
 http://www.yorku.ca/mack/GI92.html
Precision Pointing Movement Time
►
Study: Sears and Shneiderman ’91
 Broke down task into gross and fine components for small targets
 Precision Point Mean Time = a + b log2(D/W+1) + c log2(d/W)
►c
– speed for short distance movement
► d – minor distance
 Notice how the overall time changes with a smaller target.
►
Other factors
 Age (Pg. 369)
►
Research: How can we design devices that produce smaller
constants for the predictive equation
 Two handed
 Zooming
Affordance
► Quality
of an object, or an environment,
that allows an individual to perform an
action.
► Gibson (’77) – perceived action
possibilities
► Norman – The Design of Everyday Things
Affordance Examples
Affordance Examples
http://jared-donovan.com/teaching/blog/hci
Affordances Matter?
► When
would affordances matter?
 Languages
 Emergencies
http://jared-donovan.com/teaching/blog/hci
Novel Devices
►
Themes:
 Make device more diverse
Users
► Task
►
 Improve match between task
and device
 Improve affordance
 Refine input
 Feedback strategies
►
Foot controls
 Already used in music where
hands might be busy
 Cars
 Foot mouse was twice as slow
as hand mouse
 Could specify ‘modes’
Novel Devices
►
Eye-tracking
 Accuracy 1-2 degrees
 selections are by constant
stare for 200-600 ms
 How do you distinguish w/ a
selection and a gaze?
 Combine w/ manual input
►
Multiple degree of freedom
devices
 Logitech Spaceball and
SpaceMouse
 Ascension Bird
 Polhemus Liberty and
IsoTrack
Novel Devices
►
Boom Chameleon
 Pros: Natural, good spatial
understanding
 Cons: limited applications,
hard to interact (very
passive)
►
DataGlove
 Pinch glove
 Gesture recognition
 American Sign Language,
musical director
 Pros: Natural
 Cons: Size, hygiene,
accuracy, durability
Novel Devices
►
Haptic Feedback
 Why is resistance useful?
 SensAble Technology’s
Phantom
 Cons: limited applications
 Sound and vibration are
easier and can be a good
approximation
► Rumble
►
pack
Two-Handed input
 Different hands have
different precision
 Non-dominant hand selects
fill, the other selects objects
Ubiquitous Computing and
Tangible User Interfaces
► Active
Badges allows
you to move about the
house w/ your profile
► Which sensors could
you use?
► Elderly, disabled
► Research: Smart
House
► Myron Kruger – novel
user participation in art
(Lots of exhibit art at
siggraph)
http://
www.linuxjournal.com/files/linuxjournal.com/linuxjournal/articles/030/3047/3047f2.png
Novel Devices
►
Paper/Whiteboards
 Video capture of annotations
 Record notes (special tracked pens
Logitech digital pen)
►
Handheld Devices
 PDA
 Universal remote
 Help disabled
► Read
LCD screens
► Rooms in building
► Maps
 Interesting body-context-sensitive.
► Ex.
hold PDA by ear = phone call
answer.
Novel Devices
► Miscellaneous
 Shapetape – reports 3D
shape.
► Tracks
limbs
► Engineer
for specific
app (like a gun trigger
connected to serial
port)
 Pros: good affordance
 Cons: Limited general
use, time
Speech and Auditory Interfaces
►
►
►
There’s the dream
Then there’s reality
Practical apps don’t really require
freeform discussions with a
computer
 Goals:
► Low
► Low
►
cognitive load
error rates
Smaller goals:
 Speech Store and Forward (voice mail)
 Speech Generation
 Currently not too bad, low cost,
available
Speech and Auditory Interfaces
►
►
►
►
►
Ray Kurzweil (’87) – first commercial
speech recognition software
Bandwidth is much lower than visual
displays
Ephemeral nature of speech (tone, etc.)
Difficulty in parsing/searching (Box 9.2)
Types





►
Discrete-word recognition
Continuous speech
Voice information
Speech generation
Non-speech auditory
If you want to do research here, review
research in:
 Audio
 Audio psychology
 Digital signal processing
http://www.kurzweiltech.com/raybio.html
Discrete-Word Recognition
►
►
►
►
Individual words spoken by a specific person
Command and control
90-98% for 100-10000 word vocabularies
Training
 Speaker speaks the vocabulary
 Speaker-independent
►
Still requires





Low noise operating environment
Microphones
Vocabulary choice
Clear voice (language disabled are hampered, stressed)
Reduce most questions to very distinct answers (yes/no)
Discrete-Word Recognition
►
Helps:





►
Disabled
Elderly
Cognitive challenged
User is visually distracted
Mobility or space restrictions
Apps:
 Telephone-based info
►
►
Study: much slower for cursor movement than mouse or keyboard
(Christian ’00)
Study: choosing actions (such as drawing actions) improved
performance by 21% (Pausch ’91) and word processing (Karl ’93)
 However acoustic memory requires high cognitive load (> than hand/eye)
►
►
Toys are successful (dolls, robots). Accuracy isn’t as important
Feedback is difficult
Continuous Speech Recognition
►
►
►
►
Dictation
Error rates and error repair are still poor
Higher cognitive load, could lower overall quality
Why is it hard?
 Recognize boundaries (normal speech blurs them)
 Context sensitivity
 “How to wreck a nice beach”
►
►
►
Much training
Specialized vocabularies (like medical or legal)
Apps:




Dictate reports, notes, letters
Communication skills practice (virtual patient)
Automatic retrieval/transcription of audio content (like radio, CC)
Security/user ID
Voice Information Systems
Use human voice as a source of info
► Apps:
►
 Tourist info
 Museum audio tours
 Voice menus (Interactive Voice Response IVR
systems)
►
Use speech recognition to also cut through
menus
 If menus are too long, users get frustrated
 Cheaper than hiring 24 hr/day reps
►
Voice mail systems
 Interface isn’t the best
►
Get email in your car
 Also helps with non-tech savvy like the elderly
►
Potentially aides with
 Learning (engage more senses)
 Cognitive load (hypothesize each sense has a
limited ‘bandwidth’)
►
Think ER, or fighter jets
Speech Generation
► Play
back speech (games)
► Combine text (navigation systems)
► Careful evaluation!
 Speech isn’t always great
► Door
is ajar – now just a tone
► Use flash
► Supermarket scanners
 Often times a simple tone is better
 Why? Cognitive load
► Thus
cockpits and control rooms need speech
► Competes w/ human-human communication
Speech Generation
►
►
Ex: Text-to-Speech (TTS)
Latest TTS uses multiple syllabi to make generated speech sound
better
 Robotic speech could be desirable to get attention
 All depends on app
 Thus don’t assume one way is the best, you should user test
►
►
Apps: TTS for blind, JAWS
Web-based voice apps: VoiceXML and SALT (tagged web pages).
 Good for disabled, and also for mobile devices
►
Use if
 Message is short
 Requires dynamic responses
 Events in time
►
Good when visual displays aren’t that useful. When?
 Bad lighting, vibrations (say liftoff)
Non-speech Auditory Interface
► Audio
► Major
tones that provide information
Research Area
 Sonification – converting information into audio
 Audiolization
 Auditory Interfaces
► Browsers
link




produced a click when you clicked on a
Increases confidence
Can do tasks without visual cognitive load
Helps figure out when things are wrong
Greatly helps visually impaired
Non-speech Auditory Interface
►
Terms:
 Auditory icons – familiar sounds
(record real world sound and
play it in your app)
 Earcons – new learned sounds
(door ajar)
►
Role in video games is huge
 Emotions, Tension, set mood
►
To create 3D sound
 Need to do more than stereo
 Take into account Head-related
transfer function (HRTF)
►
►
Ear and head shape
New musical instruments
 Theremin
►
New ways to arrange music
Displays
►
►
Primary Source of
feedback
Properties:









Physical Dimension
Resolution
Color Depth and correctness
Brightness, contrast, glare
Power
Refresh rate
Cost
Reliability
# of users
Display
Technology
►
Monochrome displays
(single color)
 Low cost
 Greater intensity range
(medical)
►
Color
Raster Scan CRT
LCD – thin, bright
Plasma – very bright, thin
LED – large public displays
Electronic Ink – new product
w/ tiny capsules of negative
black particles and positive
white
 Braille – refreshable cells
with dots that rise up





Large Displays
► Wall
displays
 Informational
► Control
rooms, military, flight
control rooms, emergency
response
► Provides
 System overview
 Increases situational awareness
 Effective team review
 Interactive
► Require
new interaction methods
(freehand sketch, PDAs)
► Local and remote collaboration
► Art, engineering
Large Displays
► Multiple
Desktop Displays
 Multiple CRTs or Flat panels for
large desktops
 Cheap
 Familiar
 Spatial divide up tasks
 Comparison tasks are easier
 Too much info?
► Eventually
pixel
-> Every surface a
Mobile device displays
►
Personal
 Reprogrammable picture
frames
► Digital
family portrait
(GaTech)
►
Medical
 Monitor patients
►
Research: Modality
Translation Services (Trace
Center – University of
Wisconsin)
 As you move about it auto
converts data, info, etc. for
you
Mobile device displays guidlines
►
►
►
►
►
►
►
►
Bergman ’00, Weiss, ’02
Industry led research and design case
studies (Lindholm ’03)
Typically short in time usage (except
handheld games)
Optimize for repetitive tasks (rank functions
by frequency)
Research: new ways to organize large
amounts of info on a small screen
Study: Rapid Serial Visual Presentation
(RSVP) presents text at a constant speed
(33% improvement Oquist ’03)
Searching and web browsing still very poor
performance
Promising: Hierarchical representation (show
full document and allow user to select where
to zoom into)
3D Printing
► Create
custom objects from
3D models
► Create physical models for
 Design review
 Construction