Transcript Precise Selection Techniques for Multi-Touch Screens Hrvoje Benko Andy D. Wilson Patrick Baudisch Columbia University and Microsoft Research CHI 2006

Precise Selection
Techniques for
Multi-Touch Screens
Hrvoje Benko
Andy D. Wilson
Patrick Baudisch
Columbia University and Microsoft Research
CHI 2006
Selecting a small target is
very HARD!
CHI 2006
2
Small target size comparison

Target
UI element
Average finger ~ 15 mm wide
Width
(abstract screen)
Width
17” screen
1024x768
Width
30” screen
1024x768
Close
button
18 pixels
6 mm
(40% of finger)
10.8 mm
(66% of finger)
Resize
handle
4 pixels
1.34 mm
(9% of finger)
2.4 mm
(16% of finger)
CHI 2006
3
Touchscreen Issues
1.
2.
3.
4.
5.
Finger >>> Target
Finger occludes the target
Fingers/hands shake and jitter
Tracking can be noisy (e.g. video)
No hover state (hover == drag)
CHI 2006
4
Previous Work

Solutions based on single touch interfaces and
complex on-screen widgets:
Sears, A. and Shneiderman, B.
“High Precision Touchscreens:
Design Strategies and
Comparisons with a Mouse.”
(’91)
Albinsson, P. A. and Zhai, S.
“High Precision Touch Screen
Interaction.” (CHI ’03)
CHI 2006
5
Dual Finger Selections


Multi-touch techniques
Single fluid interaction


no lifting/repositioning of fingers
Design guidelines:



Keep simple things simple.
Provide an offset to the cursor when so
desired.
Enable user controlled control-display
ratio.
CHI 2006
6
Simulating Hover State


Extension of the “area==pressure” idea
(MacKenzie and Oniszczak, CHI 1998)
Problem:


LARGE area difference  reliable clicking
SMALL movement (i.e. SMALL area
difference)  precise and accurate clicking
CHI 2006
7
SimPress (Simulated Pressure)


Clicking gesture –
“finger rocking”
Goal:


Maximize ∆ touch
area
Minimize ∆ cursor
location
CHI 2006
8
SimPress Cursor Placement
Center-of-Mass Cursor


Large ∆ touch area
Large ∆ cursor loc.
CHI 2006
Top Middle Cursor


Large ∆ touch area
Small ∆ cursor loc.
9
SimPress in Action
CHI 2006
10
Dual Finger Selections
1.
2.
3.
4.
5.
Offset
Midpoint
Stretch
X-Menu
Slider
Primary finger  cursor position & click
Secondary finger  cursor speed or C/D
CHI 2006
11
Dual Finger Offset


Fixed offset WRT
finger
Ambidextrous
control
CHI 2006
12
Dual Finger Midpoint




Cursor  ½
distance between
fingers
Variable speed
control
Max speed
reduction is 2x
Dead spots on
screen!
CHI 2006
13
Dual Finger Stretch

Inspired by ZoomPointing
(Albinsson & Zhai,‘03)


Primary finger  anchor
Secondary finger


defines the zooming area
scales the area in all
directions away from the
anchor
CHI 2006
14
Dual Finger Stretch

Offset is
preserved after
selection!
CHI 2006
15
Zooming Comparison

Bounding Box Zoom


Fingers placed OFF
target
Target distance
increases w/ zoom
CHI 2006

“Stretch” Zoom


Primary finger placed
ON target
Same motion = 2x
zoom
16
Dual Finger X-Menu

Crossing Menu (no buttons/no clicks)



Cursor notification widget


Eyes-free interaction
Freezing cursor



4 speed modes
2 helper modes
Quick offset setup
Eliminate errors in noisy conditions
Helpers:


Snap – Remove offset
Magnification Lens
CHI 2006
17
Dual Finger X-Menu
CHI 2006
18
Dual Finger X-Menu
with Magnification Lens
CHI 2006
19
Dual Finger Slider
Freeze
Slow 10X
Slow 4X
Normal
Snap
CHI 2006
20
Dual Finger Slider
CHI 2006
21
Multi-Touch Table Prototype



Back projected
diffuse screen
IR vision-based
tracking
Similar to
TouchLight
(Wilson, ICMI’04)
CHI 2006
22
User Experiments


Measure the impact of a particular technique on
the reduction of error rate while clicking
2 parts:



Task:




Evaluation of SimPress clicking
Comparison of Four Dual Finger Techniques
Reciprocal target selection
Varying the square target width
Fixed distance (100 pixels)
12 paid participants (9 male,3 female, ages 20–
40), frequent computer users, various levels of
touchscreen use
CHI 2006
23
Part 1: SimPress Evaluation
Percent of Trials ± SEM

100
90
80
70
60
50
40
30
20
10
0

Within subjects
repeated measures
design
5 target widths:


1
2
4
8
16
Target Width (pixel)
F(4,44)=62.598, p<0.001
CHI 2006

1,2,4,8,16 pxls
Hypothesis: only 16
pxls targets are
reliably selectable
Results: 8 pixel
targets still have
~10% error rate
24
Part 2: Comparison of 4 Dual
Finger Selection Techniques


Compare: Offset, Stretch, X-Menu, Slider
Varying noise conditions


Within subjects repeated measures design:



Inserted Gaussian noise: σ=0, 0.5, 2
3 noise levels x 4 techniques x 4 target widths
(1,2,4,8 pxls)
6 repetitions  288 trials per user
Hypotheses:


Techniques that control the C/D will reduce the
impact of noise
Slider should outperform X-Menu
CHI 2006
25
Part 2: Error Rate Analysis
Interaction of Noise x Technique

low
ErrorRate (%) ± SEM
70
medium
high
60
50
40
30
20
10
0
Ofset
X-Menu
Slider
Stretch
F(6,66)= 8.025, p<0.001
CHI 2006
26
Part 2: Error Rate Analysis

Interaction of Width x Technique
Offset
X-Menu
Slider
Stretch
Error Rate (%) ± SEM
100
80
60
40
20
0
W-1
W-2
W-4
W-8
F(9,99)=29.473, p<0.001
CHI 2006
27
Part 2: Movement Time
Analysis

Offset
X-Menu
Slider
Stretch
Movement Time (s) ± SEM

7
Missing
Analysis on median
times
Stretch is ~ 1s faster
than Slider/X-Menu
(t(11)=5.011, p<0.001)
6

5
4
3
Slider similar
performance to XMenu
2
1
0
W-1
W-2
W-4
W-8
CHI 2006
28
Subjective Evaluation

Post-experiment questionnaire (5 pt Likert
scale)




Most mental effort: X-Menu (~2.88)
Hardest to learn: X-Menu ( ~2.09)
Most enjoyable: Stretch (~4.12), Slider (~4.08)
No significant differences WRT fatigue
Overall Preference
Best Technique for Noise Condition
Offset
XMenu
Slider
Stretch
8
7
6
5
4
3
2
1
0
12
10
8
6
4
2
0
Low Noise
Medium Noise
Offset
High Noise
CHI 2006
X-Menu
Slider
Stretch
29
Conclusions and Future Work


Top performer & most preferred: Stretch
Slider/X-Menu




Freezing the cursor (positive feedback)


Comparable error rates to Stretch
No distortion of user interface
Cost: ~1s extra
Like “are you sure?” dialog for clicking…
Possible future SimPress extensions:


Detect user position/orientation
Stabilization of the cursor
CHI 2006
30
Questions
Multi-Touch Tabletops



MERL DiamondTouch (Dietz & Lehigh,
’01)
SmartSkin (Rekimoto, ’02)
PlayAnywhere and TouchLight (Wilson,
’04, ’05)
CHI 2006
32
ANOVA Table
Source
df
F
p
Noise (N)
(2,22)
20.24
<0.001
Technique (T)
(3,33)
169.14
<0.001
Width (W)
(3,33)
150.40
<0.001
NxT
(6,66)
8.03
<0.001
TxW
(9,99)
29.47
<0.001
NxW
NxTxW
CHI 2006
33