Precise Selection Techniques for Multi-

Touch Screens

Hrvoje BenkoAndy D. Wilson

Patrick Baudisch

Columbia University and Microsoft ResearchCHI 2006

CHI 2006 2

Selecting a small target is very HARD!

CHI 2006 3

Small target size comparison Average finger ~ 15 mm wide

TargetUI element

Width(abstract screen)

Width17” screen1024x768

Width 30” screen1024x768

Close button 18 pixels

6 mm(40% of finger)

10.8 mm(66% of finger)

Resize handle 4 pixels

1.34 mm(9% of finger)

2.4 mm(16% of finger)

CHI 2006 4

Touchscreen Issues

1. Finger >>> Target2. Finger occludes the target3. Fingers/hands shake and jitter4. Tracking can be noisy (e.g. video) 5. No hover state (hover == drag)

CHI 2006 5

Previous Work

Solutions based on single touch interfaces and complex on-screen widgets:

Albinsson, P. A. and Zhai, S. “High Precision Touch Screen Interaction.” (CHI ’03)

Sears, A. and Shneiderman, B. “High Precision Touchscreens: Design Strategies and Comparisons with a Mouse.” (’91)

CHI 2006 6

Dual Finger Selections

Multi-touch techniques Single fluid interaction

no lifting/repositioning of fingers Design guidelines:

Keep simple things simple. Provide an offset to the cursor when so

desired. Enable user controlled control-display

ratio.

CHI 2006 7

Simulating Hover State

Extension of the “area==pressure” idea (MacKenzie and Oniszczak, CHI 1998)

Problem: LARGE area difference reliable clicking SMALL movement (i.e. SMALL area

difference) precise and accurate clicking

CHI 2006 8

SimPress (Simulated Pressure)

Clicking gesture – “finger rocking”

Goal: Maximize ∆ touch

area Minimize ∆ cursor

location

CHI 2006 9

Top Middle Cursor

Large ∆ touch area Small ∆ cursor loc.

Center-of-Mass Cursor

Large ∆ touch area Large ∆ cursor loc.

SimPress Cursor Placement

CHI 2006 10

SimPress in Action

CHI 2006 11

Dual Finger Selections

1. Offset2. Midpoint3. Stretch4. X-Menu5. Slider

Primary finger cursor position & clickSecondary finger cursor speed or C/D

CHI 2006 12

Dual Finger Offset

Fixed offset WRT finger

Ambidextrous control

CHI 2006 13

Dual Finger Midpoint

Cursor ½ distance between fingers

Variable speed control

Max speed reduction is 2x

Dead spots on screen!

CHI 2006 14

Dual Finger Stretch

Inspired by ZoomPointing (Albinsson & Zhai,‘03)

Primary finger anchor Secondary finger

defines the zooming area scales the area in all

directions away from the anchor

CHI 2006 15

Dual Finger Stretch

Offset is preserved after selection!

CHI 2006 16

Zooming Comparison

Bounding Box Zoom Fingers placed OFF

target Target distance

increases w/ zoom

“Stretch” Zoom Primary finger

placed ON target Same motion = 2x

zoom

CHI 2006 17

Dual Finger X-Menu

Crossing Menu (no buttons/no clicks) 4 speed modes 2 helper modes

Cursor notification widget Eyes-free interaction

Freezing cursor Quick offset setup Eliminate errors in noisy conditions

Helpers: Snap – Remove offset Magnification Lens

CHI 2006 18

Dual Finger X-Menu

CHI 2006 19

Dual Finger X-Menu with Magnification Lens

CHI 2006 20

Dual Finger Slider

NormalSlow 4X

Slow 10XFreeze

Snap

CHI 2006 21

Dual Finger Slider

CHI 2006 22

Multi-Touch Table Prototype

Back projected diffuse screen

IR vision-based tracking

Similar to TouchLight (Wilson, ICMI’04)

CHI 2006 23

User Experiments

Measure the impact of a particular technique on the reduction of error rate while clicking

2 parts: Evaluation of SimPress clicking Comparison of Four Dual Finger Techniques

Task: Reciprocal target selection Varying the square target width Fixed distance (100 pixels)

12 paid participants (9 male,3 female, ages 20–40), frequent computer users, various levels of touchscreen use

CHI 2006 24

Part 1: SimPress Evaluation

Within subjects repeated measures design

5 target widths: 1,2,4,8,16 pxls

Hypothesis: only 16 pxls targets are reliably selectable

Results: 8 pixel targets still have ~10% error rate

0102030405060708090

100

1 2 4 8 16

Target Width (pixel)

Per

cent

of

Tria

ls ±

SE

M

F(4,44)=62.598, p<0.001

CHI 2006 25

Part 2: Comparison of 4 Dual Finger Selection Techniques

Compare: Offset, Stretch, X-Menu, Slider Varying noise conditions

Inserted Gaussian noise: σ=0, 0.5, 2 Within subjects repeated measures design:

3 noise levels x 4 techniques x 4 target widths (1,2,4,8 pxls)

6 repetitions 288 trials per user Hypotheses:

Techniques that control the C/D will reduce the impact of noise

Slider should outperform X-Menu

CHI 2006 26

Part 2: Error Rate Analysis

Interaction of Noise x Technique

0

10

20

30

40

50

60

70

Ofset X-Menu Slider Stretch

Err

orR

ate

(%

) ±

SE

M

low medium high

F(6,66)= 8.025, p<0.001

CHI 2006 27

Part 2: Error Rate Analysis

0

20

40

60

80

100

W-1 W-2 W-4 W-8

Err

or

Rate

(%

) ± S

EM

Offset X-Menu Slider Stretch

Interaction of Width x Technique

F(9,99)=29.473, p<0.001

CHI 2006 28

Part 2: Movement Time Analysis

Analysis on median times

Stretch is ~ 1s faster than Slider/X-Menu (t(11)=5.011, p<0.001)

Slider similar performance to X-Menu

0

1

2

3

4

5

6

7

W-1 W-2 W-4 W-8

Mov

emen

t T

ime

(s)

± S

EM

Offset X-Menu Slider Stretch

Missing

CHI 2006 29

Subjective Evaluation

Post-experiment questionnaire (5 pt Likert scale) Most mental effort: X-Menu (~2.88) Hardest to learn: X-Menu ( ~2.09) Most enjoyable: Stretch (~4.12), Slider (~4.08) No significant differences WRT fatigue

Best Technique for Noise Condition

02

46

810

12

Low Noise Medium Noise High Noise

Offset XMenu Slider Stretch

Overall Preference

012345678

Offset X-Menu Slider Stretch

CHI 2006 30

Conclusions and Future Work

Top performer & most preferred: Stretch Slider/X-Menu

Comparable error rates to Stretch No distortion of user interface Cost: ~1s extra

Freezing the cursor (positive feedback) Like “are you sure?” dialog for clicking…

Possible future SimPress extensions: Detect user position/orientation Stabilization of the cursor

Questions

CHI 2006 32

Multi-Touch Tabletops

MERL DiamondTouch (Dietz & Lehigh, ’01)

SmartSkin (Rekimoto, ’02) PlayAnywhere and TouchLight (Wilson,

’04, ’05)

CHI 2006 33

ANOVA Table

Source df F p

Noise (N) (2,22) 20.24 <0.001

Technique (T) (3,33) 169.14 <0.001

Width (W) (3,33) 150.40 <0.001

N x T (6,66) 8.03 <0.001

T x W (9,99) 29.47 <0.001

N x W

N x T x W