Embed Size (px)
Citation preview
Ken Hinckley Seongkook Heo (KAIST & MSR)Michel Pahud Christian HolzHrvoje Benko Abigail Sellen Gavin SmythRichard Banks Kenton O’Hara Bill Buxton
for MOBILE INTERACTION
PRE-TOUCH SENSING
Michel Pahud Christian HolzHrvoje Benko Abigail Sellen Gavin SmythRichard Banks Kenton O’Hara Bill Buxton
Ken Hinckley Seongkook Heo (KAIST & MSR)
for MOBILE INTERACTION
PRE-TOUCH SENSING
TOUCH IS
ANALOGand
CONTINUOUS…
IF NATURAL HUMAN
(x,y)
Then why do we boil the
down to flatland?entire vocabulary of mobile touch
Much of what characterizes ‘touch’ startsbefore contact
and originates
beyond the confinesof the screen.
AN INSTANT IN TIME…FROM TOUCH AS A POINT,
TO A WHOLE WORLD THAT SURROUNDS IT…
This treasury of contextual detail—
the combination of hover and gripthat we collectively refer to as
pre-touch—has been lost to mobile interaction.
SENSOR
/technology
– Self-capacitance is well-known. Not our contrib.
fogale-sensation.com
You’ve probably heard about hover, or seen examples of control gestures in-air.
You’ve probably heard about hover, or seen examples of control gestures in-air.
The focus of pre-touch isn’t on such overt, ‘foreground’ gestures.
You’ve probably heard about hover, or seen examples of control gestures in-air.
The focus of pre-touch isn’t on such overt, ‘foreground’ gestures.
Instead, pre-touch is primarily about the ‘background’ of the interaction—
You’ve probably heard about hover, or seen examples of control gestures in-air.
The focus of pre-touch isn’t on such overt, ‘foreground’ gestures.
Instead, pre-touch is primarily about the ‘background’ of the interaction—
what happens before, the framing, the context.
This is a design space paper.
This is a design space paper.
We explore an emerging modality, breadth-first across a number of
possibilities.
This is a design space paper.
We explore an emerging modality, breadth-first across a number of
possibilities.
But we do not go depth-first into any one technique in exhaustive detail.
WE EXPLORED:
1) ANTICIPATORY REACTIONS
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
WE EXPLORED:
1) ANTICIPATORY REACTIONS
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
WE EXPLORED:
1) ANTICIPATORY REACTIONS
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
WE EXPLORED:
1) ANTICIPATORY REACTIONS
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch, simultaneous with hover
WE EXPLORED:
1) ANTICIPATORY REACTIONS
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
WE EXPLORED:
1) ANTICIPATORY REACTIONS
– when fingers away, no UI. Only content.
– so we fade in (0.2s) … later fade out (1.2s)
– since we’re fading in anyway… take liberty– to fade in context-appropriate controls.
TO ILLUSTRATE THIS DESIGN SPACE,
ad-lib interface : key ideas
– approach suggests intent to engage
but…
WE EXPLORED:
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
WE EXPLORED:
– when fingers away, no UI. Only content.
– so we fade in (0.2s) … later fade out (1.2s)
– since we’re fading in anyway… take liberty– to fade in context-appropriate controls.
TO ILLUSTRATE THIS DESIGN SPACE,
ad-lib interface : key ideas
– approach suggests intent to engage
but…
WE EXPLORED:
– when fingers away, no UI. Only content.
– so we fade in (0.2s) … later fade out (1.2s)
– since we’re fading in anyway… take liberty– to fade in context-appropriate controls.
TO ILLUSTRATE THIS DESIGN SPACE,
ad-lib interface : key ideas
– approach suggests intent to engage
but…
WE EXPLORED:
– when fingers away, no UI. Only content.
– so we fade in (0.2s) … later fade out (1.2s)
– since we’re fading in anyway… take liberty– to fade in context-appropriate controls.
TO ILLUSTRATE THIS DESIGN SPACE,
ad-lib interface : key ideas
– approach suggests intent to engage
but…
WE EXPLORED:
– when fingers away, no UI. Only content.
– so we fade in (0.2s) …
– since we’re fading in anyway… take liberty– to fade in context-appropriate controls.
TO ILLUSTRATE THIS DESIGN SPACE,
ad-lib interface : key ideas
– approach suggests intent to engage
but…
WE EXPLORED:
– when fingers away, no UI. Only content.
– so we fade in (0.2s) … later fade out (1.2s)
– since we’re fading in anyway… take liberty– to fade in context-appropriate controls.
TO ILLUSTRATE THIS DESIGN SPACE,
ad-lib interface : key ideas
– approach suggests intent to engage
but…
WE EXPLORED:
– when fingers away, no UI. Only content.
– so we fade in (0.2s) … later fade out (1.2s)
– since we’re fading in anyway… take liberty– to fade in context-appropriate controls.
TO ILLUSTRATE THIS DESIGN SPACE,
ad-lib interface : key ideas
– approach suggests intent to engage
but…
WE EXPLORED:
– when fingers away, no UI. Only content.
– so we fade in (0.2s) … later fade out (1.2s)
– since we’re fading in anyway, take liberty– to fade in context-appropriate controls.
TO ILLUSTRATE THIS DESIGN SPACE,
ad-lib interface : key ideas
– approach suggests intent to engage
but…
WE EXPLORED:
– when fingers away, no UI. Only content.
– so we fade in (0.2s) … later fade out (1.2s)
– since we’re fading in anyway, take liberty– to fade in context-appropriate controls.
TO ILLUSTRATE THIS DESIGN SPACE,
ad-lib interface : key ideas
– approach suggests intent to engage
but…
WE EXPLORED:
grip + hover
TO ILLUSTRATE THIS DESIGN SPACE,
multiple
contexts
(“dialing”)
even adaptation ofcontrol gestures
WE EXPLORED:
1) ANTICIPATORY REACTIONS
calm revelation
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
ad-lib interface
WE EXPLORED:
1) ANTICIPATORY REACTIONS
calm revelation
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
ad-lib interface
WE EXPLORED:
1) ANTICIPATORY REACTIONS
calm revelation
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
ad-lib interface
WE EXPLORED:
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
WE EXPLORED:
1) ANTICIPATORY REACTIONS
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
WE EXPLORED:
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
WE EXPLORED:
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
WE EXPLORED:
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
0ms(touch down)
temporal trajectoryfor a small target
temporal trajectoryfor a large target
-200ms
30mm
single-finger distance above touchscreen
20mm
10mm
-400ms-600ms-800ms-1s
WE EXPLORED:
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
WE EXPLORED:
1) ANTICIPATORY REACTIONS
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch simultaneous with hover
WE EXPLORED:
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
WE EXPLORED:
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
WE EXPLORED:
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
INFORMALEVALUATION
and they’re predictive.”
“feels very natural to my hand,” allows
– adapt to hand size?
Ad-lib interface (video player)
“I like the transparent controls,
“a single hand in a comfortable position.”
– controls should appear more quickly?
– per user adaptations may also be helpful
Overall
– The one-handed adaptations stood out
– But all techniques had proponents
– Ballistic vs. fine taps: great for some users,
– current heuristic doesn’t consider grip.– not so much for others.
CONCLUSION
CONCLUSION
Pre-touch holds great promise for fresh approaches to many challenges in mobile touch
interaction.
Our work has only just begun.
Seongkook
TO ILLUSTRATE THIS DESIGN SPACE,
Heo
WE EXPLORED:
2) RETROACTIVE INTERPRETATIONS
3) HYBRID TOUCH-HOVER GESTURES
TO ILLUSTRATE THIS DESIGN SPACE,
interface contingent on grip and hover
approach trajectory informs touch semantics
on-screen touch + above-screen aspects
Questions?
