A SHIFT TO

• Handhelds• Slates• Drafting Boards• Tabletops• Wall Displays

• Pen, or• Touch.

DIRECT INPUT A SHIFT TO

DETAILED LITERATURE

REVIEW

(ABRIDGED VERSION)

pen

t(ouch!)

one modality =

PEN + TOUCH

DESIGN STUDY

PAPER NOTEBOOK OBSERVATIONS

B1: Tuck the penB2: Hold ClippingsB3: Hold While WritingB4: Framing

PAPER NOTEBOOK OBSERVATIONS

B1: Tuck the penB2: Hold ClippingsB3: Hold While WritingB4: FramingB5: ScrapsB6: New Stuff from AboveB7: Piling

Fingers

B1: Tuck the penB2: Hold ClippingsB3: Hold While WritingB4: FramingB5: ScrapsB6: New Stuff from AboveB7: PilingB8: Draw along EdgesB9: Tearing: Bimanual w/

PAPER NOTEBOOK OBSERVATIONS

OF HANDSAND

HANDEDNESS

Q: Which hand do you write with, Right or Left?

A: Yes

• Guiard, 1987: NPH defines frame of reference.• Right vs. Left is the wrong question to ask.

What is the logic of the division of labor

between pen and touch in interface design?

Pen WritesTouch Manipulates

fluidity of annotation + manipulation navigation zooming object creation

Pen + Touch Yields New Tools

• Video

PEN + TOUCH GESTURES

staple

straightedge stamp

Xacto cut

brush

peel off copy

• Holding touch integrates Object Selection and Pen Mode Switching into a single action.

• Holding object…• provides salient feedback of temporary state. • unambiguously phrases together multiple

elementary pen and touch inputs.• Extends the space of interactions w/out hitting

heavyweight modes & ambiguities.• No physical buttons (NPH modes…)• Minimal on-screen UI & widgets.

SOME INTRIGUING PROPERTIES…

INFORMALUSER

STUDY

• Users easily interleaved pen & touch – obvious• Staple, Xacto Cut, and Peel off Copy

particularly resonated with users• The particular tools were not always of interest• Some example comments:

REACTIONS FROM USERS

“The way it works is just like the way I already work in my notebook.” “I wouldn’t have guessed the gestures worked that way, but once I tried it, it felt pretty natural.”

SUMMARY

Not pen. Not touch.

Not pen OR touch.Pen AND Touch,

designed in concert

Why Pen? I can type faster.

Why Pen? I can type faster.

• It’s not the same message.• Different thought process: text mires you in detail.

In other words…

32

devices without buttons…

PEN vs.

TOUCH

CAGED DEATHMATCH

pen writes touch manipulates

PEN + TOUCH = NEW TOOLS

INTRIGUINGDESIGN

PROPERTIES