Embed Size (px)
Citation preview
People, Pens, and Computers
François GuimbretièreHCIL, University of Maryland
Ken HinckleyMSR
New devices, old tasks
Middle picture from Sellen et al.
People, Pens, and Tablet PC
The New Yorker
Illustration from Ken Hinckley presentation at Stanford
Typical setting for today’s interface
• Fixed stable environment, with a keyboard,
• Indirect interaction,
• High precision pointing system
Typical Tablet PC use
• Portable, unstable environment, without a keyboard
• Direct interaction,
• Low precision aiming
Pen based interface for Tablet PC
Document areaHow to create a more fluid interface?
Interface frameworkHow to make it more pen-friendly?
Pen based interface for Tablet PC
Interface frameworkHow to make it more pen-friendly?
• Empirical foundations• Use of strokes to cross target is more pen friendly
• Crossing is as efficient as point-and-click [Accot & Zhai, 2002]
• The basic interactor
• How expressive is it?
CrossY: crossing-based GUI[Apitz & Guimbretière 04]
CrossY
video
Previous Work
• Theoretical basis• Steering Law, Trajectory-Based Tasks [Accot & Zhai 97-02]
• Limited scope examples• Toggle Map [Baudish 98]
• Lotus Notes: multiple e-mail selection
• Conceptual design• Visual Instruments: [Winograd & Guimbretière 98]
• Overloading• Gedrics: [Geißler 95]
Command composition
• From stroke-by-stroke interaction• Borders are used to validate/cancel
Command composition
• From stroke-by-stroke interaction• Borders are used to validate/cancel
• To multi-command stroke
Scrolling
Line by line area
Page by page area
Absolute area
CrossY scrollbar
• Overloading simplify interactions• Shorter distances to issue commands
• Not as much precision necessary
CrossY scrollbar
• Overloading simplify interactions• Shorter distances to issue commands
• Not as much precision necessary
• Extending stroke for repeat• No need to wait for a timeout
Cursor control
• Cross to jump to an absolute position
Cursor control
• Cross to jump to an absolute position
Cursor control
• Cross to jump to an absolute position
• Near drag for coarse adjustment
Cursor control
• Cross to jump to an absolute position
• Near drag for coarse adjustment
Cursor control
• Cross to jump to an absolute position
• Near drag for coarse adjustment
• Far drag for fine adjustment• Similar to FineSlider [Masui 95]
• But one single stroke
Use of directionality
• Continuous find and replace
Use of directionality
• Continuous find and replace• Reverse direction for undo
What we learned
• Very well received by users• HCIL open house
• UIST
• Space requirements• Similar to point-and-click
• Trade-off with command combination due to sloppiness
• Overloading vs. easy discovery• Consistency helps with adoption
• Known in Windowing systems
Pen based interface for Tablet PC
Document areaHow to create a more fluid interface?
Scriboli and Stitching
Ken’s presentation
My desk
Affordances of paper documents[Sellen 01]
• Easy to navigate• Two-handed interactions and tactile feedback
• Reading across more than one document at once
• Easy to annotate• Directly on the document or on a nearby pad
• Well accepted during meetings• Socially accepted conventions
• Very difficult to modify• Printed documents are created and edited as digital documents
• Expensive to distribute and archive
Bridging the gap: previous work
• Digital emulation• FreeStyle system [Wang 89]
• MATE [Hardock 93]
• XLibris [Schilit 98], [Golovchinsky 02]
• Tight coupling• DigitalDesk [Wellner 93], Ariel [Mackay 95]
• A-Book [Mackay 02]
• PaperLink [Arai 97]
• Intelligent Paper [Dymetman 98]
• Paper as input device• Xax [Johnson 93]
• Anoto
• Paper PDA [Heiner 99], [Avrahami 01]
Cohabitation[Guimbretière 03]
Stroke capture
From Anoto documentation
• Requirement• Stroke coordinates on the page
• Page ID
• Large address space
• Possible technologies• Anoto
• DataGlyphs [Hecht 94]
• MEMO pen [Nabeshima 95]
System architecture
My Desk with PADD
PapierCraft: command system for PADD[Liao, Guimbretiere and Hinckley 05]
PADD document
Anoto pen
PADD notepad
PapierCraft: using PADD as proxies
Copy Paste
Synchronization On your Tablet PC
PapierCraft
• Gesture/Ink• Use the command button
• Scopes
• Command selection• Marking menu
• Writing down an unambiguous prefix of a command
• Batch mode processing
Early feedback and future work
• Small scale evaluation• Commands with written words were very popular
• Higher recognition rate needed for strokes only feedback
• Contextual information might provide enough feedback during pastes
• Future work• More reliable recognition engine
• Pen-based feedback• Haptic feedback, LED
• Real time processing : streaming stroke
Conclusions
• Four pen-based interfaces• CrossY explores crossing-based interfaces
• Scriboli explores high performance direct manipulation interfaces
• Stitching explores multi-devices interactions
• PapierCraft uses PADD as document proxies
• One goal• Bring the ease of use of pen and paper to digital interfaces
• Future• Seamless integration of both media
Acknowledgments
• Collaborators• Jim Hollan
• Students• Georg Apitz, Nicholas Chen (CrossY/ScribolY)
• Kevin Conroy, Dave Levin, Chunyuan Liao (PADD/ProofRite)
• Sponsors• Microsoft
• HP
• Anoto, Logitech, Maxell
• Colleagues and friends• Corinna Löckenhoff
• Samrat (Bobby) Bhattacharjee
• Ben Bederson
