Informal PUIs: No Recognition Required, at AAAI 2002 workshop

Informal PUIs:No Recognition Required

James LandayJason HongScott KlemmerJames LinMark Newman

Mar 25 2001 2

WIMPy User Interfaces

• Windows, Icons, Menus, Pointer­ Hitting limitations of this kind of interface

• WIMP designed for limited audience­ Full use of eyes­ Literate­ Full use of hands

• WIMP designed for limited situations­ Fixed location (often offices)­ Sitting down­ Single person

Mar 25 2001 3

Perceptual User Interfaces

• PUIs leverage our innate perceptual, motor, and cognitive abilities­ Speech recognition, computer vision,

sketching, handwriting

• Sketching is one part of this bigger push • PUIs more useful for different audiences

and wider range of situations­ Mobile workers or Home­ Speech => Hands­free­ Location Tracking => Physical motion

Mar 25 2001 4

PUIs and Recognition

• Key question:­ When should perceptual input be recognized?

• Immediate recognition can interfere with creativity and communications

Mar 25 2001 5

Communication and Creativity

• Informal visual representation­ communicates “unfinished”­ encourages creativity­ faster to create­ higher level comments

• Formal visual representation­ communicates “finished”­ inhibits creativity (detailed)­ slower to create­ lower­level comments

Mar 25 2001 6

Informal User Interfaces

• Historic bias towards formal user interfaces­ Computation over creativity and communication­ Structured input to simplify computation

• Position:­ Recognition interferes with creativity and

communication due to errors and perception­ Recognition should be minimized or deferred

for these kinds of apps

Mar 25 2001 7

Informal User Interfaces

• Sketching• Speech• Handwriting

Mar 25 2001 8

Informal User Interfaces

• Sketching• Speech• Handwriting

Mar 25 2001 9

Slide

Landay (1996)

Mar 25 2001 10

Slide

Gross and Do (UIST96)

Mar 25 2001 11

Slide

Lin, Newman, Hong, Landay (CHI2000)

Mar 25 2001 12

DENIM Video

Mar 25 2001 13

Informal User Interfaces

• Sketching• Speech• Handwriting

Mar 25 2001 14

Slide

Stifelman, Arons, Schmandt (CHI2001)

Mar 25 2001 15

SUEDE:Informal Prototyping for Speech-based UIs

• Supports design practice- example scripts- Wizard of Oz- error simulation- iterative design

• Informal user interface- no speech recognition or

synthesis- need not be programming expert- fast & fluid design

Read my important

email

Mar 25 2001 16

Slide

Klemmer et al (UIST2000)

Mar 25 2001 17

SUEDE Video

Mar 25 2001 18

Informal User Interfaces

• Sketching• Speech• Handwriting

Mar 25 2001 19

NotePals:Informal Handwriting Capture

Davis et al (CHI99)

Mar 25 2001 20

NotePals:Informal Handwriting Capture

Mar 25 2001 21

Takeaway Ideas

• Perceptual user interfaces useful for more audiences and more places

• Informal User Interfaces valuable for PUIs­ Minimize or defer recognition­ Creative or communications­oriented tasks

• Informal User Interfaces can be applied to a range of perceptual input­ Sketching­ Speech­ Handwriting

Informal PUIs:No Recognition Required

James LandayJason HongScott KlemmerJames LinMark Newman

Mar 25 2001 23

Backup Slides

Mar 25 2001 24

Backup Slides

Igarashi, Edwards, LaMarca, and Mynatt (AVI2000)