Modeling and Specifying Tangible User Interfaces

1Comp150 TUI Spring 2007

Modeling and Specifying Modeling and Specifying Tangible User InterfacesTangible User Interfaces


Tangible User Interfaces - Design Tangible User Interfaces - Design SpaceSpace

Designers’ Outpost [Klemmer et al.2001]

Navigation Blocks [Camarata et al. 2002]

Media Blocks [Ullmer et al. 2003]

Senseboard [Jacob et al. 2001]QuickTime™ and a

TIFF (LZW) decompressorare needed to see this picture.

ComTouch [Jacob et al. 2001]

Tangible Query Interfaces [Ullmer et al. 2003]

URP[Underkoffler and Ishii. 1999]

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Tangible Video Editor [Zigelbaum et al. 2005]

PeoplePretzelShaer et al. 2004]

Quetzal [Horn et al. 2006]


Why to Model?Why to Model?

Highlight system properties: observability, predicatability, access points, etc.

Expose design issues: what is physical what is digital, physical syntax, expected and unexpected interactions.

Explore and compare alternative designs.

Communicate design within multidisciplinary teams Design documentation

Guide implementation Identify boundary cases

Analysis

Communication

Implementation


TUIML: Tangible User Interface modeling TUIML: Tangible User Interface modeling LanguageLanguage

Interaction diagram

3

TAC Palette

1 2







TAC Palette Dialogue diagram

Dialogue Interaction

structure behavior


Tokens: Physical objects that represent digital information. Users interact with tokens to access and manipulate digital information.

Constraints: Physical objects that provide context to tokens manipulation by constraining the ways in which tokens can be manipulated. A constraint constrain the manipulation of a token by: Suggesting how to manipulate a token Physically constraining Providing a frame of reference

TAC: A relationship between a token and a set of constraints. TACs encapsulate a set of manipulation actions that can be performed upon a token in respect to a set of constraints.

Describing the Structure of a TUI using TUIMLDescribing the Structure of a TUI using TUIML


TUIML Representation of TUIML Representation of Tokens and ConstraintsTokens and Constraints

Constraint Representation Conceptual Relations Action/Manipulation

Surface Identity, presence, position (x,y,z), spatial relations, order, number, group, containment

Add, remove, move, stack, line up, cluster, separate etc.

Rack Identity, presence, position (x), spatial relations, order, number, proximity

Add, remove, slide

Indentation Presence, identity Add, remove

Knob Position (θ), identity Rotate

Length slider Position (x1,x2), identity

Slide

Connector Connection, identity Connect, disconnect

Tokens Constraints

Attributes:

ShapeColorSizeTexture


Combining Tokens and ConstraintsCombining Tokens and Constraints into TACs into TACs

m

Building model

Wind tool

Message

Play indentation

QueryParameter


URPURP[Underkoffler and Ishii 1999][Underkoffler and Ishii 1999]

A TUI for urban planning. Combines physical building models with physical interactive tools to help urban planners

perform analysis of shadows, proximities, reflection, wind and visual space. Physical interaction objects include:

Building models A wind tool A material wand A clock dial A distance measuring tool


The TAC PaletteThe TAC Palette

TAC Representation Association Manipulation

Variable Token Constraint TAC graphics Action Response

1 building building model surfaceother buildings

Add displays shadow according to time.

Remove Removes related info from display

move Updates display

2 distance Distance tool Two buildings Surface

Add Displays distance

remove Hides distance

m


The TAC PaletteThe TAC Palette

TAC Representation Association Manipulation

Variable Token Constraint TAC graphics Action Response

1 building building model surfaceother buildings

Add displays shadow according to time.

Remove Removes related info from display

move Updates display

2 distance Distance tool Two buildings Surface

Add Displays distance

remove Hides distance

3 wind simulation

Wind tool Buildingssurface

Add Displays wind

Remove Hides wind

move Updates wind

m

m


Describing Behaviors Using TUIMLDescribing Behaviors Using TUIML

Interaction diagram

3

TAC Palette

1 2









structure behavior


Interaction Model for TUIsInteraction Model for TUIs


Dialogue diagram - URPDialogue diagram - URP

wind

move

time

mater.

orien.

addBuilding /shadow(T)

addBuilding /shadow(T)

removeBuilding /shadow(F)

addBuilding /shadow(F)

removeBuilding numOfBuilding=2/shadow(F)

removeBuilding numOfBuilding>2/shadow(F)

wind

move

time

mater.

orien.

dist.


Task Diagram - URP Wind SimulationTask Diagram - URP Wind Simulation


Analyzing URPAnalyzing URP

Observability and predictability:• Does the physical state of the system indicate to the user the internal state of the system? • Does the physical state of the system contain enough information to enable the user to

determine what tasks are available for him to perform? • Assuming multiple users interact with the system at the same time. Does the physical state of

the system indicate to users what tasks they can or cannot perform in parallel? Modes

• Certain actions are meaningful only in certain contexts. Bring an example of a user action that is meaningless in one context and has a meaning in another.

Physical Syntax• Does the physical state of the system indicate to users which actions are legal¥illegal? Does

the physical state of the system indicate to users how actions can be combined into sequences in order to perform a task?


The Marble Answering Machine (MAM)The Marble Answering Machine (MAM)(Durrel Bishoph 1992)(Durrel Bishoph 1992)


MAM - TAC PaletteMAM - TAC Palette


MAM - Dialogue diagramMAM - Dialogue diagram


MAM - Task diagramsMAM - Task diagrams

Call back Play


Tangible Query InterfacesTangible Query InterfacesUllmer, Ishii, Jacob 2003Ullmer, Ishii, Jacob 2003

Using physically constrained tokens to express, manipulate and visualize parameterized database queries.


Tangible Queries - TAC PaletteTangible Queries - TAC Palette

Representation Association Manipulation

TAC Variable Token Constraints TAC Graphics Action Response

1 Query Parameter(upper bound)

Upper slider

Bar, lower slider

Slide setUpperBound()

2 Query Parameter(lower bound)

Lower slider

Bar, upper slider

Slide setLowerBound()

3 Query parameter

Parameter bar

RackOther bars

Add

Slide

Remove

displayQueryRes()

displayQueryRes()

displayQueryRes()


Tangible Queries - Dialogue DiagramTangible Queries - Dialogue Diagram


Tangible Queries - Task DiagramsTangible Queries - Task Diagrams

Slide bar

Adjust upper bound


Summary TUIMLSummary TUIML

Interaction diagram

3

TAC Palette

1 2









structure behavior

Documents

Modeling and Specifying Tangible User Interfaces