Shoogle Multimodal Excitatory Interfaces on Mobile Devices John Williamson Roderick Murray-Smith...

Shoogle Multimodal Excitatory Interfaces on Mobile Devices

John Williamson Roderick Murray-Smith Stephen Hughes

“Shoogle, v., n., (c. 1724) Also shogle, schogle, shougle

1. v. intr. To sway, move unsteadily, to rock, wobble, swing

2. v. tr. To shake, joggle, to cause to totter or rock, to swing backwards and forwards”

Dictionary of the Scots Language

An Example Scenario

Would like to know if any messages have arrived

Standard alerts can be intrusive and annoying

Visual attention is limited

Solution: Simply pick up the device and shake it, hear and feel contents rattling about

For example, get a quick overview of the quantity and type of messages

Model-based Interaction

Excite – listen – feel

Reveal contents of a mobile device through shaking Active perception: output in context of input motion User driven feedback

Intuitive display of abstract data

Exploit human understanding of natural physical processes

Easy to extend interaction in natural ways Physically based but not physically limited

Features

Mobile and enclosed : no moving parts

Eyes-free : vibration and audio only

Rich feedback : communicate as densely as possible

Non-disruptive : do not interrupt

Respect privacy : do not reveal private information

Concept

Shake device to summarise contents

Device becomes physical “container” Holding “content balls”

Model-based audio and haptic display

Natural sounding audio

Inertial sensing for movement

Can be passive or active

“change in a pocket”

Sonification

Takes advantage of impact perception

Dense communication in extremely short bursts

Many attributes can be perceived from a single impact.

Mass, material, velocity...

A Video (1:58)

Sensing Hardware

Use inertial sensing

3 DoF accelerometers

Cheap

Small and internal

Rich signals

Prototypes: iPaq 5550 / MESH inertial sensor pack

Could also run on std. phones

Bluetooth SHAKE sensor

Physics: Springs, Friction and Collisions

Physics

Virtual “balls” anchored via Hooke-law springs

Subject to nonlinear frictional forces

Collide with walls, losing some energy and rebounding

Simple Euler integration model Measured accelerations incorporated directly

Audio Synthesis

Granular approach

Impact sounds triggered on wall collisions

Samples of real-world impacts

Many impacts recorded for each type to maintain variations Wood, metal, glass, water, ice, gravel, ping-pong balls...

Vibrotactile Feedback

Produced on impact

But fixed pattern

Presence, not identity

Greatly increases “solidness” offeedback

Mass decides frequency division

Large balls -> loudspeaker + motor >Mmax

Summary

Totally eyes-free interface

User driven display – works on demand

Physically compact

Intuitive and compelling to use

Come try it if you don't believe me!

Summary

Applicable in many interaction contexts:

Messages, files, memory space remaining, battery life, IM contacts online...

Several possible extensions of the interaction style

Tapping, sieving

More sophisticated linking of content to physics

Shoogle Multimodal Excitatory Interfaces on Mobile Devices

John Williamson Roderick Murray-Smith Stephen Hughes

“Shoogle, v., n., Also shogle, schogle, shougle

1. v. intr. To sway, move unsteadily, to rock, wobble, swing

2. v. tr. To shake, joggle, to cause to totter or rock, to swing backwards and forwards”

Dictionary of the Scots Language