Interaction with Surfaces

AimsLast week focused on looking at

interaction with keyboard and mouse

This week◦Surface Interaction◦Gestures

Surface

What do we mean by surface?

4

Touch ScreensVery durable – only input device

to survive at Disney WorldArm fatigue can be a problem

◦Tilt the screen (lighting)◦Surface to rest arms

Types of SurfacesLots of different types:

◦Resistive – two layers when press down they collide

◦Capacitive - layer that stores electrical charge, touches the monitor with his or her finger, some of the charge is transferred to the user

◦Surface acoustic wave system - two transducers and reflector can sense when wave has been disturbed

Modelling User InteractionWould be nice if you could

calculate the time required to point at an object.

7

Fitts’ Law (1954)Predictive Model of time required

to point at an object – nice formulae

Can determine optimal location and size of buttons

Time required to complete hand movement was dependant on the distance users had to move (D)

The target Size (W)

8

Fitts’ LawIf it took 2 seconds to select a

target that was 10 cm away might expect it to double if 20 cm away to 4 seconds not the case

Increasing size of the target from 1cm2 to 2cm2 enables users to point more rapidly

Start and Stop times are constantMT = a + b log2 (D/W+1)

9

Fitts Law and usersKnow your users!Fitt’s law well established for

adultsNeeds refinement for other users

such as children and elderlyAge can effect speed and

accuracy

Study performed using mouseRepeating action ofSelecting two

targets1= Young adults2= 5 year olds3= 4 year olds

Study performed using mouseRepeating action ofSelecting two targets1= Young adults2= 5 year olds3= 4 year olds

Juan Pablo Hourcade, Benjamin B. Bederson, Allison Druin, and Francois Guimbretiere. 2004. Differences in pointing task performance between preschool children and adults using mice. ACM Trans. Comput.-Hum. Interact. 11, 4 (December 2004), 357-386

Social InteractionEven with this knowledge design

for interaction problematic when you have multi-user interaction

Example multi-player games on large displays (Video)

From GUIs to NUIsNatural User InterfaceFocus on natural way of

interacting with devicesConcept around for long timeNewton Message Pad Apple 1989Hand writing recognitionAWFUL!

13

GesturesNatural ways to interactVision recognition used to

capture hand movement and gestures in front of a white board

Widely researched area in HCIWii Accelerometer based gesture recognition

State-Transition ModelDirect touch

Idle Active

Finger lands

Finger lifted

Finger moving in air

Finger moving on device

15

Gestures

The actual shape Filtered to interpret a circle

16

Gestures Shaking – basic note

entryTilting - subdividing

note durations in the stored sequence.

Tapping - tap for lower and higher notes.

Tossing – used to pass music to another deviceWeinberg, G., Godfrey, M., and Beck, A. 2010. ZOOZbeat: mobile music recreation. In Proceedings of the 28th of the

international Conference Extended Abstracts on Human Factors in Computing Systems (Atlanta, Georgia, USA, April 10 - 15, 2010). CHI EA '10. ACM, New York, NY, 4817-4822

17

GesturesIssues

◦Training usually required◦Remembering gestures = Gulf of execution

◦Nothing happens = Gulf of evaluation Need to understand cause movement

constraints are reached

◦No real standards – Apple defined gestures ?

◦Can programme your own

18

What’s this gesture?

19

What’s this gesture?

20

GesturesSurface established gestures

21

GesturesSurface user defined gestures

What about Fat FingersUser touches

device large part of finger contacts surface (system not realise it)

Can’t see the pixels that are being targeted – finger is in the way

Video

Gulf of CompetenceGulf occurs when

learn something one way and then have to relearn it another

For example mouse to keyboard shortcuts

Mouse to gesture interaction

24

Summary GesturesHow many could a child

remember?Would they need training?Is this part of the system or one

you created?Need to discuss issues for your

assignmentOne of major input methods to

consider