User Interface Engineering HS 2013 - AIT Lab · Research perspective Overview of own research as motivation 29.09.2013 2 Last Week ETH Zürich –Departement Computer Science User

||

Input Fundamentals

29.09.2013 1

User Interface Engineering – HS 2013

ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges

||

Brief Overview of HCI as a discipline

History of the UI

Product perspective

Research perspective

Overview of own research as motivation

29.09.2013 2

Last Week


Design

PrototypeEvaluation

||

Admin

Input fundamentals

Buttons, switches

Keyboards

Mice

Basics of human capabilities and performance

29.09.2013 3

Overview


|| 29.09.2013 4

Exercise location (for hands on work)


CNB D101

|| 29.09.2013 5ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges

https://www1.ethz.ch/immobilien/gmis/do_enter

• Lab access

• Predefined hours

(more details in

today’s exercise)

• Electronic building

access

• Download form

• Print, sign and return

• Barbara von Allmen

• Me

• Matthias Kovatsch

• CNB H floor

https://www1.ethz.ch/immobilien/gmis/do_enter

||

Date still tba – We will communicate this before the next lecture.

Will be roughly same date as last homework assignment's due date (30.10.)

29.09.2013 6

Midterm exam date


||

In-Class vote

Possible dates:

Last lecture: 18.12.13

After Christmas: 15.1.14

During exam period: 22.1.14

29.09.2013 7

Final Project Date?


|| 29.09.2013 8

Today




|| 29.09.2013 11

Bop It – What’s in it?


|| 29.09.2013 12

Bop It – What’s in it?



https://www.sparkfun.com/products/9136

https://www.sparkfun.com/products/9136

||

Buttons are everywhere

29.09.2013 14

Buttons & Switches


Especially on interactive devices

||

What’s in a Switch?

29.09.2013 15ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges

|| 29.09.2013 16

Switches


+

-

Interrupter Switch

|| 29.09.2013 17

Switches


+

-

Two way diverter switch

||

Number of

Poles

Throws

Default Behavior for Push Buttons

Normally Open (NO)

Normally Closed (NC)

Duration of contact

Long-term (Switch)

Momentary (Button)

29.09.2013 18

Types of Switches


|| 29.09.2013 19

Magnetic Reed Switch


|| 29.09.2013 20

Mercury Tilt-Over Switch


||

Designing with Switches - Essentials


|| 29.09.2013 22

An interactive system – Conceptual Overview


Mechanical

elements

Enclosure

Circuits Microcontroller Host-PC

Sensors

Actuators

ADC

DAC

|| 29.09.2013 23

Microcontroller


Atmel ATmega328

Used in Arduino and other electronics

frameworks

|| 29.09.2013 24

Arduino Board


|| 29.09.2013 25

Arduino Pin Mapping


|| 29.09.2013 26

Interactivity


D

LED

1K

Output 12

5V

SP

ST

Input 13

10K


/* * Switch and LED test program */

int ledPin = 12; // LED is connected to pin 12

int switchPin = 13; // switch is connected to pin 13

int val; // variable for reading the pin status

void setup()

pinMode(ledPin, OUTPUT); // Set the LED pin as output

pinMode(switchPin, INPUT); // Set the switch pin as input

void loop()

val = digitalRead(switchPin); // read input value and store it in val

If (val == LOW) // check if the button is pressed

digitalWrite(ledPin, HIGH); // turn LED on

if (val == HIGH) // check if the button is not pressed

digitalWrite(ledPin, LOW); // turn LED off



|| 29.09.2013 30

Key cap construction


Key Switch Membrane

Conductive Layer

Conductive Layer

Spacer Non Conductive

|| 29.09.2013 31

Key cap construction


Key Switch Membrane

Conductive Layer

Conductive Layer

Spacer Non Conductive

||

IBM compatible keyboard has 104(+) keys

To expensive to read each key individually

Depending on controller 20-100 IO pins

Needed for many different tasks

Solution:

Connect buttons in matrix fashion

Keyboard: 16x8 matrix requires 24 pins

29.09.2013 32

Keyboard Switch Matrix



||

Rows & Columns connected to

microcontroller

Controller drives circuitry and generates key

events (key codes)

Each row and column pin

Can be used as in- or output pin

Output pins are driven (low / high)

Input pins are read (low / high)

29.09.2013 34

Key Matrix


A B

C D

Mic

rocontr

olle

r

||

Drive rows, read columns

Drive columns, read rows

29.09.2013 35

Single Key


A B

E F

C D

G H

I J

M N

K L

O P

||

Drive columns sequentially

Read rows simultaneously

29.09.2013 36

Multiple Keys


A B

E F

C D

G H

I J

M N

K L

O P

||

Certain constellations may lead to erroneous key press/release events

“C” is detected – pressed or not

Release of “B” can’t be detected

29.09.2013 37

Multiple Keys - Ghosting and Masking


A B

C D

A B

D

||

Diodes restrict flow direction of current

29.09.2013 38

Eliminating Ghosting and Masking


D



||

Many other devices (can) produce binary events just like buttons

Proximity detector

Touch sensor

Light barrier

…

29.09.2013 41

Things that act like switches



||

Optoelectronic sensor

essentially a super fast low-res camera

Typically 18x18 pixel

Sometimes >3000Hz (gaming mice)

LED at grazing angle

Onboard microprocessor computes 2D

coordinates

29.09.2013 43

Optical Mice


||

Mouse sensor operates under very

constraint circumstances

Translation only

Fixed illumination

Simple flow methods are sufficient

29.09.2013 44

Optical Flow


||

Cross-Correlation

Computes similarity between signals (offset in

some dimension(s))

CC-based Optical Flow computation

Shift frame t0 relative to frame t1

n times in all directions

Highest correlation score signifies motion

direction

29.09.2013 45

Onboard Optical Flow


||

Implemented on ASIC

Integrates imaging sensor and processor

Compute optical flow via correlation

Store previous frame

(Row- and column-wise) sums of XORs of

single pixels

Integrate decisions to track movement

Massive parallel implementation

Very fast in hardware

29.09.2013 46

Optical Flow in Hardware


||

Uses infrared laser diode rather than LED

for illumination.

Exploit laser speckle

Pattern produced by mutual interference of

several wavefronts of the same wavelength

but different phase and amplitudes

Observable when coherent lightsource (i.e.

laser) is shone onto a surface (each point on

the surface acts as secondary lightsource)

Increases observable structure in the

environment

Allows for much higher sensing fidelity and

faster update rate

Works on more surfaces (even glass)

29.09.2013 47

Laser Optical Mice





IR Emitter

Photo Diode


Forward / Backward rotation is ambiguous!

|| 29.09.2013 52

Quadrature Encoding


Clockwise

Rotation

Phase A B

1 0 0

2 0 1

3 1 1

4 1 0

C-Clockwise

Rotation

Phase A B

1 1 0

2 1 1

3 0 1

4 0 0

||

15 Minute Break


||

Continuous Input


||

Convert variable resistance to measurable signal

Resistance-varying and fixed resistor in series,

measure voltage between them

No discrete event

Polling required

ADC necessary

29.09.2013 55

Voltage divider


𝑉𝑂𝑢𝑡 = 𝑉𝑠𝑢𝑝𝑝𝑙𝑦 ×𝑅2

𝑅1 + 𝑅2

||

Resistive element

Strip or arc

Sliding contact (Wiper)

Electrical terminals at each end of resistive element

Electrical terminal at the wiper

29.09.2013 56

Example: Potentiometer


R

|| 29.09.2013 57

Example: Rotary Encoder


||

Force Sensing Resistor

29.09.2013 58

Further examples


http://www.openmusiclabs.com/learning/sensors/fsr/

http://www.openmusiclabs.com/learning/sensors/fsr/


||

Resistance straight: 10kΩ

Resistance bent: 40kΩ

29.09.2013 60

Further examples: Flex sensor




|| 29.09.2013 63

Basic Human Cognition and Perception


||

How fast / accurate can we perform a task?

Depends on relatively stable and well

understood human capabilities

Can be modelled using basic cognitive

processes

Allows us to predict task completion time

Can complement / replace usability studies

Allows to compare different design choices

29.09.2013 64

Human Cognition and Targeting Performance


||

Developed by S. Card, T. Moran, A. Newell (The Psychology of Human-

Computer Interaction, 1983)

Basic model that gives us an abstracted understanding of

Perception

Memory

Motor system

Can be applied to estimate:

Execution time

Error rates

Training effects for simple stimulation / reaction interactions

29.09.2013 65

Model Human Processor


||

Three processors with associated memory

Perceptual system

Sensors and buffers

Cognitive system

Working memory content symbolically coded

Cognitive processes

Motor system

Movements

Overall ‘runtime’ determines task completion

time

29.09.2013 66

Model Human Processor - Overview


29.09.2013ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges 67

29.09.2013ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges 68

||

Receives sensor signals and stores in buffers

One buffer for each sensory channel

Perception time 𝜏𝑝 ≈ 100 𝑚𝑠

Range from 50 − 200 𝑚𝑠

Bloch’s Law: 𝑅 = 𝐼 ∙ 𝑡 𝑅 is response, 𝐼 is Intensity and 𝑡 is exposure time

𝑅 is constant for 𝑡 < 100 𝑚𝑠

Explains why movies appear as fluid animation

29.09.2013 69

Perceptual processor


||

Operates on chunks of information

Processing time 𝜏𝐶 ≈ 70 𝑚𝑠

Short Term Memory as working memory

Conscious control of cognitive processes

Serial “Tunnel of consciousness”

Examples: multiply, read

Automatic cognitive processes

Parallel

Example: automatic visual searching

Repetition and building of connections stores

information in long-term memory

29.09.2013 70

Cognitive processor


||

0446321111

29.09.2013 71

Chunking Example


vs

044 632 11 11

||

Short-term storage of limited data

Maximum capacity: 7 [5,9] units, duration: 2 sec

Typical capacity: 3 [2,4] units, duration: 7 sec

Storage capacity & duration depend on cognitive load

Even modest cognitive load increases error rate

STM limited by number but not complexity of units

Chunking (grouping of symbols)

29.09.2013 72

Cognitive processor


|| 29.09.2013 73

Long-term Memory


Memory of the past

Declarative memory

Associative network of representations of facts

Episodic memory: experienced events

Semantic memory: facts, knowledge

Procedural memory

Operative knowledge, typically not conscious

Skills (type writing, piano playing, skiing)

Memory capacity practically unlimited

Problem is organization for retrieval

Duration depends on quality and intensity of elaboration

Fast read, slow write => commiting to LTM maxes out at 7s / chunk


|| 29.09.2013 75

Motor System (Processing time)


Experiment: Move pen between lines as fast as possible for 5 sec

Open loop

Without perceptual control

ca. 70 pen reversals in 5 sec

𝜏𝑀 ≈ 70 𝑚𝑠

Closed loop

Perceptual system controls movement

ca. 20 corrections in 5 sec

Processing time

𝜏𝑃 + 𝜏𝐶 + 𝜏𝑀 ≈ 240 𝑚𝑠

|| 29.09.2013 76

Motor System: Fitt’s Law


Directed movement as an information processing task

Not limited by muscles, but by ability to process sensory input

Predict time to hit a target (to press a button) as function of distance and size

Robust performance model (Fitts, 1954): 𝑇𝑝𝑜𝑠 = 𝐼𝑀 ∙ 𝐼𝐷

Index of difficulty 𝐼𝐷 = 𝑙𝑜𝑔2(2𝐷

𝑊)

Index of Movement 𝐼𝑀 = 100 ms / bit (approx. constant)

W

D

|| 29.09.2013 77

Fitts’ Law: Experiment


4 cm

16 cm

8 cm

For each target width & distance:

tap for 10s, count taps afterwards

||

Doubling the distance adds roughly a constant to execution time

Logarithmic nature of the law

Doubling the target width (within limits) is roughly equal to halving the distance

Clear by D/W term in the formulation

29.09.2013 78

Fitts’ Law Summary


|| 29.09.2013 79

Interpreting Results


Draw graph with 𝐼𝐷 values on the x-axis and average 𝐼𝑀 values on the y-axis

Perform a linear regression (e.g., spreadsheet program)

Fixed distance D

Varying width W

Slope = 𝐼𝑀

MT = -0.4595 + 0.8092 ID

R2 = 0.93

0

0.5

1

1.5

2

2.5

3

3.5

4

0 1 2 3 4 5 6

ID

MT

[sec]

||

Shannon’s Formulation:

𝑇𝑝𝑜𝑠 = 𝑎 + 𝑏 ∗ log2𝐷

𝑊+ 1

a,b depend on device, need to be empirically derived

Use 𝑎 = 0, 𝑏 = 𝐼𝑀 = 100 𝑚𝑠 for quick and dirty estimates

Improved curve fit

And?

29.09.2013 80

Improvements


||

Find uses of Fitts’ Law in modern software design.

Email screenshot and a one paragraph explanation

on how Fitts’ Law (and Shannon’s Formulation in

particular) might have influenced design decisions.

We will discuss your examples next week.

29.09.2013 81

Homework (not for credit)


|| 29.09.2013 82

Summary


Overview of input devices built from simple buttons and

switches

Discrete (Game controller buttons, Keyboards)

Analog(-ish) (Mouse-Wheel, Data Glove)

Working principle of several fundamental input devices

Mouse

Keyboard

Basic model of human cognitive processes and pointing

performance

|| 29.09.2013 83

Next Week


||

Scherz, Paul (2006). Practical Electronics for Inventors. McGraw-Hill

http://www.andyholtin.com/links/Practical_Electronics_for_Inventors.pdf

Tucker, Allen B. (2004). Computer Science Handbook (Fundamental Input and Output Devices).

Chapman and Hall. 2nd Ed.

Lyon, Richard F. (1981). The Optical Mouse and an Architectural Methodology for Smart Digital

Sensors. Xerox PARC TechReport

Card, S.K; Moran, T. P; and Newell, A. The Model Human Processor: An Engineering Model of

Human Performance. In K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of Perception

and Human Performance. Vol. 2: Cognitive Processes and Performance, 1986, pages 1–35.

29.09.2013 84

Reading suggestions


http://www.andyholtin.com/links/Practical_Electronics_for_Inventors.pdf

Documents

User Interface Engineering HS 2013 - AIT Lab · Research perspective Overview of own research as motivation 29.09.2013 2 Last Week ETH Zürich –Departement Computer Science User