The HumanIntroduction, Modeling, Vision

Human Computer Interaction, 2nd Ed.

Dix, Finlay, Abowd, and Beale

Chapter 1

Introduction

• About IT, HCI and humans …

• Interaction captured in figure below with “system, task, user”– So far, have talked about

“interface design” of system• Guidelines, principles,

evaluation, etc.

• Now will look at user– Will consider “model” of human

– Models capture elements of whatever is being modeled that are relevant to some endeavor

• E.g., “cognitive engineering”Task

“wor

k on

task

” “comm

ands”

System

User

“gives”“per

form

s”

“feedback”and its representation

(possibly manipulable)

Introduction

• On describing humans– To understate, … a question of long standing …

• One of the big questions …– Has been and can be done in various ways and at different levels of analysis

• Moral, spiritual, …., psychological, … physiological, … chemical• Reductionist

• Psychology focuses on the individual

• Many orientations through psychology’s short history– Note: Clinical/personality orientations, e.g., Freud, Jung, have their own utility in

explanation, but are rarely subject to scientific investigation– Structural, turn of 20th century, relations among elements– Gestalt, 1930’s and on, general, especially of perception, still useful– Behaviorist, all S->R, which is fine for some things, but fall short for explanation

of mental phenomena– Information processing/Cognitive, current orientation

• Influences from Shannon’s ideas on information, shortcomings of behaviorism, successes in codifying information in computing

• “Human information processor”

Modeling Humans

• Any theory or model is an abstraction

• For HCI, goals are primarily in “Computer” and “Interaction”– Utility of human model lies in how well it helps with interfaces

• Card, Moran, and Newell (1983, 1986) Model Human Processor – “Classic” example of cognitive architecture with focus on humans interacting with

computers• Perceptual system• Motor system• Cognitive system

– Each has own processor and memory– Principles of operation dictate system behavior under

certain conditions– A very simple model

• Dix et. al use similar information processing division of elements in chapter

Overview

• Dix et al. model:1. Information i/o …

• visual, auditory, haptic, movement

2. Information stored in memory• sensory, short-term, long-term

3. Information processed and applied• reasoning, problem solving, skill, error

• Idea tonight – Give broad overview of elements of human – Show that for some elements of user interface design, detailed knowledge is at

least useful and perhaps critical

• For design - sensory, perceptual, and cognitive guidelines and principles

Some Terminology

• Transduction of energy, etc. by sensory receptors

– E.g., light to neural impulses by retinal cells

Sensation Perception Cognition

• Latin – • cognitio – knowledge

• “Higher level” mental representations and procedures

• Process of thought

• Forming a “mental image” or awareness and representation

• To “see a window”

• “Top down” and “bottom up” process

• Demo, next slide

What’s below?

Demo

Model Human Processor + AttentionOverview

• A “useful” big picture - Card et al. ’83, ‘86 … plus attention– Senses/input f(attention, processing) motor/output– Notion of “processors”

• Purely an engineering abstraction

• Detail next slide– And in lectures!

Model Human Processor + AttentionDetail

• Sensory store– Rapid decay “buffer” to hold

sensory input for later processing

• Perceptual processor– Recognizes symbols, phonemes– Aided by LTM, cf. demo

• Cognitive processor– Uses recognized symbols– Makes comparisons and

decisions– Problem solving– Interacts with LTM and WM

• Motor processor– Input from cog. proc. for action– Instructs muscles– Feedback

• Results of muscles by senses

• Attention– Allocation of resources

Model Human Processor - Original

• Card et al. ‘83, ‘86

• An architecture with parameters for cognitive engineering …

– Will see visual image store, etc. tonight

• Memory properties– : Decay time: how long memory lasts– : Size: number of things stored– : Encoding: type of things stored

Model Human Processor - Original

• Memory properties– : Decay time: how

long memory lasts– : Size: number of

things stored– : Encoding: type of

things stored

Dix: Information Input and Output

• Again, human considered as information processor

• Input channels are the five senses– With some more important

than others– For hci, vision primarily

• Output channels are human effectors– E.g., limbs, fingers, head,

vocal system

Vision

Vision

• Vision and visual perception studied across a range of disciplines

• Points tonight meant to highlight usefulness for HCI in knowing about vision

• For vision consider (Dix):– Physical reception of stimulus– Processing and interpretation of stimulus

The Eye - Physical Reception

• Mechanism for receiving light and transforming it into nerve transmissions

– A transducer: light -> nerve “firings”

• Light reflects from objects– Some strikes retina– Images are focused upside-down on

retina

• Ganglion cells (brain!) detect pattern and movement

• As camera, has– equivalent of lens + aperture (pupil)– and film (retina)

• Note that image is upside down on retina!

– … perception

Environment: Visible Light

• Generally, the body’s sensory system is the way it is because it had survival value

– Led to success • survival and reproduction

• Here, focus on human vision (because of computer), but all share basic notions

• Humans have receptors for (a small part of) the electromagnetic spectrum

– Have receptors sensitive to (fire when excited by) energy 400-700nm

– Snakes “see” infrared, some insects ultraviolet

• i.e., have receptors that fire– What would life be like if humans

could see other parts of electromagnetic spectrum???

Environment: Visible Light

• Generally, the body’s sensory system is the way it is because it had survival value

– Led to success • survival and reproduction

• Here, focus on human vision (because of computer), but all share basic notions

• Humans have receptors for (a small part of) the electromagnetic spectrum

– Have receptors sensitive to (fire when excited by) energy 400-700nm

– Snakes “see” infrared, some insects ultraviolet

• i.e., have receptors that fire– What would life be like if humans

could see other parts of electromagnetic spectrum???

Human Eye: Retinal Receptors

• Two types of (photo) receptors on retina: rods and cones– Rods look like, well, rods …– Will later look at color blindness … when cones fail

• Rods:– spread all over the retinal surface (75 - 150 million)– low resolution, no color vision, but very sensitive to low light (scotopic or dim-light vision)

• Cones:– dense array around central portion of retina - fovea centralis (6 - 7 million)– high-resolution, color vision, but require brighter light (photopic or bright-light vision)

Human Eye: Lens

• Eye has compound lens: – cornea (power) and lens (adjust focal length)

f = focal length of lens

d = distance to object

r = distance to image that is formed

– Flexibility of lens changes with age, approaching 0 at 60 years

Human Eye: Depth of Field and Focus

• Depth of focus – will see interesting affect …– Distance over which objects are “in focus” without change in lens (focus)

• Note: Different colors have different depths of focus• Varies with size of pupil

– Range of focus:Distance Near Far Depth of focus 50 cm 43 cm 60 cm 17 cm 1 m 75 cm 1.5 m 75 cm 2 m 1.2m 6.0m 1.8 m 3 m 1.5m Infinity Large

– Rarely do computer systems model either depth of field or depth of focus

Human Eye: Depth of Field - Example

• Photographic Images:– Depth of field longer with small aperture (f stop)– Range of focus:

Distance Near Far Depth of focus 50 cm 43 cm 60 cm 17 cm 1 m 75 cm 1.5 m 75 cm 2 m 1.2m 6.0m 1.8 m 3 m 1.5m Infinity Large

Human Eye: Chromatic Aberration

Human Eye: Chromatic Aberration

• Different wavelengths of light focus at different distances within eye

– Short-wavelength blue light refracted more than long-wavelength red light

– Focusing on a red patch, an adjacent blue patch will be significantly out of focus

• Strong illusory depth effects

• Human eye has no correction for chromatic aberration!

– Inadvisable: fine blue patterns in visualizations!

– Visual effects in soap bubbles, crystal sculptures, etc.

Using Physiology for Design

• Chromatic Aberration

• Different wavelengths focus differently– Highly separated wavelengths (red & blue) can’t be focused simultaneously!!

• Guideline: Don’t use red-on-blue text– It looks fuzzy and hurts to read

FYI: Human Eye: Receptors

• Lens focuses image on mosaic of photoreceptor cells lining retina

• From 1 – 100 receptors feed into 1 ganglion cell

• Fovea– Small area in center of

retina densely packed with cones

– Vision sharpest– ½o - 2o of arc

Receptor mosaic in fovea

Interpreting the Visual Signal

Interpreting the Visual Signal

• Size and depth– Visual angle indicates how much of view object occupies

(relates to size and distance from eye)

• Visual acuity is ability to perceive detail (limited)– familiar objects perceived as constant size

(in spite of changes in visual angle when far away)

– cues like overlapping help perception of size and depth

• Brightness– Human system not map directly to physical

• Color– Different sensory elements responsible for perception of color

– Implications for color use, e.g., color blindness

Human Eye: Visual Angle

• Visual angle - angle subtended by object at eye of viewer– In degrees, minutes, seconds of arc

– Thumbnail at arm’s length subtends ~1o of visual angle

– 1cm (2/5”) object at 57 cm (20”), monitor distance, ~1o of visual angle

FYI:Human Eye: Acuities - Grating Acuity

• Acuities– Measurements of abilities to

see detail– Provide ultimate limits of

information densities can perceive

• Resolve to about 1 min. of arc– Roughly corresponds to with

receptor spacing in fovea– E.g., to see 2 lines as distinct

blank space between should lie on receptor

– So, should be able to perceive lines separated by twice receptor spacing!

• Superacuities– Resolution above what expected

by receptor density due to integration of signals

Grating Acuity - 1-2 minutes of arc

1 minute = 60 seconds - Ability to distinguish a pattern of bright and dark bars from a uniform gray background

FYI:Acuities: Point, Letter, Stereo, Vernier

• Point acuity– 1 minute of arc– Ability to resolve two distinct point

targets

• Letter acuity – 5 minute of arc– Ability to resolve letters– Snellen eye chart

• 20/20 means a 5-minute letter target can be seen 90% of time

• Stereo acuity– 10 seconds of arc– Ability to resolve objects in depth– Measured as difference between 2

angles (a and b) for a just-detectable difference

• Vernier acuity– 10 seconds of arc– Ability to see if two lines are collinear

FYI:Acuity Distribution and Visual Field

• Again, receptors densely packed at fovea

• Binocular overlap – Region of visual field viewed by both

eyes– Only here (not in periphery)

stereopsis

• Visual acuity non-uniformly distributed over visual field

– E.g., Can resolve only about 1/5 detail at 10o

• Next slide (tries) demonstrate “equi-resolvability” of characters as a function of distance from fovea

– r = f(dist. fovea)– (stare at center and see smaller

characters at center as well or better than those at periphery)

Human Eye: Resolution

Brightness, Luminance, Lightness

Brightness, Luminance, Lightness

• Brightness– subjective reaction to levels of light– affected by luminance (energy) of object– measured by just noticeable difference – Ecologically, need to be able to manipulate objects in environment

• Information about quantity of light, of relatively little use– Rather, what need to know about its use

• Human visual system evolved to extract surface properties– Loose information about quantity and quality of light– E.g., experience colored objects, not color light

• Color constancy

– Similarly, overall reflectance of a surface• Lightness constancy

Luminance, Brightness, Lightness

• Luminance– Amount of light (energy) coming from region of space,

• Measured as units energy / unit area• E.g., foot-candles / square ft, candelas / square m• Physical

• Brightness– Perceived amount of light coming from a source– Here, will refer to things perceived as self-luminous

• Lightness– Perceived reflectance of a surface– E.g., white surface is light, black surface is dark

• Again,– Physical - Luminance

– Number of photons coming from a region of space

– Perceptual - Brightness– Amount of light coming from a glowing source

• Lightness– Reflectance of a surface, paint shade

FYI:Luminance

• Amount of light (energy) hitting the eye• To take into account human observer:

– Weighted by the sensitivity of the photoreceptors to each wavelength• Spectral sensitivity function:

• E.g., humans about 100 times less sensitive to light at 450nm than at 510nm• Note, use of blue for detail, e.g., text, not seem good

– Compounded by chromatic aberration in which blue focuses at different point

• Later, will examine difference cone sensitivities

700

400

EVL

FYI: Brightness

• Perceived amount of light coming from a glowing (self-luminous) object– E.g., instruments

• Perceived brightness very non-linear function of the amount of light– Shine a light of some intensity on a surface, and ask an observer, “How bright?” Intensity = How bright is the point?” 1 1 4 2 16 4

• Stevens power law– Perceived sensation, S, is proportional to the stimulus intensity, I, raised to a power, n– S = I n

– Here, Brightness = Luminancen

– With n = 0.333 for patches of light, 0.5 for points– Applies only to lights in relative isolation in dark, so application more complicated

• Applies to many other perceptual channels– Loudness (dB), smell, taste, heaviness, force, friction, touch, etc.

• Enables high sensitivity at low levels without saturation at high levels• Just-noticeable difference depends on value

Dix: Interpreting the Signal - Color

• Color– made up of hue, intensity, saturation– cones sensitive to color wavelengths– blue acuity is lowest– ~8% males and ~1% females color blind

Trichromacy Theory

• Recall, that there are 2 types of retinal receptors:– Rods, low light, monochrome

• So overstimulated at all but low levels contribute little

• So only consider cones for color vision

– Cones, high light, color– Not evenly distributed on retina

Distribution of receptors across the retina, left eye shown; the cones are concentrated in the fovea,

which is ringed by a dense concentration of rods http://www.handprint.com/HP/WCL/color1.html#oppmodel

Trichromacy Theory• Cones (3 types) differentially

sensitive to wavelengths– “trichromacy”

• Each type cone has different peak sensitivity:– S: 450 nm “blue”– M: 540 nm “green”– L: 580 nm “red”– More later

– No accident 3 colors in monitor

• Color space:– An arrangement of colors in a 3-

dimensional space• Monitor: R,G,B• Primary paint colors: R,Y,B• Printer: cyan, magenta, yellow

– There are many, each designed for different purposes

– Will consider several– Can match all colors perceived with 3 colors

• Does not matter that spectral composition of that patch of light may be completely different

– But, chickens have 12 …– Different gamut, more later

Cone sensitivity functions

Cone response space, defined byresponse of each of the three conetypes. Becomes 2d with color deficiency

FYI:Color Space Ex.: RGB Color Cube

• Again, can specify color with 3– Will see other way

• RGB Color Cube– Neutral Gradient Line– Edge of Saturated Hues– ppt example

http://graphics.csail.mit.edu/classes/6.837/F01/Lecture02/ http://www.photo.net/photo/edscott/vis00020.htm

Cone Sensitivity Functions

• Cone receptors least sensitive to (least output for) to blue

Relative sensitivity curves for the three types of cones, log vertical scale, cone spectral curves from Vos & Walraven, 1974

Relative sensitivity curves for the three types of cones, the Vos & Walraven curves on a normal vertical scale

Color Blindness

• ~8% male, and ~1% females have some form of color vision deficiency!

• Most common:– Lack of long wave length sensitive

receptors (red, protanopia) • See figure at right bottom

– Lack of mid wave length receptors (green, deuteranopia)

• Results in inability to distinguish red and green

• E.g., cherries in 1st figure hard to see

• Trichromatic vs. dichromatic vision

Color Blindness Examples

• Normal:– trichromatic

• No red, green, blue– dichromatic

Using Physiology for DesignAn example guideline based on physiology

• Fovea has few blue (short wavelength) cones– Can’t resolve small blue features (unless they have high contrast with

background)

• Lens and aqueous humor turn yellow with age– Blue wavelengths are filtered out

• Lens weakens with age– Blue is harder to focus

• Guideline: don’t use blue against dark backgrounds where small details matter (like text!)

FYI: Perceived Color

• Color perceived relative to context– Are the “X”s in the figure below the same color?

FYI: Perceived Color

• Color perceived relative to context– Are the “X”s in the figure below the same color?

– Easy implications for use in maps

• Contrast illusion – An illusion is an extreme case

• Somewhat “surprising” because it leads to error

– Appears to be different color X!

FYI: Perceived Color

• With color of x touching …

FYI: Afterimage

• Occurs due to bleaching of photopigments– (demo next slide)

• Implications for misperceiving (especially contiguous colors – and black and white)– “I thought I saw …”

• To illustrate:– On next slide

– Stare at + sign on left• May see colors around circle

– Move gaze to right– See yellow and desaturated red

Afterimage Example

Afterimage Example

Dix: Interpreting the signal (continued)

• The visual system compensates for:– movement– changes in luminance

• Context is used to resolve ambiguity

• Optical illusions sometimes occur due to over compensation

FYI: Simultaneous Brightness Contrast

• Gray patch on a dark background looks lighter than the same patch on a light background

– Predicted by DOG model of concentric opponent receptive fields

• Again, illusion, system overcompensates for context

FYI: Mach Bands

• Like, “light stripes” where two areas meet

• At point where uniform area meets a luminance ramp, bright band is perceived – Said another way, appear where abrupt change in first derivative of

brightness profile– Simulated by DOG model– Particularly a problem for uniformly shaded polygons in computer graphics

• Hence, various methods of smoothing are applied

Ernst Mach

FYI: Mach Bands

• At point where uniform area meets a luminance ramp, bright band is perceived – Said another way, appear where abrupt change in first derivative of

brightness profile– Simulated by DOG model– Particularly a problem for uniformly shaded polygons in computer graphics

• Hence, various methods of smoothing are applied

Ernst Mach

Psychology, Physiology and Design of Interactive Systems

• Some direct applications– e.g. blue acuity is poor

blue should not be used for important detail

• However, correct application generally requires understanding of context in psychology, and an understanding of particular experimental conditions

• A lot of knowledge has been distilled in– guidelines – cognitive models – experimental and analytic evaluation techniques

End ?

Optical Illusions

the Ponzo illusion the Mueller-Lyer illusion

Optical Illusions

Preattentive Processing and Illusions

• What is wrong with this triangle? – Impossible (or at least

difficult) to build

• Cues for perception misleading– Must rely on conscious

(rational) processes intelligence to figure it out,

– Conscious/rational processes much slower

The Other Senses …

• But not smell and taste– Long latency, practical challenges

• Hearing

• Touch

• Kinesthetic – Movement– Fitt’s law

Hearing

• Provides information about environment:distances, directions, objects etc.

• Physical apparatus:– outer ear – protects inner and amplifies sound– middle ear – transmits sound waves as

vibrations to inner ear– inner ear – chemical transmitters are released

and cause impulses in auditory nerve

• Sound– pitch – sound frequency– loudness – amplitude– timbre – type or quality

Hearing (cont)

• Humans can hear frequencies from 20Hz to 15kHz

– less accurate distinguishing high frequencies than low.

• Auditory system filters sounds– can attend to sounds over background noise. – for example, the cocktail party phenomenon.

Touch

• Provides important feedback about environment

• May be key sense for someone who is visually impaired

• Stimulus received via receptors in the skin– thermoreceptors – heat and cold– nociceptors – pain– mechanoreceptors – pressure

(some instant, some continuous)

• Some areas more sensitive than others, e.g., fingers

• Kinethesis - awareness of body position – affects comfort and performance.

Movement

• Time taken to respond to stimulus:reaction time + movement time

• Movement time dependent on age, fitness etc.

• Reaction time - dependent on stimulus type:– visual ~ 200ms– auditory ~ 150 ms– pain ~ 700ms

• Increasing reaction time decreases accuracy in the unskilled operator but not in the skilled operator.

Movement – Fitt’s Law

• Fitts' Law, 1954– Demo: http://www.tele-actor.net/fitts/

Movement – Fitt’s Law

• Fitts' Law, 1954–Fundamental law of human sensory-motor system–Practical application in interface design–Describes the time taken to hit a screen target

–Time, Mt, to move hand to a target of size, S, at distance, D, away

Mt = a + b log2(D/S + 1)

where: a and b are empirically determined constantsMt is movement timeD is Distance S is Size of target

– “index of difficulty”: log2(D/S + 1)• Same performance at greater distance with greater size

targets as large as possible, distances as small as possible

Fitt’s Law – Example Implication

• Hierarchical menus are hard to hit– Especially when it takes two actions …

Psychology and Design of Interactive Systems

• Some direct applications– e.g. blue acuity is poor

blue should not be used for important detail

• However, correct application generally requires understanding of context in psychology, and an understanding of particular experimental conditions

• A lot of knowledge has been distilled in– guidelines – cognitive models – experimental and analytic evaluation techniques

End