SFWR ENG 4GC3Sensory Perception, Cognition and

Human/Computer Interfaces for Game Design

First Lecture

Course Description

� Time� Lectures: Tue, Wed, Fri 12:30–13:20� Labs: Mon 14:30–17:20 EOW

� Instructor� Andriy Pavlovych, PhD

� Pronunciation: Ahn-DREE PavlOHvych

� E-mail: andriyp@cse.yorku.ca|gmail.com

� Course Web-Page� TBA

� Notes, updates

� Textbook (tentative)� Cognitive Psychology: A Student's Handbook, 6th Edition.

by Michael Eysenck, Mark T. Keane

What the course is about

� Human sensory perception

� Learning and cognition

� Precise control and feedback mechanisms

� Alternate input devices

� Use of music and sounds

� Game aesthetics

� Critical analysis of existing interfaces

Evaluation (tentative)

� Assignments* 20%� Project 20%� Midterm 25%� Final 35%

� * – number of assignments to be determined (approx. 4)

� Missed work� Makeups of missed assignments, exams will not be possible

� Need extension?� Contact the instructor well ahead of time explaining why you

should be granted an extension

Possible Challenges

� Strong programming skills

� Some understanding of computer architecture and human perception

� Ability to pay attention to detail

� Ability to be open-minded

Human limits and capabilities

Human Performance Limits

� Sensory limits� thresholds

� deficiencies

� Responder limits

� Cognitive processing limits� reaction time (next slide)

� speed

� accuracy

� estimating

� multitasking

Sensory Limits

Senses:� Sighta

� Hearinga

� Tastea

� Smella

� Toucha

� Kinesthetic� Cutaneous� Temperature� Vestibular

a The five “classical senses”

Sensory Thresholds

Sense Detection ThresholdSight Candle flame seen 30 miles on a dark clear nightHearing Tick of a watch under quiet conditions at 20 feetTaste Teaspoon of sugar in 2 gallons (7.5 L) of waterSmell Drop of perfume diffused into a three-room

apartmentTouch Wing of a bee falling on your neck from a

distance of 1 cm

Vision Frequency Limits

Visual-Cognitive Interaction“The Stroop Effect” (Stroop, 1935)

Say the words (orcolours) in each list asquickly as possible

Record the speed andaccuracy of responses

Are the results thesame for each list?

Stroop Effect in HCI

Is there Stroop interference when entering aphone number, such as 1-800-HELLO, on atelephone keypad?

Stereogram Example (Just for fun)

Stereogram Example "shown"

Another Example

From…http://en.wikipedia.org/wiki/Stereogram

Hearing Intensity Limits

Tick of a watchunder quietconditions at 20 feet

Hearing Frequency Limits

Sound Localization

� Interaural Level Differences

� Dominate for higher frequencies (> 1600 Hz)

� Interaural Time Differences

� Dominate for lower frequencies (< 800 Hz)

� Phase difference for sound arrival

� Group Delay Differences

� Sound onset, etc. Higher frequencies

� Spectral Differences

� Shape of head, pinna + ear canal modify the sound

Kinesthesia and Proprioception

� Definition � Kinesthetic sense:

� a sense mediated by receptors located in muscles, tendons, and joints and stimulated by bodily movements and tensions; also : sensory experience derived from this sense

� Proprioception:

� the reception of stimuli produced within the organism

� Probably 3rd most important, next to vision and audition

� To control our actions, we need to know the position of body parts both before and after movements

� Kinesthesia and proprioception provide information on the position of limbs, how far they have moved, etc.

Proprioception

Responder Limits

Fitts' Law

Serial task(aka reciprocal tapping task)

Discrete task

Task Difficulty

� Fitts proposed the following "Index of

� Difficulty" for target selection tasks:

� Where

� A is the amplitude of movement

� W is the width of the target

Unit: bits

Movement Time

� proposed that the movement time (MT) to select a target is linearly related to ID:

Throughput

� Slope of the ID-MT relationship is a performance measure

� Depends on…

� Human

� Activity

� Context

E.g., mouse vs trackball vs …

Throughput in HCI

� Mouse

� Trackball

� Joystick

� Touchpad

But…Controlling for, or understanding, other effects is extremelydifficult (e.g., learning, individual differences,apparatus, procedures)

Cognitive Limits

Short Term Memory Limit

� Miller’s “magic number 7±2”?

� Experiment

� Sequence of numbers is said

� Subjects transcribe it after it’s said

� Sequence started with 4 numbers and increased to 13 numbers

Sequences

A 7 4 9 2

B 3 0 5 8 2

C 9 3 7 1 4 6

D 3 6 5 0 7 2 4

E 2 1 9 7 8 5 4 3

F 3 7 5 6 2 5 4 5 0

G 3 5 2 7 0 8 9 3 2 5

H 0 8 7 3 9 1 2 3 5 1 6

I 3 5 2 4 9 0 6 5 8 2 0 4

J 7 5 3 9 1 8 4 5 1 3 4 3 0

Results

Designing for Humans

Control-Display Relationships

Controls and Displays

� “Control”

� An input device actuated/manipulated by a human

� Examples: keyboard, mouse, joystick, button, microphone, etc

� “Display”

� An output device stimulating a human sense

� Visual display (e.g., LCD, CRT, any light)

� Auditory display (e.g., speaker)

� Tactile display (e.g., a solenoid-driven pins)

� Smell display (?)

Tactile Displays

� Sense of touch extremely important in user interfaces

� Passive tactile displays are everywhere� E.g., contour of keys on a keyboard

� Active tactile displays not as common, but still possible� E.g., vibro-tactile feedback on mobile phones

� Two more examples� Tactile feedback added to a mouse

� Tactile feedback added to a touchpad

Next few slides

Tactile Mouse

� A mouse re-engineered to include tactile feedback on the button

� Improved target selection

� Employs a solenoid-driven pin embedded in the button

http://www.yorku.ca/mack/Ergonomics.html

Tactile Touchpad

� Button clicks without separate buttons or tapping the pad surface

� Just press down (like a mouse button)

Auditory Displays

� Useful…

� For the visually challenged

� To reduce visual demand in mobile computing

� Example

� Eyes-free text entry on a touchscreen phone