1

Psy280: PerceptionPsy280: Perception

Prof. AndersonProf. AndersonDepartment of PsychologyDepartment of Psychology

Audition 1 & 2Audition 1 & 2

2

Hearing: WhatHearing: What’’s it good for?s it good for?

nn Remote sensingRemote sensingnn Not restricted like visual fieldNot restricted like visual fieldnn Can sense object not visibleCan sense object not visible

3

Hearing: The sound of silenceHearing: The sound of silencenn A tree in the forestA tree in the forest

nn Physical signal but no perceptionPhysical signal but no perceptionnn One hand clappingOne hand clapping

nn No physical signal, no perceptionNo physical signal, no perceptionnn Separate physical quantity from perceptualSeparate physical quantity from perceptual

qualityqualitynn Sound is the perceptual correlate of the physicalSound is the perceptual correlate of the physical

changes in air pressurechanges in air pressurenn Or water pressure when under waterOr water pressure when under water

nn John CageJohn Cage’’s 4:33 No. 2, 1962s 4:33 No. 2, 1962

4

What are the physical attributesWhat are the physical attributesassociated with sound?associated with sound?

nn LoudnessLoudnessnn Amplitude or height of pressure waveAmplitude or height of pressure wave

nn PitchPitchnn Frequency of times per second (Hz) a pressure waveFrequency of times per second (Hz) a pressure wave

repeats itselfrepeats itself

5

What is sound quality?What is sound quality?nn Pure tonesPure tones

nn Single frequency (f)Single frequency (f)nn Rarely exist in real worldRarely exist in real world

nn Complex tonesComplex tonesnn More than one fMore than one fnn Due to resonanceDue to resonancenn Air pressure causes reverberationsAir pressure causes reverberations

nn E.g., tuning forksE.g., tuning forksnn E.g., Plucking the A string on a guitarE.g., Plucking the A string on a guitar

nn Fundamental frequency 440 Hz (cycles/s)Fundamental frequency 440 Hz (cycles/s)nn HarmonicsHarmonics

nn Reverberations at multiples of the fundamentalReverberations at multiples of the fundamentalnn E.g., 880, 1320E.g., 880, 1320nn Creates fullness of complex soundsCreates fullness of complex sounds

nn Timbre is the relative amplification of harmonicsTimbre is the relative amplification of harmonics

6

The human earThe human earnn Outer earOuter ear

nn Focusing of soundFocusing of soundnn Resonance amplifies 2000-Resonance amplifies 2000-

5000 Hz range5000 Hz rangenn Converts from air toConverts from air to

mechanical vibrationmechanical vibrationnn Middle earMiddle ear

nn AmplificationAmplificationnn Fluid denser than airFluid denser than airnn Focus vibrations ontoFocus vibrations onto

stapes/oval windowstapes/oval windownn Increased leverage fromIncreased leverage from

ossiclesossiclesnn Inner earInner ear

nn Sensory transductionSensory transductionnn Physical to neural energyPhysical to neural energy

nn Fluid pressure changesFluid pressure changesnn Bending of hair cellsBending of hair cells

7

Auditory sensory transduction:Auditory sensory transduction:The inner earThe inner ear

nn CochleaCochleann Coiled and liquid filledCoiled and liquid filled

nn 3 layers3 layersnn Cochlear partitionCochlear partition

nn Contains organ of cortiContains organ of cortinn Organ of cortiOrgan of corti

nn Cilia (hair) cellsCilia (hair) cellsnn Between basilar andBetween basilar and

tectorial membranestectorial membranesnn Transduction Transduction

nn Movement of ciliaMovement of ciliabetween membranesbetween membranes

8

Auditory transductionAuditory transduction

nn BendingBending——>physical energy>physical energynn Converted to neural signalsConverted to neural signals

nn Bend one direction Bend one direction ——> depolarization> depolarizationnn More likely to fire APMore likely to fire AP

nn Other direction Other direction ——> > hyperpolarizationhyperpolarizationnn Less likely to fire APLess likely to fire AP

9

Auditory pathwaysAuditory pathways

10

Audition: What and whereAudition: What and where

nn What is it?What is it?nn *Pitch*Pitchnn IdentificationIdentification

nn Surprisingly, little isSurprisingly, little isknown beyond speechknown beyond speech

nn Where is it?Where is it?nn *location*location

11

What: PitchWhat: Pitch

nn How does neural firing signal differentHow does neural firing signal differentpitches?pitches?nn 1) Timing codes1) Timing codesnn 2) Place codes2) Place codes

12

Pitch: Temporal codingPitch: Temporal codingnn Idea: Diff fIdea: Diff f’’s signaleds signaled

by rate of neuronalby rate of neuronalfiringfiring

nn Hair cell responseHair cell responsenn Bend one direction Bend one direction ——>>

depolarizationdepolarizationnn Other direction Other direction ——>>

hyperpolarizationhyperpolarizationnn Result?Result?

nn Bursting pattern ofBursting pattern ofneural response relatedneural response relatedto frequency ofto frequency ofoscillationoscillation

13

Problems with temporal codingProblems with temporal codingnn Problem: A single neuron canProblem: A single neuron can’’t fire at the rate necessary tot fire at the rate necessary to

represent higher f tonesrepresent higher f tonesnn E.g., 1000-20,000 Hz (i.e., 1000-20000 per second)E.g., 1000-20,000 Hz (i.e., 1000-20000 per second)nn Max neuron firing rate: 500-800 per secondMax neuron firing rate: 500-800 per second

nn Solution: volley principleSolution: volley principlenn No single neuron represents fNo single neuron represents fnn Coding across many neurons with staggered firing ratesCoding across many neurons with staggered firing rates

nn Evidence: Phase lockingEvidence: Phase lockingnn Diff neurons respond toDiff neurons respond todiff peaksdiff peaksnn Not every peakNot every peaknn Pool across multiple neurons toPool across multiple neurons torepresent high frepresent high f’’ss

14

Pitch: Place codingPitch: Place codingnn Related to doctrine of specificRelated to doctrine of specific

nerve energiesnerve energiesnn What is pitch?What is pitch?

nn Activation of different places inActivation of different places inauditory systemauditory system

nn Frequency specificFrequency specificnn TonotopyTonotopy

nn CochlearCochlearnn BrainstemBrainstemnn CorticalCortical

nn Stimulate these regionsStimulate these regionsnn Should result in pitchShould result in pitch

perceptionperception

Owlbrainstem

Human auditory cortex

15

Place coding starts in cochleaPlace coding starts in cochleann Von Von Bekesy Bekesy studiedstudied

basilar membrane inbasilar membrane incadaverscadaversnn Base more narrow andBase more narrow and

stifferstiffernn Apex wider and moreApex wider and more

flexibleflexiblenn Observed traveling wavesObserved traveling wavesnn Diff frequencies (f) resultDiff frequencies (f) result

in waves w/ diffin waves w/ diffenvelopesenvelopesnn Higher f: Peak closer to baseHigher f: Peak closer to basenn Lower f: Peak closer to apexLower f: Peak closer to apex

nn Thus, f related to Thus, f related to ““placeplace””where peak fluctuationwhere peak fluctuationoccursoccurs

16

Frequency tuning:Frequency tuning:Neural place codingNeural place coding

nn Tonotopic arrangement of hair cell nervesTonotopic arrangement of hair cell nervesnn Diff nerves innervate diff parts of basilar membraneDiff nerves innervate diff parts of basilar membrane

nn Allows for Allows for ““placeplace”” code for frequency code for frequency

Frequency tuning curves ofsingle hair cells

17

Complex tones:Complex tones:Fourier decompositionFourier decomposition

nn BasilarBasilarmembrane actsmembrane actsas f analyzeras f analyzer

nn Breaks downBreaks downcomplex f inputscomplex f inputsinto constituentinto constituentpure tonepure tonecomponentscomponents

18

Auditory masking: EvidenceAuditory masking: Evidencefor cochlear place codingfor cochlear place coding

nn Auditory maskingAuditory maskingnn Presence of certain tonesPresence of certain tones

decreases perception ofdecreases perception ofnearby tonesnearby tones

nn Similar f result in greaterSimilar f result in greatermaskingmasking

nn Asymmetry in spread ofAsymmetry in spread ofmaskingmaskingnn Consistent with basilarConsistent with basilar

vibrational overlapvibrational overlapnn E.g. 400 Hz mask overlapsE.g. 400 Hz mask overlaps

more with 800 than 200 Hzmore with 800 than 200 Hz

400 Hz maskIncreases thresholdfor 800 more than200 Hz

19

Mystery of the missingMystery of the missingfundamentalfundamental

nn 400 Hz fundamental plus harmonics (800,400 Hz fundamental plus harmonics (800,1200, 1600, 2000)1200, 1600, 2000)nn Sounds like 400 Hz pitch with complexSounds like 400 Hz pitch with complex

timbretimbrenn What if remove fundamental f (400Hz)?What if remove fundamental f (400Hz)?

nn Perceived pitch doesnPerceived pitch doesn’’t change!t change!nn Hence: The missing fundamentalHence: The missing fundamental

nn Problem for place codingProblem for place codingnn No direct stimulation of 400 Hz on basilarNo direct stimulation of 400 Hz on basilar

membranemembrane

f

nn Harmonic structure determines perceived pitch Harmonic structure determines perceived pitchnn Not what is present on basilar membraneNot what is present on basilar membranenn What we hear is not what the basilar membrane tell us, but whatWhat we hear is not what the basilar membrane tell us, but whatour brain doesour brain does

20

What does Barry White soundWhat does Barry White soundlike on the telephone?like on the telephone?

nn Telephone carries 300-Telephone carries 300-3400Hz3400Hz

nn Typical male voiceTypical male voicenn Fundamental f = 120 HzFundamental f = 120 Hz

nn Barry whiteBarry whitenn 30 Hz?30 Hz?

nn CanCan’’t speak to Barry on thet speak to Barry on thetelephone?telephone?

nn Missing fundamentalMissing fundamentalallows us to hear allows us to hear ““virtualvirtual””pitch of voicepitch of voice

21

If its too loud your too oldIf its too loud your too oldnn Db (SPL) scaleDb (SPL) scale

nn Loudness doubles aboutLoudness doubles aboutevery 10 db at 1000 Hzevery 10 db at 1000 Hz

nn Audibility curvesAudibility curvesnn Loudness varies with fLoudness varies with f

nn Low volumeLow volumenn Attenuated low and high fAttenuated low and high f

relative to midrangerelative to midrangenn High volumeHigh volume

nn Less frequency attenuationLess frequency attenuationnn Low volume soundsLow volume sounds

muddymuddynn Mostly mid rangeMostly mid range

nn I like my music loudI like my music loud

Pain and pleasure

Each curve represents equalloudness

22

Otoacoustic emissions:Otoacoustic emissions:Talking earsTalking ears

nn Ears donEars don’’t only receive sounds, they maket only receive sounds, they makethem!them!nn Discovered in 1978Discovered in 1978nn Tiny microphonesTiny microphones

nn Occur spontaneously and also in response toOccur spontaneously and also in response tosoundsoundnn It like your ears are talking back!It like your ears are talking back!

nn Created by movement of outer hair cells (ohc)Created by movement of outer hair cells (ohc)nn Part of auditory sensitivity is movement of ohc toPart of auditory sensitivity is movement of ohc to

change change region specific flexibility of basilar membraneregion specific flexibility of basilar membranenn Allows tuning curves to be so narrowAllows tuning curves to be so narrow

nn Hearing impairments often start with loss ofHearing impairments often start with loss ofohc functionohc function

23

Auditory localizationAuditory localizationnn Where is the sound coming from?Where is the sound coming from?

nn DistanceDistancenn Elevation (vertical)Elevation (vertical)nn Azimuth (horizontal)Azimuth (horizontal)

nn Localization not nearly as precise as visionLocalization not nearly as precise as visionnn Localization within 2-3.5 degrees in front of headLocalization within 2-3.5 degrees in front of headnn 20 degrees behind head20 degrees behind headnn Suggests important role of visionSuggests important role of vision

nn Tunes auditory localizationTunes auditory localization

24

Why is is auditory localizationWhy is is auditory localizationnot obvious?not obvious?

nn VisionVisionnn Stimulate different photoreceptors in eyeStimulate different photoreceptors in eye

nn AuditionAuditionnn No such separation of sounds sources onNo such separation of sounds sources on

sensory surfacesensory surfacenn Sources combine to equally stimulate earSources combine to equally stimulate ear

receptorsreceptors

25

Why have two ears?Why have two ears?

nn Two aural perspectives on the worldTwo aural perspectives on the world

nn Like vision, can be used to get differentLike vision, can be used to get differentsound pictures of environmentsound pictures of environment

nn Binaural cuesBinaural cuesnn The disparities between ears is used forThe disparities between ears is used for

localizationlocalization

26

AzimuthAzimuthnn Interaural (between ears) TimeInteraural (between ears) Time

Difference (ITD)Difference (ITD)nn Air pressure changes are very slowAir pressure changes are very slow

relative to speed of lightrelative to speed of lightnn ITD at side = max 600 ITD at side = max 600 µµSSnn ITD at front = 0ITD at front = 0nn Can induce perception of location byCan induce perception of location by

varying ITD using headphonesvarying ITD using headphonesnn Interaural Level (intensity) DifferenceInteraural Level (intensity) Difference

(ILD)(ILD)nn Amplitude decreases w/ distanceAmplitude decreases w/ distancenn Head casts sound/acoustic shadowHead casts sound/acoustic shadow

nn Reduced amplitude due to reflectionReduced amplitude due to reflectionnn Measure w/ tiny microphonesMeasure w/ tiny microphonesnn f dependentf dependent

nn Greater shadow for higher fGreater shadow for higher f

27

ElevationElevationnn ITD/ILD not very usefulITD/ILD not very usefulnn Use spectral cuesUse spectral cuesnn Frequency informationFrequency information

can result in differentcan result in differentperceptual qualiaperceptual qualiann Monaural: f serves asMonaural: f serves as

signal for pitchsignal for pitchnn Binaural: f serves as signalBinaural: f serves as signal

for locationfor locationnn Pinna differentiallyPinna differentially

absorb fabsorb fnn Result: Notches inResult: Notches in

frequency spectrafrequency spectra

Above

Level

Below

28

DistanceDistancenn At close distances (< 1 meter)At close distances (< 1 meter)

nn ILD can discriminate near and farILD can discriminate near and farnn At very close distances ILD is very large (e.g. 20 Db)At very close distances ILD is very large (e.g. 20 Db)

nn But whatBut what’’s that going to do for us?s that going to do for us?nn At far distancesAt far distances

nn We are very poor judges for unfamiliar soundsWe are very poor judges for unfamiliar soundsnn Suggests that sound serves as signal for visual searchSuggests that sound serves as signal for visual search

nn Use sound level for familiar sourcesUse sound level for familiar sourcesnn Frequency: Auditory atmospheric hazeFrequency: Auditory atmospheric haze

nn Absorption of high fAbsorption of high fnn Sound muffledSound muffled

nn Auditory parallaxAuditory parallaxnn Sounds move faster across ears at near relative to farSounds move faster across ears at near relative to far

distancesdistances

29

Brain basis for localizationBrain basis for localizationnn ITD detectorsITD detectors

nn Brainstem: SuperiorBrainstem: Superiorolivary nucleusolivary nucleus

nn Primary auditory cortexPrimary auditory cortexnn Coincidence detectionCoincidence detection

nn Neurons fire maximallyNeurons fire maximallywhen signals arrive atwhen signals arrive atsame timesame time

nn Thus: Thus: ““coincidencecoincidence””nn Axonal distance createAxonal distance create

input delaysinput delays

Sound to right

Sound to left

30

Auditory scene analysisAuditory scene analysisnn How do we segregate different sounds beingHow do we segregate different sounds being

produced by many sources simultaneously?produced by many sources simultaneously?

nn How do we tell what frequencies belong to whatHow do we tell what frequencies belong to whatsource?source?nn E.g., Cocktail partyE.g., Cocktail partynn DonDon’’t perceive an unorganized jumble of frequenciest perceive an unorganized jumble of frequencies

nn Not simply high vs low fNot simply high vs low fnn Most f ranges overlapMost f ranges overlap

nn How do we segregate information as belonging toHow do we segregate information as belonging todistinct auditory objects?distinct auditory objects?

31

Principles of auditoryPrinciples of auditorygroupinggrouping

nn Like gestalt visual principlesLike gestalt visual principlesnn Auditory stream segregationAuditory stream segregationnn SimilaritySimilaritynn TimbreTimbrenn LocationLocationnn PitchPitchnn TimeTime

1 stream

2 streams

32

Auditory-visual interactions:Auditory-visual interactions:Location and pitchLocation and pitch

nn Visual capture of soundVisual capture of sound

nn Location: VentriloquismLocation: Ventriloquismeffecteffect

nn Pitch: McGurk effectPitch: McGurk effectnn ““BaBa””nn ““VaVa””nn ““ThaTha””nn ““DaDa””

nn Visual information isVisual information isintegrated with auditionintegrated with auditionnn Creates fused auditory visualCreates fused auditory visual

perceptionperception

33

Auditory-visual interactions:Auditory-visual interactions:Location and pitchLocation and pitch

nn Auditory experience is much more thanAuditory experience is much more thanpressure level changespressure level changes